IST Austria Distributed Algorithms and Systems Lab

university

https://ist.ac.at/en/research/alistarh-group/

https://github.com/IST-DASLab

Activity Feed

AI & ML interests

None defined yet.

Recent Activity

anm2211 new activity about 18 hours ago

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF:[Request] Harder benchmarks!

helcig updated a model 1 day ago

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

anm2211 new activity 1 day ago

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF:Updated chat template

View all activity

Papers

Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts

WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms

View all Papers

ISTA-DASLab 's collections 12

GSQ-RCO-GGUF

Non-uniform GGUF quantizations via GSQ + RCO: per-tensor mixed precision in standard GGUF form

ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

Text Generation • 197B • Updated 1 day ago • 1.58M • 479
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

Image-Text-to-Text • 177B • Updated about 1 month ago • 7.6M • 1.09k
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Image-Text-to-Text • 27B • Updated Sep 2 • 1.28M • 2.46k
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

Image-Text-to-Text • 117B • Updated 10 days ago • 1.06M • 540

GSQ

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling, https://huggingface.co/papers/2604.18556

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling

Paper • 2604.18556 • Published Apr 20 • 24
ISTA-DASLab/Llama-3.1-70B-Instruct-2Bit-GSQ

Text Generation • 7B • Updated May 18 • 76 • 2
ISTA-DASLab/Llama-3.1-70B-Instruct-3Bit-GSQ

Text Generation • 9B • Updated May 18 • 69 • 1
ISTA-DASLab/Qwen3.6-35B-A3B-2Bit-GSQ

Image-Text-to-Text • 36B • Updated Aug 14 • 1.15k • 31

FP-Quant QAT

High-quality QAT FP4 models to use with the fp_quant vLLM/Transformers integration on Blackwell NVIDIA GPUs. See https://arxiv.org/abs/2509.23202

ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-NVFP4

2B • Updated Oct 27, 2025 • 27 • 1
ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-MXFP4

2B • Updated Oct 27, 2025 • 8
ISTA-DASLab/Llama-3.1-8B-Instruct-FPQuant-QAT-NVFP4

5B • Updated Oct 27, 2025 • 24
ISTA-DASLab/Llama-3.1-8B-Instruct-FPQuant-QAT-MXFP4

5B • Updated Oct 27, 2025 • 25

MR-GPTQ

MXFP4 and NVFP4 quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-nvfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 32
ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-mxfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 30

QuEST

https://arxiv.org/abs/2502.05003

QuEST: Stable Training of LLMs with 1-Bit Weights and Activations

Paper • 2502.05003 • Published Feb 7, 2025 • 48
ISTA-DASLab/QuEST-800M-INT1

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-sparse-INT4

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-INT4

Text Generation • Updated Feb 19, 2025

AQLM+PV

Official AQLM quantizations for "PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression": https://arxiv.org/abs/2405.14852

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

Paper • 2405.14852 • Published May 23, 2024 • 2
ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16

Text Generation • 11B • Updated Sep 17, 2024 • 35 • 45
ISTA-DASLab/Mistral-Nemo-Instruct-2407-AQLM-PV-2Bit-1x16-hf

3B • Updated Aug 16, 2024 • 16 • 3
ISTA-DASLab/Meta-Llama-3.1-8B-Instruct-AQLM-PV-2Bit-1x16-hf

Text Generation • 2B • Updated Aug 28, 2024 • 48 • 8

MoESQ

Paired-4:8 + NVFP4 W4A4 expert compressed MoE models for NVIDIA Blackwell Sparse Tensor Cores.

ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ

Text Generation • 646B • Updated 25 days ago • 556 • 1
ISTA-DASLab/Qwen3-30B-A3B-P48NVFP4-MoESQ

Text Generation • 20B • Updated 25 days ago • 882 • 6
ISTA-DASLab/Qwen3.5-397B-A17B-P48NVFP4-MoESQ

Image-Text-to-Text • 258B • Updated 25 days ago • 79 • 1
ISTA-DASLab/Qwen3.8-Flash-Next-P48NVFP4-MoESQ

Text Generation • 135B • Updated 25 days ago • 1.26k • 13

Optimizers

DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

Paper • 2602.02016 • Published Feb 2 • 14

MatGPTQ

MatGPTQ quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MatGPTQ

Text Generation • 8B • Updated 9 days ago • 193
ISTA-DASLab/Qwen3-8B-MatGPTQ

Text Generation • 8B • Updated 9 days ago • 150 • 3
MatGPTQ: Accurate and Efficient Post-Training Matryoshka Quantization

Paper • 2602.03537 • Published Feb 3 • 6

Gemma3-GPTQ

GPTQ-quantized Gemma3 models

ISTA-DASLab/gemma-3-4b-it-GPTQ-4b-128g

Image-Text-to-Text • 4B • Updated Apr 13, 2025 • 625 • 7
ISTA-DASLab/gemma-3-12b-it-GPTQ-4b-128g

Image-Text-to-Text • 12B • Updated Apr 13, 2025 • 2.24k • 6
ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g

Image-Text-to-Text • 28B • Updated Mar 20, 2025 • 384k • 44

HIGGS

Models prequantized with [HIGGS](https://arxiv.org/abs/2411.17525) zero-shot quantization. Requires the latest `transformers` to run.

Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

Paper • 2411.17525 • Published Nov 26, 2024 • 6
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-4bit

Text Generation • 3B • Updated Dec 10, 2024 • 31
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-3bit

Text Generation • 2B • Updated Dec 10, 2024 • 34
ISTA-DASLab/Llama-3.1-8B-HIGGS-GPTQ-4bit

Text Generation • 3B • Updated Dec 10, 2024 • 29

AQLM

AQLM quantized LLMs

Extreme Compression of Large Language Models via Additive Quantization

Paper • 2401.06118 • Published Jan 11, 2024 • 15
ISTA-DASLab/Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16

Text Generation • 11B • Updated May 13, 2024 • 33 • 20
ISTA-DASLab/Meta-Llama-3-8B-Instruct-AQLM-2Bit-1x16

Text Generation • 2B • Updated Nov 8, 2024 • 114 • 12
ISTA-DASLab/Meta-Llama-3-8B-AQLM-2Bit-1x16

Text Generation • 2B • Updated Jul 2, 2024 • 35 • 8