metadata
title: Quantization Explorer
emoji: ⚙️
colorFrom: blue
colorTo: indigo
sdk: static
pinned: false
license: mit
short_description: 'Explore quantization: FP8, INT8, INT4 and trade-offs.'
Quantization Explorer
Quantization Explorer is an educational Hugging Face Space by the open-weight organization.
It explains how model quantization reduces memory requirements by representing weights at lower precision, and how common approaches such as FP8, INT8, INT4, bitsandbytes, GPTQ, AWQ and GGUF quantization differ in purpose and trade-offs.
What you can explore
- What model quantization is
- FP16/BF16 vs. FP8 vs. INT8 vs. INT4
- Theoretical raw weight memory
- Post-training quantization
- On-the-fly quantization
- Calibration-based methods
- bitsandbytes
- GPTQ
- AWQ
- GGUF / llama.cpp quantization
- Quality, speed and compatibility trade-offs
- A simple quantization decision helper
Core idea
Higher-precision weights
↓
Quantization method
↓
Lower-bit representation
↓
Lower memory / storage
↓
Potential speed benefits
+
Possible quality / compatibility trade-offs
Primary references
- Hugging Face Transformers — Quantization overview: https://huggingface.co/docs/transformers/quantization/overview
- bitsandbytes: https://huggingface.co/docs/transformers/en/quantization/bitsandbytes
- GPTQ: https://huggingface.co/docs/transformers/quantization/gptq
- AWQ: https://huggingface.co/docs/transformers/quantization/awq
- llama.cpp quantization: https://github.com/ggml-org/llama.cpp/tree/master/tools/quantize
Related organization
Open Weight
https://huggingface.co/open-weight
Related project
Open Weights
https://huggingface.co/open-weights
Collaboration
Open-weight AI, model infrastructure, inference, deployment, research and ecosystem partnerships.
Contact: agenten@magenta.de