Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- quantized
|
| 5 |
+
- gptq
|
| 6 |
+
- awq
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# quantbench-artifacts — the exact quantized weights behind the QuantBench rows
|
| 10 |
+
|
| 11 |
+
These are the **actual artifacts** produced by the 2026-07-30/31 QuantBench
|
| 12 |
+
sweep, salvaged off the Modal volume so leaderboard rows can be verified
|
| 13 |
+
against real weights instead of taken on trust.
|
| 14 |
+
|
| 15 |
+
Canonical set: each (model x method) at calibration `wikitext2-train`,
|
| 16 |
+
n=128, seed 0. The other 68 artifacts from the sweep were left on the
|
| 17 |
+
volume; these four cover the headline comparisons.
|
| 18 |
+
|
| 19 |
+
| Directory | Base model | Method | Calibration | Base licence |
|
| 20 |
+
|---|---|---|---|---|
|
| 21 |
+
| `qwen25-1p5b__gptq__wikitext2__n128__s0` | `Qwen/Qwen2.5-1.5B-Instruct` | GPTQ | wikitext2-train n=128 seed 0 | Apache-2.0 |
|
| 22 |
+
| `qwen25-1p5b__awq__wikitext2__n128__s0` | `Qwen/Qwen2.5-1.5B-Instruct` | AWQ | wikitext2-train n=128 seed 0 | Apache-2.0 |
|
| 23 |
+
| `smollm2-1p7b__gptq__wikitext2__n128__s0` | `HuggingFaceTB/SmolLM2-1.7B-Instruct` | GPTQ | wikitext2-train n=128 seed 0 | Apache-2.0 |
|
| 24 |
+
| `smollm2-1p7b__awq__wikitext2__n128__s0` | `HuggingFaceTB/SmolLM2-1.7B-Instruct` | AWQ | wikitext2-train n=128 seed 0 | Apache-2.0 |
|
| 25 |
+
|
| 26 |
+
## Provenance and honesty notes
|
| 27 |
+
|
| 28 |
+
- Quantization ran on an NVIDIA A10; evaluation ran on A10 and T4.
|
| 29 |
+
- GPTQ: `g128 desc_act=True damp=0.01 calib_len=2048`, `gptqmodel-7.3.2`,
|
| 30 |
+
torch 2.8.0+cu128 / transformers 5.14.1.
|
| 31 |
+
- AWQ: `zero_point g128 GEMM calib_len=512`, `autoawq-0.2.9`,
|
| 32 |
+
torch 2.6.0+cu124 / transformers 4.51.3 (autoawq is deprecated upstream and
|
| 33 |
+
was pinned to its last-tested combination).
|
| 34 |
+
- Calibration windows were drawn with a seeded RNG from wikitext-2-raw-v1
|
| 35 |
+
train. Seed 0 here; seeds 1 and 2 exist in the measured rows.
|
| 36 |
+
- **Every GPTQ row in this sweep loaded via `torch-fallback`, not an
|
| 37 |
+
optimized kernel** — GPTQ throughput numbers in the leaderboard measure a
|
| 38 |
+
dequantize-in-PyTorch path and are not a kernel benchmark.
|
| 39 |
+
- These weights are derivatives of Apache-2.0 base models; the base licence
|
| 40 |
+
and attribution carry over. Nothing here was human-reviewed for quality.
|
| 41 |
+
|
| 42 |
+
Measurements: `Mohaaxa/quantbench-sweep` (rows.csv + per-row JSON + logs).
|