Spaces:
Sleeping
Sleeping
Add 'When to pick KakeyaLattice over HQQ / Quanto / KIVI' comparison block
Browse files
README.md
CHANGED
|
@@ -59,6 +59,24 @@ fixed by the E8 codec design and do not depend on head_dim.)
|
|
| 59 |
|
| 60 |
Wall-clock latency per config is also reported.
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
## Caveats
|
| 63 |
|
| 64 |
- The cache roundtrips K/V but stores the reconstructed tensor in the
|
|
|
|
| 59 |
|
| 60 |
Wall-clock latency per config is also reported.
|
| 61 |
|
| 62 |
+
## When to pick KakeyaLattice over HQQ / Quanto / KIVI
|
| 63 |
+
|
| 64 |
+
- **HQQ / AWQ / GPTQ** are *weight* quantisers. KakeyaLattice is a
|
| 65 |
+
*KV-cache* quantiser. They are **orthogonal** — stack them.
|
| 66 |
+
- **QuantoQuantizedCache / HQQQuantizedCache** in transformers are
|
| 67 |
+
per-channel scalar quantisers. At ≤ 1 % |Δppl| KakeyaLattice
|
| 68 |
+
compresses the KV cache **9 %–38 % harder** across Qwen3-4B,
|
| 69 |
+
GLM-4-9B-Chat, Gemma-4-E4B, and DeepSeek-R1-Distill-Qwen-1.5B
|
| 70 |
+
(real vLLM, H200, 128 k context, WikiText-103 n=8; see the
|
| 71 |
+
[GitHub README](https://github.com/FluffyAIcode/LLM-KV--Cache-compress#headline-numbers)
|
| 72 |
+
for the full table).
|
| 73 |
+
- **KIVI (2-bit KV)** hits similar bit budgets but cannot gaussianise
|
| 74 |
+
heavy-tailed KV distributions; KakeyaLattice's Sylvester–Hadamard
|
| 75 |
+
rotation does, giving lower |Δppl| at matched bits.
|
| 76 |
+
- **SnapKV / H2O / Scissorhands** are *eviction* (which KV to keep),
|
| 77 |
+
not *quantisation* (how to store). They compose multiplicatively
|
| 78 |
+
with KakeyaLattice.
|
| 79 |
+
|
| 80 |
## Caveats
|
| 81 |
|
| 82 |
- The cache roundtrips K/V but stores the reconstructed tensor in the
|