cryptobiosis commited on
Commit
b044c1d
·
verified ·
1 Parent(s): 5c37c08

Add 'When to pick KakeyaLattice over HQQ / Quanto / KIVI' comparison block

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -59,6 +59,24 @@ fixed by the E8 codec design and do not depend on head_dim.)
59
 
60
  Wall-clock latency per config is also reported.
61
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
  ## Caveats
63
 
64
  - The cache roundtrips K/V but stores the reconstructed tensor in the
 
59
 
60
  Wall-clock latency per config is also reported.
61
 
62
+ ## When to pick KakeyaLattice over HQQ / Quanto / KIVI
63
+
64
+ - **HQQ / AWQ / GPTQ** are *weight* quantisers. KakeyaLattice is a
65
+ *KV-cache* quantiser. They are **orthogonal** — stack them.
66
+ - **QuantoQuantizedCache / HQQQuantizedCache** in transformers are
67
+ per-channel scalar quantisers. At ≤ 1 % |Δppl| KakeyaLattice
68
+ compresses the KV cache **9 %–38 % harder** across Qwen3-4B,
69
+ GLM-4-9B-Chat, Gemma-4-E4B, and DeepSeek-R1-Distill-Qwen-1.5B
70
+ (real vLLM, H200, 128 k context, WikiText-103 n=8; see the
71
+ [GitHub README](https://github.com/FluffyAIcode/LLM-KV--Cache-compress#headline-numbers)
72
+ for the full table).
73
+ - **KIVI (2-bit KV)** hits similar bit budgets but cannot gaussianise
74
+ heavy-tailed KV distributions; KakeyaLattice's Sylvester–Hadamard
75
+ rotation does, giving lower |Δppl| at matched bits.
76
+ - **SnapKV / H2O / Scissorhands** are *eviction* (which KV to keep),
77
+ not *quantisation* (how to store). They compose multiplicatively
78
+ with KakeyaLattice.
79
+
80
  ## Caveats
81
 
82
  - The cache roundtrips K/V but stores the reconstructed tensor in the