metadata
license: gpl-3.0
tags:
- kv-cache
- llm-inference
- optimization
- compression
- mla
- deepseek
language:
- zh
- en
📄 KV Cache Compression for Text-Only LLMs
纯文本大语言模型 KV 缓存压缩 — 完整技术报告 | Full Technical Reports
Contents
| File | Description |
|---|---|
mla_absorbed_cache_report.md |
🚀 MLA 吸收式缓存优化（中文） |
mla_absorbed_cache_report_en.md |
🚀 Absorbed MLA Cache Optimization (English) |
kv_cache_compression_report.md |
📊 KV 缓存压缩完整方案（中文） |
kv_compress_plan.md |
🗺 详细实施计划（中文） |
🚀 Highlight: Absorbed MLA Cache (270KB → 8.4KB/token, 32×)
DeepSeek-V2-Lite MLA 优化，在 L40S 实测：
| Approach | KV/token | Compression | Error |
|---|---|---|---|
| Standard MHA | 270 KB | 1x | — |
| Absorbed MLA | 30.4 KB | 8.9x | 0 |
| + per-channel INT8 | 15.2 KB | 17.8x | 0.011 |
| + INT4 (extreme) | 8.4 KB | 32x | 0.112 |
Single L40S: 1.24M tokens context.
Key Insight
Text-only models can be aggressively compressed (quantization + eviction), while reasoning models (R1-class) must be conservative (quantization + sliding window + tiered storage). Scenario classification matters more than the algorithm itself.
Cloud LTE Studio · 2026-08-08 · GPL-3.0 License