Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -5,6 +5,8 @@ tags:
|
|
| 5 |
- llm-inference
|
| 6 |
- optimization
|
| 7 |
- compression
|
|
|
|
|
|
|
| 8 |
language:
|
| 9 |
- zh
|
| 10 |
- en
|
|
@@ -12,33 +14,34 @@ language:
|
|
| 12 |
|
| 13 |
# 📄 KV Cache Compression for Text-Only LLMs
|
| 14 |
|
| 15 |
-
**纯文本大语言模型 KV 缓存压缩 — 完整技术报告**
|
| 16 |
|
| 17 |
-
##
|
| 18 |
|
| 19 |
-
|
|
| 20 |
-
|------|------|
|
| 21 |
-
| `
|
| 22 |
-
| `
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
##
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
|
| 35 |
-
**
|
| 36 |
|
| 37 |
-
##
|
| 38 |
|
| 39 |
-
>
|
| 40 |
-
> **场景判定比算法本身更重要。**
|
| 41 |
|
| 42 |
---
|
| 43 |
|
| 44 |
-
*Cloud LTE Studio · 2026-08-
|
|
|
|
| 5 |
- llm-inference
|
| 6 |
- optimization
|
| 7 |
- compression
|
| 8 |
+
- mla
|
| 9 |
+
- deepseek
|
| 10 |
language:
|
| 11 |
- zh
|
| 12 |
- en
|
|
|
|
| 14 |
|
| 15 |
# 📄 KV Cache Compression for Text-Only LLMs
|
| 16 |
|
| 17 |
+
**纯文本大语言模型 KV 缓存压缩 — 完整技术报告** | **Full Technical Reports**
|
| 18 |
|
| 19 |
+
## Contents
|
| 20 |
|
| 21 |
+
| File | Description |
|
| 22 |
+
|------|-------------|
|
| 23 |
+
| `mla_absorbed_cache_report.md` | 🚀 MLA 吸收式缓存优化(中文) |
|
| 24 |
+
| `mla_absorbed_cache_report_en.md` | 🚀 **Absorbed MLA Cache Optimization (English)** |
|
| 25 |
+
| `kv_cache_compression_report.md` | 📊 KV 缓存压缩完整方案(中文) |
|
| 26 |
+
| `kv_compress_plan.md` | 🗺 详细实施计划(中文) |
|
| 27 |
|
| 28 |
+
## 🚀 Highlight: Absorbed MLA Cache (270KB → 8.4KB/token, 32×)
|
| 29 |
|
| 30 |
+
DeepSeek-V2-Lite MLA 优化,在 L40S 实测:
|
| 31 |
|
| 32 |
+
| Approach | KV/token | Compression | Error |
|
| 33 |
+
|----------|----------|-------------|-------|
|
| 34 |
+
| Standard MHA | 270 KB | 1x | — |
|
| 35 |
+
| Absorbed MLA | 30.4 KB | 8.9x | 0 |
|
| 36 |
+
| + per-channel INT8 | 15.2 KB | 17.8x | 0.011 |
|
| 37 |
+
| + INT4 (extreme) | **8.4 KB** | **32x** | 0.112 |
|
| 38 |
|
| 39 |
+
**Single L40S: 1.24M tokens context.**
|
| 40 |
|
| 41 |
+
## Key Insight
|
| 42 |
|
| 43 |
+
> Text-only models can be aggressively compressed (quantization + eviction), while reasoning models (R1-class) must be conservative (quantization + sliding window + tiered storage). **Scenario classification matters more than the algorithm itself.**
|
|
|
|
| 44 |
|
| 45 |
---
|
| 46 |
|
| 47 |
+
*Cloud LTE Studio · 2026-08-08 · MIT License*
|