Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,44 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
base_model: karpathy/tinyllamas
|
| 4 |
+
tags:
|
| 5 |
+
- llama2
|
| 6 |
+
- gguf
|
| 7 |
+
- tinyllamas
|
| 8 |
+
- validation
|
| 9 |
+
- test-suite
|
| 10 |
---
|
| 11 |
+
|
| 12 |
+
# TinyStories Llama2 GGUF Validation Suite
|
| 13 |
+
|
| 14 |
+
This repository provides a collection of ultra-lightweight GGUF models converted and quantized from Andrej Karpathy's `llama2.c` project.
|
| 15 |
+
|
| 16 |
+
### Why this repository exists?
|
| 17 |
+
When developing a custom LLM inference engine from scratch (C/C++, Vulkan, WebAssembly, etc.), debugging with a full-sized 7B model is slow and inefficient. This suite offers **10MB - 60MB scale GGUF models** across various quantization levels, allowing developers to implement and validate their loaders and quantization kernels step-by-step.
|
| 18 |
+
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
## 📦 Included Formats & Testing Roadmap
|
| 22 |
+
|
| 23 |
+
| Filename | Type | Size | Purpose / Validation Target |
|
| 24 |
+
| :--- | :--- | :--- | :--- |
|
| 25 |
+
| **`stories15M.F32.gguf`** | `F32` | ~60 MB | **Baseline Test.** Validates GGUF parsing, tensor layout, matrix multiplication, RoPE, and Attention logic without any dequantization overhead. |
|
| 26 |
+
| **`stories15M.F16.gguf`**<br>`stories15M.BF16.gguf` | `F16`<br>`BF16` | ~30 MB | **Half-Precision Test.** Validates 16-bit floating point loading, type casting, and inference stability. |
|
| 27 |
+
| **`stories15M.Q8_0.gguf`** | `Q8_0` | ~16 MB | **Quantization Level 1.** Validates the simplest linear quantization logic (block-based uniform scaling with 32 elements). |
|
| 28 |
+
| **`stories15M.Q4_0.gguf`**<br>`stories15M.Q4_1.gguf` | `Q4_0`<br>`Q4_1` | ~10 MB | **Quantization Level 2.** Validates classic 4-bit linear quantization and bit-unpacking logic. |
|
| 29 |
+
| **`stories15M.Q2_K`** 〜 **`Q6_K.gguf`** | `K-Quants` | 9~15 MB | **Standard Quants.** Validates modern super-block structural parsing with mixed precision. |
|
| 30 |
+
| **`stories15M.IQ3_XXS`** 〜 **`IQ4_XS.gguf`** | `I-Quants` | 8~12 MB | **Advanced Quants.** Non-linear quantization targeting lookup table (codebook) decoding logic. |
|
| 31 |
+
| **`stories15M.TQ1_0.gguf`**<br>`stories15M.TQ2_0.gguf` | `Ternary` | 7~9 MB | **Experimental.** Ternary (-1, 0, 1) state quantization for cutting-edge engine testing. |
|
| 32 |
+
| **`stories260K.F32.gguf`**<br>`stories260K.F16.gguf` | `F32`<br>`F16` | ~1 MB | **Ultra-Mini Check.** Extreme low-resource baseline utilizing a tiny 512-token vocabulary (`tok512`). |
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
|
| 36 |
+
## 📝 Model Specifications
|
| 37 |
+
- **Architecture:** Llama 2 (with scaled-down dimensions)
|
| 38 |
+
- **Dataset:** TinyStories (synthetic text dataset focused on 3 to 4-year-old vocabulary)
|
| 39 |
+
- **Vocabulary Size:** 32,000 for 15M models, 512 for 260K models.
|
| 40 |
+
|
| 41 |
+
## 📜 Acknowledgments & License
|
| 42 |
+
- **Original Weights:** Trained by Andrej Karpathy ([karpathy/tinyllamas](https://huggingface.co/karpathy/tinyllamas)).
|
| 43 |
+
- **License:** **MIT License** (inherited from the original `llama2.c` repository).
|
| 44 |
+
|