Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,51 @@
+# 5M Text Generator
+
+A small 5M parameter text generation model trained on educational/scientific text data.
+
+## Model Details
+
+- **Parameters**: 4,983,808 (4.98M)
+- **Architecture**: Transformer Decoder with GQA, RoPE, SwiGLU
+- **Training**: 1000 steps on 5000 text samples
+- **Vocabulary**: 8000 tokens (BPE)
+
+## Usage
+
+```python
+from transformers import PreTrainedTokenizerFast
+import torch
+import sys
+sys.path.insert(0, ".")  # if needed
+
+from model import TextDecoder
+
+# Load model
+model = TextDecoder.from_pretrained("CodeDevX/5m-text-generator")
+tokenizer = PreTrainedTokenizerFast.from_pretrained("CodeDevX/5m-text-generator")
+
+# Generate text
+input_ids = tokenizer("The quick brown fox", return_tensors="pt").input_ids
+output = model.generate(input_ids, max_new_tokens=50, temperature=0.8)
+print(tokenizer.decode(output[0]))
+```
+
+## Architecture
+
+- Hidden size: 256
+- Layers: 6
+- Attention heads: 8 (KV heads: 2)
+- Intermediate size: 768
+- Max sequence length: 512
+
+## Training Results
+
+| Step | Loss |
+|------|------|
+| 100 | 12.68 |
+| 200 | 2.17 |
+| 500 | 0.15 |
+| 1000 | 0.13 |
+
+## License
+
+MIT