docs/TRAINING.md

# NeuralAI — Training Metrics & Documentation

**Last Updated: May 18, 2026**

---

## 📈 Training Results

| Metric | Value |
| --- | --- |
| **Final Loss** | 0.040 |
| **Loss Reduction** | 98% from baseline |
| **Training Samples** | 347 (SFT) + 159 (DPO) |
| **Validation Split** | 20% |
| **Training Time** | \~45 min (Colab T4 GPU) |
| **Perplexity** | Tracked per epoch |
| **Hardware** | NVIDIA T4 (Google Colab) |

---

## 🧠 Dataset Breakdown (347 samples)

| Category | Count | Topics |
| --- | --- | --- |
| **Coding** | 80 | Python, JavaScript, SQL, REST APIs, debugging, code review |
| **ML/AI** | 45 | Transformers, RAG, fine-tuning, NLP, neural networks |
| **Data Science** | 40 | Pandas, NumPy, visualization, statistics, data cleaning |
| **Web Dev** | 35 | HTML/CSS, React, Flask, APIs, deployment |
| **General Q&A** | 50 | Concepts, explanations, comparisons, how-it-works |
| **Writing** | 35 | Emails, essays, reports, documentation |
| **System Admin** | 25 | Linux, Docker, networking, troubleshooting |
| **Math/Logic** | 37 | Algorithms, data structures, calculus, proofs |

---

## 📈 DPO Alignment Results (v5.0)

| Metric | Value | Notes |
| --- | --- | --- |
| **DPO Phase 1** | COMPLETE | 159 preference pairs |
| **Loss (DPO)** | 0.69 → 0.29 | Significant convergence |
| **Margin** | +1.10 | Model clearly distinguishes "chosen" vs "rejected" |
| **Accuracy** | 100% | Final batch accuracy on training set |
| **Categories** | 8 | Code, Logic, Reasoning, etc. |

---

## 🏗️ Proposed DPO Categories for Expansion

To improve model alignment and personality, the following categories are proposed for the next DPO phase:

1. **Memphis Culture** - Knowledge about Memphis, TN, history, music (blues/soul), and the Founder's background.
2. **AI Ethics & Safety** - Refusing harmful requests, maintaining helpful yet bounded behavior.
3. **Code Optimization** - Preferring performant, idiomatic code over naive implementations.
4. **System Architecture** - Designing scalable systems and proper abstractions.
5. **Multi-step Reasoning** - Better handling of complex, multi-part instructions.
6. **Creative Writing & Roleplay** - More engaging and personality-driven responses.
7. **Advanced Debugging** - Identifying subtle bugs and suggesting fixes with rationale.

---

## 🏗️ Training Configuration

```python
# Model
base_model = "HuggingFaceTB/SmolLM2-360M-Instruct"

# Quantization (QLoRA)
load_in_4bit = True
bnb_4bit_quant_type = "nf4"
bnb_4bit_compute_dtype = "float16"

# LoRA
lora_rank = 16
lora_alpha = 32
lora_dropout = 0.05
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

# Training
learning_rate = 2e-4
lr_scheduler = "cosine"  # with warmup
warmup_ratio = 0.1
per_device_train_batch_size = 4
gradient_accumulation_steps = 4
num_train_epochs = 3
max_grad_norm = 1.0  # gradient clipping
fp16 = True  # mixed precision

# Data
max_length = 512
train_validation_split = 0.2
```

---

## 📉 Loss Progression

| Epoch | Train Loss | Val Loss | Notes |
| --- | --- | --- | --- |
| 1 | 2.10 | 0.85 | Warmup complete |
| 2 | 0.65 | 0.18 | Rapid learning phase |
| 3 | 0.12 | 0.04 | Convergence |

**Final:** Loss 0.040 — model converges well with no overfitting (val loss close to train loss).

---

## 🔧 Training Pipeline

```markdown
1. Data Preparation
   └── train.jsonl (347 JSON samples, ChatML format)

2. Environment Setup
   └── pip install torch transformers peft bitsandbytes accelerate datasets

3. Script Execution
   └── python train_neuralai.py --epochs 3 --batch-size 4 --lr 2e-4

4. Model Output
   └── checkpoints/final_model/
       ├── adapter_model.safetensors  (LoRA weights)
       ├── adapter_config.json        (LoRA config)
       ├── tokenizer.json
       └── tokenizer_config.json

5. Deployment
   └── Upload adapter to HuggingFace Hub
   └── Deploy via Flask web UI
```

---

## ✅ Features Implemented

- ✅ LR scheduler with warmup
- ✅ 20% validation split
- ✅ Perplexity metrics (logged per epoch)
- ✅ Gradient clipping (max_norm=1.0)
- ✅ InstructionDataset class (ChatML format)
- ✅ QLoRA fine-tuning (4-bit NF4)
- ✅ Float16 training (no bitsandbytes CUDA issues)
- ✅ Flash Attention / SDPA fallback

---

## 🚨 Colab Issues Fixed (For Reference)

| Error | Fix |
| --- | --- |
| `output.input_ids[..., -1]` shape mismatch | Used `attn_implementation="eager"` |
| SDPA `torch.compile` compatibility | Added `torch.compile` fallback |
| Unused column removal crash | Set `remove_unused_columns=False` |
| `bitsandbytes` CUDA errors | Switched to float16 (no quantization) |
| Template mismatch on generation | Used `apply_chat_template` with try/except |

---

## 📦 Dependencies

```markdown
torch>=2.0
transformers>=4.40
peft>=0.10
datasets>=2.18
chromadb>=0.4
sentence-transformers>=2.3
pypdf>=4.0
python-docx>=1.0
flask>=3.0
```

---

## 🎯 Next Steps

1. **DPO Expansion** — Add more preference pairs in the proposed categories (Target: 500+ pairs).
2. **Streaming Optimization** — Refine the streaming UI for faster perceived latency.
3. **Tool Use Alignment** — Train specifically on `<tool>` tag usage and results.
4. **Automated Evaluation** — Implement a "Model vs Model" evaluation pipeline.
5. **GPU Serving** — Migrate to a persistent GPU-enabled environment.