NeuralAI — Training Metrics & Documentation

Last Updated: May 18, 2026

📈 Training Results

Metric	Value
Final Loss	0.040
Loss Reduction	98% from baseline
Training Samples	347 (SFT) + 159 (DPO)
Validation Split	20%
Training Time	~45 min (Colab T4 GPU)
Perplexity	Tracked per epoch
Hardware	NVIDIA T4 (Google Colab)

🧠 Dataset Breakdown (347 samples)

Category	Count	Topics
Coding	80	Python, JavaScript, SQL, REST APIs, debugging, code review
ML/AI	45	Transformers, RAG, fine-tuning, NLP, neural networks
Data Science	40	Pandas, NumPy, visualization, statistics, data cleaning
Web Dev	35	HTML/CSS, React, Flask, APIs, deployment
General Q&A	50	Concepts, explanations, comparisons, how-it-works
Writing	35	Emails, essays, reports, documentation
System Admin	25	Linux, Docker, networking, troubleshooting
Math/Logic	37	Algorithms, data structures, calculus, proofs

📈 DPO Alignment Results (v5.0)

Metric	Value	Notes
DPO Phase 1	COMPLETE	159 preference pairs
Loss (DPO)	0.69 → 0.29	Significant convergence
Margin	+1.10	Model clearly distinguishes "chosen" vs "rejected"
Accuracy	100%	Final batch accuracy on training set
Categories	8	Code, Logic, Reasoning, etc.

🏗️ Proposed DPO Categories for Expansion

To improve model alignment and personality, the following categories are proposed for the next DPO phase:

Memphis Culture - Knowledge about Memphis, TN, history, music (blues/soul), and the Founder's background.
AI Ethics & Safety - Refusing harmful requests, maintaining helpful yet bounded behavior.
Code Optimization - Preferring performant, idiomatic code over naive implementations.
System Architecture - Designing scalable systems and proper abstractions.
Multi-step Reasoning - Better handling of complex, multi-part instructions.
Creative Writing & Roleplay - More engaging and personality-driven responses.
Advanced Debugging - Identifying subtle bugs and suggesting fixes with rationale.

🏗️ Training Configuration

# Model
base_model = "HuggingFaceTB/SmolLM2-360M-Instruct"

# Quantization (QLoRA)
load_in_4bit = True
bnb_4bit_quant_type = "nf4"
bnb_4bit_compute_dtype = "float16"

# LoRA
lora_rank = 16
lora_alpha = 32
lora_dropout = 0.05
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

# Training
learning_rate = 2e-4
lr_scheduler = "cosine"  # with warmup
warmup_ratio = 0.1
per_device_train_batch_size = 4
gradient_accumulation_steps = 4
num_train_epochs = 3
max_grad_norm = 1.0  # gradient clipping
fp16 = True  # mixed precision

# Data
max_length = 512
train_validation_split = 0.2

📉 Loss Progression

Epoch	Train Loss	Val Loss	Notes
1	2.10	0.85	Warmup complete
2	0.65	0.18	Rapid learning phase
3	0.12	0.04	Convergence

Final: Loss 0.040 — model converges well with no overfitting (val loss close to train loss).

🔧 Training Pipeline

1. Data Preparation
   └── train.jsonl (347 JSON samples, ChatML format)

2. Environment Setup
   └── pip install torch transformers peft bitsandbytes accelerate datasets

3. Script Execution
   └── python train_neuralai.py --epochs 3 --batch-size 4 --lr 2e-4

4. Model Output
   └── checkpoints/final_model/
       ├── adapter_model.safetensors  (LoRA weights)
       ├── adapter_config.json        (LoRA config)
       ├── tokenizer.json
       └── tokenizer_config.json

5. Deployment
   └── Upload adapter to HuggingFace Hub
   └── Deploy via Flask web UI

✅ Features Implemented

✅ LR scheduler with warmup
✅ 20% validation split
✅ Perplexity metrics (logged per epoch)
✅ Gradient clipping (max_norm=1.0)
✅ InstructionDataset class (ChatML format)
✅ QLoRA fine-tuning (4-bit NF4)
✅ Float16 training (no bitsandbytes CUDA issues)
✅ Flash Attention / SDPA fallback

🚨 Colab Issues Fixed (For Reference)

Error	Fix
`output.input_ids[..., -1]` shape mismatch	Used `attn_implementation="eager"`
SDPA `torch.compile` compatibility	Added `torch.compile` fallback
Unused column removal crash	Set `remove_unused_columns=False`
`bitsandbytes` CUDA errors	Switched to float16 (no quantization)
Template mismatch on generation	Used `apply_chat_template` with try/except

📦 Dependencies

torch>=2.0
transformers>=4.40
peft>=0.10
datasets>=2.18
chromadb>=0.4
sentence-transformers>=2.3
pypdf>=4.0
python-docx>=1.0
flask>=3.0

🎯 Next Steps

DPO Expansion — Add more preference pairs in the proposed categories (Target: 500+ pairs).
Streaming Optimization — Refine the streaming UI for faster perceived latency.
Tool Use Alignment — Train specifically on <tool> tag usage and results.
Automated Evaluation — Implement a "Model vs Model" evaluation pipeline.
GPU Serving — Migrate to a persistent GPU-enabled environment.