Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention Paper • 2606.20945 • Published Jun 18 • 80
FrontiersMind/Nandi-Mini-V1.1-600M-Intermediate-Checkpoint-400GT Text Generation • 0.6B • Updated May 30 • 67 • 8
view article Article How I contributed a new model to the Transformers library using Codex nielsr • Mar 30 • 52
Running on CPU Upgrade 265 The Synthetic Data Playbook: Generating Trillions of the Finest Tokens 📝 265 Visualize synthetic‑data experiments as an interactive bookshelf
view article Article The 1 Billion Token Challenge: Finding the Perfect Pre-training Mix codelion • Nov 3, 2025 • 65
Running Featured 1.39k FineWeb: decanting the web for the finest text data at scale 🍷 1.39k Explore and download the FineWeb web‑scale text dataset
Running 3.94k The Ultra-Scale Playbook 🌌 3.94k The ultimate guide to training LLM on large GPU Clusters
The Instruction Gap: LLMs get lost in Following Instruction Paper • 2601.03269 • Published Dec 19, 2025 • 8
Running on CPU Upgrade Featured 3.24k The Smol Training Playbook 📚 3.24k The secrets to building world-class LLMs