README.md

metadata

title: README
emoji: 🏃
colorFrom: purple
colorTo: blue
sdk: static
pinned: false

Training Small Language Models with Knowledge Distillation

Official pre-trained models and baselines in

• MiniLLM: Knowledge distillation of LLMs during instruction tuning.
• MiniPLM: Knowledge distillation of LLMs during pre-training.