Rick Holmberg PRO
AI & ML interests
Recent Activity
Organizations
Interpreting early update behaviour in SSM activations
and the @liquidai blog comes with some nice details about the training procedure, so let's analyze it.
basically, a full agent training pipeline but compressed into 2.6B
base model → SFT → specialized teachers per domain (SFT + RLVR) → on-policy distillation back into one student → agentic RL
the two most interesting stages
→ MOPD: the student generates, each prompt routes to its domain teacher for token-level feedback. teachers branch from the same SFT checkpoint, so their signal stays close to the student's distribution
→ agentic RL: multi-turn GRPO inside real harnesses (OpenClaw, Hermes Agent), one sandbox per rollout, a proxy captures token-level trajectories while the harness stays a black box
this makes a 2.6B that beats much larger models on instruction following and tool use
SFT, distillation, RL, RL envs: exactly what we're covering in our Training Agents livestream series (next one coming soon!)
→ model: LiquidAI/LFM2.5-2.6B
→ blog: https://www.liquid.ai/blog/lfm2-5-2-6b
→ live series: https://www.youtube.com/playlist?list=PLo2EIpI_JMQvQZm-kVlz4wY1vWF0LBcf5
Yo convolution layers... You caused me to pay attention hahaha get it. An AI model would laugh at this joke and say "That joke is doing more work than the 22 conv layers, this is a pun on the causal convolution layers." Ahhh dad jokes aside I'm actually kinda stoked to run a little atlas on your model. It's like 22 people sitting at a bar eavesdropping on a few people adjacent to the left of them and predicting based on that narrow view point with 8 patrons listening to the entire thing with clarity and scratching their heads. Ima see what saving KV cache is doing here. To hell with the benchmarks and arXiv papers! Let's weigh it measure it and properly fit it in a suit. I wanna see if it looks properly proportioned or if it's a saw toothed oddball with wide shoulders a narrow midriff big ol' booty tiny calves huge shins and stiletto sized feet.