Learning to Route — coding router benchmark + anchor tables
Companion artifacts for Learning to Route: a 27-task coding benchmark (medium-hard exact tasks plus quality-scored optimization tasks, all with adversarial hidden tests), per-model outcome logs from cheap LLMs plus one frontier tier (deepseek-v4-flash, gpt-5.4-nano/mini, gemini-3.5-flash, gpt-5.5), and trained router anchor tables (static-embedding vectors plus per-model score/cost stats).
Headline result: a verify and escalate cascade over the four cheap models solves 100% of benchmark tasks at 26% of the cost of the best single model and 4% of the cost of gpt-5.5, which ties a $0.03 model at 77.8% pass. Per solved task: cascade $0.0041 vs frontier $0.1213. Full numbers in report.json.
Routing = one static embed (~0.15ms CPU, 16MB model) + k-NN over these anchors. Serve from Python (pybed), Go (gobed), or Zig (zbed).
tasks.jsonl— benchmark tasks (prompt, entry_point, difficulty, hidden tests)results.jsonl— (task, model, passed, cost, latency) outcome logrouter.json— trained anchor table:{text, vec[512], stats: {model: {n, pass, cost}}}
See the whitepaper.