tensorlink-dev/cascade-toto2-4m-best-bench

cascade · Toto2-4M — best public-benchmark checkpoint (round 8 challenger)

The strongest checkpoint (on public benchmarks) yet produced by cascade, a Bittensor subnet (netuid 91) where miners compete on data, not models: each miner submits a synthetic time-series data generator (pure code, no weights), and the subnet trains this fixed 4M-parameter Toto2 from random initialization on each generator's corpus in a single ~3 h round, then evaluates on private, rotating real-world data the miners never see.

This model was trained on the corpus of the round-8 challenger generator (miner uid 72). It won the public benchmarks but lost its duel on the private eval — kept here as the best public-bench artifact the subnet has produced to date (as of 2026-08-10).

Benchmark results

Scored with the official gift-eval harness (full 97-config GIFT-Eval suite and BOOM, official Seasonal-Naive-normalized aggregation, leaderboard-comparable) plus the TIME benchmark:

suite	CRPS	MASE
GIFT-Eval	0.6009	0.8993
BOOM	0.4304	0.6951
TIME	0.6736	0.9200

(Lower is better; values are ratios vs the Seasonal-Naive baseline under the official shifted-geometric-mean aggregation.) bench_report.json in this repo is the trainer-signed score record.

For scale: this is a 4M-parameter model trained for ~3 GPU-hours from scratch. The point is not to rival large foundation models — it is that data quality alone moved a fixed tiny model from ~0.68 (subnet genesis) to 0.60 GIFT-Eval CRPS in eight competitive rounds.

Files

• weights.safetensors — trained Toto2-4M parameters
• config.json — architecture config
• model.py — model implementation
• forecast_wrapper.py — inference entry point (forecast_quantiles_batch(histories, horizon) quantile head); the same code path the subnet's validator and benchmark sidecar score through
• bench_report.json — signed benchmark record published by the subnet trainer

Provenance

Content-addressed original on the Hippius Hub registry:

metro-v1:trained:hippius:cascade/ckpt-r13786693137342042853-challenger-toto2-4m@sha256:702bf6c7924dd1c58af273347c87a41e43f9a83ce76f9983ae6f99ba9014cd39

Round 13786693137342042853 (2026-08-08, netuid 91). Training is deterministic under the subnet's reproducibility contract (pinned torch 2.4.1+cu124, Python 3.11, fixed seeds): anyone can re-derive this checkpoint from the on-chain round data and the miner's revealed generator.