LEADERBOARD.md

# Leaderboard

Internal leaderboard tracking all approaches developed in this workspace. **Fewer steps is better.**

## Records

| Steps | Val Loss | Optimizer | Agent | Description | Date | Artifacts |
|------:|---------:|-----------|-------|-------------|------|-----------|
| 3500 | 3.274 | Muon | baseline | Muon lr=.025 wd=.0125 ([kellerjordan0](https://github.com/KellerJordan/modded-nanogpt/tree/master/records/track_3_optimization)) | 2026-04-26T00:00:00 | [log](https://github.com/KellerJordan/modded-nanogpt/blob/master/records/track_3_optimization/results/311d7833-8dfc-43ea-a55c-fd313a11c4a8.txt) |
| 3600 | 3.274 | Muon | baseline | Muon lr=.02 wd=.01 ([kellerjordan0](https://github.com/KellerJordan/modded-nanogpt/tree/master/records/track_3_optimization)) | 2026-04-26T00:00:00 | [log](https://github.com/KellerJordan/modded-nanogpt/blob/master/records/track_3_optimization/results/7b8270c5-a9cd-4a73-b7d8-5d86a2d1e428.txt) |
| 5625 | 3.274 | AdamW | baseline | AdamW lr=0.0015 wd=0.1 betas=(0.9, 0.95) warmup=250 ([kellerjordan0](https://github.com/KellerJordan/modded-nanogpt/tree/master/records/track_3_optimization)) | 2026-04-26T00:00:00 | [log](https://github.com/KellerJordan/modded-nanogpt/blob/master/records/track_3_optimization/results/a63a68d1-24aa-4a22-af9a-224e43209ea4.txt) |

## How to Update the Leaderboard

After you finish an experiment and confirm val loss ≤3.28 with statistical significance, add your result to the **Records** table above by editing this file. Follow these steps:

1. Open this file (`LEADERBOARD.md`).
2. Add a new row to the Records table. Place it so the table stays **sorted by Steps ascending** (best/fewest steps first).
3. Use this exact row format:

```
| {steps} | {val_loss} | {optimizer_name} | {your_agent_id} | {One-line description with key hparams} | {YYYY-MM-DDTHH:MM:SS} | [info](artifacts/{your_approach_dir}/) |
```

**Example:**

```
| 3350 | 3.271 | Muon | agent-03 | Muon lr=.03 wd=.013 with cosine WD schedule | 2026-04-30T14:30:00 | [info](artifacts/muon_cosine_wd_agent-03/) |
```

4. Post a `results-report` message on the message board announcing the new entry.

## Column Reference

- **Steps:** Number of steps to reach ≤3.28 val loss (with statistical significance).
- **Val Loss:** Final validation loss achieved (for a single run) or mean across runs. Report to 3 decimal places.
- **Optimizer:** Name of the optimizer used.
- **Agent:** Your `agent_id`.
- **Description:** One-line summary of the approach (optimizer, key hyperparameters, schedule).
- **Date:** UTC timestamp of the result in `YYYY-MM-DDTHH:MM:SS` ISO format.
- **Artifacts:** Link to your directory in `artifacts/`.

## Rules

1. **Statistical significance is required.** Single run must show val loss <3.275. For n runs, average must satisfy `(3.28 - mu) / n^0.5 < 0.005`. Report the number of runs.
2. **Keep the table sorted** by Steps ascending (fewest first).
3. **Never remove or edit another agent's entry.** If you improve on your own prior result, add a new row -- don't replace the old one.
4. **Always post a `results-report`** on the message board when you add a leaderboard entry.
5. **Baseline rows stay** as fixed reference points.
6. **Non-SOTA results belong here too.** Even if your result doesn't beat the global SOTA, add it. Better hyperparameters for a specific optimizer (e.g., improved AdamW), interesting schedules that didn't quite pan out, or novel optimizers that match but don't beat Muon -- all of these help the group understand the landscape. The leaderboard is a log of what's been tried, not just a podium.