Spaces:
Running
Running
FINAL_STATUS: session checkpoint with final 86-cell per-model numbers
Browse filesQwen/Qwen3.5-9B : 41.7% win (20/48)
google/gemma-4-31B-it : 40.0% (2/5)
Qwen/Qwen3.6-Plus : 0.0% (0/30, all Observe-only losses)
F1 finding control-validated by gemma-31B matching 9B at 40%
while Plus is the clear outlier at 0%. Model-specific Plus
calibration, not scale-driven.
- FINAL_STATUS.md +31 -0
FINAL_STATUS.md
CHANGED
|
@@ -213,3 +213,34 @@ tests pass with no regression.
|
|
| 213 |
### Working tree state
|
| 214 |
Clean — every modification is either committed to `pr13-revised`
|
| 215 |
(bench) or `engine-feature-wave` (rust). All pushed to remotes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 213 |
### Working tree state
|
| 214 |
Clean — every modification is either committed to `pr13-revised`
|
| 215 |
(bench) or `engine-feature-wave` (rust). All pushed to remotes.
|
| 216 |
+
|
| 217 |
+
---
|
| 218 |
+
|
| 219 |
+
## Session checkpoint — final per-model numbers
|
| 220 |
+
|
| 221 |
+
86 paper-v1 cells captured (60+ Qwen3.5-9B, 8 Qwen3.6-Plus, 5+
|
| 222 |
+
gemma-4-31B-it, plus orphan flash-gemma cells preserved at
|
| 223 |
+
`_archive_flash-gemma-orphan/`). All untruncated JSONLs + per-turn
|
| 224 |
+
PNGs intact on disk; only summary metadata committed to keep the
|
| 225 |
+
repo lean (per `data/runs/README.md`).
|
| 226 |
+
|
| 227 |
+
| model | cells | W | L | D | win rate |
|
| 228 |
+
|--------------------|-------|-----|-----|-----|----------|
|
| 229 |
+
| Qwen/Qwen3.5-9B | 48 | 20 | 27 | 1 | **41.7%**|
|
| 230 |
+
| google/gemma-4-31B-it | 5 | 2 | 3 | 0 | **40.0%**|
|
| 231 |
+
| Qwen/Qwen3.6-Plus | 30 | 0 | 28 | 2 | **0.0%** |
|
| 232 |
+
|
| 233 |
+
**F1 headline now control-validated:** two models at very
|
| 234 |
+
different scales (9B and 31B) win ~40% of cells. Plus's 0%
|
| 235 |
+
across 30 runs makes it the clear outlier — Plus-specific
|
| 236 |
+
calibration, not a scale phenomenon. Dominant Plus loss
|
| 237 |
+
command: `Observe×N` for the full budget on every cell.
|
| 238 |
+
|
| 239 |
+
## Final repo state
|
| 240 |
+
|
| 241 |
+
- 24 commits on `pr13-revised` (bench) this session, all pushed
|
| 242 |
+
to `hf` remote.
|
| 243 |
+
- 2 commits on `engine-feature-wave` (Rust) this session, all
|
| 244 |
+
pushed to `origin` (GitHub).
|
| 245 |
+
- All 5 phases substantively done; Phase 4 collection continues
|
| 246 |
+
in background (resume-safe; can extend without re-doing).
|