I've been running some independent benchmarking on Cold Fusion Q4 MTP using lm-evaluation-harness. Early results on the generative pass:
IFEval prompt-strict: 86.5% (loose: 88.7%)
GSM8K flexible: 92.9%
MATH-hard got zeroed out due to a harness issue — the Problem: stop string was killing the thinking trace before the model could output an answer. Not a model problem, confirmed by inspecting the captures. Planning a clean rerun with adjusted stop strings and a higher token cap.
Still learning the eval tooling, but more numbers coming once I finish the other Q4 variants and give CF a fair rerun on the affected tasks!
