Upload web/TEST_RESULTS.md with huggingface_hub
Browse files- web/TEST_RESULTS.md +29 -3
web/TEST_RESULTS.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
-
# Test results β updated 2026-07-
|
| 2 |
|
| 3 |
-
All suites run with `npm test` (chains all
|
| 4 |
Hardware for GPU numbers: NVIDIA via WebGPU, DP4A int8 dot path, exact-gated
|
| 5 |
against the verified units at init.
|
| 6 |
|
|
@@ -11,7 +11,8 @@ against the verified units at init.
|
|
| 11 |
| `test_ieee.js` | the JS epilogue mirror is IEEE-754 spec-correct, not merely agreeable | PASS β 500k+ checks, 0 disagreements; rejects the old round-once mirror on 34% of inputs |
|
| 12 |
| `test_gates.js` | the exact kernel gate rejects real bugs, accepts the real kernel | PASS β 5/5 injected bugs rejected, clean kernel accepted, audit sees the sign of zero |
|
| 13 |
| `test_metamorphic.js` | correctness properties that need no reference implementation | PASS β 6/6 properties hold; catches stride/swap/acc= mutants |
|
| 14 |
-
| `test_corpus.js` | mutation-scores the oracles with an external bug taxonomy | PASS β properties
|
|
|
|
| 15 |
| `test_optimizer.js` | DaisyAdam beats SGD through the units, deterministic replicas | PASS β 1.59 vs 1.95, replica diff 0.000e+0 |
|
| 16 |
| `test_transformer.js` | the transformer LM trains through the units end to end | PASS β loss 4.75 β 1.26 (baseline 4.56), replica diff 0.000e+0 |
|
| 17 |
| `test_unit_backward.js` | int8 STE gradients do not damage convergence | PASS β units/float loss ratio 1.007 |
|
|
@@ -59,6 +60,31 @@ uniform 2Γ. Still no reference implementation anywhere in the property suite;
|
|
| 59 |
the suite is now relations for the loop plus spec-pinned absolutes for the
|
| 60 |
values, and the differential gate remains an independent second opinion.
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
## Backward rework: bit-identity + GPU wall clock
|
| 63 |
|
| 64 |
The backward was reworked for dispatch efficiency: independent GEMMs
|
|
|
|
| 1 |
+
# Test results β updated 2026-07-17
|
| 2 |
|
| 3 |
+
All suites run with `npm test` (chains all eleven). Every suite exits 0.
|
| 4 |
Hardware for GPU numbers: NVIDIA via WebGPU, DP4A int8 dot path, exact-gated
|
| 5 |
against the verified units at init.
|
| 6 |
|
|
|
|
| 11 |
| `test_ieee.js` | the JS epilogue mirror is IEEE-754 spec-correct, not merely agreeable | PASS β 500k+ checks, 0 disagreements; rejects the old round-once mirror on 34% of inputs |
|
| 12 |
| `test_gates.js` | the exact kernel gate rejects real bugs, accepts the real kernel | PASS β 5/5 injected bugs rejected, clean kernel accepted, audit sees the sign of zero |
|
| 13 |
| `test_metamorphic.js` | correctness properties that need no reference implementation | PASS β 6/6 properties hold; catches stride/swap/acc= mutants |
|
| 14 |
+
| `test_corpus.js` | mutation-scores the oracles with an external bug taxonomy | PASS β properties 4/4 (2/2 loop, 2/2 math), differential 4/4, control clean |
|
| 15 |
+
| `test_selfcorpus.js` | scores every instrument against MY OWN four bugs from July 2026 | PASS β properties 0/2, differential 2/2, metaTest 1/1, liveness 1/1 |
|
| 16 |
| `test_optimizer.js` | DaisyAdam beats SGD through the units, deterministic replicas | PASS β 1.59 vs 1.95, replica diff 0.000e+0 |
|
| 17 |
| `test_transformer.js` | the transformer LM trains through the units end to end | PASS β loss 4.75 β 1.26 (baseline 4.56), replica diff 0.000e+0 |
|
| 18 |
| `test_unit_backward.js` | int8 STE gradients do not damage convergence | PASS β units/float loss ratio 1.007 |
|
|
|
|
| 60 |
the suite is now relations for the loop plus spec-pinned absolutes for the
|
| 61 |
values, and the differential gate remains an independent second opinion.
|
| 62 |
|
| 63 |
+
## Scoring my own bugs (`test_selfcorpus.js`)
|
| 64 |
+
|
| 65 |
+
The external corpus measures the oracles against kernel bugs someone else
|
| 66 |
+
wrote down. But this month's four REAL bugs (each with a name and a fix) are
|
| 67 |
+
a different population:
|
| 68 |
+
|
| 69 |
+
| bug | lives in | properties | differential | metaTest | liveness |
|
| 70 |
+
|---|---|---|---|---|---|
|
| 71 |
+
| stripped binding (scales ignored) | data/scale | MISSED | CAUGHT | β | β |
|
| 72 |
+
| round-once sum (wrong rounding schedule) | data/rounding | MISSED | CAUGHT | β | β |
|
| 73 |
+
| dead gate (vacuous pass) | the checker | β | β | CAUGHT | β |
|
| 74 |
+
| roster-gradient stall | the protocol | β | β | β | CAUGHT |
|
| 75 |
+
|
| 76 |
+
Properties score **0/2** on the data-plane pair β the cΒ·out theorem again:
|
| 77 |
+
one bug is a per-column scalar, the other a last-ulp rounding change, and the
|
| 78 |
+
unit-scale anchor sits exactly where both are invisible. The differential
|
| 79 |
+
gate catches both. But half the bugs did not live in the kernels at all: the
|
| 80 |
+
dead gate is a bug in a CHECKER (only mutation-testing the gate sees it), and
|
| 81 |
+
the stall is a bug in the PROTOCOL (every computed value on every peer was
|
| 82 |
+
correct, so no data oracle can fire β a liveness simulation with an
|
| 83 |
+
asymmetric gradient drop catches it, and verifies the repair protocol
|
| 84 |
+
finishes with identical weights). The instruments that caught this month's
|
| 85 |
+
bugs are not better oracles; they are different instruments. The population
|
| 86 |
+
defines the instrument, not the other way round.
|
| 87 |
+
|
| 88 |
## Backward rework: bit-identity + GPU wall clock
|
| 89 |
|
| 90 |
The backward was reworked for dispatch efficiency: independent GEMMs
|