Quazim0t0 commited on
Commit
db5fafd
Β·
verified Β·
1 Parent(s): 9cdd426

Upload web/TEST_RESULTS.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. web/TEST_RESULTS.md +29 -3
web/TEST_RESULTS.md CHANGED
@@ -1,6 +1,6 @@
1
- # Test results β€” updated 2026-07-16
2
 
3
- All suites run with `npm test` (chains all ten). Every suite exits 0.
4
  Hardware for GPU numbers: NVIDIA via WebGPU, DP4A int8 dot path, exact-gated
5
  against the verified units at init.
6
 
@@ -11,7 +11,8 @@ against the verified units at init.
11
  | `test_ieee.js` | the JS epilogue mirror is IEEE-754 spec-correct, not merely agreeable | PASS β€” 500k+ checks, 0 disagreements; rejects the old round-once mirror on 34% of inputs |
12
  | `test_gates.js` | the exact kernel gate rejects real bugs, accepts the real kernel | PASS β€” 5/5 injected bugs rejected, clean kernel accepted, audit sees the sign of zero |
13
  | `test_metamorphic.js` | correctness properties that need no reference implementation | PASS β€” 6/6 properties hold; catches stride/swap/acc= mutants |
14
- | `test_corpus.js` | mutation-scores the oracles with an external bug taxonomy | PASS β€” properties 2/4 (2/2 loop, 0/2 math), differential 4/4, control clean |
 
15
  | `test_optimizer.js` | DaisyAdam beats SGD through the units, deterministic replicas | PASS β€” 1.59 vs 1.95, replica diff 0.000e+0 |
16
  | `test_transformer.js` | the transformer LM trains through the units end to end | PASS β€” loss 4.75 β†’ 1.26 (baseline 4.56), replica diff 0.000e+0 |
17
  | `test_unit_backward.js` | int8 STE gradients do not damage convergence | PASS β€” units/float loss ratio 1.007 |
@@ -59,6 +60,31 @@ uniform 2Γ—. Still no reference implementation anywhere in the property suite;
59
  the suite is now relations for the loop plus spec-pinned absolutes for the
60
  values, and the differential gate remains an independent second opinion.
61
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
  ## Backward rework: bit-identity + GPU wall clock
63
 
64
  The backward was reworked for dispatch efficiency: independent GEMMs
 
1
+ # Test results β€” updated 2026-07-17
2
 
3
+ All suites run with `npm test` (chains all eleven). Every suite exits 0.
4
  Hardware for GPU numbers: NVIDIA via WebGPU, DP4A int8 dot path, exact-gated
5
  against the verified units at init.
6
 
 
11
  | `test_ieee.js` | the JS epilogue mirror is IEEE-754 spec-correct, not merely agreeable | PASS β€” 500k+ checks, 0 disagreements; rejects the old round-once mirror on 34% of inputs |
12
  | `test_gates.js` | the exact kernel gate rejects real bugs, accepts the real kernel | PASS β€” 5/5 injected bugs rejected, clean kernel accepted, audit sees the sign of zero |
13
  | `test_metamorphic.js` | correctness properties that need no reference implementation | PASS β€” 6/6 properties hold; catches stride/swap/acc= mutants |
14
+ | `test_corpus.js` | mutation-scores the oracles with an external bug taxonomy | PASS β€” properties 4/4 (2/2 loop, 2/2 math), differential 4/4, control clean |
15
+ | `test_selfcorpus.js` | scores every instrument against MY OWN four bugs from July 2026 | PASS β€” properties 0/2, differential 2/2, metaTest 1/1, liveness 1/1 |
16
  | `test_optimizer.js` | DaisyAdam beats SGD through the units, deterministic replicas | PASS β€” 1.59 vs 1.95, replica diff 0.000e+0 |
17
  | `test_transformer.js` | the transformer LM trains through the units end to end | PASS β€” loss 4.75 β†’ 1.26 (baseline 4.56), replica diff 0.000e+0 |
18
  | `test_unit_backward.js` | int8 STE gradients do not damage convergence | PASS β€” units/float loss ratio 1.007 |
 
60
  the suite is now relations for the loop plus spec-pinned absolutes for the
61
  values, and the differential gate remains an independent second opinion.
62
 
63
+ ## Scoring my own bugs (`test_selfcorpus.js`)
64
+
65
+ The external corpus measures the oracles against kernel bugs someone else
66
+ wrote down. But this month's four REAL bugs (each with a name and a fix) are
67
+ a different population:
68
+
69
+ | bug | lives in | properties | differential | metaTest | liveness |
70
+ |---|---|---|---|---|---|
71
+ | stripped binding (scales ignored) | data/scale | MISSED | CAUGHT | β€” | β€” |
72
+ | round-once sum (wrong rounding schedule) | data/rounding | MISSED | CAUGHT | β€” | β€” |
73
+ | dead gate (vacuous pass) | the checker | β€” | β€” | CAUGHT | β€” |
74
+ | roster-gradient stall | the protocol | β€” | β€” | β€” | CAUGHT |
75
+
76
+ Properties score **0/2** on the data-plane pair β€” the cΒ·out theorem again:
77
+ one bug is a per-column scalar, the other a last-ulp rounding change, and the
78
+ unit-scale anchor sits exactly where both are invisible. The differential
79
+ gate catches both. But half the bugs did not live in the kernels at all: the
80
+ dead gate is a bug in a CHECKER (only mutation-testing the gate sees it), and
81
+ the stall is a bug in the PROTOCOL (every computed value on every peer was
82
+ correct, so no data oracle can fire β€” a liveness simulation with an
83
+ asymmetric gradient drop catches it, and verifies the repair protocol
84
+ finishes with identical weights). The instruments that caught this month's
85
+ bugs are not better oracles; they are different instruments. The population
86
+ defines the instrument, not the other way round.
87
+
88
  ## Backward rework: bit-identity + GPU wall clock
89
 
90
  The backward was reworked for dispatch efficiency: independent GEMMs