Quazim0t0 commited on
Commit
aaeaa6e
·
verified ·
1 Parent(s): e3c56c6

Upload web/TEST_RESULTS.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. web/TEST_RESULTS.md +41 -0
web/TEST_RESULTS.md CHANGED
@@ -153,6 +153,47 @@ path runs. A note on method: the init backend race makes step totals vary
153
  run-to-run (LUT vs DP4A can win), so uncontrolled before/after numbers are
154
  not comparable — hence the same-session A/B.
155
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
156
  ## The dirty-buffer gate (and the assumption it falsified)
157
 
158
  Buffer pooling introduced a bug class the gates were built before: a pooled
 
153
  run-to-run (LUT vs DP4A can win), so uncontrolled before/after numbers are
154
  not comparable — hence the same-session A/B.
155
 
156
+ ## Fixing the audit's coverage (the weak spot, named and closed)
157
+
158
+ The live audit's two constants — `cells: 6`, `due: 2% of GEMMs` — were chosen
159
+ to bound overhead and never justified against a bug class. That sampled ~1.2
160
+ cells per step, ~360 over a 300-step run: a coverage number sitting in exactly
161
+ the seat `allclose` used to occupy.
162
+
163
+ **The overhead premise was false.** An audit costs `k` multiply-adds per cell
164
+ in JS. Measured at the live logits shape:
165
+
166
+ ```
167
+ 6 cells: 2.1 us/audit -> 0.021 ms/step if EVERY GEMM is audited
168
+ 16 cells: 2.8 us/audit -> 0.028 ms/step
169
+ 64 cells: 11.6 us/audit -> 0.116 ms/step
170
+ ```
171
+
172
+ Against a ~320 ms step that is under 0.01%. The 2% rate was throttling
173
+ coverage to save nothing, so the audit now runs on **every** GEMM.
174
+
175
+ **But the rate was the smaller half.** The bugs that actually occur here —
176
+ a bounds-guard off-by-one, a pack-tail padding bug — corrupt the **last
177
+ row/column only**. Uniform random sampling finds that with probability ~1/n
178
+ per cell, and at the 16512-wide unembed that is never. So sampling is now
179
+ **stratified**: the first 6 cells are the structural danger points (last
180
+ row, last column, all four corners, last batch) chosen deterministically, the
181
+ rest random interior cells for diffuse bugs.
182
+
183
+ Measured against that named bug class (last column zeroed, n=512, 300 audits):
184
+
185
+ | strategy | caught |
186
+ |---|---|
187
+ | uniform, 12 cells | **5 / 300** (predicted ~2.3%) |
188
+ | stratified, 12 cells | **300 / 300** |
189
+ | stratified, clean kernel | 0 false positives / 300 |
190
+
191
+ Note the cell count is identical in both rows: **the strategy changed, not the
192
+ budget**. Widen n and the uniform number goes to zero while the stratified one
193
+ stays at 100%. This is the same move as poisoning the buffer pool — construct
194
+ the dangerous case instead of waiting to land on it — applied to the sampler
195
+ instead of the gate.
196
+
197
  ## The dirty-buffer gate (and the assumption it falsified)
198
 
199
  Buffer pooling introduced a bug class the gates were built before: a pooled