Upload web/TEST_RESULTS.md with huggingface_hub
Browse files- web/TEST_RESULTS.md +41 -0
web/TEST_RESULTS.md
CHANGED
|
@@ -153,6 +153,47 @@ path runs. A note on method: the init backend race makes step totals vary
|
|
| 153 |
run-to-run (LUT vs DP4A can win), so uncontrolled before/after numbers are
|
| 154 |
not comparable — hence the same-session A/B.
|
| 155 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 156 |
## The dirty-buffer gate (and the assumption it falsified)
|
| 157 |
|
| 158 |
Buffer pooling introduced a bug class the gates were built before: a pooled
|
|
|
|
| 153 |
run-to-run (LUT vs DP4A can win), so uncontrolled before/after numbers are
|
| 154 |
not comparable — hence the same-session A/B.
|
| 155 |
|
| 156 |
+
## Fixing the audit's coverage (the weak spot, named and closed)
|
| 157 |
+
|
| 158 |
+
The live audit's two constants — `cells: 6`, `due: 2% of GEMMs` — were chosen
|
| 159 |
+
to bound overhead and never justified against a bug class. That sampled ~1.2
|
| 160 |
+
cells per step, ~360 over a 300-step run: a coverage number sitting in exactly
|
| 161 |
+
the seat `allclose` used to occupy.
|
| 162 |
+
|
| 163 |
+
**The overhead premise was false.** An audit costs `k` multiply-adds per cell
|
| 164 |
+
in JS. Measured at the live logits shape:
|
| 165 |
+
|
| 166 |
+
```
|
| 167 |
+
6 cells: 2.1 us/audit -> 0.021 ms/step if EVERY GEMM is audited
|
| 168 |
+
16 cells: 2.8 us/audit -> 0.028 ms/step
|
| 169 |
+
64 cells: 11.6 us/audit -> 0.116 ms/step
|
| 170 |
+
```
|
| 171 |
+
|
| 172 |
+
Against a ~320 ms step that is under 0.01%. The 2% rate was throttling
|
| 173 |
+
coverage to save nothing, so the audit now runs on **every** GEMM.
|
| 174 |
+
|
| 175 |
+
**But the rate was the smaller half.** The bugs that actually occur here —
|
| 176 |
+
a bounds-guard off-by-one, a pack-tail padding bug — corrupt the **last
|
| 177 |
+
row/column only**. Uniform random sampling finds that with probability ~1/n
|
| 178 |
+
per cell, and at the 16512-wide unembed that is never. So sampling is now
|
| 179 |
+
**stratified**: the first 6 cells are the structural danger points (last
|
| 180 |
+
row, last column, all four corners, last batch) chosen deterministically, the
|
| 181 |
+
rest random interior cells for diffuse bugs.
|
| 182 |
+
|
| 183 |
+
Measured against that named bug class (last column zeroed, n=512, 300 audits):
|
| 184 |
+
|
| 185 |
+
| strategy | caught |
|
| 186 |
+
|---|---|
|
| 187 |
+
| uniform, 12 cells | **5 / 300** (predicted ~2.3%) |
|
| 188 |
+
| stratified, 12 cells | **300 / 300** |
|
| 189 |
+
| stratified, clean kernel | 0 false positives / 300 |
|
| 190 |
+
|
| 191 |
+
Note the cell count is identical in both rows: **the strategy changed, not the
|
| 192 |
+
budget**. Widen n and the uniform number goes to zero while the stratified one
|
| 193 |
+
stays at 100%. This is the same move as poisoning the buffer pool — construct
|
| 194 |
+
the dangerous case instead of waiting to land on it — applied to the sampler
|
| 195 |
+
instead of the gate.
|
| 196 |
+
|
| 197 |
## The dirty-buffer gate (and the assumption it falsified)
|
| 198 |
|
| 199 |
Buffer pooling introduced a bug class the gates were built before: a pooled
|