Upload web/TEST_RESULTS.md with huggingface_hub
Browse files- web/TEST_RESULTS.md +49 -0
web/TEST_RESULTS.md
CHANGED
|
@@ -153,6 +153,55 @@ path runs. A note on method: the init backend race makes step totals vary
|
|
| 153 |
run-to-run (LUT vs DP4A can win), so uncontrolled before/after numbers are
|
| 154 |
not comparable — hence the same-session A/B.
|
| 155 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 156 |
## Where the remaining GPU time goes (a negative result, kept on purpose)
|
| 157 |
|
| 158 |
After the shared-operand fix, `bgemm` was the biggest line (~95 ms/step).
|
|
|
|
| 153 |
run-to-run (LUT vs DP4A can win), so uncontrolled before/after numbers are
|
| 154 |
not comparable — hence the same-session A/B.
|
| 155 |
|
| 156 |
+
## The dirty-buffer gate (and the assumption it falsified)
|
| 157 |
+
|
| 158 |
+
Buffer pooling introduced a bug class the gates were built before: a pooled
|
| 159 |
+
buffer is **not zero-initialized**, so a kernel that assumes zeros (the rowmax
|
| 160 |
+
`atomicMax` accumulator does) is correct on step one and wrong on step two.
|
| 161 |
+
That is a **state** bug — no single call is wrong, the *sequence* is — the
|
| 162 |
+
family no oracle can reach, because an oracle is a function of one call.
|
| 163 |
+
|
| 164 |
+
The premise was that the gate suite is structurally blind to this, since every
|
| 165 |
+
gate runs on first-acquisition (zeroed) memory. **Mutation-testing the gate
|
| 166 |
+
falsified that.** Deleting the `clearBuffer` and re-running:
|
| 167 |
+
|
| 168 |
+
```
|
| 169 |
+
B2B MLP chain (LUT) failed verification: out mismatch @0 (5x16x6x3)
|
| 170 |
+
^ the SECOND shape
|
| 171 |
+
```
|
| 172 |
+
|
| 173 |
+
The plain gate catches it — not by design, but because the sweep's own four
|
| 174 |
+
shapes recycle each other's buffers (they land in one power-of-2 pool bucket)
|
| 175 |
+
and an uncleared max only grows. The suite had **incidental** dirty coverage
|
| 176 |
+
nobody designed and nobody documented.
|
| 177 |
+
|
| 178 |
+
Incidental is the problem. It depends on the sweep having ≥2 shapes, on those
|
| 179 |
+
shapes colliding in one bucket, and on residue exceeding the real value.
|
| 180 |
+
Shorten the shape list and the coverage silently evaporates — with the gate
|
| 181 |
+
still green. So the coverage is now deliberate: `poisonPool` runs the chain at
|
| 182 |
+
**1e4 magnitude** first, so the residue dominates any value the gate can
|
| 183 |
+
produce, releases it, then sweeps again. Detection no longer depends on
|
| 184 |
+
ordering luck, and it covers the first shape too.
|
| 185 |
+
|
| 186 |
+
Verified:
|
| 187 |
+
|
| 188 |
+
| check | result |
|
| 189 |
+
|---|---|
|
| 190 |
+
| correct kernel, dirty re-gate | admitted (no false positive) |
|
| 191 |
+
| mutant (`clearBuffer` deleted), plain gate | caught at shape 2 (incidental) |
|
| 192 |
+
| mutant, poisoned gate | caught at shape 1 (deliberate) |
|
| 193 |
+
| init cost | 1546 → 1636 ms, **~90 ms one-time** |
|
| 194 |
+
| eleven suites + live 300-step run | green, no console errors |
|
| 195 |
+
|
| 196 |
+
The re-gate deliberately skips the 800-trial respec hunt — that hunt is about
|
| 197 |
+
the quantize spec and has nothing to do with buffer state, and skipping it is
|
| 198 |
+
what keeps the added cost at 90 ms instead of doubling gate time.
|
| 199 |
+
|
| 200 |
+
The lesson worth more than the gate: an instrument can have coverage it was
|
| 201 |
+
never designed for, and coverage you did not design is coverage you cannot
|
| 202 |
+
rely on. The only way to find out was to mutate the gate and watch which
|
| 203 |
+
shape it failed on.
|
| 204 |
+
|
| 205 |
## Where the remaining GPU time goes (a negative result, kept on purpose)
|
| 206 |
|
| 207 |
After the shared-operand fix, `bgemm` was the biggest line (~95 ms/step).
|