Quazim0t0 commited on
Commit
fa4d1d4
·
verified ·
1 Parent(s): ca0e49c

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +54 -4
README.md CHANGED
@@ -126,9 +126,13 @@ it is **not** a substitute for a real GPU on large models. Full envelope in
126
  differential gate.
127
  - **RDNA2 ISA audit hardenings** — bit-level (−0-aware) gate comparisons;
128
  proof that FMA contraction cannot change the quantize.
129
- - **Ten-suite test chain** (`cd web && npm test`) — convergence, replicas,
130
- oracle, gates, properties, corpus, B2B, optimizer, transformer LM, int8
131
- backward; results in [web/TEST_RESULTS.md](web/TEST_RESULTS.md).
 
 
 
 
132
 
133
  ### Documentation
134
  - Python: [QUICKSTART](docs/QUICKSTART.md), [LIMITS](docs/LIMITS.md),
@@ -245,7 +249,53 @@ guard still stops anything that would fork the weights.
245
  via a WGSL-exact respec (`floor(f32(x·invScale)+0.5)` — no GPU division)
246
  whose fround-stepped JS mirror keeps mixed GPU/CPU fleets bit-identical.
247
 
248
- All ten test suites (`cd web && npm test`) pass; results with methodology in
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
249
  [`web/TEST_RESULTS.md`](web/TEST_RESULTS.md).
250
 
251
  ---
 
126
  differential gate.
127
  - **RDNA2 ISA audit hardenings** — bit-level (−0-aware) gate comparisons;
128
  proof that FMA contraction cannot change the quantize.
129
+ - **Eleven-suite test chain** (`cd web && npm test`) — convergence, replicas,
130
+ oracle, gates, properties, external corpus, **self-corpus** (the instruments
131
+ scored against my own bugs), B2B, optimizer, transformer LM, int8 backward;
132
+ results in [web/TEST_RESULTS.md](web/TEST_RESULTS.md).
133
+ - **Dirty-buffer gate** — the pool is poisoned before a re-sweep so state bugs
134
+ (a kernel assuming zeroed memory) are caught deterministically rather than
135
+ by ordering luck.
136
 
137
  ### Documentation
138
  - Python: [QUICKSTART](docs/QUICKSTART.md), [LIMITS](docs/LIMITS.md),
 
249
  via a WGSL-exact respec (`floor(f32(x·invScale)+0.5)` — no GPU division)
250
  whose fround-stepped JS mirror keeps mixed GPU/CPU fleets bit-identical.
251
 
252
+ **Profile-driven speed work** every change below is bit-identical (gradient
253
+ and loss hashes unchanged), so none of it trades correctness for wall clock:
254
+ - **Buffer pooling**: GPU buffers are recycled by size bucket instead of being
255
+ created and destroyed per dispatch (~19 per MLP call, per layer, per step).
256
+ **6–10% faster**, every hash unchanged.
257
+ - **Shared-operand embedding GEMMs**: profiling put two f32 backward GEMMs at
258
+ **55% of the entire step**, and both consumed the same `dlogits` operand —
259
+ ~17 MB at the 16512-token vocab, uploaded *twice*. One upload, one encoder,
260
+ one submit: that pair went 205 → 90 ms and the step **12% faster**. The
261
+ fusion is gated bit-for-bit against the two calls it replaces.
262
+ - **A negative result, kept on purpose**: the remaining hot kernel looked
263
+ cache-hostile (adjacent lanes wrote 66 KB apart), but making the writes
264
+ contiguous changed nothing. Two probes explain why — holding the output at
265
+ 17 MB while cutting compute 32× barely moved the time. The logits GEMM is
266
+ **transfer-bound, not compute-bound**, and the readback cannot be removed
267
+ because softmax must stay in JS (WGSL's `exp` is not correctly rounded, and
268
+ a per-vendor `exp` would fork replicas). The real lever there is the
269
+ vocabulary, not the kernel.
270
+ - **An init backend race was tried and removed**: it tied on the shipped path,
271
+ cost ~430 ms of init, and made the backend vary between page loads, which
272
+ silently invalidated three A/B comparisons before it was caught. A knob that
273
+ changes what you are measuring is worse than a fixed choice.
274
+
275
+ **Dirty-buffer gate — and the assumption it falsified.** Pooling introduced a
276
+ bug class the gates predate: a pooled buffer is *not* zero-initialized, so a
277
+ kernel that assumes zeros is right on step one and wrong on step two. That is
278
+ a **state** bug, where no single call is wrong and the *sequence* is, which is
279
+ the family no oracle can reach. The assumption was that the gates were blind
280
+ to it. Mutation-testing the gate proved otherwise: deleting the zeroing made
281
+ the plain gate fail at its *second* shape, because the sweep's own shapes
282
+ recycle each other's buffers. The suite had **incidental** coverage nobody
283
+ designed, which is coverage nobody can rely on — shorten the shape list and it
284
+ evaporates with the gate still green. It is now deliberate: the pool is
285
+ poisoned with 1e4-magnitude residue before a re-sweep, so detection no longer
286
+ depends on ordering luck. ~90 ms one-time.
287
+
288
+ **Scoring the oracles against my OWN bugs** (`web/test_selfcorpus.js`) — the
289
+ external corpus measures kernel bugs someone else wrote down, so this suite
290
+ asks the harder question: what do the instruments score against the four real
291
+ bugs of the month? Properties **0/2** on the data-plane pair (the `c·out`
292
+ theorem again), differential **2/2** — but half the bugs were not in the
293
+ kernels at all. A dead gate is a bug in a *checker*, caught only by mutating
294
+ the gate; a stalled roster gradient is a bug in the *protocol*, where every
295
+ computed value on every peer was correct, so no data oracle could fire. Those
296
+ needed different instruments, not better oracles.
297
+
298
+ All eleven test suites (`cd web && npm test`) pass; results with methodology in
299
  [`web/TEST_RESULTS.md`](web/TEST_RESULTS.md).
300
 
301
  ---