Upload web/docs/TROUBLESHOOTING.md with huggingface_hub

web/docs/TROUBLESHOOTING.md

@@ -0,0 +1,85 @@
+# Troubleshooting
+
+Common failure modes, what the log messages mean, and what to do.
+
+## Connection problems
+
+**Devices don't see each other.**
+- Same network? Auto-grouping is by **public IP** — devices on different
+  networks (e.g. phone on cellular) will never auto-group. Use
+  `?room=CODE` on every device instead.
+- Room joins need **host approval** — the first device in the room must click
+  Accept for each newcomer.
+- Corporate/CGNAT networks can block WebRTC (only STUN is configured, no TURN
+  relay). Try a room over the hosted Space, or another network.
+
+**"WebGPU not available" / backend shows CPU.**
+- WebGPU needs a **secure context**: `localhost` or HTTPS. Plain
+  `http://192.168.x.x` never gets WebGPU — tunnel it or use the Space.
+- Old hardware/drivers without a modern GPU backend fall back to CPU. That's
+  supported — CPU devices compute the identical bits, just slower.
+
+**A kernel warning at startup** (e.g. "failed verification — using CPU LUT
+mirrors"). The device's GPU produced values that don't match the verified
+units bit-for-bit, so that kernel was refused and a slower-but-exact path is
+used. This is the system working as designed; the run stays correct.
+
+## Training problems
+
+**"SYNC GUARD: stopped — weight hash mismatch."** Replicas forked — the run
+is stopped rather than trained past a divergence. The most common cause is
+**mixed builds**: devices running different versions of the page whose math
+specs differ (the quantize respec is a real, bounded spec change — old and
+new builds must not co-train). Make sure every device has the same version
+(hard-refresh: Ctrl+Shift+R), then restart the run or restore from a
+checkpoint.
+
+**"missing a roster gradient … requesting repair from the leader."** A
+gradient reached some peers but not others (asymmetric mesh). The follower
+re-requests it from the leader, who retains the last 8 steps — normally the
+run continues by itself. If repair fails repeatedly, the sync guard stops the
+run; check whether a device went offline.
+
+**"probe hash mismatch."** A device's *kernel* is producing wrong values at
+runtime (the seeded probe GEMM no longer matches the fleet). That device
+should rejoin — its gates will re-run at init and demote it to CPU if its GPU
+misbehaves.
+
+**Loss looks stuck / falls very slowly.** Small widths and short runs are
+modest models — try more steps, a wider model, or more devices (each adds
+its batch to the effective batch). Remember the envelope: this pools
+compute, not memory; it is a proof of concept for small models.
+
+**A slow device paces everyone.** Steps are synchronous: the group advances
+when every roster gradient arrives. Removing (or upgrading) the slowest
+device speeds up everyone.
+
+## Data problems
+
+**"dataset unreachable — using built-in corpus."** The `/data` endpoint
+couldn't read the FineWeb-Edu shards (offline, or the CDN hiccuped — the
+server reads parquet slices straight off the HF CDN with range requests).
+Training continues on the built-in fallback text; generation quality will
+reflect the smaller corpus.
+
+## Checkpoint problems
+
+**"tokenizer mismatch."** The checkpoint was trained with a different vocab
+than the tokenizer this page loaded — it cannot be resumed here.
+
+**"bad dims in checkpoint."** Width/sequence outside the supported 16–128
+range — likely a corrupted or foreign file.
+
+Loading a valid checkpoint on **one** device broadcasts it to the whole
+group; you don't need to load it everywhere.
+
+## Verifying your install
+
+```bash
+npm test
+```
+
+All ten suites should pass on a clean clone (Node only, no browser/GPU
+required). If a suite fails on unmodified code, please report it — the tests
+are deterministic apart from bounded random sweeps that are designed not to
+flake. See [TEST_RESULTS.md](../TEST_RESULTS.md) for what each suite proves.