Quazim0t0 commited on
Commit
28a864a
Β·
verified Β·
1 Parent(s): 82c61b1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +102 -0
README.md CHANGED
@@ -38,6 +38,108 @@ it is **not** a substitute for a real GPU on large models. Full envelope in
38
 
39
  ---
40
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  ## Quick start
42
 
43
  ### Docker (most reliable β€” one command)
 
38
 
39
  ---
40
 
41
+ ## Feature list
42
+
43
+ ### Python cluster trainer (`daisychain/`)
44
+ - **Data-parallel training across mixed machines** β€” each node trains its own
45
+ shard; gradients combine into the exact full-batch gradient, replicas stay
46
+ bit-identical.
47
+ - **Capacity-weighted sharding** β€” faster machines automatically take a
48
+ bigger share of the batch.
49
+ - **Emulated GPU compute (verified INT8 units)** β€” `VerifiedLinear` layers
50
+ run every forward multiply / requantize / ReLU through the bundled trained
51
+ units; cluster-wide unit-invocation counts printed by rank 0.
52
+ - **Bring your own model** β€” any `Task` (`build_model` / `sample` / `loss`)
53
+ via `DAISY_TASK`; template in `examples/my_task_template.py`.
54
+ - **Plain-float alternative task** β€” same cluster and pooling with ordinary
55
+ float math.
56
+ - **Live dashboard** (`daisychain-dashboard`) β€” readiness banner, P2P
57
+ connectivity scan, pooled cores/RAM, per-node capacity plan, live loss.
58
+ - **SpikeWhale control panel** (`spikewhale_panel`, `localhost:8899`) β€”
59
+ sliders for model size / training settings, any HF dataset you can access
60
+ (default streamed FineWeb-Edu), start/stop/re-adjust, live loss.
61
+ - **Docker demo cluster** β€” 3 nodes + dashboard in one command.
62
+ - **Windows helper** (`scripts\setup.bat`) and Tailscale mesh guide.
63
+
64
+ ### DaisyChain-Web (`web/`) β€” browser P2P training
65
+ - **Zero-install nodes** β€” opening the page IS joining; devices on one
66
+ network auto-group (Snapdrop-style, by public IP).
67
+ - **Private cross-network rooms** β€” `?room=CODE` with **host approval** for
68
+ every join.
69
+ - **Full WebRTC mesh** β€” gradients travel peer-to-peer; the server only
70
+ signals and serves static files, it never sees weights or gradients.
71
+ - **Leader-follower runs** β€” whoever presses Start sets width / sequence /
72
+ batch-per-device / steps / learning rate for the whole group; config
73
+ broadcast on the wire.
74
+ - **Mid-run join** β€” late devices are synced in (weights + step) and
75
+ contribute from the next step.
76
+ - **Bit-identical replicas** β€” same seeded init, strict roster-order gradient
77
+ averaging, deterministic Adam with identical state on every peer; verified
78
+ live by per-step weight hashes.
79
+ - **Sync guard** β€” any weight-hash mismatch stops the run instead of
80
+ training past a fork; the step roster forbids silent partial averages.
81
+ - **Gradient repair** β€” a follower missing a roster gradient re-requests it
82
+ from the leader (8 steps retained), bit-exact, and the run continues.
83
+ - **Cross-device kernel probe** β€” every step, every device re-hashes a fixed
84
+ seeded int8 GEMM through its live kernel; catches broken arithmetic that
85
+ weight hashes cannot see.
86
+ - **Hardcoded FineWeb-Edu streaming** β€” the server reads random slices of
87
+ the 10BT parquet shards straight off the HF CDN via HTTP range requests
88
+ (pure-JS `hyparquet`); built-in corpus fallback offline.
89
+ - **Checkpoints** β€” download `.pt`, upload β†’ **broadcast to the whole
90
+ group**; validated (magic, dims, tokenizer vocab) before accepting.
91
+ - **Inference kit** β€” one self-contained HTML file with the trained weights
92
+ baked in; generations offline, anywhere.
93
+ - **In-page generation** β€” prompt box on the trained model.
94
+ - **Old-hardware tier** β€” no WebGPU? The identical units run on CPU (same
95
+ bits, so CPU and GPU devices co-train in one group). There is no
96
+ plain-float path.
97
+ - **Large-message fragmentation** β€” multi-MB gradients/checkpoints chunked
98
+ at 48 KB over the data channels.
99
+
100
+ ### Verified compute & kernels (web)
101
+ - **Verified INT8 units everywhere** β€” block-scaled int8 GEMM: exact LUT
102
+ products, exact int32 accumulation, bit-exact f32 epilogue with a pinned
103
+ rounding schedule; scales derived in JS f64 (division never runs on GPU).
104
+ - **Backends, best-first** β€” DP4A hardware int8 dot β†’ LUT compute shader β†’
105
+ CPU mirror; every kernel **exact-gated at init** (bit-level compares) and
106
+ demoted to the mirror on any mismatch.
107
+ - **Continuous random-cell audit** at live training shapes.
108
+ - **Fused attention kernels** β€” gather/scatter head-strided qΒ·kα΅€ and aΒ·v
109
+ straight from BTΓ—C layout (CUTLASS ex. 36/52 style).
110
+ - **QKV dual-GEMM fusion** β€” shared left operand quantized once, one batch-3
111
+ dispatch (ex. 45); bit-identical.
112
+ - **B2B MLP chain** β€” both MLP GEMMs back-to-back on GPU with fused per-row
113
+ absmax reduction and on-device quantize (ex. 13 + 23); WGSL-exact respec
114
+ with a fround-stepped JS mirror; fma-contraction-immune by construction.
115
+ - **Dispatch-optimized backward** β€” overlapped independent GEMMs, batch-3
116
+ sibling fusions (ex. 05/24); bit-identical gradients; optional int8 STE
117
+ backward path (dormant, 1.21Γ— vs float).
118
+
119
+ ### Verification stack (web)
120
+ - **Exact init gates** on every kernel, every device, every boot β€” including
121
+ gates that "gate the gate" with discriminating boundary inputs.
122
+ - **IEEE-754 binary32 oracle** in exact BigInt arithmetic β€” proves the JS
123
+ epilogue mirror is spec-correct (rejects the old mirror on 34% of inputs).
124
+ - **Metamorphic property suite** β€” reference-free relations + definitional
125
+ absolutes; **4/4** on an externally-authored bug corpus, matching the
126
+ differential gate.
127
+ - **RDNA2 ISA audit hardenings** β€” bit-level (βˆ’0-aware) gate comparisons;
128
+ proof that FMA contraction cannot change the quantize.
129
+ - **Ten-suite test chain** (`cd web && npm test`) β€” convergence, replicas,
130
+ oracle, gates, properties, corpus, B2B, optimizer, transformer LM, int8
131
+ backward; results in [web/TEST_RESULTS.md](web/TEST_RESULTS.md).
132
+
133
+ ### Documentation
134
+ - Python: [QUICKSTART](docs/QUICKSTART.md), [LIMITS](docs/LIMITS.md),
135
+ [CUSTOM_TASK](docs/CUSTOM_TASK.md), [TAILSCALE](docs/TAILSCALE.md).
136
+ - Web: [Getting started](web/docs/GETTING_STARTED.md),
137
+ [Architecture](web/docs/ARCHITECTURE.md),
138
+ [Verification](web/docs/VERIFICATION.md),
139
+ [Troubleshooting](web/docs/TROUBLESHOOTING.md).
140
+
141
+ ---
142
+
143
  ## Quick start
144
 
145
  ### Docker (most reliable β€” one command)