Run one model across several devices. Each device downloads and holds only its own layers — straight from Hugging Face, never from a peer — and passes the hidden state around the ring over WebRTC. The group can run a model that no single device could hold. Every multiply goes through the verified INT8 units. Runs locally: npm start.
Any Hugging Face repo with .safetensors weights whose architecture is Llama-style (Llama, Mistral, Qwen2, SmolLM, TinyLlama) or GPT-2-style. Loading reads only config.json, the tokenizer, and the weight headers — a few tens of KB, whatever the model's size. Gated or private repos will ask for a token, once.
load a model to see how it would be split
Layers are apportioned by measured speed — each device times a real GEMM through its own kernel. The MB figure is what that device will actually download and hold.
There is no KV cache — every token re-runs the whole window, so context length is the main cost per token.
0 = greedy, which is deterministic and therefore checkable. Above 0, sampling follows the seed so runs stay reproducible.
Needs a secure context (localhost or HTTPS) for WebGPU and cross-device WebRTC. No WebGPU? The identical verified INT8 units run on CPU — slower, but a CPU device and a GPU device produce the same bits, so they can share a ring. There is no plain-float path.
Heads up: peers connect directly, so devices in your ring can see each other's IP address. A middle stage never receives the embedding table or the vocabulary, but it does see the hidden states passing through — and the head sees your prompt and output. Run rings with devices and people you trust. Proof of concept, not a hardened service.
This model needs a token to download.
Kept in memory for this tab only: never written to disk or storage, never put in a URL, never logged, and never sent to other devices — each device is asked for its own. Create a read token at huggingface.co/settings/tokens.