Getting started with DaisyChain-Infer

This is a local project. There is no hosted version — you clone it and run it on your own machines.

Run the server

npm install
npm start            # http://localhost:8788

Open it in two tabs to try the ring on one machine, or on two devices to try it for real.

HTTPS matters. WebGPU and cross-device WebRTC need a secure context: localhost or HTTPS. Plain http://192.168.x.x from another device will not get WebGPU and may not connect at all. For real multi-device runs on your own network, put it behind a tunnel (cloudflared, ngrok) or a local TLS reverse proxy.

Pick a model

Type any Hugging Face repo id — owner/name — and press Load.

Works with .safetensors repos whose architecture is:

Llama-style: Llama, Mistral, Qwen2/2.5, SmolLM, TinyLlama
GPT-2-style: GPT-2 and close relatives

Good ones to start with:

Repo	Size	Notes
`HuggingFaceTB/SmolLM-135M`	513 MB f32	30 layers — splits nicely
`openai-community/gpt2`	523 MB f32	the GPT-2 path
`Qwen/Qwen2.5-0.5B`	942 MB f32	GQA 14/2, and it ships QKV biases

Loading reads only config.json, tokenizer.json and the weight headers — a few tens of KB, whatever the model's size. No weights move until you press Generate, and then each device fetches only its own layers.

Weights are held as f32, so budget ~4 bytes per parameter across the group. The ring plan shows exactly how many MB each device will download and hold.

An unsupported architecture or tokenizer is refused with a message naming what it found. That is deliberate: approximating one would produce fluent, confident, wrong text.

Gated or private models

You are asked for a Hugging Face token once, and only when a request actually fails for want of one. Create a read token at huggingface.co/settings/tokens.

The token is held in memory for that tab and nowhere else — not localStorage, not sessionStorage, not a cookie, not the URL, never logged, and never sent to another device. Each device is prompted for its own, because each device downloads its own layers. Reloading the tab forgets it, and Forget token clears it immediately.

For a gated model, accept its licence on the model page first — a token alone will not get past a licence you have not accepted.

Rooms

Devices on one network group automatically (by public IP, Snapdrop-style). To include devices on other networks, everyone opens:

https://<host>/?room=MY-SECRET-CODE

The first person in is the host and approves each device individually.

Run it

Wait until the devices see each other in the peer list.
On one device, load the model. That device becomes the head: it holds the embedding table, embeds your prompt, and turns the returning hidden state back into words.
Check the ring plan — who holds which layers, and how much each will download.
Type a prompt and press Generate. Each device fetches its layers (this takes a moment the first time), reports ready, and the ring starts.

Setting	Default	What it does
Tokens	60	how many to generate
Context length	64	window size; the main cost per token (there is no KV cache)
Temperature ÷100	0	0 = greedy, deterministic and therefore checkable
Seed	1234	drives sampling above temperature 0, so runs stay reproducible
Verify	off	afterwards, re-run with every layer here and compare

Every device shows the token stream as it arrives — including ones holding a few layers and no vocabulary.

Reading the numbers

tokens/sec — the whole ring's rate. Expect it to fall as you add stages: each one adds a round trip per token. More stages buy capacity, not speed.
ring hops — stages × tokens.
kernel probe (in the log) — the same number on every honest device, whatever its backend. A peer reporting a different one is computing different arithmetic, and the log says so.
my slice — which layers this device holds and how much memory they take.

Verify

Tick Verify before generating. Afterwards the head downloads the whole model and re-runs the identical prompt locally, then compares token ids:

VERIFIED: the distributed run and the single-device run produced identical token ids.

In a pipeline nothing recomputes anything, so this is the only check that can show a distributed answer is right rather than merely self-consistent. It needs a model one device can hold, which is why it is optional.

Without a browser:

npm test

When it goes wrong

"the ring stalled at token N" — a stage went quiet. Press Generate again to re-plan around whoever is still connected.
"unsupported architecture" — the model is not Llama-style or GPT-2-style.
"tokenizer type … is not supported" — the repo uses SentencePiece or WordPiece; only byte-level BPE is implemented.
401 / 403 — the model needs a token, or the token lacks access. For gated repos, accept the licence on the model page first.
"the server ignored the byte range" — the whole file came back instead of a slice; refused rather than accepted, since that defeats the point.
"REFUSED activation … belongs to model X" — a stage still holds a slice of a previous model. Reload that device.
"⚠ … disagrees with this device's kernel probe" — that device's arithmetic differs from yours. Do not give it layers.
out of memory — the stage's slice does not fit. Add devices, or pick a smaller model; the plan shows the per-device cost before you start.

ARCHITECTURE.md — the ring, the plan, the wire protocol.
The parent projects: DaisyChain-Train · DaisyChain-Web