README.md

metadata

title: Phox — Wave-Rider Brain
emoji: 🌀
colorFrom: gray
colorTo: purple
sdk: gradio
pinned: false
short_description: Transformer forward pass as one equilibrium, int8 on CPU

Phox — Wave-Rider Inference Engine

A transformer forward pass expressed as a single equilibrium rather than a stack of layers, running int8-resident on a CPU. Every operation has a wave form, and each one is measured against the reference rather than asserted.

Model: Llama-3.2-3B (Hermes-3-Llama-3.2-3B-abliterated) — 28 layers, D=3072, 3.21B parameters. Weights quantised to int8 with per-output-column scales: 2.82 GB, fully resident, widened to f32 inside AVX2 registers so the padding is never written to memory.

Verified

claim	measurement
Full forward in wave form	5/5 identical token IDs vs reference, 28 layers
The forward pass is ONE fixed point	Jacobi (unsequenced) and Gauss-Seidel reach identical logits — error 0.000e+00 at the critical path
RMSNorm == saturable gain medium	direction cosine 1.000000000000 — an identity, not a fit
softmax == amplification + power pool	1.3e-07 rel-err
RoPE == free-running oscillator phase	2.6e-08 rel-err
Amplitude coupling reproduces W·x	3.2e-08; phase-only Kuramoto 9.75e-01 — the encoding is load-bearing
int8 per-column fidelity	y cosine 0.999949; tokens unchanged through all 28 layers
Throughput	1.43 tok/s (0.70 s/token) — see the note below; this is NOT a speedup

Honest limitations

• It performs matmuls. The wave form is an equivalence proof about the arithmetic, not a substitute for it: relaxation converges to W·x, which is a value you have already computed. There is no speedup from the physics.

• And there is no speedup at all, against a real baseline. On this same machine, CPU-only, same model family, in the same units:

  	tok/s	s/token
  this engine (int8, 2.82 GB)	1.43	0.70
  llama.cpp (Q4_K_M, 1.87 GiB)	3.02	0.33

  We are 2.1x slower than llama.cpp. Earlier drafts framed this as "68 s -> 0.7 s"; that 68 s was an unoptimised numpy path re-reading 6.4 GB from disk per token, i.e. our own worst case, and quoting it as a baseline was not honest. The work here bought a correct and resident int8 CPU engine, not a fast one. The value is in the equivalence proofs and the negative results, not the throughput.

• Ternary does not work. Measured engine-free against real weights on both Llama-3.2-3B and Gemma-4-12B: y cosine 0.877 against int8's 0.99995. Per-column scaling rescues int2 relative to a global scale (0.52 vs 0.036) but neither is usable. Quantisation fidelity turns out to be a property of the bit width rather than the model — the two families agree to three decimals at every width tested.

• Scale is unproven. Verified on 3B. The 12B needs int4 (5.45 GB) and does not fit in this machine's free RAM.

• This Space ships no weights. The dashboard renders and the visualisation is wired, but the engine cannot start without a local model. Build the cache with bqsm_int8.py --build on a machine that has one.

• The tokenizer is whitespace-split, not BPE, so punctuation and rare words are spelled character by character. bqsm_tokenizer.py implements real byte-level BPE but is not yet wired in.

• The SRP popcount readout was removed. Once the bf16 kernel did the dense vocabulary scan in ~60 ms, SRP was both slower (82–159 ms) and approximate — 512-bit Hamming distances tie heavily, and a true argmax with 884 tokens strictly closer had 1,135 tied-or-closer, falling outside k=1024.

• Training, backprop, cross-request batching, and sampling above greedy argmax are not addressed.

Architecture notes

The forward pass is a DAG of 311 blocks with a critical path of 227 — so the available parallelism is 1.37×, and 171 of those depth levels are strictly serial. Depth is irreducible: 757 data-dependent nonlinearities (RMSNorm, the saturating gate, softmax) break matrix associativity at every stage. Remove them and all 28 layers collapse into a single 128256×3072 matrix that computes nothing interesting.

At one token the arithmetic intensity is 1.78 FLOP/byte — memory-bound by roughly 6×. Fusing 197 matrix-vector products down to 113 moves zero bytes. Only fewer bytes per weight, or more work per byte (batching), change the time.

Running locally

python3 bqsm_assist/bqsm_int8.py --build          # quantise once, ~2.82 GB
python3 bqsm_assist/bqsm_int8.py --n 5 --verify   # check against golden tokens
python3 phoenix_dashboard.py                      # dashboard on :8765

Code: github.com/compunerd/basin-quotient-machine