README.md

---
title: Phox — Wave-Rider Brain
emoji: 🌀
colorFrom: gray
colorTo: purple
sdk: gradio
pinned: false
short_description: Transformer forward pass as one equilibrium, int8 on CPU
---
# Phox — Wave-Rider Inference Engine

A transformer forward pass expressed as a **single equilibrium** rather than a
stack of layers, running int8-resident on a CPU. Every operation has a wave
form, and each one is measured against the reference rather than asserted.

**Model:** Llama-3.2-3B (Hermes-3-Llama-3.2-3B-abliterated) — 28 layers,
D=3072, 3.21B parameters. Weights quantised to int8 with per-output-column
scales: **2.82 GB, fully resident**, widened to f32 inside AVX2 registers so the
padding is never written to memory.

## Verified

| claim | measurement |
|---|---|
| Full forward in wave form | **5/5 identical token IDs** vs reference, 28 layers |
| The forward pass is ONE fixed point | Jacobi (unsequenced) and Gauss-Seidel reach identical logits — **error 0.000e+00** at the critical path |
| RMSNorm == saturable gain medium | direction cosine **1.000000000000** — an identity, not a fit |
| softmax == amplification + power pool | **1.3e-07** rel-err |
| RoPE == free-running oscillator phase | **2.6e-08** rel-err |
| Amplitude coupling reproduces `W·x` | **3.2e-08**; phase-only Kuramoto **9.75e-01** — the encoding is load-bearing |
| int8 per-column fidelity | y cosine **0.999949**; tokens unchanged through all 28 layers |
| Throughput | **1.43 tok/s** (0.70 s/token) — see the note below; this is NOT a speedup |

## Honest limitations

- **It performs matmuls.** The wave form is an *equivalence proof* about the
  arithmetic, not a substitute for it: relaxation converges to `W·x`, which is a
  value you have already computed. There is no speedup from the physics.
- **And there is no speedup at all, against a real baseline.** On this same
  machine, CPU-only, same model family, in the same units:

  |  | tok/s | s/token |
  |---|---|---|
  | this engine (int8, 2.82 GB) | **1.43** | 0.70 |
  | llama.cpp (Q4_K_M, 1.87 GiB) | **3.02** | 0.33 |

  We are **2.1x slower than llama.cpp**. Earlier drafts framed this as "68 s -> 0.7 s"; that
  68 s was an unoptimised numpy path re-reading 6.4 GB from disk per token, i.e.
  our own worst case, and quoting it as a baseline was not honest. The work here
  bought a *correct and resident* int8 CPU engine, not a fast one. The value is
  in the equivalence proofs and the negative results, not the throughput.
- **Ternary does not work.** Measured engine-free against real weights on both
  Llama-3.2-3B and Gemma-4-12B: y cosine **0.877** against int8's **0.99995**.
  Per-column scaling rescues int2 relative to a global scale (0.52 vs 0.036) but
  neither is usable. Quantisation fidelity turns out to be a property of the bit
  width rather than the model — the two families agree to three decimals at
  every width tested.
- **Scale is unproven.** Verified on 3B. The 12B needs int4 (5.45 GB) and does
  not fit in this machine's free RAM.
- **This Space ships no weights.** The dashboard renders and the visualisation is
  wired, but the engine cannot start without a local model. Build the cache with
  `bqsm_int8.py --build` on a machine that has one.
- **The tokenizer is whitespace-split, not BPE**, so punctuation and rare words
  are spelled character by character. `bqsm_tokenizer.py` implements real
  byte-level BPE but is not yet wired in.
- **The SRP popcount readout was removed.** Once the bf16 kernel did the dense
  vocabulary scan in ~60 ms, SRP was both slower (82–159 ms) and approximate —
  512-bit Hamming distances tie heavily, and a true argmax with 884 tokens
  strictly closer had 1,135 tied-or-closer, falling outside k=1024.
- Training, backprop, cross-request batching, and sampling above greedy argmax
  are not addressed.

## Architecture notes

The forward pass is a DAG of **311 blocks with a critical path of 227** — so the
available parallelism is **1.37×**, and 171 of those depth levels are strictly
serial. Depth is irreducible: 757 data-dependent nonlinearities (RMSNorm, the
saturating gate, softmax) break matrix associativity at every stage. Remove them
and all 28 layers collapse into a single 128256×3072 matrix that computes
nothing interesting.

At one token the arithmetic intensity is **1.78 FLOP/byte** — memory-bound by
roughly 6×. Fusing 197 matrix-vector products down to 113 moves zero bytes.
Only fewer bytes per weight, or more work per byte (batching), change the time.

## Running locally

```bash
python3 bqsm_assist/bqsm_int8.py --build          # quantise once, ~2.82 GB
python3 bqsm_assist/bqsm_int8.py --n 5 --verify   # check against golden tokens
python3 phoenix_dashboard.py                      # dashboard on :8765
```

Code: [github.com/compunerd/basin-quotient-machine](https://github.com/compunerd/basin-quotient-machine)