bob-int: bob 2 ck_13000 weights, optimizer state, and the full registration

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,122 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- fixed-point
+- integer-arithmetic
+- deterministic
+- reproducibility
+---
+
+# bob-int
+
+The weights and every file in this repository are licensed
+**Apache-2.0**. The bob name and marks are not part of that grant.
+
+Model: bob 2-ck_13000 (bob-talk)
+
+A small integer language model. The same, every time.
+
+Try it: **https://bob-talk.org** ; the model in this repository is the
+model answering that page, verifiable by digest (below).
+
+Bob exists because I asked the question: "If fixed point buys exactness, can exact multithreading/parallelism then provide speed?"
+I'm still working on that question. This project was inspired by Norm Winningstead, and my own dumb questions.
+Bob has demonstrated the ability to resume his own training, cross-ISA and byte-exact reproducible. I do not know if Bob's training model 
+can do this on *every* possible combination- and make no claims to his ability outside of what I have already demonstrated.
+
+## What is in this repository
+
+```text
+ck_13000.weights.bin        the final weights, training step 13,000
+                            sha256 53ae7609f22e580ac1d67ad1995dfc2d87c0ebcd5f60f3ab7136345f502c2890
+ck_13000.opt.bin            the optimizer state at the same step, so the
+                            trajectory can be resumed, not only replayed
+registration/exponents.json the registered fixed-point exponents and bit
+                            widths for every activation and gradient site
+registration/width-sites.json  the accumulator width registrations
+registration/init/          the initial weights and ordering- the
+                            trajectory is replayable from step 0
+SHA256SUMS                  digests for every file above
+```
+
+## The property these files carry
+
+Every number in training and inference is fixed-point integer
+arithmetic under a registered contract: each site has a registered
+exponent and width, and arithmetic that would leave its registration
+**refuses**- it stops with a named site and an exit code rather than
+saturating or wrapping silently.
+
+Because the arithmetic is integer, results do not depend on
+floating-point summation order. The digest of the reply tokens for a
+given (prompt, seed, effort) is the same on every conforming build.
+Four pre-published examples with their digests are on
+https://bob-talk.org and embedded in its page source.
+
+Two portability results are in the project record, each stated no
+wider than its measurement:
+
+**bob 2's own training** (the weights in this repository) was saved
+mid-run and resumed on different rented machines across two GPU
+generations of one vendor, with checkpoint digests matching at the
+joints. All of bob 2's GPU lineages are one vendor; no cross-vendor
+claim is made for this trajectory.
+
+**A separate 80-step synthetic trial (2026-08-17, not these weights)**
+ran the same integer training on an Apple M4 via Metal (arm64, macOS)
+and a Pascal-era NVIDIA GTX 1070 via Vulkan (x86_64, Linux)-> produced
+byte-identical weights AND optimizer state- sha256
+`cddab458c1e2e8b9…` / `436114216b977be2…`- hashed independently on
+each machine. The two devices also made byte-identical refusal
+decisions: the enforcement machinery itself is deterministic across
+vendors. The trial is synthetic and unregistered; it demonstrates the
+arithmetic contract's portability, not this model's history.
+
+I know of no shipped system combining an end-to-end integer training
+loop with integer optimizer state, overflow semantics that refuse with
+a named site and typed exit, and demonstrated mid-run save-and-resume
+with matching digests- resting on a documented 2026-08-17 survey of
+eleven named systems: unoccupied as of the search date, not
+impossible, not permanent. Please direct me toward prior art if available.
+
+## Verify this is the model behind bob-talk.org
+
+```
+sha256sum ck_13000.weights.bin
+# 53ae7609f22e580ac1d67ad1995dfc2d87c0ebcd5f60f3ab7136345f502c2890
+```
+
+That digest is compiled into the public server, which refuses to start
+if its embedded weights do not hash to it. Ask the live page
+(POST /ask, `{"prompt": "Alice was", "seed": "7"}`) and compare the
+reply digest against the published table on the page.
+
+## Format
+
+Weights are raw little-endian integer tensors in the registered
+layout; `registration/exponents.json` gives each site's fixed-point
+interpretation. This is not a transformers/safetensors checkpoint and
+will not load with `from_pretrained`. Geometry: d_model 512, 8 layers,
+8 heads, head_dim 64, d_mlp 2048, context 8192, vocab 8192; parameter
+count 33,608,704 with the learned position table (proved at both ends
+of the pipeline; receipts in the project record).
+
+Inference code: [TBD - LIVE GH REPO NOT RELEASED; https://bob-talk.org/ for live inference]
+
+## What is NOT claimed
+
+bob 2 is deterministic in the sense stated above. He is not thereby
+accurate, clever, or safe, and no such claim is made. He is small
+(33.6M parameters), his prose is what a 33.6M-parameter model's prose
+is, and the property I claim to be interesting is the arithmetic, 
+not the model's vocabulary.
+
+## Provenance
+
+Trained August of 2026 by f32. Training step 13,000; two legs; the enforcement
+regime is part of the record (one leg ended in a registered latch on a
+named site — the refusal machinery demonstrated live, then the
+registration corrected and the run completed).

SHA256SUMS

@@ -0,0 +1,9 @@
+4fb65611354d4c19fad49c90588283cadbfed77c50e5cb533355e54b5213aac8  ./ck_13000.opt.bin
+53ae7609f22e580ac1d67ad1995dfc2d87c0ebcd5f60f3ab7136345f502c2890  ./ck_13000.weights.bin
+f2a008fb4295b7d9ff0093e56a01fd24f1dba7773f40485cd69f5f176c2f6c6c  ./README.md
+87c73b8f34990c8db9a9a48dbc66bf7b1f3502cea83913bc3679e8912ddd8ecb  ./registration/exponents.json
+79bb7297951e9667305ebac3498d89206bf8b7cf22a1d7169009dd0bdc337762  ./registration/init/config.json
+45a69cc283257a2e591b1f5335992507395f3f83259b838247d9ed7fa9cf3f70  ./registration/init/INIT.json
+058e1b063a6725e14ab950d6eb276ad8fff6052aaf64bade25b64f9e4e3b1d16  ./registration/init/init.weights.bin
+a780f6e86eb15514ad25f04eae0d44d0f6f628c2555312369d69d776879672e7  ./registration/init/order.u32.le.bin
+68c5945ff941d0d7a439ed3ae740805a0aa7089e664a5cfe8728960e3bfd4c94  ./registration/width-sites.json

ck_13000.opt.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fb65611354d4c19fad49c90588283cadbfed77c50e5cb533355e54b5213aac8
+size 41033728

ck_13000.weights.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53ae7609f22e580ac1d67ad1995dfc2d87c0ebcd5f60f3ab7136345f502c2890
+size 20516864

registration/exponents.json

@@ -0,0 +1,703 @@
+{
+  "act": {
+    "attn_av": {
+      "bits": 23,
+      "exp": -19
+    },
+    "attn_proj_out": {
+      "bits": 23,
+      "exp": -22
+    },
+    "ln1_out": {
+      "bits": 23,
+      "exp": -17
+    },
+    "ln2_out": {
+      "bits": 23,
+      "exp": -17
+    },
+    "lnf_out": {
+      "bits": 23,
+      "exp": -17
+    },
+    "logits": {
+      "bits": 23,
+      "exp": -16
+    },
+    "mlp_h": {
+      "bits": 23,
+      "exp": -19
+    },
+    "mlp_out": {
+      "bits": 23,
+      "exp": -21
+    },
+    "qkv": {
+      "bits": 23,
+      "exp": -18
+    },
+    "resid": {
+      "bits": 23,
+      "exp": -19
+    }
+  },
+  "adam": {
+    "eps_int": 131072,
+    "m_bits": 27,
+    "sqrt_shift": 7,
+    "v_bits": 48
+  },
+  "backoff_bits": 0,
+  "bwd": {
+    "d_attn_av": {
+      "bits": 23,
+      "exp": -18
+    },
+    "d_attn_proj_out": {
+      "bits": 23,
+      "exp": -15
+    },
+    "d_ln1_out": {
+      "bits": 23,
+      "exp": -20
+    },
+    "d_ln2_out": {
+      "bits": 23,
+      "exp": -20
+    },
+    "d_ln_g": {
+      "bits": 23,
+      "exp": -20
+    },
+    "d_lnf_out": {
+      "bits": 23,
+      "exp": -22
+    },
+    "d_logits": {
+      "bits": 23,
+      "exp": -20
+    },
+    "d_mlp_h": {
+      "bits": 23,
+      "exp": -19
+    },
+    "d_mlp_out": {
+      "bits": 23,
+      "exp": -16
+    },
+    "d_p": {
+      "bits": 23,
+      "exp": -17
+    },
+    "d_qkv": {
+      "bits": 23,
+      "exp": -18
+    },
+    "d_resid": {
+      "bits": 23,
+      "exp": -14
+    },
+    "d_score": {
+      "bits": 23,
+      "exp": -22
+    }
+  },
+  "eps_raw": [
+    2748779,
+    2748779,
+    2748779,
+    2748779,
+    2748779,
+    2748779,
+    2748779,
+    2748779,
+    2748779,
+    2748779,
+    2748779
+  ],
+  "format": "bob-int-exponents-1",
+  "grad": [
+    {
+      "bits": 23,
+      "exp": -7
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -10
+    },
+    {
+      "bits": 23,
+      "exp": -10
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -7
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -14
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -14
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -13
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -8
+    },
+    {
+      "bits": 23,
+      "exp": -9
+    },
+    {
+      "bits": 23,
+      "exp": -12
+    },
+    {
+      "bits": 23,
+      "exp": -10
+    },
+    {
+      "bits": 23,
+      "exp": -11
+    },
+    {
+      "bits": 23,
+      "exp": -6
+    }
+  ],
+  "loss_exp": -20,
+  "note": "geometry-derived carrier targets: weight=20 activation=20 gradient=20; inclusive-Q30 attention row at ctx=512 closes at 62 of 62 accumulator bits; shadow_contract={\"batch\":16,\"config_path\":\"/Volumes/F32WORK/training-scratch/bob/mvc/software/internal/ml/bob/train-int/mvc/config.json\",\"config_sha256\":\"79bb7297951e9667305ebac3498d89206bf8b7cf22a1d7169009dd0bdc337762\",\"config_source_sha256\":\"79bb7297951e9667305ebac3498d89206bf8b7cf22a1d7169009dd0bdc337762\",\"corpus_path\":\"/Volumes/F32WORK/training-scratch/bob/mvc/software/internal/ml/detcorpus/freeze/streams/all/tokens.u16.le.bin\",\"corpus_sha256\":\"5b7343b6415ce15dafba5ce951c3da546be64d28f8124a5a6fbea22196630502\",\"format\":\"bob-shadow-contract-1\",\"lr_den\":16,\"lr_num\":1,\"optimizer\":\"sgd\",\"order_path\":\"/Volumes/F32WORK/training-scratch/bob/mvc/.scratch/mvc-registration-a18/init.bootstrap/order.u32.le.bin\",\"order_sha256\":\"a780f6e86eb15514ad25f04eae0d44d0f6f628c2555312369d69d776879672e7\",\"schedule_total\":3128,\"seed\":2827,\"shadow_sha256\":\"0a318217280f44468e43e891aa3cd35eb8f97a53b2f262cb0be776d08ff64c0e\",\"steps\":450,\"warmup\":200}",
+  "tensors": [
+    "h.0.attn.proj.bias",
+    "h.0.attn.proj.weight",
+    "h.0.attn.qkv.bias",
+    "h.0.attn.qkv.weight",
+    "h.0.ln1.bias",
+    "h.0.ln1.weight",
+    "h.0.ln2.bias",
+    "h.0.ln2.weight",
+    "h.0.mlp.fc.bias",
+    "h.0.mlp.fc.weight",
+    "h.0.mlp.proj.bias",
+    "h.0.mlp.proj.weight",
+    "h.1.attn.proj.bias",
+    "h.1.attn.proj.weight",
+    "h.1.attn.qkv.bias",
+    "h.1.attn.qkv.weight",
+    "h.1.ln1.bias",
+    "h.1.ln1.weight",
+    "h.1.ln2.bias",
+    "h.1.ln2.weight",
+    "h.1.mlp.fc.bias",
+    "h.1.mlp.fc.weight",
+    "h.1.mlp.proj.bias",
+    "h.1.mlp.proj.weight",
+    "h.2.attn.proj.bias",
+    "h.2.attn.proj.weight",
+    "h.2.attn.qkv.bias",
+    "h.2.attn.qkv.weight",
+    "h.2.ln1.bias",
+    "h.2.ln1.weight",
+    "h.2.ln2.bias",
+    "h.2.ln2.weight",
+    "h.2.mlp.fc.bias",
+    "h.2.mlp.fc.weight",
+    "h.2.mlp.proj.bias",
+    "h.2.mlp.proj.weight",
+    "h.3.attn.proj.bias",
+    "h.3.attn.proj.weight",
+    "h.3.attn.qkv.bias",
+    "h.3.attn.qkv.weight",
+    "h.3.ln1.bias",
+    "h.3.ln1.weight",
+    "h.3.ln2.bias",
+    "h.3.ln2.weight",
+    "h.3.mlp.fc.bias",
+    "h.3.mlp.fc.weight",
+    "h.3.mlp.proj.bias",
+    "h.3.mlp.proj.weight",
+    "h.4.attn.proj.bias",
+    "h.4.attn.proj.weight",
+    "h.4.attn.qkv.bias",
+    "h.4.attn.qkv.weight",
+    "h.4.ln1.bias",
+    "h.4.ln1.weight",
+    "h.4.ln2.bias",
+    "h.4.ln2.weight",
+    "h.4.mlp.fc.bias",
+    "h.4.mlp.fc.weight",
+    "h.4.mlp.proj.bias",
+    "h.4.mlp.proj.weight",
+    "ln_f.bias",
+    "ln_f.weight",
+    "wpe",
+    "wte"
+  ],
+  "weight": [
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -27
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -28
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -29
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -29
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -30
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -24
+    },
+    {
+      "bits": 23,
+      "exp": -30
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -30
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -30
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -30
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -29
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -31
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -30
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -30
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -26
+    },
+    {
+      "bits": 23,
+      "exp": -25
+    },
+    {
+      "bits": 23,
+      "exp": -24
+    },
+    {
+      "bits": 23,
+      "exp": -19
+    },
+    {
+      "bits": 23,
+      "exp": -23
+    },
+    {
+      "bits": 23,
+      "exp": -22
+    }
+  ]
+}

registration/init/INIT.json

@@ -0,0 +1,484 @@
+{
+  "batch": 16,
+  "config": {
+    "ctx": 512,
+    "d_mlp": 1024,
+    "d_model": 256,
+    "head_dim": 32,
+    "n_head": 8,
+    "n_layer": 5,
+    "vocab": 4097
+  },
+  "config_sha256": "79bb7297951e9667305ebac3498d89206bf8b7cf22a1d7169009dd0bdc337762",
+  "corpus_sha256": "5b7343b6415ce15dafba5ce951c3da546be64d28f8124a5a6fbea22196630502",
+  "corpus_tokens": 25627271,
+  "env": {
+    "LC_ALL": "C",
+    "MKL_NUM_THREADS": "1",
+    "NUMEXPR_NUM_THREADS": "1",
+    "OMP_NUM_THREADS": "1",
+    "OPENBLAS_NUM_THREADS": "1",
+    "PYTHONHASHSEED": "0"
+  },
+  "exponents_sha256": "87c73b8f34990c8db9a9a48dbc66bf7b1f3502cea83913bc3679e8912ddd8ecb",
+  "format": "bob-int-init-1",
+  "init_weights_sha256": "058e1b063a6725e14ab950d6eb276ad8fff6052aaf64bade25b64f9e4e3b1d16",
+  "note": "initialization lives in the registration lane, where float is lawful; the training binary loads these bytes and draws no randomness",
+  "numpy": "2.4.6",
+  "order_sha256": "a780f6e86eb15514ad25f04eae0d44d0f6f628c2555312369d69d776879672e7",
+  "quantization": {
+    "h.0.attn.proj.bias": {
+      "exp": -25,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.0.attn.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4900825086186856e-08,
+      "max_abs_value": 0.028547046461096078,
+      "max_mantissa": 957880
+    },
+    "h.0.attn.qkv.bias": {
+      "exp": -27,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.0.attn.qkv.weight": {
+      "exp": -23,
+      "max_abs_error": 5.960445941671777e-08,
+      "max_abs_value": 0.08745060120487574,
+      "max_mantissa": 733589
+    },
+    "h.0.ln1.bias": {
+      "exp": -28,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.0.ln1.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.0.ln2.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.0.ln2.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.0.mlp.fc.bias": {
+      "exp": -29,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.0.mlp.fc.weight": {
+      "exp": -23,
+      "max_abs_error": 5.9604571268218054e-08,
+      "max_abs_value": 0.0902318703807739,
+      "max_mantissa": 756920
+    },
+    "h.0.mlp.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.0.mlp.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4901081767364904e-08,
+      "max_abs_value": 0.02785039083381541,
+      "max_mantissa": 934504
+    },
+    "h.1.attn.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.1.attn.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4900512158334822e-08,
+      "max_abs_value": 0.030010076930930364,
+      "max_mantissa": 1006971
+    },
+    "h.1.attn.qkv.bias": {
+      "exp": -29,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.1.attn.qkv.weight": {
+      "exp": -23,
+      "max_abs_error": 5.960463195231469e-08,
+      "max_abs_value": 0.0868514430738508,
+      "max_mantissa": 728563
+    },
+    "h.1.ln1.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.1.ln1.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.1.ln2.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.1.ln2.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.1.mlp.fc.bias": {
+      "exp": -30,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.1.mlp.fc.weight": {
+      "exp": -23,
+      "max_abs_error": 5.9604310903571545e-08,
+      "max_abs_value": 0.09860658932161066,
+      "max_mantissa": 827172
+    },
+    "h.1.mlp.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.1.mlp.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4901131004455004e-08,
+      "max_abs_value": 0.026966789496377968,
+      "max_mantissa": 904855
+    },
+    "h.2.attn.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.2.attn.proj.weight": {
+      "exp": -24,
+      "max_abs_error": 2.980231248936316e-08,
+      "max_abs_value": 0.030661219476549803,
+      "max_mantissa": 514410
+    },
+    "h.2.attn.qkv.bias": {
+      "exp": -30,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.2.attn.qkv.weight": {
+      "exp": -23,
+      "max_abs_error": 5.960367261900745e-08,
+      "max_abs_value": 0.09253014902603585,
+      "max_mantissa": 776199
+    },
+    "h.2.ln1.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.2.ln1.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.2.ln2.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.2.ln2.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.2.mlp.fc.bias": {
+      "exp": -30,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.2.mlp.fc.weight": {
+      "exp": -23,
+      "max_abs_error": 5.960426689016751e-08,
+      "max_abs_value": 0.0952345986171801,
+      "max_mantissa": 798886
+    },
+    "h.2.mlp.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.2.mlp.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4901160072348929e-08,
+      "max_abs_value": 0.02909195431374654,
+      "max_mantissa": 976164
+    },
+    "h.3.attn.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.3.attn.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4901043606484199e-08,
+      "max_abs_value": 0.027527185692206652,
+      "max_mantissa": 923659
+    },
+    "h.3.attn.qkv.bias": {
+      "exp": -30,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.3.attn.qkv.weight": {
+      "exp": -23,
+      "max_abs_error": 5.9604490555004164e-08,
+      "max_abs_value": 0.08902847695681987,
+      "max_mantissa": 746825
+    },
+    "h.3.ln1.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.3.ln1.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.3.ln2.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.3.ln2.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.3.mlp.fc.bias": {
+      "exp": -30,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.3.mlp.fc.weight": {
+      "exp": -23,
+      "max_abs_error": 5.9604436383059456e-08,
+      "max_abs_value": 0.08617036348075562,
+      "max_mantissa": 722849
+    },
+    "h.3.mlp.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.3.mlp.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.490113704649687e-08,
+      "max_abs_value": 0.03025808203863395,
+      "max_mantissa": 1015293
+    },
+    "h.4.attn.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.4.attn.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4900754712575404e-08,
+      "max_abs_value": 0.03118358869308788,
+      "max_mantissa": 1046348
+    },
+    "h.4.attn.qkv.bias": {
+      "exp": -29,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.4.attn.qkv.weight": {
+      "exp": -23,
+      "max_abs_error": 5.960446929163116e-08,
+      "max_abs_value": 0.09162640165799685,
+      "max_mantissa": 768618
+    },
+    "h.4.ln1.bias": {
+      "exp": -31,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.4.ln1.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.4.ln2.bias": {
+      "exp": -30,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.4.ln2.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "h.4.mlp.fc.bias": {
+      "exp": -30,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.4.mlp.fc.weight": {
+      "exp": -23,
+      "max_abs_error": 5.960430381896087e-08,
+      "max_abs_value": 0.09193171455344683,
+      "max_mantissa": 771179
+    },
+    "h.4.mlp.proj.bias": {
+      "exp": -26,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "h.4.mlp.proj.weight": {
+      "exp": -25,
+      "max_abs_error": 1.4901079483655658e-08,
+      "max_abs_value": 0.027933995770124034,
+      "max_mantissa": 937309
+    },
+    "ln_f.bias": {
+      "exp": -24,
+      "max_abs_error": 0.0,
+      "max_abs_value": 0.0,
+      "max_mantissa": 0
+    },
+    "ln_f.weight": {
+      "exp": -19,
+      "max_abs_error": 0.0,
+      "max_abs_value": 1.0,
+      "max_mantissa": 524288
+    },
+    "wpe": {
+      "exp": -23,
+      "max_abs_error": 5.960389527423504e-08,
+      "max_abs_value": 0.08881911151199198,
+      "max_mantissa": 745069
+    },
+    "wte": {
+      "exp": -22,
+      "max_abs_error": 1.1920920599955975e-07,
+      "max_abs_value": 0.09259805303077025,
+      "max_mantissa": 388384
+    }
+  },
+  "seed": 2827,
+  "steps": 3128,
+  "tensors": [
+    "h.0.attn.proj.bias",
+    "h.0.attn.proj.weight",
+    "h.0.attn.qkv.bias",
+    "h.0.attn.qkv.weight",
+    "h.0.ln1.bias",
+    "h.0.ln1.weight",
+    "h.0.ln2.bias",
+    "h.0.ln2.weight",
+    "h.0.mlp.fc.bias",
+    "h.0.mlp.fc.weight",
+    "h.0.mlp.proj.bias",
+    "h.0.mlp.proj.weight",
+    "h.1.attn.proj.bias",
+    "h.1.attn.proj.weight",
+    "h.1.attn.qkv.bias",
+    "h.1.attn.qkv.weight",
+    "h.1.ln1.bias",
+    "h.1.ln1.weight",
+    "h.1.ln2.bias",
+    "h.1.ln2.weight",
+    "h.1.mlp.fc.bias",
+    "h.1.mlp.fc.weight",
+    "h.1.mlp.proj.bias",
+    "h.1.mlp.proj.weight",
+    "h.2.attn.proj.bias",
+    "h.2.attn.proj.weight",
+    "h.2.attn.qkv.bias",
+    "h.2.attn.qkv.weight",
+    "h.2.ln1.bias",
+    "h.2.ln1.weight",
+    "h.2.ln2.bias",
+    "h.2.ln2.weight",
+    "h.2.mlp.fc.bias",
+    "h.2.mlp.fc.weight",
+    "h.2.mlp.proj.bias",
+    "h.2.mlp.proj.weight",
+    "h.3.attn.proj.bias",
+    "h.3.attn.proj.weight",
+    "h.3.attn.qkv.bias",
+    "h.3.attn.qkv.weight",
+    "h.3.ln1.bias",
+    "h.3.ln1.weight",
+    "h.3.ln2.bias",
+    "h.3.ln2.weight",
+    "h.3.mlp.fc.bias",
+    "h.3.mlp.fc.weight",
+    "h.3.mlp.proj.bias",
+    "h.3.mlp.proj.weight",
+    "h.4.attn.proj.bias",
+    "h.4.attn.proj.weight",
+    "h.4.attn.qkv.bias",
+    "h.4.attn.qkv.weight",
+    "h.4.ln1.bias",
+    "h.4.ln1.weight",
+    "h.4.ln2.bias",
+    "h.4.ln2.weight",
+    "h.4.mlp.fc.bias",
+    "h.4.mlp.fc.weight",
+    "h.4.mlp.proj.bias",
+    "h.4.mlp.proj.weight",
+    "ln_f.bias",
+    "ln_f.weight",
+    "wpe",
+    "wte"
+  ],
+  "width_sites_sha256": "68c5945ff941d0d7a439ed3ae740805a0aa7089e664a5cfe8728960e3bfd4c94"
+}

registration/init/config.json

@@ -0,0 +1,9 @@
+{
+  "ctx": 512,
+  "d_mlp": 1024,
+  "d_model": 256,
+  "head_dim": 32,
+  "n_head": 8,
+  "n_layer": 5,
+  "vocab": 4097
+}

registration/init/init.weights.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:058e1b063a6725e14ab950d6eb276ad8fff6052aaf64bade25b64f9e4e3b1d16
+size 20516864

registration/init/order.u32.le.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a780f6e86eb15514ad25f04eae0d44d0f6f628c2555312369d69d776879672e7
+size 200192