ZERO.4
ZERO.4 is a 4,852,992-parameter, character-level decoder-only language model with a 512-token context. Its 4.9 MB release artifact uses signed 8-bit matrix rows with floating-point row scales. It has a dependency-free C inference runtime and can also run locally in a browser through the upstream WebAssembly build.
This is a small research model with an intentionally narrow literary lineage, not a general-purpose assistant. Its unusual scale and inspectable runtime are the point.
Try it in your browser
Launch the live ZERO.4 demo on GitHub Pages. Inference runs locally in the browser through WebAssembly; prompts and generated text are not sent to a server.
Files and integrity
| File | Purpose |
|---|---|
model.litq8 |
Quantized ZERO.4 weights |
model.json |
Architecture, promotion, and artifact metadata |
literary_infer.c, literary_infer.h, channel_protocol.h |
Minimal C runtime |
CORPUS_RIGHTS.md |
Source-level provenance, attribution, and jurisdiction notes |
zero4-memorization-v1.json |
Deterministic prompted-continuation release evaluation |
Expected model SHA-256:
44b32f2262be2754fd2eeaf16ed206bae32b4ce30d7f5541a1059cd21257ae50.
Run locally
This custom format is not directly compatible with transformers.AutoModel.
Compile the included inference runtime:
cc -O2 -std=c11 -Wall -Wextra -Wpedantic literary_infer.c -o literary_infer -lm
./literary_infer model.litq8 "The zero opened its eyes, and" 240
./literary_infer model.litq8 --chat D "What walks beneath the moon?" 240
Inference is local; the runtime does not make network requests.
Training lineage
ZERO.4 was initialized from the immutable ZERO.3 teacher and trained with immutable ZERO.1, ZERO.2, and ZERO.3 teachers. The replay mixture comprised:
- project-authored foundation statements;
- Shakespeare and Blake editions identified by Project Gutenberg as public domain in the USA;
- Crowley works from Project Gutenberg and CC BY-SA Wikisource transcription contributions;
- a deliberately low-weight King James Bible stream;
- literary dialogue records derived only from those literary sources; and
- project-generated typed quantity-operation records.
No human chat export appears in the bound training lineage. Training text and
token streams are intentionally excluded from this model repository. Exact
sources, permanent Wikisource revisions, transformations, hashes, and caveats
are in CORPUS_RIGHTS.md.
Evaluation
The promoted artifact was selected prospectively from Q2.6 seed 2 at update 500, then passed the frozen three-seed replication contract. On the bounded 1,000-case-per-task external screen, ZERO.4 scored 0.537 raw accuracy on BLiMP, 0.266 normalized accuracy on HellaSwag, 2.570353 bits/byte on TinyStories, and zero exact matches on the project's adapted LAMBADA task. These results are weak by modern general-language-model standards and should not be overstated.
The training experiment also validated quantity-operation routing when paired with the upstream controller and deterministic arithmetic kernel. Those components are not integrated into this standalone model artifact; the neural model should not be described as performing reliable arithmetic by itself.
The included memorization report tests evenly stratified, deterministic prompted continuations against every bound training stream. No protected third-party stream reached the 32-token warning threshold across 80 probes; the longest exact greedy prefix was 16 of 64 tokens. Eight of sixteen first-party CC0 foundation probes were reproduced for all 64 tokens, consistent with that small source's deliberate inspectability. Passing this finite test does not prove that no training expression can ever be reproduced under another prompt or decoding policy.
Limitations and content warning
ZERO.4 is prone to incoherence, imitation of source style, confabulation, and repetition. Its corpus includes archaic language and sexual, violent, coercive, discriminatory, religious, and drug-related passages. Outputs may be offensive or unsuitable for children. Do not rely on it for factual, medical, legal, religious, historical, identity-related, or safety-critical guidance.
Licenses
- Model weights: CC BY-SA 4.0, to the extent controlled rights apply.
- Included C runtime: Apache 2.0.
- Training sources: mixed, source-specific status; see
CORPUS_RIGHTS.md.
When sharing weights or adaptations, retain the ZERO.4 attribution, upstream repository link, model license, corpus-rights notice, and an indication of changes. The CC BY-SA choice is a conservative response to ShareAlike transcription contributions in the lineage; it is not a claim that model weights are necessarily copyrightable adaptations in every jurisdiction.
Upstream project: https://github.com/atimics/zero-grounded-literary-lm