Slayer1B 1.09
Arkadiusz Sลota, Anna Sieczka, Piotr Styลa, Aleksander Ogrodzki, Paweล Puzio, Marcin Kuลmierski / Fabryka AI (SlayerLab)
Intermediate base checkpoints
Continued pretraining is in progress; these are base models, not instruction-tuned models. The checkpoints below were taken in the constant-learning-rate phase. The main-base plan does not select an in-run terminal decay; the configured later schedule boundary still requires operator review. Allocation pauses do not mean the campaign is complete.
| Update | Positions in this continuation | Lineage lower bound, positions | Native weights file |
|---|---|---|---|
| 5,000 | 5,242,880,000 | 23,991,510,554 | wagi_5000.pt |
| 10,000 | 10,485,760,000 | 29,234,390,554 | wagi_10000.pt |
| 15,000 | 15,728,640,000 | 34,477,270,554 | wagi_15000.pt |
| 20,000 | 20,971,520,000 | 39,720,150,554 | wagi_20000.pt |
| 25,000 | 26,214,400,000 | 44,963,030,554 | wagi_25000.pt |
| 30,000 | 31,457,280,000 | 50,205,910,554 | wagi_30000.pt |
| 35,000 | 36,700,160,000 | 55,448,790,554 | wagi_35000.pt |
Files are managed in the separate checkpoint archive. Archive snapshot, 12 November 2026, 18:51 UTC: upload of all eight native checkpoint files completed at 18:16 UTC. The archive is public; verify its exact revision, file list, SHA256SUMS and file sizes before use. This main repository contains documentation during training.
The archive distinguishes seven weights-only .pt files from the full update-35,000 recovery checkpoint ckpt_35000.pt, which also contains optimizer and queue state. These are native training checkpoints, not a Transformers AutoModel or safetensors release. Check the archive's exact revision, file list and checksums before use; its publication status is separate from the evaluation table below.
Token counts: positions in this continuation = updates ร 1,048,576, counting each logical update once, not repeated compute after recovery. Lineage lower bound = 18,863,257,682 inherited positions + positions in this continuation โ 114,627,128 identified-copy positions. The lineage figure is a source- and file-disjoint lower bound, not globally unique tokens. Exact evaluated safetensors identities in EVALUATION.json are distinct from the native .pt file digests in the archive.
Execution receipt reported by the operators, 12 November 2026, 18:51 UTC: training has been paused since approximately 16:48 UTC, at reported progress around update 40,000, while allocation prices exceed the approved ceilings. The public native archive currently ends at update 35,000. The CPT Track history provides later telemetry; reported progress, saved recovery state and published files have separate receipts.
Objective and starting point
Funding update โ 11 November 2026, 15:29 UTC: the operator reported USD 500 received for continued main-base training. This is an additional funding receipt, not a measurement of the current account balance or a funded final update count. The earlier USD 425 campaign envelope belongs to the historical 12:07 UTC decision. The objective remains maximum useful affordable exposure with constant learning rates after warmup and no planned in-run terminal decay, retaining cost guards and the operational reserve for a recoverable checkpoint, transfer and safe shutdown. An extension approaching the configured decay boundary requires review. Stop at a verified full checkpoint when the real budget or allocation requires it; a 5,000-update boundary is not required. Positions per dollar measure operating efficiency, not model quality.
Recipe adoption โ 11 November 2026: operators confirmed the new recipe at 16:23:47 UTC, resuming from update 29,800 on H200, with --steps 120000 --ckpt-every 200, LR 0.0006 and configured WSD decay beginning at update 96,000. The operational target is 80,000 updates, corresponding to 102,634,710,554 source- and file-disjoint lineage positions under the stated accounting. The 120,000-update argument controls the scheduler. Allocation and funding can interrupt progress before the target.
A separate, continuously curated option targets approximately 10B tokens of quality-selected โgoldโ data at context 4,096. It remains isolated from main-base inputs and evaluation material, with its own manifest and licensing/admission gate; books selected with Arkadiusz Sลota require title-level license approval. A later finishing run with decay needs its own run and budget decision. Preliminary exclusion lists are not a complete packed 10B corpus or a release, and quality-based selection is not evidence of a measured quality gain. See the data card for the dated inventory and isolation boundary.
The starting weights are SlayerLab/Slayer1B-1.08, not random initialization or the abandoned attempt's checkpoint. Operators verified step-zero INIT-FROM, fresh optimizer state, approved c01 bytes, fresh numerical control and successful preflight before the observed updates. The selected public v5 launcher uses a separate output history for this continuation.
The unchanged architecture is specified as 1,073,912,851 parameters, 20 layers, width 2,048, 16 heads, SwiGLU, RMSNorm, RoPE base 500,000, QK normalization and value residuals. The reused v6 tokenizer has vocabulary 32,768 and pin 0640d3bd3674d7a6e59c540d945a2ad275a12e1bcaeb6007c37a90751f9815dd.
Architecture and tokenizer lineage includes SlayerLab/Slayer1B-1.07. The selected 1.08 base and the separate historical 1.08b ablation are different checkpoints; the continuation described here starts from 1.08.
The training configuration records the 120,000-update scheduling recipe and its adoption. The operational target of 80,000 updates and the actual completed count are separate fields. Completed 1.08 and the abandoned first attempt retain their own historical records.
Current CPT configuration and budget-limited endpoint
| Setting | Operator-selected recipe |
|---|---|
| Selected scheduling argument | KROKI=120000; confirmed runtime --steps 120000 adoption at 11 November, 16:23:47 UTC, resuming from update 29,800 on H200; configured decay begins at update 96,000 |
| New-source allocation | Initially 20% PL / 80% EN target. On 12 November at 18:44 UTC, the owner approved 30% PL / 70% EN for future, not-yet-built segments from c16 onward; this is not a runtime-adoption receipt or a measured cumulative model-language ratio |
| Optimizer / schedule | Muon peak 0.02 plus AdamW peak 0.0006; 2,000-update warmup, then constant learning rates under the main-base plan; adoption receipt confirms LR 0.0006. The configured final-20% WSD decay to 0.00006 remains at update 96,000; an extension approaching it requires review |
| Checkpoints | Runtime --ckpt-every 200 confirmed at the 16:23:47 UTC adoption; weights-only milestones every 5,000 updates. Native milestone files and full-state recovery files are distinguished in the separate checkpoint archive |
| Code | Launcher v5 SHA256 796c71016c1e167bfd3ca98b78518f91c3502a96eac233dd7186ef6639fcc656; exact command and converter pins in the configuration document |
Operators must review actual available funds, prices and allocation changes without assuming that the decay boundary is permanently unaffordable. An extended run must be reviewed before entering decay; do not shorten KROKI to a budget estimate and bring decay forward. Runtime changes need the operator's approved restart procedure and receipt. The trainer and v5 launcher are unchanged; the operator wrapper's argument ceiling was separately approved for the new recipe.
The reviewed inherited count is 18,863,257,682 source/file-disjoint positions. The conservative c01 correction is 114,627,128 positions, computed from two exclusion-list counts. The training target is over 100B source- and file-disjoint positions in the 1.00โ1.09 lineage; the formula reaches that threshold at update 77,488, and the 80,000-update plan corresponds to approximately 102.6B. Historical content-level near-duplicates remain unresolved. The data card separately records prepared input capacity.
Initial CPT execution evidence
The separate CPT Track history preserves real training aggregates. Its 02:27 UTC registration timestamp is not the 04:50 UTC training start reported by operators. Fresh hardware control, initialization from 1.08, world size eight and preflight were independently checked for that start.
| First checked public sample | Value |
|---|---|
| Update | 20 |
| Training loss | 2.486659 |
| Interval throughput | 578,293 positions/s |
| New processed positions | 20,971,520 |
| AdamW learning rate during warmup | 0.000006 |
The step-20 sample is retained as initial evidence; use Track for current progress. Interval throughput is not full-window or billed throughput. Training loss is measured on the current input segment: the change from c02 to c03 at update 10,000 changes the data distribution, so a lower training loss across that boundary does not establish a sudden quality gain. Later checkpoint-copy and recovery evidence is recorded separately above.
Archived from-scratch attempt โ not the CPT recipe
| Setting | Abandoned first-attempt configuration |
|---|---|
| Trainer / optimizer | Existing r24.2; Muon with AdamW; fresh optimizer |
| Batch / context | 256 ร 4,096 = 1,048,576 positions per update |
| Layout | Eight GPU ranks; microbatch 8 ร accumulation 4; activation checkpointing off |
| Budget | 95,000 updates; nineteen slots of 5,000 |
| WSD | 2,000 warmup updates; constant LR 0.0006, Muon 0.02; final 20% uses 1โsqrt decay to nonzero LR floor 0.00006 |
| Checkpoints | Every 500 updates and retained slot-end milestones |
The abandoned plan's 95,000 ร 1,048,576 = 99,614,720,000 positions was not exactly 100B and is not the approved length of the revised continuation. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.
Slot-1 GPU execution was observed before that historical interruption. Do not resume the from-scratch state as the requested CPT. The revised continuation has its own output history and Track identity; historical points have not been relabelled as training from 1.08. Each queue extension still requires approved byte ranges, manifest digests and preserved sampler identity within its experiment.
Archived first-attempt evidence โ not CPT progress
The public Track run is separate from 1.08. At update 30 it records training loss 10.3959, interval throughput 576,593 positions/s and 31,457,280 processed positions (30 ร 1,048,576). This is an early interval, not end-to-end billed throughput or a quality evaluation. The first slot targets 5,000 updates; its completion is not claimed.
The last received public sample before the pause is update 450, training loss 4.3910, with 471,859,200 processed positions. Its ingestion timestamp is 22:36:50 UTC. It is not a live GPU reading, a final exposure count or proof of a checkpoint at update 500. The trainer's recoverable step must be checked from the saved state, not inferred from elapsed time or the tracker label.
Fresh numerical control and step-zero INIT-FROM preflight were verified for the CPT start described above. The earlier permission for the abandoned slot was not reused as that evidence. Any later physical-host restart still requires its own numerical control; successful start does not prove a checkpoint restore.
Weights-only milestone exports and full-state recovery checkpoints serve different purposes. The linked archive distinguishes both kinds; this main repository hosts neither during training. A milestone file does not establish that the newest complete optimizer state was transferred or that a cloud allocation is still running.
Data and exposure
The first CPT reference, c01, contains 5,347,737,900 input positions and TRAIN SHA256 3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1. One 5,000-update stage processes 5,242,880,000 positions, less than that input capacity. The historical c01โc06 queue reference supplied six stages through update 30,000; it is not the whole later prepared pool. Available input capacity, processed positions and historical content uniqueness remain distinct. See the data card for dated queue identities, source changes and the c01 duplicate correction.
The PL/EN balance will follow the approved fresh-data inventory. The user explicitly allows a different ratio; source buckets still do not establish the language share of actually consumed windows. Missing ancestral data must be reported as a limitation, not silently counted as verified unique exposure.
New material requires licensing, PII filtering, contamination scans, grouped held-outs and deduplication against the base model's full data ancestry. A 1% document holdout is not necessarily 1% of tokens. Historical exposure in the starting weights cannot be undone by filtering new inputs; retrospective benchmark hits must be handled explicitly in evaluation denominators.
Standard benchmark measurements โ updates 5,000โ35,000
We thank BenchTeam / Fabryka AI for benchmark implementations, protocol work and evaluation proposals. Its independent full benchmark report is distinct from the SlayerLab measurements below, published in the CPT Track history. The seven listed 1.09 checkpoints are intermediate weights of the ongoing continuation, not its eventual budget-limited endpoint.
The same nine full splits were scored zero-shot in float32 for 1.07 update 22,327, 1.08 update 1,709 and 1.09 updates 5,000, 10,000, 15,000, 20,000, 25,000, 30,000 and 35,000. Evaluator SHA, dataset revisions, file digests, task protocols and sample counts match across these rows. Scores are percentages with pointwise 95% confidence intervals; the named metric is accuracy (acc) or length-normalized accuracy (acc_norm). WinoGrande uses partial-sentence likelihood; MMLU-PL is micro-averaged. Polish HellaSwag and MMLU use the OpenGPTX task definitions.
| Task | Metric | n | 1.07 | 1.08 | 1.09, update 5,000 | 1.09, update 10,000 | 1.09, update 15,000 | 1.09, update 20,000 | 1.09, update 25,000 | 1.09, update 30,000 | 1.09, update 35,000 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ARC-Challenge | acc_norm | 1,172 | 27.30 [24.74, 29.86] | 30.03 [27.39, 32.68] | 29.35 [26.79, 32.00] | 30.46 [27.82, 33.11] | 32.51 [29.86, 35.24] | 33.11 [30.46, 35.84] | 33.79 [31.06, 36.52] | 33.02 [30.29, 35.67] | 35.15 [32.42, 37.88] |
| HellaSwag | acc_norm | 10,042 | 43.38 [42.43, 44.34] | 43.42 [42.43, 44.39] | 46.53 [45.55, 47.52] | 48.93 [47.97, 49.91] | 49.77 [48.80, 50.76] | 50.38 [49.39, 51.35] | 51.17 [50.18, 52.14] | 52.00 [51.02, 52.98] | 52.48 [51.50, 53.46] |
| PIQA | acc_norm | 1,838 | 66.97 [64.80, 69.15] | 66.21 [64.15, 68.34] | 68.28 [66.21, 70.40] | 68.99 [66.92, 71.11] | 68.99 [66.87, 71.06] | 70.78 [68.72, 72.85] | 71.55 [69.48, 73.61] | 70.67 [68.61, 72.74] | 71.27 [69.15, 73.29] |
| WinoGrande | acc | 1,267 | 53.59 [50.83, 56.35] | 53.67 [50.91, 56.43] | 54.93 [52.17, 57.70] | 54.93 [52.17, 57.62] | 56.75 [54.06, 59.43] | 55.01 [52.25, 57.77] | 55.88 [53.12, 58.64] | 57.14 [54.46, 59.83] | 56.75 [54.06, 59.43] |
| BoolQ | acc | 3,270 | 58.29 [56.61, 59.97] | 59.97 [58.26, 61.59] | 61.80 [60.18, 63.49] | 61.13 [59.45, 62.78] | 59.88 [58.20, 61.53] | 51.41 [49.69, 53.12] | 52.20 [50.52, 53.91] | 60.67 [58.99, 62.35] | 59.88 [58.20, 61.56] |
| OpenBookQA | acc_norm | 500 | 33.60 [29.40, 37.80] | 34.80 [30.60, 39.00] | 35.60 [31.40, 40.00] | 34.40 [30.20, 38.60] | 36.00 [31.80, 40.20] | 35.20 [31.00, 39.40] | 37.20 [32.80, 41.40] | 36.20 [32.00, 40.40] | 34.40 [30.40, 38.60] |
| Belebele, Polish | acc | 900 | 26.00 [23.11, 28.89] | 24.00 [21.22, 26.89] | 24.11 [21.33, 26.89] | 24.11 [21.33, 27.00] | 22.56 [19.89, 25.33] | 24.67 [21.89, 27.56] | 25.33 [22.44, 28.22] | 24.44 [21.67, 27.33] | 24.33 [21.56, 27.11] |
| HellaSwag-PL | acc_norm | 10,042 | 43.83 [42.84, 44.79] | 43.90 [42.90, 44.87] | 44.73 [43.76, 45.71] | 45.51 [44.51, 46.48] | 44.18 [43.21, 45.16] | 44.24 [43.26, 45.22] | 43.94 [42.94, 44.91] | 45.75 [44.77, 46.73] | 46.63 [45.68, 47.63] |
| MMLU-PL | acc | 14,042 | 24.01 [23.30, 24.71] | 23.04 [22.34, 23.74] | 24.10 [23.39, 24.82] | 23.32 [22.62, 24.03] | 23.50 [22.80, 24.21] | 23.86 [23.17, 24.58] | 24.26 [23.57, 24.96] | 24.71 [23.99, 25.42] | 23.28 [22.58, 23.97] |
Reference accuracies are context, not additional trained models or release thresholds:
| Reference strategy | Scope | Accuracy (%) |
|---|---|---|
| Uniform choice among four answers | Four-option reference for ARC-Challenge, HellaSwag/PL, OpenBookQA, Belebele-PL and MMLU-PL | 25.00 |
| Uniform choice among two answers | BoolQ, PIQA and WinoGrande | 50.00 |
| Always answer yes | BoolQ validation majority class: 2,033 of 3,270 labels | 62.17 |
The majority-class reference is different from uniform guessing. A score near a reference does not prove that checkpoints are equal or that a model has no knowledge; individual confidence intervals are not paired tests. Class-balanced accuracy and answer-frequency diagnostics, when available, are auxiliary and do not replace the primary metrics above.
Intervals use the percentile bootstrap with 10,000 replicates and seed 20261110. The updated approved aggregate does not supply the random-stream derivation or CI-generator SHA; the previous aggregate documented model-specific streams. Items are treated as independent; subject, template and source clusters are not modeled. These are intervals for individual results, not paired differences or training-seed variance; overlap is not a test of a difference.
The machine-readable aggregate record supplies exact values, intervals, counts, dataset revisions and file digests, checkpoint and evaluator identities, and metric selection, without prompts, answers or per-item predictions. ARC-Challenge, OpenBookQA and MMLU-PL use test splits; HellaSwag, PIQA, WinoGrande, BoolQ and HellaSwag-PL use validation splits; Belebele uses the full Polish evaluation set.
The approved aggregate SHA-256 is d91456c7480f8031288afd5056c002ce68ef7d7751556d222ae54d7ab6fc648a. All eight earlier model records retain their scores, intervals, counts and dataset pins; the update-35,000 record extends the nine-task comparison without replacing the independent two-task protocol below.
Descriptive results; lineage contamination checks are incomplete. New-input chunks c03 onward were cleaned against these nine Track tasks, c01โc02 were only partially checked, and the inherited 1.00โ1.08 data was not scanned. This is not a guarantee that these tasks are held out from the model's whole training history. Belebele and MMLU-PL are near their four-choice 25% random baseline; this is not proof of absent knowledge. No overall average, ranking or release threshold is inferred. Different evaluators must not be silently substituted.
MMLU-PL and Belebele-PL score the answer letter AโD given listed options; ARC-Challenge, HellaSwag/PL and PIQA score the answer text. This protocol difference is observed. Its contribution to the results requires a separate comparison.
There is no replay-only control for this continuation, and initialization, exposure and schedule differ between checkpoints. These measurements do not isolate a causal effect of the new datasets or quantify training-seed variance. Earlier internal diagnostics are separate historical records, not a criterion for naming or publishing this model.
Auxiliary class-balanced diagnostic
Balanced accuracy is the unweighted mean of recall across gold-answer classes. It was collected after the primary measurements. The values below are descriptive; their source does not provide confidence intervals. They retain a separate method and source digest in EVALUATION.json.
| Update | BoolQ balanced accuracy (%) | MMLU-PL balanced accuracy (%) | Belebele-PL balanced accuracy (%) |
|---|---|---|---|
| 5,000 | 53.39 | 25.34 | 25.60 |
| 10,000 | 52.42 | 25.09 | 25.40 |
| 15,000 | 53.98 | 24.99 | 24.42 |
| 20,000 | 57.34 | 25.06 | 24.91 |
| 25,000 | 57.44 | 24.77 | 26.25 |
| 30,000 | 59.10 | 25.00 | 25.39 |
| 35,000 | 60.29 | 24.70 | 25.64 |
The corresponding uniform-choice reference is 50% for BoolQ and 25% for the two four-class tasks. These auxiliary values support inspection of class imbalance. They leave the primary metrics and their published intervals unchanged; a checkpoint-difference claim requires a separate paired analysis.
Additional GSM8K and SST-2 measurements โ update 10,000
SlayerLab measured this two-task subset on 11 November 2026 using the pinned BenchTeam code, lm-eval 0.4.9.1, the slayer adapter, float32, batch 1 and context limit 4,096. These are full evaluated splits, not a full 50-task run or a copy of the community's independent report. This separate two-task protocol measures 1.09 update 10,000 only; it does not supply GSM8K or SST-2 results for updates 15,000, 20,000, 25,000, 30,000 or 35,000. Releasing another checkpoint does not supply a missing measurement.
| Task | Metric / prompting | n | 1.07 | 1.08 | 1.09, update 10,000 |
|---|---|---|---|---|---|
| GSM8K | strict exact match, 5-shot | 1,319 | 1.52 [0.91, 2.20] | 0.91 [0.45, 1.44] | 0.99 [0.45, 1.52] |
| SST-2 | accuracy, 0-shot | 872 | 51.03 [47.71, 54.36] | 50.46 [47.13, 53.67] | 58.26 [54.93, 61.35] |
Scores are percentages with reported pointwise 95% bootstrap intervals (10,000 draws, seed 20261028). These intervals describe individual results, not model differences or training-seed variance. This protocol and its intervals are recorded separately under additional_measurements in EVALUATION.json; the preceding nine-task protocol is unchanged.
Descriptive results; training-lineage contamination scan pending. GSM8K strict-match scores are near zero. SST-2 zero-shot accuracy is calibration-sensitive; these scores alone do not establish a general reasoning or sentiment-understanding gain. No paired-difference or internal calibration analysis is published here.
Cost, licensing and limitations
Continued training is budget-limited and may pause with spot availability. The funding receipt above is not the current balance or a guarantee of a final update count. Metered attribution is not a settled invoice; hardware choice must account for effective retained progress, interruptions, storage and evaluation, not only an advertised GPU-hour price. This page does not allocate a new budget to gold-data training or change the existing cost guards.
The model's declared license is Apache-2.0, aligned with the selected 1.08 base. Source-text rights and dataset publication require separate review; a model license does not grant rights to redistribute its training texts. Privacy filters are heuristic; generated text may be inaccurate, biased, offensive or personal. No reasoning quality, instruction-following ability or professional reliability has been demonstrated for the revised 1.09.