--- library_name: coreai license: apache-2.0 base_model: convaiinnovations/laya tags: - coreai - apple-silicon - on-device - modernbert - laya - calibrated-decisions - text-classification language: - multilingual - en - ja pipeline_tag: text-classification --- Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's `coreai-torch` (LLMs: `coreai.llm.export`) into `.aimodel` bundles that run on the GPU or the Neural Engine. This model has no row on [DeviceMark](https://devicemark.github.io/), the on-device LLM leaderboard. # laya multilingual — Core AI export Zoo card, recipe and gate transcript: [coreai-model-zoo/models/laya-multilingual](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/laya-multilingual/README.md). A **typed decision model** that answers in one forward pass: give it a state (a message, an email, a JSON record, a conversation) and a typed question — `choice` (pick one of named options), `score` (an ordered scale) or `noul` (yes/no) — and it returns a probability for every option, no generation. [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya), subfolder `multilingual/` (Apache-2.0, revision `1c5edc17…`, laya 0.3.4), as a static `.aimodel` for macOS 27 and for the iPhone 17 Pro (run there 2026-09-23: the same 201 rows pass, 47 ms per decision on the GPU). It is the catalog's first **encoder-type decision model**: the other decision models here are language models read out after a prefill; this one scores every option at its own marker position in a single encoder call. Architecture (from `model.safetensors` and `encoder/config.json`; 321,908,998 parameters, all stored in F16 but the three-value temperature buffer): an mmBERT-base encoder — ModernBERT layout, 22 layers, hidden 768, 12 heads × 64, GLU MLP 1152 with exact GELU, biasless attention/MLP/LayerNorm (ε 1e-5), vocabulary 256,000 — with global attention on layers 0, 3, …, 21 and a sliding window of inclusive radius 64 elsewhere, RoPE θ 160,000 for both kinds; then a type embedding (choice / score / noul), two pre-norm transformer layers (768, 12 heads, ReLU 3072), a scorer (LayerNorm → Linear → GELU → Linear) at every position, and a small act head. **This is an encoder, not a generator.** One question is one forward over a right-padded window; no KV cache, no sampling loop. It runs through raw `AIModel` calls, not the generate engine. ## Graph contract ``` function "main" input "input_ids" [1, S] int32 right-padded with PAD 0 input "attention_mask" [1, S] int32 1 over real tokens, 0 over padding input "qtype_onehot" [1, 3] fp32 choice / score / noul output "token_logits" [1, S] fp32 the scorer at every position — read at the option markers output "pooled_cls" [1, 768] fp32 input of "act" function "act" fp32 in every variant input "pooled_cls" [1, 768] fp32 input "feats" [1, 4] fp32 top1, top1 − top2, entropy / ln max(K,2), max(K,2) / 255 output "act_logits" [1, 2] fp32 class 0 = answer directly S = 256 or 512 (export-time choice); batch = 1; one bundle holds both functions ``` **Host recipe** — the sequence and the readout are the publisher's, unchanged: - Build `[CLS 2] question: [SEP 1] [MASK 4] option … [MASK 4] option … [SEP 1] [SEP 1]` with laya's `build_sequence` (option text ≤ 48 tokens, head budget 256, state right-truncated to the window; the marker positions are where the `[MASK]` ids sit). Every text piece is tokenized without special tokens by the checkpoint's `tokenizer.json` (Gemma-style BPE, byte fallback). - Gather `token_logits` at the markers (K logits), softmax them **raw** for the four `feats`, run `act`; then divide the K logits by the question's temperature (bucket by type and K first, then per type) and softmax for the answer. `metadata.json` carries the checkpoint's temperatures (T = 1) and the fitted calibration the LiteRT port published (`temperature`, `temperature_by_options`). - `reference.json` in every folder is the parity instrument: 201 question rows for that window with token ids, marker positions, the publisher's answers at T = 1 and the official model's raw logits. ## Measured **Mac** (Apple M4 Max), macOS 27.0 build **26A428**, coreai-build 3600.83.1, the JIT `.aimodel`, 2026-09-23. Every row: the 201 rows of its window through the whole pipeline (main → host features → act), 3 runs per row, then 40 warm questions; the warm time is one whole question, NumPy I/O included. The GPU rows ran alone on the GPU (the machine-wide GPU lock held, no other GPU job visible during the run). `cpu_only` is the parity option, and its milliseconds were taken while other jobs used the machine's CPU: reference values, not a speed claim. | Variant | S | Compute | Argmax (choice + score) | Max \|Δp\| at T=1 | Marker max \|Δ\| | Act relative | Repeat drift | Load | Warm median | |---|---:|---|---:|---:|---:|---:|---:|---:|---:| | wfp16 | 256 | GPU | 81/81 | 4.47e-6 | 5.53e-5 | 2.60e-6 | 0 | 570 ms | **12.1 ms** | | wfp16 | 512 | GPU | 81/81 | 4.59e-6 | 1.00e-4 | 3.47e-6 | 0 | 589 ms | **19.5 ms** | | fp32 | 256 | GPU | 81/81 | 4.47e-6 | 5.53e-5 | 2.60e-6 | 0 | 853 ms | 10.4 ms | | fp32 | 512 | GPU | 81/81 | 4.59e-6 | 1.00e-4 | 3.47e-6 | 0 | 982 ms | 17.2 ms | | wfp16 | 256 | cpu_only | 81/81 | 9.48e-6 | 7.49e-5 | 1.85e-6 | 0 | 720 ms | 45.1 ms | | wfp16 | 512 | cpu_only | 81/81 | 9.48e-6 | 6.68e-5 | 1.94e-6 | 0 | 600 ms | 86.2 ms | | fp32 | 256 | cpu_only | 81/81 | 9.48e-6 | 7.49e-5 | 1.85e-6 | 0 | 762 ms | 63.0 ms | | fp32 | 512 | cpu_only | 81/81 | 9.48e-6 | 6.68e-5 | 1.94e-6 | 0 | 720 ms | 84.2 ms | On the CPU, wfp16 and fp32 return bit-identical marker and act logits on all 402 rows: wfp16 stores the weights in fp16 — exact, the checkpoint is F16 — and computes in fp32. On this Mac's GPU both stay within 1e-4 of the CPU run as well. **Use the GPU, and request it explicitly**: with the Neural Engine preference the Mac returned the GPU's results bit for bit for fp32, but for wfp16 it returned different results that changed from run to run (next section). **iPhone 17 Pro** (iOS 27.0 24A437, the `ios/wfp16-s256` JIT bundle, 2026-09-23, a headless harness over CoreAIKit's `EncoderDecider.decideRow` on the 201 fixture rows, the fixture's own token ids, T = 1): on the GPU **201/201 rows within 1e-3, argmax 81/81, max |Δp| 8.1e-6**, act probability delta 0, no drift over three passes; CPU-only the same verdict at 9.4e-6. The kit's builder renders every row identically on the phone (201/201 tokens and markers). One decision on the phone's GPU: **53 ms median** (p90 57 ms, 30 warm-up calls, 603 timed calls) with the row's tokens given, 69–70 ms through `TypedDecisions` with the state's tokens kept (the question tokenized on the phone, 5 ms median), 73 ms CPU-only; the bundle loads in 1.2–2.5 s, the process peaks at 357 MB (670 MB CPU-only). With a Neural Engine preference the phone misses the bar the way the Mac does — 196/201 within 1e-3, argmax 80/81, max |Δp| 0.33 on five rows — at 82 ms and 1.4 GB, so the kit keeps refusing that preference for this bundle. Thermal state during those runs: "fair" for the GPU rows, "serious" for the CPU-only and Neural Engine rows; the GPU gate run again at "nominal" (2026-09-24) gave the same verdict at 54.5 ms median (p90 59.6), so the row cost is not thermal-bound. Through the kit's own `decide-cli bench --repeat 3` on the cool phone (a 109-token state, eight questions, the measure of the Mac's 11.5 ms): **47.1 ms per decision** with the state shared, 46.4 from scratch, 390 / 458 ms for the state and its eight. [`gate-laya-multilingual-iphone-gpu.json`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/laya-multilingual/gate-laya-multilingual-iphone-gpu.json) in the zoo repository is the first GPU run's record. The AOT `ios-h18p/wfp16-s256` bundle, run the same way on the cool phone (2026-09-24): the same verdict (201/201, argmax 81/81, max |Δp| 8.1e-6) at **51.0 ms** median (p90 54.4), first call 68.6 ms after a 1.7 s load, peak 357 MB — no faster to load or to run than the JIT bundle the catalog ships. The publisher's own model on the SemIf `authored144` fixture (144 three-option evidence / rule / candidate questions it was not trained for), as `laya.load(...).predict` answers them: mean family balanced accuracy **0.6114**, accuracy 0.5903 (the same at T = 1 and with the fitted calibration, and at both windows). That is the reference a port's decisions are compared with, not a claim about the port. ### JevBench public 231 (Mac, 2026-09-24) | easy 48 | standard 72 | hard 111 | ECE hard | p50 | p95 | hard max | |---:|---:|---:|---:|---:|---:|---:| | 0.896 | 0.403 | 0.342 | 0.273 | 0.02 s | 0.20 s | 0.3 s | The benchmark's own harness ([fstandhartinger/jevbench](https://github.com/fstandhartinger/jevbench) `2fa63fa`, v1.4.0, `typesafe` adapter) ran the 231 public items against coreai-kit `adbc755` `decide-cli serve` with the `macos/wfp16-s256` bundle, one question per request. The bundle's input window is 256 tokens: 95 of the 111 hard items were truncated to it (hard states are up to 3,677 tokens long), and no easy or standard item was (the longest is 107 tokens). Accuracy per tier and the hard tier's ECE are JevBench's own scoring (argmax of the returned probabilities); p50 and p95 are per-request latency over all 231 requests, hard max the maximum over the hard tier. Latency was measured without an exclusive GPU window (contended), with this model's server running alone. JevBench's published scores (Intelligence and the rest) are chance-corrected over 534 items, sealed ones included, and are not comparable to these accuracies. **iPhone 17 Pro (2026-09-24)** | | easy 48 | standard 72 | hard 111 | |---|---:|---:|---:| | accuracy | 0.896 | 0.403 | 0.342 | | p50 | 0.05 s | 0.05 s | 0.07 s | | p95 | 0.05 s | 0.05 s | 0.26 s | | p50 / p95 over | 48 rows, nominal | 72 rows, nominal | 111 rows, nominal | The phone (iOS 27.0 24A437) received the same request bodies as the Mac run, one question per request. A headless harness app answered each with coreai-kit 0.7.1, through the call the kit's System One server makes. It ran the `ios/wfp16-s256` bundle; the Mac run used `macos/wfp16-s256`. Every bundle file on the phone matched the Hub revision by hash. Every answer's argmax equals the Mac run's. p50 and p95 are the kit's time per request. A nominal row started and ended with the phone on its battery at thermal state nominal. ## Numerics gate The oracle is the publisher's package itself (`laya.load(, subfolder="multilingual")`, transformers 5.17.0, CPU fp32): it reproduces the frozen fixture of the LiteRT port exactly (token ids 402/402; batched answers and logits bit-identical), and each row alone, right-padded to its window, is the tensor reference. Bars, at every stage: argmax identical on every choice and score row, max |Δp| ≤ 1e-3 at T = 1 over the options, |Δ act probability| ≤ 1e-3; marker logits ≤ 1e-3 and act logits ≤ 1e-4 relative to the official batch-1 run; repeat drift ≤ 1e-6 on CPU; a wrong-pairing control (every row judged against another same-shape row's outputs) must fail — it does, on 162–166 of 200 pairs. - **Authoring** (`gate_laya_authoring.py`): the 201 rows of each window, plus every hidden state of 14 rows (embeddings, 22 layers, final norm, after the type embedding, both head layers) against the official model — absolute max |err| ≤ 1e-4 on the embeddings and layers 0–9, relative max |err| / max |ref| ≤ 2e-4 on layer 10 through the head (measured: 5.0e-5 absolute, 5.7e-5 relative at S=256, 7.8e-6 at S=512). Five mutations must be caught and are, by the layer, tensor and answer gates: a local radius of 63 (layer 1, 1.24 / 0.84 at S=256 / 512), all-global, all-local, ignoring padding, and dropping the type embedding (after the type embedding, 6.6e-2 relative). A pad-isolation check replaces every pad id with a random token: real positions stay bit-identical. - **Export**: the torch-exported, decomposed `main` and `act` pass the same 201-row gate before conversion. - **Runtime**: the tables above. [`gate-laya-multilingual.json`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/laya-multilingual/gate-laya-multilingual.json) in the zoo repository is the transcript: every stage's summary with the sha256 of the full record it came from; the records keep each row's raw marker and act logits, unrounded. ## Neural Engine and fp16 The checkpoint is F16, so fp16 **storage** is exact. fp16 **compute** is not good enough here. The recipe that runs everything in fp16 except the RoPE application and the attention softmax misses the answer bar everywhere it was measured — torch max |Δp| 3.4e-3 / 5.0e-3 (S=256 / 512); Mac CPU 4.38e-2 with one choice flipped (80/81); Mac GPU 5.84e-3; Mac Neural Engine 1.85e-2 with one choice flipped (both windows) — and keeping the residual stream and LayerNorm in fp32 as well still leaves 4.9e-3 in torch (S=256). Placement follows the compute precision here. Compiled for the iPhone 17 Pro with `--preferred-compute neural-engine`, the fp16 recipe gets 47 Neural Engine regions; wfp16, which computes in fp32, gets a single region of 5.7 KB of IR and everything else stays in the GPU package. On the Mac, the Neural Engine preference gave these results: | Variant | S | Argmax | Max \|Δp\| | Marker max \|Δ\| | Repeat drift | Load | Warm median | Reading | |---|---:|---:|---:|---:|---:|---:|---:|---| | fp32 | 256 / 512 | 81/81 | 4.47e-6 / 4.59e-6 | 5.53e-5 / 1.00e-4 | 0 | 986 / 1,026 ms | 10.5 / 20.4 ms | identical to the GPU run on every row: it ran on the GPU | | wfp16 | 256 | 81/81, 81/81 | 0.290, 0.164 | 0.82, 1.66 | **90, 90** | 1,186 ms | 12.7 ms | two runs, different answers each time: a defect of this specialization, do not use | | wfp16 | 512 | 81/81, **80/81** | 0.343, 0.238 | 7.76, 11.4 | **202, 212** | 1,188 ms | 20.1 ms | the same, and the second run flipped a choice | | fp16 recipe | 256 | 80/81 | 1.85e-2 | 0.211 | 0 | 5,919 ms | 42.9 ms | deterministic, but outside the bar | | fp16 recipe | 512 | 80/81 | 1.85e-2 | 0.681 | 0 | 5 ms (cached) | 67.7 ms | the same | `coreai.runtime` reports no placement, so the reading column is inferred from the numbers. The iPhone folder is therefore the GPU compile, named for what it is (`ios-h18p/`); the neural-engine compile's region count is recorded in its manifest. ## ⬇️ Bundle This repository holds one folder per variant, each self-contained: the bundle, `tokenizer/` (the checkpoint's files, unmodified), `metadata.json` (the decision contract), `reference.json` and `provenance/` (export manifest with per-file sha256, the export and runtime gate records). | Folder | Platform | Format | Bundle | Bytes | |---|---|---|---|---:| | `macos/wfp16-s256/` | macOS 27 | JIT `.aimodel` | `laya_ml_wfp16_s256.aimodel` | 644,855,189 | | `macos/wfp16-s512/` | macOS 27 | JIT `.aimodel` | `laya_ml_wfp16_s512.aimodel` | 645,772,737 | | `macos/fp32-s256/` | macOS 27 (reference) | JIT `.aimodel` | `laya_ml_fp32_s256.aimodel` | 1,288,156,432 | | `macos/fp32-s512/` | macOS 27 (reference) | JIT `.aimodel` | `laya_ml_fp32_s512.aimodel` | 1,289,073,976 | | `ios/wfp16-s256/` | iOS 27 (portable JIT) | JIT `.aimodel` | `laya_ml_wfp16_s256.aimodel` | 644,855,189 | | `ios/wfp16-s512/` | iOS 27 (portable JIT) | JIT `.aimodel` | `laya_ml_wfp16_s512.aimodel` | 645,772,737 | | `ios-h18p/wfp16-s256/` | iOS 27, **h18p only** | AOT `.aimodelc` | `laya_ml_wfp16_s256.h18p.aimodelc` | 645,117,160 | | `ios-h18p/wfp16-s512/` | iOS 27, **h18p only** | AOT `.aimodelc` | `laya_ml_wfp16_s512.h18p.aimodelc` | 646,034,970 | The `ios-h18p/` bundles are compiled for one device architecture (`h18p`, the iPhone 17 Pro) with `xcrun coreai-build compile --platform iOS --min-deployment-version 27.0 --preferred-compute gpu --architecture h18p` (coreai-build 3600.83.1). **Never load an iOS bundle on a Mac.** Convert yourself: [`conversion/laya/`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/laya/README.md) — staged scripts, and [`recipe.toml`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/laya-multilingual/recipe.toml) names the commands. ## CoreAIKit (Swift) Catalog id `laya-multilingual` (active once the kit's encoder backend is merged). `TypedDecisions` loads the bundle as an encoder backend — the same `decide` / `prefill` calls as the kit's language-model decision models — and runs it on the GPU; a Neural Engine preference is refused at load. Measured through the kit on the same Mac (M4 Max, macOS 27.0 26A428, the wfp16 bundles above, GPU lock held, 2026-09-23): - Parity on the 201 rows of each window: token ids and marker positions 201/201, argmax 81/81, max |Δp| 5e-6 on the GPU and 9e-6 with `cpuOnly`. - `decide-cli bench` (a 109-token state, 8 questions, warm): **11.5 ms** per decision at S=256 when the state is tokenized once and shared, 12.1 ms when every decision tokenizes its row; 18.9 / 19.4 ms at S=512. - Load 1.06–1.16 s, of which the tokenizer (the 256,000-entry `tokenizer.json`) takes about 1.0 s and 145–162 MB; the whole process is 474–498 MB after the first decision. - SemIf authored144 at the kit's default temperature (the fitted calibration): argmax 144/144 with the publisher's model at the same temperature, max |Δp| 2.9e-6; mean family balanced accuracy 0.6114, accuracy 0.5903. - With a Neural Engine preference (measured before it was refused): 168–183 of 201 rows within the 1e-3 bar over four runs, max |Δp| 0.61. [`measurements-coreai-kit.json`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/laya-multilingual/measurements-coreai-kit.json) holds these numbers and the records they came from. ## The port in one lesson: a massive activation moves the layer bar From layer 11 the encoder parks a value of about 14,000 on a few dimensions of the first token. One fp32 step there is about 0.001, so the publisher's own two attention paths (SDPA, which the package uses, and eager) already disagree by up to 0.048 on those layers while their marker logits agree to 3.5e-5. The publisher's own eager path does not meet a per-layer bar of 1e-4 above layer 10, so it cannot be the bar for a reimplementation there; the gate keeps 1e-4 where it still separates a correct graph from a wrong one (the embeddings and layers 0–9, where a window of 63 instead of 64 shows up at 0.8–1.2) and switches to a relative bar of twice the official spread above. In the fp16 recipe the largest relative error sits at the final norm (0.13 at S=256 in torch), the state that normalizes those large values. ## License and limits Apache-2.0 at the pinned upstream revision. Not tested: the 512 window on the phone (the 256 window was, JIT and AOT h18p, on one iPhone 17 Pro), other Macs or OS builds, dynamic or batched shapes, windows other than 256 and 512, more than 20 options, the fitted calibration's quality at S = 512, languages beyond the fixture's English, Japanese and mixed rows, sustained thermals. The act probability is saturated at 1.0 on every fixture row, in the publisher's model as here: it is carried through the graph, not evidence of when to escalate.