| --- |
| license: apache-2.0 |
| library_name: coreml |
| pipeline_tag: text-classification |
| base_model: convaiinnovations/laya |
| base_model_relation: quantized |
| tags: |
| - coreml |
| - laya |
| - apple-silicon |
| - neural-engine |
| - decision-model |
| - fluidaudio |
| --- |
| |
| # laya-coreml |
|
|
| Core ML conversion of **laya-multilingual** (Convai Innovations, Apache-2.0): a 322M-parameter |
| mmBERT-base encoder with a typed decision head that answers `choice`, `score`, and `noul` |
| questions about a text state in one forward pass, returning calibrated probabilities and no |
| generated tokens. Weights are unchanged from |
| [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) `multilingual/` at |
| revision `1c5edc17a7acd8701df6fc341c0d179f1c62c982`. |
|
|
| Runs through [FluidUse](https://github.com/FluidInference/FluidUse) (`LayaManager`) on macOS 14+. |
|
|
| ```swift |
| let laya = try await LayaManager.load() // downloads the 128 + 512 buckets and tokenizer.json |
| let answer = try await laya.answer( |
| state: "The T piece dropped at column 3 leaves one hole under it.", |
| question: .noul("Is this a clean placement?")) |
| print(answer.noul!) // P(true) |
| ``` |
|
|
| ```bash |
| swift run -c release FluidUseLaya answer --state "…" --type choice \ |
| --instructions "What does the customer want?" --options "refund|order status|technical help" |
| swift run -c release FluidUseLaya tetris # headless Tetris played by laya decisions |
| swift run -c release LayaTetrisDemo # SwiftUI demo |
| ``` |
|
|
| ## Files |
|
|
| | File | Tokens | Notes | |
| | --- | ---: | --- | |
| | `laya_multilingual_fp16_L128_options32.mlmodelc` | 128 | Short prompts; runs on CPU + Neural Engine | |
| | `laya_multilingual_fp16_L256_options32.mlmodelc` | 256 | | |
| | `laya_multilingual_fp16_L512_options32.mlmodelc` | 512 | Long states; GPU is faster than ANE here | |
| | `laya_multilingual_fp16_L1024_options32.mlmodelc` | 1024 | Upstream `max_len`; GPU | |
| | `laya_multilingual_e8_L{128,256,512,1024}_options32.mlmodelc` | | Same buckets with an int8 embedding table: 448–453 MB each, accuracy within 0.5 points of fp16 on the full benchmark | |
| | `tokenizer.json` | | mmBERT / Gemma vocabulary (256k), byte fallback | |
|
|
| Each fp16 bucket is a complete model (614 MB, 393 MB of which is the embedding table) with |
| 32 option slots; the `e8` buckets store that table as int8 per-channel. Encoder-weight int8 and |
| 6-/4-bit palettes fail the parity gates (the ANE in particular), so they are not published. `FluidUse` picks the smallest loaded bucket that fits a prompt and truncates |
| the state on the right for the largest one, exactly like laya's `max_len`. |
|
|
| Inputs: `input_ids` int32 `[1, L]`, `attention_mask` int32 `[1, L]`, `marker_map` float32 |
| `[1, 32, L]` (one-hot `[MASK]` position per option), `question_type` float32 `[1, 3]`. |
| Outputs: `logits` `[1, 32]`, `probabilities` `[1, 32]`, `action_probabilities` `[1, 2]`. |
| Sequence format: `[CLS] <type> question: <instructions> [SEP] ([MASK] <option>)* [SEP] <state> [SEP]`. |
|
|
| ## Parity and latency |
|
|
| Apple M5 Pro, macOS 27.0, 16 fixture questions vs. the unmodified PyTorch FP32 runtime: |
| 16/16 argmax agreement on every bucket and compute-unit setting, max probability error 0.0021 |
| (`ALL`) / 0.0126 (`CPU_AND_NE`). Per-question latency, warm: |
|
|
| | Bucket | CPU + ANE | All units | |
| | --- | ---: | ---: | |
| | L128 | **3.6 ms** | 3.9 ms | |
| | L256 | 9.9 ms | **5.2 ms** | |
| | L512 | 27.5 ms | **9.0 ms** | |
| | L1024 | 80.1 ms | **17.9 ms** | |
|
|
| On laya's published application suites (3,899 questions, seed 13, rebuilt from upstream's scripts), |
| the Core ML buckets answered from Swift match the PyTorch reference's accuracy on every suite at |
| 5.2 ms median per question (p95 18 ms): |
|
|
| | Suite | Upstream (T4, PyTorch) | Core ML (M5 Pro) | |
| | --- | ---: | ---: | |
| | jev.ag_news | 0.930 | **0.935** | |
| | jev.emotion | 0.530 | **0.537** | |
| | massive_intent.en | 0.657 | **0.657** | |
| | app.support_triage | 0.522 | **0.542** | |
| | app.email_spam | 0.993 | **0.993** | |
| | app.phishing | 0.993 | **0.993** | |
| | app.guardrails_jailbreak | 0.755 | **0.808** | |
| | app.moderation_toxicity | 0.525 | **0.535** | |
| | app.rag_relevance | 0.657 | **0.672** | |
| | app.model_routing_domain | 0.123 | **0.441** | |
| |
| Conversion pipeline, verification reports, and Swift parity fixtures: |
| [mobius `models/computer-use/laya/coreml`](https://github.com/FluidInference/mobius). |
| |
| ## License |
| |
| Apache-2.0, following the upstream weights and code by Convai Innovations |
| ([NandhaKishorM/laya](https://github.com/NandhaKishorM/laya)). Independent conversion; not an |
| official Convai Innovations release. |
| |
