--- license: apache-2.0 library_name: coreml pipeline_tag: text-classification base_model: convaiinnovations/laya base_model_relation: quantized tags: - coreml - laya - apple-silicon - neural-engine - decision-model - fluidaudio --- # laya-coreml Core ML conversion of **laya-multilingual** (Convai Innovations, Apache-2.0): a 322M-parameter mmBERT-base encoder with a typed decision head that answers `choice`, `score`, and `noul` questions about a text state in one forward pass, returning calibrated probabilities and no generated tokens. Weights are unchanged from [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) `multilingual/` at revision `1c5edc17a7acd8701df6fc341c0d179f1c62c982`. Runs through [FluidUse](https://github.com/FluidInference/FluidUse) (`LayaManager`) on macOS 14+. ```swift let laya = try await LayaManager.load() // downloads the 128 + 512 buckets and tokenizer.json let answer = try await laya.answer( state: "The T piece dropped at column 3 leaves one hole under it.", question: .noul("Is this a clean placement?")) print(answer.noul!) // P(true) ``` ```bash swift run -c release FluidUseLaya answer --state "…" --type choice \ --instructions "What does the customer want?" --options "refund|order status|technical help" swift run -c release FluidUseLaya tetris # headless Tetris played by laya decisions swift run -c release LayaTetrisDemo # SwiftUI demo ``` ## Files | File | Tokens | Notes | | --- | ---: | --- | | `laya_multilingual_fp16_L128_options32.mlmodelc` | 128 | Short prompts; runs on CPU + Neural Engine | | `laya_multilingual_fp16_L256_options32.mlmodelc` | 256 | | | `laya_multilingual_fp16_L512_options32.mlmodelc` | 512 | Long states; GPU is faster than ANE here | | `laya_multilingual_fp16_L1024_options32.mlmodelc` | 1024 | Upstream `max_len`; GPU | | `laya_multilingual_e8_L{128,256,512,1024}_options32.mlmodelc` | | Same buckets with an int8 embedding table: 448–453 MB each, accuracy within 0.5 points of fp16 on the full benchmark | | `tokenizer.json` | | mmBERT / Gemma vocabulary (256k), byte fallback | Each fp16 bucket is a complete model (614 MB, 393 MB of which is the embedding table) with 32 option slots; the `e8` buckets store that table as int8 per-channel. Encoder-weight int8 and 6-/4-bit palettes fail the parity gates (the ANE in particular), so they are not published. `FluidUse` picks the smallest loaded bucket that fits a prompt and truncates the state on the right for the largest one, exactly like laya's `max_len`. Inputs: `input_ids` int32 `[1, L]`, `attention_mask` int32 `[1, L]`, `marker_map` float32 `[1, 32, L]` (one-hot `[MASK]` position per option), `question_type` float32 `[1, 3]`. Outputs: `logits` `[1, 32]`, `probabilities` `[1, 32]`, `action_probabilities` `[1, 2]`. Sequence format: `[CLS] question: [SEP] ([MASK]