README.md

---
license: apache-2.0
base_model:
  - fastino/GLiNER2.5-Decide
  - microsoft/deberta-v3-large
pipeline_tag: text-classification
language:
  - en
tags:
  - core-ai
  - coreai
  - coreaikit
  - apple
  - on-device
  - zero-shot-classification
  - text-classification
  - gliner
  - deberta
  - typed-decisions
library_name: coreai
base_model_relation: quantized
---

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's `coreai-torch` (LLMs: `coreai.llm.export`) into `.aimodel` bundles that run on the GPU or the Neural Engine.

<!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) -->
This model has no row on [DeviceMark](https://devicemark.github.io/), the on-device LLM leaderboard.
<!-- gen-cards:devicemark end -->

# GLiNER2.5-Decide — Core AI

Sort a text into labels you name at call time, on an iPhone or a Mac, with nothing sent anywhere:
an intent, a queue, a severity, a yes/no, or several of these about one text in a single forward.
There is no training for your label set and no generated text; every label gets a probability.

[`fastino/GLiNER2.5-Decide`](https://huggingface.co/fastino/GLiNER2.5-Decide) (Apache-2.0) is a
DeBERTa-v3-large encoder with a trained label head, 486M parameters with its 128k-token embedding
table (340M on Fastino's card). Its classification path is one static Core AI graph in fp16; the
tokenizer, the schema layout and the softmax / sigmoid run in the host. One call takes 37.8 ms on an
iPhone 18 Pro (256-token graph compiled ahead of time for it, thermal state nominal), and its decisions
equal the fp32 `gliner2` library's on every one of 787 decisions over 454 texts.

## Use it

With [CoreAIKit](https://github.com/john-rocky/coreai-kit)'s `TextClassifier` (branch `gliner-decide`
until the next release), pointed at the `macos/` or `ios/` directory of this repo:

```swift
import CoreAIKitEmbeddings

let classifier = try await TextClassifier(bundleAt: bundleURL)

let answers = try await classifier.classify(
    "Battery dies before lunch, but the keyboard and the screen are the best I have used on a laptop.",
    tasks: [
        ClassificationTask("sentiment", labels: ["positive", "negative", "mixed", "neutral"]),
        ClassificationTask("aspects", labels: ["battery", "keyboard", "screen", "camera", "price", "support"],
                           multiLabel: true, threshold: 0.4),
    ])
answers["sentiment"]?.labels        // ["mixed"]
answers["aspects"]?.labels          // ["battery", "keyboard", "screen"]
answers["aspects"]?.probabilities   // every label with its probability
```

One question has a shorter form: `try await classifier.classify(text, labels: ["spam", "ham"])`
returns the best label and its probability. A task may carry a `prompt` (a question to answer about the
text) and `descriptions` (what each label means), exactly as `gliner2`'s `classify_text` takes them.

The same class runs from the command line:
[Examples/TextClassify ↗](https://github.com/john-rocky/coreai-kit/tree/gliner-decide/Examples/TextClassify)
(`swift run textclassify-cli --bundle <dir> --text "…" --task intent=order_status,refund_request`).

## How it works

`gliner2` lays the tasks out ahead of the text and reads one hidden state per label:

```
( [P] intent ( [L] order_status [L] refund_request … ) ) [SEP_STRUCT] ( [P] urgency ( [L] low … ) ) [SEP_TEXT] the text, lowercased, word by word .
```

The graph takes that token stream and the positions of the `[L]` markers, and returns one logit per
label:

```
input_ids [1, S] int32   attention_mask [1, S] int32   label_idx [1, 32] int32   ->   logits [1, 32] float32
```

Inside: the DeBERTa-v3-large encoder (24 layers, hidden 1024, disentangled attention with the
relative-position bucket table computed at export), a gather of the `[L]` rows, and the 1024→2048→1
head. Two graphs are shipped, S = 256 and S = 512; the host runs the smallest one the tokens fit. The
host appends a `.` when the text ends without one, splits words with `gliner2`'s regex, lowercases the
words but not the labels, tokenizes piece by piece with the DeBERTa-v3 SentencePiece model, and turns
the logits into a decision the way `gliner2` does: softmax and argmax for a single-label task, a sigmoid
per label and a threshold (default 0.5) for a multi-label task, the best label alone when none passes.
Up to 32 labels per call, over all tasks. Weights are fp16, including the relative-position embeddings;
nothing is quantized.

## Verification

The reference is `gliner2` 2.0.0 (`AutoExtractor.classify_text`, fp32, CPU) on the source checkpoint at
revision `7ee5da4`. The fixture is the 21 examples of Fastino's model card plus rows of
[`fastino/fast-decisions`](https://huggingface.co/datasets/fastino/fast-decisions) (the published
development split, Apache-2.0): the first 20 rows of each of the 17 domains that fit 256 tokens, and up
to 10 more per domain that need 257–512 tokens. 454 texts, 787 decisions. The pass bar is decisions
equal to the reference on every task; the logit and probability differences are reported, not bounded.
The comparator was shown to fail: shifting every `label_idx` by one changes 229 of the 361 short texts'
decisions.

| stage | texts / decisions | decisions equal | max \|Δlogit\| | max \|Δprob\| |
|---|---|---|---|---|
| re-authored graph, fp32, torch CPU | 454 / 787 | 787 | 1.2e-5 | 2.5e-6 |
| fp16 graph S = 256, Mac GPU (M4 Max, macOS 27.0) | 361 / 606 | 606 | 0.014 | 0.0022 |
| fp16 graph S = 512, Mac GPU | 454 / 787 | 787 | 0.018 | 0.0025 |
| Swift host (CoreAIKit `TextClassifier`), Mac GPU | 454 / 787 | 787 | 0.014 | — |
| fp16 S = 256, iPhone 18 Pro GPU, AOT (h19p) | 361 / 606 | 606 | 0.019 | 0.0023 |
| fp16 S = 512, iPhone 18 Pro GPU, AOT (h19p) | 454 / 787 | 787 | 0.019 | 0.0023 |

The Swift host's token ids equal `gliner2`'s on all 454 texts, and its Mac GPU logits equal the Python
engine run's bit for bit (5,749 of 5,749 values). The iPhone's logits are within 0.016 of the Mac
GPU's, with every decision the same. iPhone rows: iOS 27.0 (build 24A437), measured 2026-09-26 with the
zoo's gate app. On the same phone, the JIT bundles now in `ios/` matched the reference on every decision
(606 of 606 at S = 256, 787 of 787 at S = 512), with logits within 0.016 of it
([knowledge/gliner25-decide.md](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/gliner25-decide.md) §7).

On the fixture's 340 short rows, one call per row with all of its heads and the default threshold, 63.5
% of the decisions equal the dataset's gold label. That is this port's number on the development
split under this protocol. Fastino's 60.2 % is on the held-out test split with its own protocol, and the
two are not comparable.

## Speed

One call, warm, median over 100 calls after 5 warm-up calls on fixture inputs:

| device | S = 256 | S = 512 | load, second time | footprint after load |
|---|---|---|---|---|
| iPhone 18 Pro GPU, AOT (h19p), thermal state nominal, phone rested 7 min | 37.8 ms (p90 38.1) | 92.9 ms (p90 94.5) | 0.14 s / 0.84 s | 220 MB / 376 MB |
| M4 Max GPU, another job on the GPU | 28 ms | 52 ms | 0.01 s | — |

The first load of the AOT bundles after installing on the iPhone took 1.3 s (S = 256) and 1.8 s
(S = 512), with a first call of 1.2 s and 0.4 s. The JIT bundles now in `ios/`, on the same phone
(rested, thermal state nominal): first load after installing 1.48 s and 2.32 s, first call 1.39 s and
0.49 s, load after a relaunch 0.36 s and 0.13 s, one call 35.9 ms and 87.2 ms (median;
[knowledge/gliner25-decide.md](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/gliner25-decide.md) §7). Several hundred calls without a pause slow the iPhone: across one loop of the
S = 512 fixture a call went from 92 to 151 ms, and a run started on a warm phone measured 84 ms at
S = 256 against 38 ms rested. Five minutes of rest brings the speed back.

## Files

| path | what | size |
|---|---|---|
| `macos/gliner25-decide_float16_s256_m32.aimodel` | JIT bundle, S = 256 | 873 MB |
| `macos/gliner25-decide_float16_s512_m32.aimodel` | JIT bundle, S = 512 | 875 MB |
| `ios/gliner25-decide_float16_s256_m32.aimodel`, `ios/gliner25-decide_float16_s512_m32.aimodel` | the same two JIT bundles, byte for byte; an iPhone specializes them on its first load | 873 MB, 875 MB |
| `ios-h19p/gliner25-decide_float16_s256_m32.h19p.aimodelc` | compiled ahead of time for the iPhone 18 Pro GPU (h19p); moved from `ios/` in revision `820d4e90` (2026-09-26) | 974 MB |
| `ios-h19p/gliner25-decide_float16_s512_m32.h19p.aimodelc` | same, S = 512 | 976 MB |
| `macos/tokenizer/`, `ios/tokenizer/`, `ios-h19p/tokenizer/` | the DeBERTa-v3 SentencePiece tokenizer, declared as `XLMRobertaTokenizer` so swift-transformers loads it | 8.3 MB |
| `macos/classifier.json`, `ios/classifier.json`, `ios-h19p/classifier.json` | the graph contract, the two shapes, the marker token ids, the host rules; `ios-h19p/`'s names the `.h19p.aimodelc` bundles | |
| `macos/reference_s*.json`, `ios/reference_s*.json`, `ios-h19p/reference_s*.json` | one fixture text with its graph inputs and fp32 logits, for a host to check itself against | |
| `gate/` | the fixture: the 21 card examples and the fast-decisions rows, each with its token ids, marker positions, fp32 logits and decision | 5 MB |
| `LICENSE`, `NOTICE`, `source/` | Apache-2.0, the origin and what was converted, the source `config.json` files | |
| `config.json` | marks the repo as Core AI `.aimodel` bundles for the zoo's tooling | |
| `SHA256SUMS` | every file's checksum; `conversion/gliner25_decide/stage_ship.py --check <dir>` in the zoo verifies a download | |

The JIT bundles in `ios/` have been run on the iPhone 18 Pro only (numbers above). The `ios-h19p/`
bundles load only on the h19p architecture (iPhone 18 Pro): the runtime refuses a compiled bundle on
another architecture ([knowledge/jit-distribution.md](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/jit-distribution.md)). An h18p bundle for the iPhone
17 Pro compiles from the same `.aimodel` with the zoo recipe, but has not been run on that device and is
not shipped.

## Limits

- 512 tokens for the tasks and the text together; a label costs about 4 tokens. Longer text is cut at a
  word boundary from the end and the result says so. Of the fast-decisions rows, 88 % fit 256 tokens
  and all fit 512.
- 32 labels per call. English only; Fastino's multilingual model is `GLiNER2.5-multi-Decide`.
- The tasks are part of the input, so asking a second question moves the first one's probabilities
  a little. Classify with the task set you will use.
- One text per call. Call the classifier sequentially on one instance.

Other conversions of this model's classification path:
[onnx-community/GLiNER2.5-Decide-ONNX](https://huggingface.co/onnx-community/GLiNER2.5-Decide-ONNX) and
[nishparadox/gliner2.5-decide-onnx](https://huggingface.co/nishparadox/gliner2.5-decide-onnx). This
port's fixture agrees with the former's published reference on all 13 of its decisions.

## License

Apache-2.0, the source model's license (`LICENSE`). `NOTICE` records the origin revision, the weight
file's checksum and what was converted; keep both with any part of this repo you redistribute.
DeBERTa-v3-large is MIT.

- Source: [fastino/GLiNER2.5-Decide](https://huggingface.co/fastino/GLiNER2.5-Decide) (Apache-2.0), on [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) (MIT).
- Export, gates, the iPhone gate app and the porting notes: [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo), `models/gliner25-decide`.

<!-- funnel:v1 -->

---

**More models in this format:** [Core AI Model Zoo](https://huggingface.co/collections/mlboydaisuke/core-ai-model-zoo-6a7ff330f753e8dcae04671a) — each with the recipe that produced it.

**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->