Add validated W8 extraction packages and ANE profiling for multi

README.md

@@ -17,7 +17,7 @@ This repository contains fixed-shape Core ML exports of the trained classificati
 
 The FP32 extraction stage packages support entities, relations, entity attributes, enum choices, natural/latent/anchorless records, and schemas mixed with classification. In a small fixed manifest of real text and schema fixtures, FP32 matched the native structured output on **15/15** cases. The largest FP32 confidence difference was 0.00000489. The multilingual manifest includes Spanish, French, Chinese, and German text. These checks are selected parity fixtures, not a Decision Index score or a full dataset evaluation.
 
-FP16 result: 14/15 structures matched; the latent-record fixture differed (10 native records versus 9 Core ML records), so FP16 extraction is not published. The multilingual extraction FP16 packages remain local experiments because one latent-record fixture differed. They are not in this repository.
+Standalone FP16 result: 14/15 structures matched; the latent-record fixture differed (10 native records versus 9 Core ML records). The optional adaptive W8 runtime below uses FP32 for any schema containing a latent record and FP16 for the other schemas.
 
 
 ```bash
@@ -36,13 +36,35 @@ The extraction bucket holds up to 128 combined subword tokens, 64 text words, 8
 
 On an M5 Pro with macOS 27.0, FP32 end-to-end median 10.56 ms with All, 10.02 ms with CPU+GPU, 36.08 ms with CPU+Neural Engine for a selected three-label entity request after 20 warmups and over 200 Python calls. Those are local end-to-end measurements for this shape, not ANE-only latency or a device-wide benchmark.
 
-Under CPU+Neural Engine, the FP32 feature graph compute plan assigned all operations to CPU on this machine; CPU+GPU is the faster measured setting. A local FP32-source LUT8 attempt stalled in k-means with numerical warnings; no compressed extraction package is included.
+Under CPU+Neural Engine, the FP32 feature graph compute plan assigned all operations to CPU on this machine; CPU+GPU is the faster measured setting for the uncompressed FP32 path. A local FP32-source LUT8 attempt stalled in k-means with numerical warnings.
+
+### Extraction with embedding W8
+
+The optional adaptive runtime uses two feature packages with per-channel int8 **weight-only** compression of the trained word embedding. The rest of each encoder and the corresponding five extraction heads retain their exported precision. It loads one precision at a time: FP16 with CPU+Neural Engine for schemas without latent records, then FP32 with All for schemas containing a latent record. Switching precision incurs a cold model load. The standalone FP16 path still differs on the latent-record fixture; use `CoreMLAdaptiveBoundaryExtractor` for complete selected parity.
+
+| Feature package | Original bytes | W8 bytes | Selected native structure parity | Largest confidence difference |
+| --- | ---: | ---: | ---: | ---: |
+| `gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage` | 578,545,680 | 387,210,787 | 14/15 standalone | see adaptive result |
+| `gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage` | 1,155,993,314 | 580,986,690 | 15/15 standalone | 0.02455 |
+| Adaptive FP16 plus FP32 | — | — | **15/15** | **0.01639** |
+
+The W8 FP16 feature graph had 620/652 executable operations assigned to ANE with forced CPU+Neural Engine on an M5 Pro; the 32 CPU operations were mainly integer indexing around embeddings, plus `cumsum` and one decompression constant. This is a private profiler count, distinct from the public compute-plan grouping. On the same Mac, an adaptive W8 entity request took 9.61 ms median over 100 calls after 10 warmups, using the FP16 path. The uncompressed FP32 All baseline took 10.56 ms in a separate 200-call run. These are small local measurements, not a paired speedup claim or a Decision Index score.
+
+```python
+from gliner2 import Schema
+from extraction_runtime import CoreMLAdaptiveBoundaryExtractor
+
+model = CoreMLAdaptiveBoundaryExtractor(".")
+print(model.extract("Alice founded Acme in Toronto.", Schema().entities(["person", "organization", "location"])))
+```
+
+Both W8 feature packages and the five FP16 head packages are included. `extraction-w8-assets.lock.json` records exact file hashes. `quantize-extraction-coreml.py` regenerates the W8 packages from the pinned feature exports; `verify-adaptive-extraction.py` checks the selected native-output manifest. Weight-only W8 does not imply int8 activation or matrix arithmetic.
 
 ## Classification
 
 The original L128/K8 classification packages remain available, with a separate `runtime.py` entry point. Up to eight labels fit that bucket. 299/300 selected choices matched in FP16; 100/100 matched in an FP32 control. See the classification report JSONs for the exact selected samples and limits. The full checkpoint's task and dataset scores have not been reproduced here.
 
-An optional **embedding-only W8 with asymmetric scaling** classifier keeps the encoder and heads FP16. It reduces the package from **576,528,829 to 385,193,894 bytes** (33.2%). On an M5 Pro plugged into AC, it agreed with FP16 on 100/100 selected eligible application requests (largest probability difference 0.01941). It matched the native classifier on 99/100 requests; the one near-tie disagreement at `jev.ag_news` row 100 is also present in the FP16 export. In a paired 100-request check, automatic-device full-request p50 was 4.283 ms FP16 versus 4.265 ms W8, so there is no established speed gain. The W8 compute plan put 496/516 executable ops on ANE under forced CPU+ANE, with 20 CPU boundary/constant ops; forced CPU+ANE was slower than automatic device choice. Symmetric embedding W8 exceeded the 0.02 probability-difference gate on the first selected manifest and is omitted. Use `runtime.py --precision embedding_w8_linear` for the compressed classification package; extraction continues to use FP32 packages.
+An optional **embedding-only W8 with asymmetric scaling** classifier keeps the encoder and heads FP16. It reduces the package from **576,528,829 to 385,193,894 bytes** (33.2%). On an M5 Pro plugged into AC, it agreed with FP16 on 100/100 selected eligible application requests (largest probability difference 0.01941). It matched the native classifier on 99/100 requests; the one near-tie disagreement at `jev.ag_news` row 100 is also present in the FP16 export. In a paired 100-request check, automatic-device full-request p50 was 4.283 ms FP16 versus 4.265 ms W8, so there is no established speed gain. The W8 compute plan put 496/516 executable ops on ANE under forced CPU+ANE, with 20 CPU boundary/constant ops; forced CPU+ANE was slower than automatic device choice. Symmetric embedding W8 exceeded the 0.02 probability-difference gate on the first selected manifest and is omitted. Use `runtime.py --precision embedding_w8_linear` for the compressed classification package; extraction has its own packages and runtime.
 
 When downloading with `huggingface_hub.snapshot_download`, pass `local_dir="./gliner2-multi-coreml"` and point the runtime there. Core ML compilation on the tested macOS release rejected the symlinked weight file in the default Hub cache snapshot; use a materialized local directory.
 

benchmark-extraction.py

@@ -11,7 +11,7 @@ import coremltools as ct
 import psutil
 from gliner2 import Schema
 
-from extraction_runtime import CoreMLBoundaryExtractor
+from extraction_runtime import CoreMLAdaptiveBoundaryExtractor, CoreMLBoundaryExtractor
 
 UNITS = {
     "cpu_only": ct.ComputeUnit.CPU_ONLY,
@@ -30,6 +30,8 @@ def main():
     parser = argparse.ArgumentParser()
     parser.add_argument("--model-dir", required=True)
     parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
+    parser.add_argument("--feature-package")
+    parser.add_argument("--adaptive-w8", action="store_true")
     parser.add_argument("--units", choices=list(UNITS), default="all")
     parser.add_argument("--warmup", type=int, default=20)
     parser.add_argument("--iterations", type=int, default=200)
@@ -37,7 +39,15 @@ def main():
     text = "Alice founded Acme in Toronto in 2020."
     schema = Schema().entities(["person", "organization", "location"])
     started = time.perf_counter()
-    runtime = CoreMLBoundaryExtractor(args.model_dir, precision=args.precision, compute_units=UNITS[args.units])
+    if args.adaptive_w8:
+        runtime = CoreMLAdaptiveBoundaryExtractor(args.model_dir, fp16_compute_units=UNITS[args.units])
+    else:
+        runtime = CoreMLBoundaryExtractor(
+            args.model_dir,
+            precision=args.precision,
+            compute_units=UNITS[args.units],
+            feature_package=args.feature_package,
+        )
     load_ms = (time.perf_counter() - started) * 1000
     for _ in range(args.warmup):
         runtime.extract(text, schema)
@@ -55,6 +65,8 @@ def main():
         "shape": "L128/W64/Q8/C192",
         "precision": args.precision,
         "compute_units": args.units,
+        "feature_package": args.feature_package,
+        "adaptive_w8": args.adaptive_w8,
         "warmup": args.warmup,
         "iterations": args.iterations,
         "load_ms": load_ms,
@@ -67,7 +79,8 @@ def main():
         "coremltools": ct.__version__,
     }
     folder = Path(args.model_dir)
-    path = folder / f"benchmark-{args.precision}-{args.units}.json"
+    variant = "adaptive-w8" if args.adaptive_w8 else "w8-embedding" if args.feature_package else "baseline"
+    path = folder / f"benchmark-{args.precision}-{args.units}-{variant}.json"
     path.write_text(json.dumps(report, indent=2) + "\n")
     print(json.dumps(report, indent=2))
 

extraction-w8-assets.lock.json

@@ -0,0 +1,161 @@
+{
+  "source_model": "fastino/gliner2.5-multi-v1",
+  "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
+  "coremltools": "9.0",
+  "quantization_scope": "trained word embedding only; int8 per-channel weight-only; activations and remaining weights retain exported precision",
+  "package_files": [
+    {
+      "package": "gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage",
+      "bytes": 387210787,
+      "files": [
+        {
+          "path": "gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel",
+          "bytes": 1046842,
+          "sha256": "25c0251fbbd22083eb156897c4dd075315f18326f6c2e5107463882175db8f5a"
+        },
+        {
+          "path": "gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
+          "bytes": 386163328,
+          "sha256": "04ec00914fcd74e7a6d6f827832621e8d4d8a317eb96a934c0ef93aa46955533"
+        },
+        {
+          "path": "gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage/Manifest.json",
+          "bytes": 617,
+          "sha256": "dbefec606e8e299c14baff9fd7aac685d4ee879bc16d8dd5a5c28d9a9cb5ffa1"
+        }
+      ],
+      "source_package": "gliner2_multi_extraction_features_fp16_L128_W64_Q8.mlpackage",
+      "source_bytes": 578545680,
+      "method": "linear int8 per-channel weight-only"
+    },
+    {
+      "package": "gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage",
+      "bytes": 580986690,
+      "files": [
+        {
+          "path": "gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel",
+          "bytes": 1013141,
+          "sha256": "1be2896ededfe989d059b021a4b3213ce4156e9756bbe85b78e867d11a0ac65c"
+        },
+        {
+          "path": "gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
+          "bytes": 579972932,
+          "sha256": "ddef5f5a28b195ce0e9ab5e4cd6f83b42e2e71c151d6d74583754e80807c8ebc"
+        },
+        {
+          "path": "gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage/Manifest.json",
+          "bytes": 617,
+          "sha256": "e2aa08eaba4755b4bea21e0ca6a85bd7fa4f451a1988e697e0c637587a717df6"
+        }
+      ],
+      "source_package": "gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage",
+      "source_bytes": 1155993314,
+      "method": "linear_symmetric int8 per-channel weight-only"
+    },
+    {
+      "package": "gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage",
+      "bytes": 614019,
+      "files": [
+        {
+          "path": "gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel",
+          "bytes": 61018,
+          "sha256": "3d31baa0cbe241bbdb574c313447ccd47856c8fd9aa10ee3f7b7887f62050a30"
+        },
+        {
+          "path": "gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
+          "bytes": 552384,
+          "sha256": "17e65c6985fd285abe612bf07fb1b282206c3070d1e229fd0bbf60285e21c169"
+        },
+        {
+          "path": "gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage/Manifest.json",
+          "bytes": 617,
+          "sha256": "a74bccb2e4a3be79751a07a57be2a9889daeff63b885fab6a76b2a228d6ddee1"
+        }
+      ]
+    },
+    {
+      "package": "gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage",
+      "bytes": 903163,
+      "files": [
+        {
+          "path": "gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel",
+          "bytes": 42962,
+          "sha256": "7989c66ba0d0282b5359243006fdf007e3d996eb7ae8e7f9b10a46f8f0fd7add"
+        },
+        {
+          "path": "gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
+          "bytes": 859584,
+          "sha256": "524efc11ca823cad23cc996d34cc7b3baec66bd2bccdf5d3047f83998e77e234"
+        },
+        {
+          "path": "gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Manifest.json",
+          "bytes": 617,
+          "sha256": "8ff09daba5da43d903677168e46189e716589ab7276d6d6c50240d457fc1deb0"
+        }
+      ]
+    },
+    {
+      "package": "gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage",
+      "bytes": 1443024,
+      "files": [
+        {
+          "path": "gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel",
+          "bytes": 6567,
+          "sha256": "1d84a0d996a5579fa1bb29da6ce2eec94e4c9e9c8da2265069f52dbc81aeec2c"
+        },
+        {
+          "path": "gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
+          "bytes": 1435840,
+          "sha256": "2bd01de3587d6e381bfc7809b59d3555c7a06f2afad05ccfb4d02e6aec6c20e2"
+        },
+        {
+          "path": "gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage/Manifest.json",
+          "bytes": 617,
+          "sha256": "4786e7ee95dd64ab0d6423f1c562d14aad65f4fbe46e49c5db67aa313721fcd9"
+        }
+      ]
+    },
+    {
+      "package": "gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage",
+      "bytes": 605167,
+      "files": [
+        {
+          "path": "gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel",
+          "bytes": 9990,
+          "sha256": "e55b74b80ff515dca663ba4495d4cb0e1dfae0793f7cb85703b13dd6e237f3f8"
+        },
+        {
+          "path": "gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
+          "bytes": 594560,
+          "sha256": "4c3714dc4a90bd2cba48e355d0d37b4bec91c9e98c1d9015b107820d8809bf8c"
+        },
+        {
+          "path": "gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage/Manifest.json",
+          "bytes": 617,
+          "sha256": "83eefaaca5795ec7b668aa647d8c7e262038f3c87e2fa0c79fbdcdb7e0a582d7"
+        }
+      ]
+    },
+    {
+      "package": "gliner2_multi_relation_fp16_W64_R4_P256.mlpackage",
+      "bytes": 11848716,
+      "files": [
+        {
+          "path": "gliner2_multi_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel",
+          "bytes": 32419,
+          "sha256": "97177682648258bb98feea081f1159d8eaf90d083c8bb32b7149db253240ac3d"
+        },
+        {
+          "path": "gliner2_multi_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
+          "bytes": 11815680,
+          "sha256": "d335d72da6645bad3de8c53f0f1cabaf872a1f8c59b24b4b88cd03fc30922e08"
+        },
+        {
+          "path": "gliner2_multi_relation_fp16_W64_R4_P256.mlpackage/Manifest.json",
+          "bytes": 617,
+          "sha256": "2fe43de8a585b5370f4e10abeffa9e4b361a198560428f091f32b2878ba6e2c9"
+        }
+      ]
+    }
+  ]
+}

extraction_runtime.py

@@ -1,6 +1,8 @@
 """GLiNER2.5 multilingual extraction runtime using only Core ML trained weights."""
 
+import gc
 import json
+import threading
 from contextvars import ContextVar
 from pathlib import Path
 
@@ -292,7 +294,13 @@ class CoreMLRecordHead(torch.nn.Module):
 class CoreMLBoundaryExtractor(BoundaryExtractor):
     """Native GLiNER2 schema/decoder with all trained extraction heads in Core ML."""
 
-    def __init__(self, model_dir: str, precision: str = "fp32", compute_units=ct.ComputeUnit.CPU_ONLY):
+    def __init__(
+        self,
+        model_dir: str,
+        precision: str = "fp32",
+        compute_units=ct.ComputeUnit.CPU_ONLY,
+        feature_package: str | None = None,
+    ):
         if precision not in ("fp16", "fp32"):
             raise ValueError("precision must be fp16 or fp32")
         torch.nn.Module.__init__(self)
@@ -306,9 +314,12 @@ class CoreMLBoundaryExtractor(BoundaryExtractor):
         self.length, self.max_words, self.max_queries = 128, 64, 8
         self._context = ContextVar("gliner2_coreml_extraction_context")
         suffix = f"{precision}_L128_W64_Q8"
-        self.features_model = ct.models.MLModel(
-            str(folder / f"{MODEL_PREFIX}_extraction_features_{suffix}.mlpackage"), compute_units=compute_units
+        feature_path = (
+            Path(feature_package) if feature_package else Path(f"{MODEL_PREFIX}_extraction_features_{suffix}.mlpackage")
         )
+        if not feature_path.is_absolute():
+            feature_path = folder / feature_path
+        self.features_model = ct.models.MLModel(str(feature_path), compute_units=compute_units)
         self.scorer_model = ct.models.MLModel(
             str(folder / f"{MODEL_PREFIX}_extraction_scorer_{suffix}.mlpackage"), compute_units=compute_units
         )
@@ -523,3 +534,44 @@ class CoreMLBoundaryExtractor(BoundaryExtractor):
             return raw
         finally:
             self._context.reset(token)
+
+
+class CoreMLAdaptiveBoundaryExtractor:
+    """Use an ANE-capable FP16 path and load FP32 for sensitive latent records."""
+
+    def __init__(
+        self,
+        model_dir: str,
+        fp16_feature_package: str = "gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage",
+        fp32_feature_package: str = "gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage",
+        fp16_compute_units=ct.ComputeUnit.CPU_AND_NE,
+        fp32_compute_units=ct.ComputeUnit.ALL,
+    ):
+        self.model_dir = model_dir
+        self.feature_packages = {"fp16": fp16_feature_package, "fp32": fp32_feature_package}
+        self.compute_units = {"fp16": fp16_compute_units, "fp32": fp32_compute_units}
+        self._lock = threading.Lock()
+        self._active_precision = None
+        self._runtime = None
+
+    @staticmethod
+    def selected_precision(schema) -> str:
+        built = schema.build() if hasattr(schema, "build") else schema
+        records = built.get("record_metadata", {})
+        return "fp32" if any(details.get("mode") == "latent" for details in records.values()) else "fp16"
+
+    def extract(self, text: str, schema, **kwargs):
+        precision = self.selected_precision(schema)
+        with self._lock:
+            if self._active_precision != precision:
+                self._runtime = None
+                self._active_precision = None
+                gc.collect()
+                self._runtime = CoreMLBoundaryExtractor(
+                    self.model_dir,
+                    precision=precision,
+                    compute_units=self.compute_units[precision],
+                    feature_package=self.feature_packages[precision],
+                )
+                self._active_precision = precision
+            return self._runtime.extract(text, schema, **kwargs)

gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d31baa0cbe241bbdb574c313447ccd47856c8fd9aa10ee3f7b7887f62050a30
+size 61018

gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17e65c6985fd285abe612bf07fb1b282206c3070d1e229fd0bbf60285e21c169
+size 552384

gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "068DC8E0-90EB-42DD-8F33-7982DACB8D19": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "50E9D2C3-E3A2-4606-84DA-545DE72B9175": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "50E9D2C3-E3A2-4606-84DA-545DE72B9175"
+}

gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1be2896ededfe989d059b021a4b3213ce4156e9756bbe85b78e867d11a0ac65c
+size 1013141

gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddef5f5a28b195ce0e9ab5e4cd6f83b42e2e71c151d6d74583754e80807c8ebc
+size 579972932

gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "4F0F74D8-656E-47AF-A5D6-AB711305E757": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "C3D0DBEC-B355-4E71-A511-43CD347367D2": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "C3D0DBEC-B355-4E71-A511-43CD347367D2"
+}

gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25c0251fbbd22083eb156897c4dd075315f18326f6c2e5107463882175db8f5a
+size 1046842

gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04ec00914fcd74e7a6d6f827832621e8d4d8a317eb96a934c0ef93aa46955533
+size 386163328

gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "937DBF58-428C-4184-8844-12BD2D883346": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "9651876E-BF7E-47F1-ACB3-1FEC7310D019": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "9651876E-BF7E-47F1-ACB3-1FEC7310D019"
+}

gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7989c66ba0d0282b5359243006fdf007e3d996eb7ae8e7f9b10a46f8f0fd7add
+size 42962

gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:524efc11ca823cad23cc996d34cc7b3baec66bd2bccdf5d3047f83998e77e234
+size 859584

gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "862F0AAB-6907-4EFE-B162-BCD0DC04BCC9": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "F75CC32C-8F22-467F-9870-F7793C4AB9F3": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "862F0AAB-6907-4EFE-B162-BCD0DC04BCC9"
+}

gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d84a0d996a5579fa1bb29da6ce2eec94e4c9e9c8da2265069f52dbc81aeec2c
+size 6567

gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bd01de3587d6e381bfc7809b59d3555c7a06f2afad05ccfb4d02e6aec6c20e2
+size 1435840

gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "7A4F6CCD-69F2-458F-8FCD-B670B28DC042": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "8FBF52FE-F3B3-4501-9097-AE8B00E84BEF": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "8FBF52FE-F3B3-4501-9097-AE8B00E84BEF"
+}

gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e55b74b80ff515dca663ba4495d4cb0e1dfae0793f7cb85703b13dd6e237f3f8
+size 9990

gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c3714dc4a90bd2cba48e355d0d37b4bec91c9e98c1d9015b107820d8809bf8c
+size 594560

gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "CC8E1A07-36C7-4543-98F7-03F7A93F2AE2": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "D1C61619-5A81-4BDB-BB3B-0AD1A79495E6": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "CC8E1A07-36C7-4543-98F7-03F7A93F2AE2"
+}

gliner2_multi_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97177682648258bb98feea081f1159d8eaf90d083c8bb32b7149db253240ac3d
+size 32419

gliner2_multi_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d335d72da6645bad3de8c53f0f1cabaf872a1f8c59b24b4b88cd03fc30922e08
+size 11815680

gliner2_multi_relation_fp16_W64_R4_P256.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "60DAEA11-225A-4AD1-8A6E-767B703A328B": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "906F9000-F74E-4C25-9045-91571D6214F4": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "60DAEA11-225A-4AD1-8A6E-767B703A328B"
+}

quantize-extraction-coreml.py

@@ -0,0 +1,95 @@
+"""Quantize the trained GLiNER2.5 encoder embedding without changing other heads."""
+
+import argparse
+import hashlib
+import json
+import shutil
+import time
+from pathlib import Path
+
+import coremltools as ct
+from coremltools.optimize.coreml import (
+    OpLinearQuantizerConfig,
+    OptimizationConfig,
+    get_weights_metadata,
+    linear_quantize_weights,
+)
+
+
+def package_files(package: Path):
+    """Return exact byte sizes and hashes for the files in a Core ML package."""
+    files = []
+    for file in sorted(item for item in package.rglob("*") if item.is_file()):
+        digest = hashlib.sha256()
+        with file.open("rb") as stream:
+            for chunk in iter(lambda: stream.read(8 * 1024 * 1024), b""):
+                digest.update(chunk)
+        files.append(
+            {"path": str(file.relative_to(package)), "bytes": file.stat().st_size, "sha256": digest.hexdigest()}
+        )
+    return files
+
+
+def embedding_weight_name(model) -> tuple[str, tuple[int, ...], str]:
+    """Locate the real checkpoint's word embedding constant in an ML Program."""
+    metadata = get_weights_metadata(model)
+    matches = [name for name in metadata if name.startswith("encoder_embeddings_word_embeddings_weight")]
+    if len(matches) != 1:
+        raise ValueError(f"Expected one trained word embedding constant, found {matches}")
+    name = matches[0]
+    weight = metadata[name].val
+    if weight.ndim != 2 or weight.shape[0] < 65_536:
+        raise ValueError(f"Unexpected word embedding shape: {weight.shape}")
+    return name, tuple(weight.shape), str(weight.dtype)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--source", required=True, type=Path)
+    parser.add_argument("--output", required=True, type=Path)
+    parser.add_argument("--mode", choices=["linear_symmetric", "linear"], default="linear_symmetric")
+    args = parser.parse_args()
+    source, output = args.source.resolve(), args.output.resolve()
+    if source == output:
+        raise ValueError("Quantized output must differ from the source package")
+    started = time.perf_counter()
+    model = ct.models.MLModel(str(source), skip_model_load=True)
+    name, shape, dtype = embedding_weight_name(model)
+    config = OptimizationConfig(
+        op_name_configs={name: OpLinearQuantizerConfig(mode=args.mode, dtype="int8", granularity="per_channel")}
+    )
+    compressed = linear_quantize_weights(model, config=config)
+    compressed.short_description = f"{model.short_description}; W8 {args.mode} per-channel word embedding"
+    compressed.author = model.author
+    compressed.license = model.license
+    compressed.user_defined_metadata.update(model.user_defined_metadata)
+    compressed.user_defined_metadata["weight_compression"] = f"W8 {args.mode} per-channel word embedding"
+    compressed.user_defined_metadata["quantized_weight"] = name
+    output.parent.mkdir(parents=True, exist_ok=True)
+    if output.exists():
+        shutil.rmtree(output)
+    compressed.save(str(output))
+    source_files, output_files = package_files(source), package_files(output)
+    source_bytes = sum(file["bytes"] for file in source_files)
+    output_bytes = sum(file["bytes"] for file in output_files)
+    if output_bytes >= source_bytes:
+        raise RuntimeError("Quantized package is not smaller than its source")
+    report = {
+        "source_package": source.name,
+        "output_package": output.name,
+        "selected_weight": name,
+        "weight_shape": shape,
+        "source_dtype": dtype,
+        "method": f"{args.mode} int8 per-channel weight-only",
+        "source_bytes": source_bytes,
+        "output_bytes": output_bytes,
+        "seconds": time.perf_counter() - started,
+        "coremltools": ct.__version__,
+        "output_files": output_files,
+    }
+    output.with_suffix(".json").write_text(json.dumps(report, indent=2) + "\n")
+    print(json.dumps({key: value for key, value in report.items() if key != "output_files"}, indent=2))
+
+
+if __name__ == "__main__":
+    main()

reports/extraction-w8-adaptive.json

@@ -0,0 +1,478 @@
+{
+  "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
+  "selected_manifest": "fifteen pinned real-text schema fixtures, not a Decision Index score",
+  "variant": "adaptive-w8-embedding",
+  "matched": 15,
+  "total": 15,
+  "cases": [
+    {
+      "name": "entities",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 4.106760025024414e-05,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.9971439242362976,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.9936610460281372,
+              "start": 14,
+              "end": 18
+            }
+          ],
+          "location": [
+            {
+              "text": "Toronto",
+              "confidence": 0.9991735816001892,
+              "start": 22,
+              "end": 29
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "entities_multi",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 5.6624412536621094e-05,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Marie Curie",
+              "confidence": 0.9977656602859497,
+              "start": 0,
+              "end": 11
+            }
+          ],
+          "city": [
+            {
+              "text": "Paris",
+              "confidence": 0.9993040561676025,
+              "start": 45,
+              "end": 50
+            },
+            {
+              "text": "Warsaw",
+              "confidence": 0.998807430267334,
+              "start": 24,
+              "end": 30
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "relations",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.003583967685699463,
+      "coreml": {
+        "relation_extraction": {
+          "founded": [
+            {
+              "head": {
+                "text": "Alice",
+                "start": 0,
+                "end": 5,
+                "confidence": 0.6171665787696838
+              },
+              "tail": {
+                "text": "Acme",
+                "start": 14,
+                "end": 18,
+                "confidence": 0.6171665787696838
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "relations_two",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.0041626691818237305,
+      "coreml": {
+        "relation_extraction": {
+          "founded": [
+            {
+              "head": {
+                "text": "Steve Jobs",
+                "start": 0,
+                "end": 10,
+                "confidence": 0.684580385684967
+              },
+              "tail": {
+                "text": "Apple",
+                "start": 19,
+                "end": 24,
+                "confidence": 0.684580385684967
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "record_natural",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.016394853591918945,
+      "coreml": {
+        "employment": [
+          {
+            "person": {
+              "text": "Alice",
+              "confidence": 0.9884026646614075,
+              "start": 0,
+              "end": 5
+            },
+            "company": {
+              "text": "Acme",
+              "confidence": 0.8344069123268127,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Bob",
+              "confidence": 0.9633687138557434,
+              "start": 21,
+              "end": 24
+            },
+            "company": {
+              "text": "Beta",
+              "confidence": 0.6842640042304993,
+              "start": 34,
+              "end": 38
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "record_latent",
+      "route": "fp32",
+      "structure_match": true,
+      "maximum_confidence_error": 0.004612326622009277,
+      "coreml": {
+        "employment": [
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.5833683609962463,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6160218119621277,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Alice",
+              "confidence": 0.9673200249671936,
+              "start": 0,
+              "end": 5
+            },
+            "company": null
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.8807083368301392,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.8964732885360718,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.7156234383583069,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6515812277793884,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Bob",
+              "confidence": 0.5077791213989258,
+              "start": 21,
+              "end": 24
+            },
+            "company": null
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.534905731678009,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6060499548912048,
+              "start": 15,
+              "end": 19
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "record_anchorless",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": null,
+      "coreml": {}
+    },
+    {
+      "name": "attributes",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.0003001689910888672,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.9936610460281372,
+              "start": 0,
+              "end": 5,
+              "role": {
+                "label": "founder",
+                "confidence": 0.9998525381088257
+              }
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.9911182522773743,
+              "start": 14,
+              "end": 18,
+              "role": {
+                "label": "founder",
+                "confidence": 0.9902054667472839
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "mixed_classification",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.00011724233627319336,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.9987744688987732,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.996292769908905,
+              "start": 14,
+              "end": 18
+            }
+          ]
+        },
+        "sentiment": {
+          "label": "positive",
+          "confidence": 0.995216965675354
+        }
+      }
+    },
+    {
+      "name": "classification_only",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.011062800884246826,
+      "coreml": {
+        "topic": {
+          "label": "sports",
+          "confidence": 0.49501627683639526
+        }
+      }
+    },
+    {
+      "name": "enum_choice",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 1.0728836059570312e-06,
+      "coreml": {
+        "product": [
+          {
+            "category": {
+              "text": "electronics",
+              "confidence": 0.9997695088386536
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "es_entities",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.00017154216766357422,
+      "coreml": {
+        "entities": {
+          "persona": [
+            {
+              "text": "María",
+              "confidence": 0.9971216320991516,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organización": [
+            {
+              "text": "Telefónica",
+              "confidence": 0.9901782274246216,
+              "start": 29,
+              "end": 39
+            }
+          ],
+          "ciudad": [
+            {
+              "text": "Madrid",
+              "confidence": 0.9994946718215942,
+              "start": 17,
+              "end": 23
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "fr_entities",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 3.17692756652832e-05,
+      "coreml": {
+        "entities": {
+          "personne": [
+            {
+              "text": "Jean Dupont",
+              "confidence": 0.9973061084747314,
+              "start": 0,
+              "end": 11
+            }
+          ],
+          "entreprise": [
+            {
+              "text": "Airbus",
+              "confidence": 0.9904406070709229,
+              "start": 35,
+              "end": 41
+            }
+          ],
+          "ville": [
+            {
+              "text": "Paris",
+              "confidence": 0.9991990923881531,
+              "start": 24,
+              "end": 29
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "zh_entities",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 0.00035566091537475586,
+      "coreml": {
+        "entities": {
+          "人物": [
+            {
+              "text": "张伟在北京大学工作",
+              "confidence": 0.9429286122322083,
+              "start": 0,
+              "end": 9
+            }
+          ],
+          "机构": [],
+          "地点": []
+        }
+      }
+    },
+    {
+      "name": "de_entities",
+      "route": "fp16",
+      "structure_match": true,
+      "maximum_confidence_error": 3.6716461181640625e-05,
+      "coreml": {
+        "entities": {
+          "Person": [
+            {
+              "text": "Angela Merkel",
+              "confidence": 0.9967142343521118,
+              "start": 0,
+              "end": 13
+            }
+          ],
+          "Stadt": [
+            {
+              "text": "Berlin",
+              "confidence": 0.9987056255340576,
+              "start": 23,
+              "end": 29
+            }
+          ]
+        }
+      }
+    }
+  ]
+}

reports/extraction-w8-fp16_cpu_ne.json

@@ -0,0 +1,470 @@
+{
+  "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
+  "selected_manifest": "pinned real-text schema fixtures, not a Decision Index score",
+  "feature_package": "gliner2_multi_extraction_features_w8_embedding_linear_fp16_L128_W64_Q8.mlpackage",
+  "precision": "fp16",
+  "compute_units": "cpu_and_neural_engine",
+  "matched": 14,
+  "total": 15,
+  "cases": [
+    {
+      "name": "entities",
+      "structure_match": true,
+      "maximum_confidence_error": 4.106760025024414e-05,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.9971439242362976,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.9936610460281372,
+              "start": 14,
+              "end": 18
+            }
+          ],
+          "location": [
+            {
+              "text": "Toronto",
+              "confidence": 0.9991735816001892,
+              "start": 22,
+              "end": 29
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "entities_multi",
+      "structure_match": true,
+      "maximum_confidence_error": 5.6624412536621094e-05,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Marie Curie",
+              "confidence": 0.9977656602859497,
+              "start": 0,
+              "end": 11
+            }
+          ],
+          "city": [
+            {
+              "text": "Paris",
+              "confidence": 0.9993040561676025,
+              "start": 45,
+              "end": 50
+            },
+            {
+              "text": "Warsaw",
+              "confidence": 0.998807430267334,
+              "start": 24,
+              "end": 30
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "relations",
+      "structure_match": true,
+      "maximum_confidence_error": 0.003583967685699463,
+      "coreml": {
+        "relation_extraction": {
+          "founded": [
+            {
+              "head": {
+                "text": "Alice",
+                "start": 0,
+                "end": 5,
+                "confidence": 0.6171665787696838
+              },
+              "tail": {
+                "text": "Acme",
+                "start": 14,
+                "end": 18,
+                "confidence": 0.6171665787696838
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "relations_two",
+      "structure_match": true,
+      "maximum_confidence_error": 0.0041626691818237305,
+      "coreml": {
+        "relation_extraction": {
+          "founded": [
+            {
+              "head": {
+                "text": "Steve Jobs",
+                "start": 0,
+                "end": 10,
+                "confidence": 0.684580385684967
+              },
+              "tail": {
+                "text": "Apple",
+                "start": 19,
+                "end": 24,
+                "confidence": 0.684580385684967
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "record_natural",
+      "structure_match": true,
+      "maximum_confidence_error": 0.016394853591918945,
+      "coreml": {
+        "employment": [
+          {
+            "person": {
+              "text": "Alice",
+              "confidence": 0.9884026646614075,
+              "start": 0,
+              "end": 5
+            },
+            "company": {
+              "text": "Acme",
+              "confidence": 0.8344069123268127,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Bob",
+              "confidence": 0.9633687138557434,
+              "start": 21,
+              "end": 24
+            },
+            "company": {
+              "text": "Beta",
+              "confidence": 0.6842640042304993,
+              "start": 34,
+              "end": 38
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "record_latent",
+      "structure_match": false,
+      "maximum_confidence_error": 0.025238752365112305,
+      "coreml": {
+        "employment": [
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.596550703048706,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.6208515167236328,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": {
+              "text": "Alice",
+              "confidence": 0.9655764698982239,
+              "start": 0,
+              "end": 5
+            },
+            "company": null
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.8875949382781982,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.9059898257255554,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.7203682661056519,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6446726322174072,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Bob",
+              "confidence": 0.5009422302246094,
+              "start": 21,
+              "end": 24
+            },
+            "company": {
+              "text": "Beta",
+              "confidence": 0.5094440579414368,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.5506979823112488,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6359010338783264,
+              "start": 15,
+              "end": 19
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "record_anchorless",
+      "structure_match": true,
+      "maximum_confidence_error": null,
+      "coreml": {}
+    },
+    {
+      "name": "attributes",
+      "structure_match": true,
+      "maximum_confidence_error": 0.0003001689910888672,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.9936610460281372,
+              "start": 0,
+              "end": 5,
+              "role": {
+                "label": "founder",
+                "confidence": 0.9998525381088257
+              }
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.9911182522773743,
+              "start": 14,
+              "end": 18,
+              "role": {
+                "label": "founder",
+                "confidence": 0.9902054667472839
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "mixed_classification",
+      "structure_match": true,
+      "maximum_confidence_error": 0.00011724233627319336,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.9987744688987732,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.996292769908905,
+              "start": 14,
+              "end": 18
+            }
+          ]
+        },
+        "sentiment": {
+          "label": "positive",
+          "confidence": 0.995216965675354
+        }
+      }
+    },
+    {
+      "name": "classification_only",
+      "structure_match": true,
+      "maximum_confidence_error": 0.011062800884246826,
+      "coreml": {
+        "topic": {
+          "label": "sports",
+          "confidence": 0.49501627683639526
+        }
+      }
+    },
+    {
+      "name": "enum_choice",
+      "structure_match": true,
+      "maximum_confidence_error": 1.0728836059570312e-06,
+      "coreml": {
+        "product": [
+          {
+            "category": {
+              "text": "electronics",
+              "confidence": 0.9997695088386536
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "es_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 0.00017154216766357422,
+      "coreml": {
+        "entities": {
+          "persona": [
+            {
+              "text": "María",
+              "confidence": 0.9971216320991516,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organización": [
+            {
+              "text": "Telefónica",
+              "confidence": 0.9901782274246216,
+              "start": 29,
+              "end": 39
+            }
+          ],
+          "ciudad": [
+            {
+              "text": "Madrid",
+              "confidence": 0.9994946718215942,
+              "start": 17,
+              "end": 23
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "fr_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 3.17692756652832e-05,
+      "coreml": {
+        "entities": {
+          "personne": [
+            {
+              "text": "Jean Dupont",
+              "confidence": 0.9973061084747314,
+              "start": 0,
+              "end": 11
+            }
+          ],
+          "entreprise": [
+            {
+              "text": "Airbus",
+              "confidence": 0.9904406070709229,
+              "start": 35,
+              "end": 41
+            }
+          ],
+          "ville": [
+            {
+              "text": "Paris",
+              "confidence": 0.9991990923881531,
+              "start": 24,
+              "end": 29
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "zh_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 0.00035566091537475586,
+      "coreml": {
+        "entities": {
+          "人物": [
+            {
+              "text": "张伟在北京大学工作",
+              "confidence": 0.9429286122322083,
+              "start": 0,
+              "end": 9
+            }
+          ],
+          "机构": [],
+          "地点": []
+        }
+      }
+    },
+    {
+      "name": "de_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 3.6716461181640625e-05,
+      "coreml": {
+        "entities": {
+          "Person": [
+            {
+              "text": "Angela Merkel",
+              "confidence": 0.9967142343521118,
+              "start": 0,
+              "end": 13
+            }
+          ],
+          "Stadt": [
+            {
+              "text": "Berlin",
+              "confidence": 0.9987056255340576,
+              "start": 23,
+              "end": 29
+            }
+          ]
+        }
+      }
+    }
+  ]
+}

reports/extraction-w8-fp32_all.json

@@ -0,0 +1,465 @@
+{
+  "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
+  "selected_manifest": "pinned real-text schema fixtures, not a Decision Index score",
+  "feature_package": "gliner2_multi_extraction_features_w8_embedding_fp32_L128_W64_Q8.mlpackage",
+  "precision": "fp32",
+  "compute_units": "all",
+  "matched": 15,
+  "total": 15,
+  "cases": [
+    {
+      "name": "entities",
+      "structure_match": true,
+      "maximum_confidence_error": 6.54458999633789e-05,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.997123658657074,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.9936854243278503,
+              "start": 14,
+              "end": 18
+            }
+          ],
+          "location": [
+            {
+              "text": "Toronto",
+              "confidence": 0.9991651773452759,
+              "start": 22,
+              "end": 29
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "entities_multi",
+      "structure_match": true,
+      "maximum_confidence_error": 1.4603137969970703e-05,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Marie Curie",
+              "confidence": 0.9977265000343323,
+              "start": 0,
+              "end": 11
+            }
+          ],
+          "city": [
+            {
+              "text": "Paris",
+              "confidence": 0.9992579817771912,
+              "start": 45,
+              "end": 50
+            },
+            {
+              "text": "Warsaw",
+              "confidence": 0.9987547397613525,
+              "start": 24,
+              "end": 30
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "relations",
+      "structure_match": true,
+      "maximum_confidence_error": 0.0022187232971191406,
+      "coreml": {
+        "relation_extraction": {
+          "founded": [
+            {
+              "head": {
+                "text": "Alice",
+                "start": 0,
+                "end": 5,
+                "confidence": 0.6158013343811035
+              },
+              "tail": {
+                "text": "Acme",
+                "start": 14,
+                "end": 18,
+                "confidence": 0.6158013343811035
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "relations_two",
+      "structure_match": true,
+      "maximum_confidence_error": 0.005145668983459473,
+      "coreml": {
+        "relation_extraction": {
+          "founded": [
+            {
+              "head": {
+                "text": "Steve Jobs",
+                "start": 0,
+                "end": 10,
+                "confidence": 0.6938887238502502
+              },
+              "tail": {
+                "text": "Apple",
+                "start": 19,
+                "end": 24,
+                "confidence": 0.6938887238502502
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "record_natural",
+      "structure_match": true,
+      "maximum_confidence_error": 0.000523686408996582,
+      "coreml": {
+        "employment": [
+          {
+            "person": {
+              "text": "Alice",
+              "confidence": 0.9879591464996338,
+              "start": 0,
+              "end": 5
+            },
+            "company": {
+              "text": "Acme",
+              "confidence": 0.8242685198783875,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Bob",
+              "confidence": 0.9649046659469604,
+              "start": 21,
+              "end": 24
+            },
+            "company": {
+              "text": "Beta",
+              "confidence": 0.6673454642295837,
+              "start": 34,
+              "end": 38
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "record_latent",
+      "structure_match": true,
+      "maximum_confidence_error": 0.004612326622009277,
+      "coreml": {
+        "employment": [
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.5833683609962463,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6160218119621277,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Alice",
+              "confidence": 0.9673200249671936,
+              "start": 0,
+              "end": 5
+            },
+            "company": null
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.8807083368301392,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.8964732885360718,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.7156234383583069,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6515812277793884,
+              "start": 15,
+              "end": 19
+            }
+          },
+          {
+            "person": {
+              "text": "Bob",
+              "confidence": 0.5077791213989258,
+              "start": 21,
+              "end": 24
+            },
+            "company": null
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Beta",
+              "confidence": 0.534905731678009,
+              "start": 34,
+              "end": 38
+            }
+          },
+          {
+            "person": null,
+            "company": {
+              "text": "Acme",
+              "confidence": 0.6060499548912048,
+              "start": 15,
+              "end": 19
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "record_anchorless",
+      "structure_match": true,
+      "maximum_confidence_error": null,
+      "coreml": {}
+    },
+    {
+      "name": "attributes",
+      "structure_match": true,
+      "maximum_confidence_error": 0.0002243518829345703,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.993528425693512,
+              "start": 0,
+              "end": 5,
+              "role": {
+                "label": "founder",
+                "confidence": 0.9998435974121094
+              }
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.9913756251335144,
+              "start": 14,
+              "end": 18,
+              "role": {
+                "label": "founder",
+                "confidence": 0.9896809458732605
+              }
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "mixed_classification",
+      "structure_match": true,
+      "maximum_confidence_error": 0.00013393163681030273,
+      "coreml": {
+        "entities": {
+          "person": [
+            {
+              "text": "Alice",
+              "confidence": 0.9987138509750366,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organization": [
+            {
+              "text": "Acme",
+              "confidence": 0.996292233467102,
+              "start": 14,
+              "end": 18
+            }
+          ]
+        },
+        "sentiment": {
+          "label": "positive",
+          "confidence": 0.9952002763748169
+        }
+      }
+    },
+    {
+      "name": "classification_only",
+      "structure_match": true,
+      "maximum_confidence_error": 0.02455449104309082,
+      "coreml": {
+        "topic": {
+          "label": "sports",
+          "confidence": 0.5306335687637329
+        }
+      }
+    },
+    {
+      "name": "enum_choice",
+      "structure_match": true,
+      "maximum_confidence_error": 4.291534423828125e-06,
+      "coreml": {
+        "product": [
+          {
+            "category": {
+              "text": "electronics",
+              "confidence": 0.9997748732566833
+            }
+          }
+        ]
+      }
+    },
+    {
+      "name": "es_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 2.181529998779297e-05,
+      "coreml": {
+        "entities": {
+          "persona": [
+            {
+              "text": "María",
+              "confidence": 0.9971477389335632,
+              "start": 0,
+              "end": 5
+            }
+          ],
+          "organización": [
+            {
+              "text": "Telefónica",
+              "confidence": 0.990371584892273,
+              "start": 29,
+              "end": 39
+            }
+          ],
+          "ciudad": [
+            {
+              "text": "Madrid",
+              "confidence": 0.9995085000991821,
+              "start": 17,
+              "end": 23
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "fr_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 3.0159950256347656e-05,
+      "coreml": {
+        "entities": {
+          "personne": [
+            {
+              "text": "Jean Dupont",
+              "confidence": 0.9973365664482117,
+              "start": 0,
+              "end": 11
+            }
+          ],
+          "entreprise": [
+            {
+              "text": "Airbus",
+              "confidence": 0.9904905557632446,
+              "start": 35,
+              "end": 41
+            }
+          ],
+          "ville": [
+            {
+              "text": "Paris",
+              "confidence": 0.9991647005081177,
+              "start": 24,
+              "end": 29
+            }
+          ]
+        }
+      }
+    },
+    {
+      "name": "zh_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 0.0006085038185119629,
+      "coreml": {
+        "entities": {
+          "人物": [
+            {
+              "text": "张伟在北京大学工作",
+              "confidence": 0.9431814551353455,
+              "start": 0,
+              "end": 9
+            }
+          ],
+          "机构": [],
+          "地点": []
+        }
+      }
+    },
+    {
+      "name": "de_entities",
+      "structure_match": true,
+      "maximum_confidence_error": 2.1457672119140625e-06,
+      "coreml": {
+        "entities": {
+          "Person": [
+            {
+              "text": "Angela Merkel",
+              "confidence": 0.9966753721237183,
+              "start": 0,
+              "end": 13
+            }
+          ],
+          "Stadt": [
+            {
+              "text": "Berlin",
+              "confidence": 0.9987298846244812,
+              "start": 23,
+              "end": 29
+            }
+          ]
+        }
+      }
+    }
+  ]
+}

reports/extraction-w8-validation.json

@@ -0,0 +1,175 @@
+{
+  "purpose": "Selected real-text extraction parity and local latency; not a Decision Index score",
+  "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
+  "hardware": {
+    "device": "arm64",
+    "chip": "Apple M5 Pro",
+    "ram": "24GB",
+    "os_version": "macOS 27.0",
+    "timestamp": "2026-09-22T23:15:03.794400+00:00"
+  },
+  "quantization": "int8 per-channel weight-only word embedding; other weights and activations retain exported precision",
+  "native_parity": {
+    "fp16_cpu_ne": {
+      "report": "reports/extraction-w8-fp16_cpu_ne.json",
+      "matched": 14,
+      "total": 15,
+      "maximum_confidence_error_on_matching_cases": 0.016394853591918945,
+      "mismatched_fixtures": [
+        "record_latent"
+      ]
+    },
+    "fp32_all": {
+      "report": "reports/extraction-w8-fp32_all.json",
+      "matched": 15,
+      "total": 15,
+      "maximum_confidence_error_on_matching_cases": 0.02455449104309082,
+      "mismatched_fixtures": []
+    },
+    "adaptive": {
+      "report": "reports/extraction-w8-adaptive.json",
+      "matched": 15,
+      "total": 15,
+      "maximum_confidence_error_on_matching_cases": 0.016394853591918945,
+      "mismatched_fixtures": []
+    }
+  },
+  "fp16_w8_feature_private_profile": {
+    "compute_units": "cpu_and_neural_engine",
+    "total_ops": 652,
+    "ane_ops": 620,
+    "gpu_ops": 0,
+    "cpu_ops": 32,
+    "ane_percent": 95.1,
+    "reasons": [
+      {
+        "reason": "Unsupported tensor data type: int32",
+        "count": 26,
+        "estimated_cpu_runtime_ms": 5.6572,
+        "op_types": {
+          "ios17.expand_dims": 8,
+          "ios17.cast": 8,
+          "tile": 3,
+          "ios17.add": 2,
+          "select": 2,
+          "ios17.gather": 1,
+          "ios17.mul": 1,
+          "ios17.reshape": 1
+        },
+        "ops": [
+          "add_0",
+          "select_0",
+          "add_0_1",
+          "select_0_1",
+          "inputs_embeds_cast_fp16",
+          "mask_3",
+          "mask_3_to_fp16",
+          "var_104",
+          "extended_attention_mask",
+          "extended_attention_mask_to_uint16",
+          "var_106_cast_int16_to_int32",
+          "var_107",
+          "attention_mask_1",
+          "var_290",
+          "var_2555",
+          "text_idx",
+          "var_2571",
+          "query_idx",
+          "text_idx_to_int16",
+          "query_idx_to_uint16",
+          "var_2599",
+          "cls_idx",
+          "cls_idx_to_uint16",
+          "var_2674_cast_fp16_to_int32",
+          "var_2705",
+          "var_2734"
+        ]
+      },
+      {
+        "reason": "Unable to resolve operation input \"y\".",
+        "count": 2,
+        "estimated_cpu_runtime_ms": 0.002,
+        "op_types": {
+          "ios17.greater_equal": 2
+        },
+        "ops": [
+          "greater_equal_0",
+          "greater_equal_0_1"
+        ]
+      },
+      {
+        "reason": "Unable to resolve operation input \"x\".",
+        "count": 2,
+        "estimated_cpu_runtime_ms": 0.002,
+        "op_types": {
+          "ios17.equal": 1,
+          "ios17.less_equal": 1
+        },
+        "ops": [
+          "var_2706",
+          "mask"
+        ]
+      },
+      {
+        "reason": "ANE not available for this op",
+        "count": 1,
+        "estimated_cpu_runtime_ms": 0,
+        "op_types": {
+          "ios16.constexpr_affine_dequantize": 1
+        },
+        "ops": [
+          "encoder_embeddings_word_embeddings_weight_to_fp16_quantized"
+        ]
+      },
+      {
+        "reason": "Unsupported MIL operation \"cumsum\".",
+        "count": 1,
+        "estimated_cpu_runtime_ms": 0.001,
+        "op_types": {
+          "ios16.cumsum": 1
+        },
+        "ops": [
+          "var_2968_cast_fp16"
+        ]
+      }
+    ]
+  },
+  "end_to_end_entity_latency": {
+    "fp32_baseline_all": {
+      "compute_units": "all",
+      "warmup": 20,
+      "iterations": 200,
+      "p50_ms": 10.563604475464672,
+      "p95_ms": 12.338125030510128,
+      "machine": "arm64",
+      "macos": "27.0"
+    },
+    "fp16_w8_cpu_ne": {
+      "compute_units": "cpu_and_neural_engine",
+      "warmup": 10,
+      "iterations": 100,
+      "p50_ms": 9.476874489337206,
+      "p95_ms": 11.658000003080815,
+      "machine": "arm64",
+      "macos": "27.0"
+    },
+    "adaptive_w8_cpu_ne": {
+      "compute_units": "cpu_and_neural_engine",
+      "warmup": 10,
+      "iterations": 100,
+      "p50_ms": 9.61008301237598,
+      "p95_ms": 10.649124975316226,
+      "machine": "arm64",
+      "macos": "27.0"
+    },
+    "fp32_w8_all": {
+      "compute_units": "all",
+      "warmup": 10,
+      "iterations": 100,
+      "p50_ms": 10.522104013944045,
+      "p95_ms": 11.859667021781206,
+      "machine": "arm64",
+      "macos": "27.0"
+    }
+  }
+}

tests/test_extraction_routing.py

@@ -0,0 +1,21 @@
+"""Schema routing for the quantized multilingual extraction runtime."""
+
+from gliner2 import Schema
+
+from extraction_runtime import CoreMLAdaptiveBoundaryExtractor
+
+
+def record_schema(mode: str):
+    schema = Schema()
+    schema.structure("employment", mode=mode).field("person", dtype="str").field("company", dtype="str")
+    return schema
+
+
+def test_latent_record_uses_fp32():
+    assert CoreMLAdaptiveBoundaryExtractor.selected_precision(record_schema("latent")) == "fp32"
+
+
+def test_other_selected_schemas_use_fp16():
+    assert CoreMLAdaptiveBoundaryExtractor.selected_precision(Schema().entities(["person"])) == "fp16"
+    assert CoreMLAdaptiveBoundaryExtractor.selected_precision(record_schema("natural")) == "fp16"
+    assert CoreMLAdaptiveBoundaryExtractor.selected_precision(record_schema("anchorless")) == "fp16"

tests/test_quantize_extraction.py

@@ -0,0 +1,21 @@
+"""Inspect the trained embedding selected for weight-only quantization."""
+
+import os
+import runpy
+from pathlib import Path
+
+import coremltools as ct
+import pytest
+
+
+def test_quantizer_selects_real_word_embedding():
+    package = os.environ.get("GLINER2_EXTRACTION_FEATURE_PACKAGE")
+    if not package:
+        pytest.skip("Set GLINER2_EXTRACTION_FEATURE_PACKAGE to a pinned real Core ML feature package")
+    namespace = runpy.run_path(str(Path(__file__).parents[1] / "quantize-extraction-coreml.py"))
+    model = ct.models.MLModel(package, skip_model_load=True)
+    name, shape, dtype = namespace["embedding_weight_name"](model)
+    assert name.startswith("encoder_embeddings_word_embeddings_weight")
+    assert shape[0] >= 250_112
+    assert shape[1] == 768
+    assert dtype in ("float16", "float32")

verify-adaptive-extraction.py

@@ -0,0 +1,63 @@
+"""Check the adaptive W8 runtime against stored pinned native outputs."""
+
+import argparse
+import json
+import runpy
+from pathlib import Path
+
+from extraction_runtime import CoreMLAdaptiveBoundaryExtractor
+
+MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model-dir", required=True)
+    parser.add_argument("--reference", default=str(Path(__file__).parent / "reports/extraction-verify-fp32.json"))
+    args = parser.parse_args()
+    reference = json.loads(Path(args.reference).read_text())
+    if reference["source_revision"] != MODEL_REVISION or reference["matched"] != reference["total"]:
+        raise ValueError("Reference must be the complete pinned native/FP32 parity manifest")
+    helpers = runpy.run_path(str(Path(__file__).parent / "verify-full-extraction.py"))
+    fixtures = helpers["fixtures"]()
+    without_confidence = helpers["without_confidence"]
+    confidence_errors = helpers["confidence_errors"]
+    reference_cases = {case["name"]: case for case in reference["cases"]}
+    if {name for name, _, _ in fixtures} != set(reference_cases):
+        raise ValueError("Fixture names differ from the pinned native reference")
+    runtime = CoreMLAdaptiveBoundaryExtractor(args.model_dir)
+    cases = []
+    for name, text, schema in fixtures:
+        baseline = reference_cases[name]
+        if text != baseline["text"]:
+            raise ValueError(f"Fixture text changed for {name}")
+        expected = baseline["native"]
+        actual = runtime.extract(text, schema, include_confidence=True, include_spans=True)
+        errors = confidence_errors(expected, actual)
+        cases.append(
+            {
+                "name": name,
+                "route": runtime.selected_precision(schema),
+                "structure_match": without_confidence(expected) == without_confidence(actual),
+                "maximum_confidence_error": max(errors, default=None),
+                "coreml": actual,
+            }
+        )
+        print(f"{name}: {cases[-1]['route']} {cases[-1]['structure_match']}", flush=True)
+    report = {
+        "source_revision": MODEL_REVISION,
+        "selected_manifest": "fifteen pinned real-text schema fixtures, not a Decision Index score",
+        "variant": "adaptive-w8-embedding",
+        "matched": sum(case["structure_match"] for case in cases),
+        "total": len(cases),
+        "cases": cases,
+    }
+    output = Path(args.model_dir) / "verify-adaptive-w8.json"
+    output.write_text(json.dumps(report, indent=2, ensure_ascii=False) + "\n")
+    print(json.dumps({"matched": report["matched"], "total": report["total"]}, indent=2))
+    if report["matched"] != report["total"]:
+        raise SystemExit(1)
+
+
+if __name__ == "__main__":
+    main()

verify-quantized-extraction.py

@@ -0,0 +1,80 @@
+"""Compare a quantized extraction package with stored pinned native outputs."""
+
+import argparse
+import json
+import runpy
+from pathlib import Path
+
+import coremltools as ct
+
+from extraction_runtime import CoreMLBoundaryExtractor
+
+MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
+UNITS = {
+    "cpu_only": ct.ComputeUnit.CPU_ONLY,
+    "cpu_and_gpu": ct.ComputeUnit.CPU_AND_GPU,
+    "cpu_and_neural_engine": ct.ComputeUnit.CPU_AND_NE,
+    "all": ct.ComputeUnit.ALL,
+}
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model-dir", required=True)
+    parser.add_argument("--feature-package", required=True)
+    parser.add_argument("--precision", choices=["fp16", "fp32"], required=True)
+    parser.add_argument("--units", choices=list(UNITS), default="cpu_and_neural_engine")
+    parser.add_argument("--reference", default=str(Path(__file__).parent / "reports/extraction-verify-fp32.json"))
+    args = parser.parse_args()
+    reference = json.loads(Path(args.reference).read_text())
+    if reference["source_revision"] != MODEL_REVISION or reference["matched"] != reference["total"]:
+        raise ValueError("Reference must be the complete pinned native/FP32 parity manifest")
+    helpers = runpy.run_path(str(Path(__file__).parent / "verify-full-extraction.py"))
+    fixtures = helpers["fixtures"]()
+    without_confidence = helpers["without_confidence"]
+    confidence_errors = helpers["confidence_errors"]
+    reference_cases = {case["name"]: case for case in reference["cases"]}
+    if {name for name, _, _ in fixtures} != set(reference_cases):
+        raise ValueError("Fixture names differ from the pinned native reference")
+    runtime = CoreMLBoundaryExtractor(
+        args.model_dir,
+        precision=args.precision,
+        compute_units=UNITS[args.units],
+        feature_package=args.feature_package,
+    )
+    cases = []
+    for name, text, schema in fixtures:
+        baseline = reference_cases[name]
+        if text != baseline["text"]:
+            raise ValueError(f"Fixture text changed for {name}")
+        expected = baseline["native"]
+        actual = runtime.extract(text, schema, include_confidence=True, include_spans=True)
+        errors = confidence_errors(expected, actual)
+        cases.append(
+            {
+                "name": name,
+                "structure_match": without_confidence(expected) == without_confidence(actual),
+                "maximum_confidence_error": max(errors, default=None),
+                "coreml": actual,
+            }
+        )
+        print(f"{name}: {cases[-1]['structure_match']}", flush=True)
+    report = {
+        "source_revision": MODEL_REVISION,
+        "selected_manifest": "pinned real-text schema fixtures, not a Decision Index score",
+        "feature_package": args.feature_package,
+        "precision": args.precision,
+        "compute_units": args.units,
+        "matched": sum(case["structure_match"] for case in cases),
+        "total": len(cases),
+        "cases": cases,
+    }
+    output = Path(args.model_dir) / f"verify-quantized-{args.precision}-{args.units}.json"
+    output.write_text(json.dumps(report, indent=2, ensure_ascii=False) + "\n")
+    print(json.dumps({"matched": report["matched"], "total": report["total"]}, indent=2))
+    if report["matched"] != report["total"]:
+        raise SystemExit(1)
+
+
+if __name__ == "__main__":
+    main()