diff --git a/README.md b/README.md index 9d1f74aeb24fdf4e4ab6fea3fe1f96ae79affcd9..6bfe486a0c8bdc66fcacbb0ad0190b1bc3f9cc9b 100644 --- a/README.md +++ b/README.md @@ -1,40 +1,44 @@ --- license: apache-2.0 library_name: coremltools -pipeline_tag: text-classification +pipeline_tag: token-classification tags: - coreml - gliner2 - apple-silicon --- -# GLiNER2.5 base Core ML classification +# GLiNER2.5 base for Core ML -Core ML FP16 conversion of the classification decision path in -[Fastino GLiNER2.5 base](https://huggingface.co/fastino/gliner2.5-base-v1), -revision `1a8bc24e00dc7300b9017c81d63e3dcdabb26596`. -The original has 193,581,591 parameters (Apache-2.0). This package includes -its trained encoder and classification head (184,945,921 parameters); it does -not export entity, relation, record or span extraction. Use the original -checkpoint for those tasks. +This repository contains fixed-shape Core ML exports of the trained classification and extraction paths from [Fastino/gliner2.5-base-v1](https://huggingface.co/fastino/gliner2.5-base-v1) at revision `1a8bc24e00dc7300b9017c81d63e3dcdabb26596`. The original Apache-2.0 checkpoint has 193,581,591 parameters. Fastino authored the source model; Fluid Inference converted it. The model packages include the learned encoder, classification, boundary, relation, explicit-span, and record heads. The Python runtime keeps the source GLiNER2 schema, candidate selection, and decoder semantics. + +## Extraction + +The FP32 extraction stage packages support entities, relations, entity attributes, enum choices, natural/latent/anchorless records, and schemas mixed with classification. In a small fixed manifest of real text and schema fixtures, FP32 matched the native structured output on **11/11** cases. The largest FP32 confidence difference was 0.00000114. The multilingual manifest includes Spanish, French, Chinese, and German text. These checks are selected parity fixtures, not a Decision Index score or a full dataset evaluation. + +FP16 result: 11/11 structures matched; the largest confidence difference was 0.1653 on a latent-record fixture. The base extraction FP16 packages are also included for speed sensitive applications; use FP32 when confidence values or latent-record decisions need closer native agreement. -The L128/K8 package is 388,981,604 bytes, with up to eight labels. It targets -iOS 17/macOS 14. Included tokenizer and `preprocessing.py` reproduce native -schema rendering without loading the original PyTorch weights at runtime. ```bash uv sync -uv run python runtime.py --model-dir . --text "The rocket launched successfully." \ - --task topic --labels '["science","sports","politics"]' +uv run python - <<'PYCODE' +from gliner2 import Schema +from extraction_runtime import CoreMLBoundaryExtractor + +model = CoreMLBoundaryExtractor('.', precision='fp32') +schema = Schema().entities(['person', 'organization', 'location']) +print(model.extract('Alice founded Acme in Toronto.', schema, include_spans=True)) +PYCODE ``` -On an M5 Pro (macOS 27.0), this FP16 artifact matched the native chosen label -on 100 of 100 eligible requests from a fixed application suite; the largest -chosen-label confidence difference was 0.002999. Median Python Core ML call -time was 8.82 ms. The selected sample skipped 300 rows with more than eight -options and seven over-length rows before collecting 100 eligible cases. -This is a smoke parity check, not a full Decision Index score or an ANE-only -latency measure. See `verify-application100.json` for details. +The extraction bucket holds up to 128 combined subword tokens, 64 text words, 8 extraction queries, and 8 classification choices. Candidate, explicit span, relation pair, and record capacities are fixed in the package names. Requests beyond the bucket raise `ValueError`; they require a larger exported bucket. The runtime uses the original tokenizer files and `gliner2==2.0.0`, but loads no PyTorch model weights. Run `verify-full-extraction.py` with the pinned native checkpoint for the selected parity check. + +On an M5 Pro with macOS 27.0, FP32 end-to-end median 9.93 ms with All. FP16 median was 8.98 ms with All, 10.26 ms with CPU+Neural Engine, and 21.39 ms with CPU Only for a selected three-label entity request after 20 warmups and over 200 Python calls. Those are local end-to-end measurements for this shape, not ANE-only latency or a device-wide benchmark. + +The feature graph compute plan assigned 53.13% of operations to ANE and 46.87% to CPU under CPU+Neural Engine on this machine. Per-tensor LUT8 compression reduced the FP16 feature package from 391 MB to 196 MB, but only 10/11 selected structures matched, so that compressed package is omitted. + +## Classification + +The original L128/K8 classification packages remain available, with a separate `runtime.py` entry point. Up to eight labels fit that bucket. 100/100 selected choices matched in FP16. See the classification report JSONs for the exact selected samples and limits. The full checkpoint's task and dataset scores have not been reproduced here. -The conversion scripts, pinned dependencies and asset hashes are included. -Fluid Inference converted the model; Fastino authored the original checkpoint. +The Core ML deployment target is iOS 17/macOS 14. Conversion scripts, pinned dependencies, asset hashes, and selected verification reports are included. The source revision is a current pinned snapshot; identity with the historical Decision Index evaluation checkpoint has not been established. diff --git a/benchmark-extraction.py b/benchmark-extraction.py new file mode 100644 index 0000000000000000000000000000000000000000..0dccde1ff6b57b48e2656e77d3d3e569e4d124d4 --- /dev/null +++ b/benchmark-extraction.py @@ -0,0 +1,76 @@ +"""Selected fixed-bucket end-to-end Core ML extraction latency on this Mac.""" + +import argparse +import json +import platform +import statistics +import time +from pathlib import Path + +import coremltools as ct +import psutil +from gliner2 import Schema + +from extraction_runtime import CoreMLBoundaryExtractor + +UNITS = { + "cpu_only": ct.ComputeUnit.CPU_ONLY, + "cpu_and_gpu": ct.ComputeUnit.CPU_AND_GPU, + "cpu_and_neural_engine": ct.ComputeUnit.CPU_AND_NE, + "all": ct.ComputeUnit.ALL, +} + + +def percentile(values, fraction): + ordered = sorted(values) + return ordered[min(round(fraction * (len(ordered) - 1)), len(ordered) - 1)] + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--model-dir", required=True) + parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32") + parser.add_argument("--units", choices=list(UNITS), default="all") + parser.add_argument("--warmup", type=int, default=20) + parser.add_argument("--iterations", type=int, default=200) + args = parser.parse_args() + text = "Alice founded Acme in Toronto in 2020." + schema = Schema().entities(["person", "organization", "location"]) + started = time.perf_counter() + runtime = CoreMLBoundaryExtractor(args.model_dir, precision=args.precision, compute_units=UNITS[args.units]) + load_ms = (time.perf_counter() - started) * 1000 + for _ in range(args.warmup): + runtime.extract(text, schema) + process = psutil.Process() + latencies = [] + peak_rss = process.memory_info().rss + for _ in range(args.iterations): + start = time.perf_counter() + runtime.extract(text, schema) + latencies.append((time.perf_counter() - start) * 1000) + peak_rss = max(peak_rss, process.memory_info().rss) + report = { + "purpose": "selected end-to-end entity extraction latency, no benchmark scoring", + "fixture": text, + "shape": "L128/W64/Q8/C192", + "precision": args.precision, + "compute_units": args.units, + "warmup": args.warmup, + "iterations": args.iterations, + "load_ms": load_ms, + "p50_ms": statistics.median(latencies), + "p95_ms": percentile(latencies, 0.95), + "mean_ms": statistics.mean(latencies), + "peak_process_rss_bytes": peak_rss, + "macos": platform.mac_ver()[0], + "machine": platform.machine(), + "coremltools": ct.__version__, + } + folder = Path(args.model_dir) + path = folder / f"benchmark-{args.precision}-{args.units}.json" + path.write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps(report, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/config.json b/config.json new file mode 100644 index 0000000000000000000000000000000000000000..b4b3fcf2b21100b475be53f44ecf5a17813ef45f --- /dev/null +++ b/config.json @@ -0,0 +1,99 @@ +{ + "architecture": "boundary", + "architecture_version": 1, + "architectures": [ + "BoundaryExtractor" + ], + "attn_implementation": "sdpa", + "boundary_head": { + "abstention_loss_weight": 0.2, + "abstention_threshold": 0.5, + "adaptive_threshold": false, + "bidirectional_proposals": true, + "boundary_attention_heads": 4, + "boundary_attention_layers": 2, + "boundary_attention_window": 128, + "boundary_dim": 128, + "boundary_ffn_multiplier": 2.0, + "boundary_focal_clip": 0.05, + "boundary_focal_gamma_negative": 2.0, + "boundary_focal_gamma_positive": 0.0, + "boundary_marginal_loss": "asymmetric_focal", + "boundary_negative_weight": 0.5, + "boundary_refinement_layers": 1, + "boundary_top_k_alpha": 0.08, + "boundary_top_k_bucket": 8, + "boundary_top_k_max": 128, + "candidate_attention_heads": 4, + "candidate_attention_layers": 0, + "candidate_budget": 192, + "candidate_pool": "shared", + "classification_loss_weight": 1.0, + "classification_temperature": 1.0, + "consistency_loss_weight": 0.1, + "consistency_warmup_steps": 2000, + "content_dim": 64, + "content_soft_max_pool": false, + "count_loss_weight": 0.2, + "directional_relation_states": true, + "dropout": 0.1, + "enable_abstention": true, + "enable_count_head": true, + "enable_records": true, + "enable_relations": true, + "enable_rotary_endpoints": true, + "enable_span_content": true, + "end_block_size": 256, + "end_top_k": 24, + "endpoint_difference_features": true, + "ends_per_start": 12, + "export_mode": "auto", + "hard_negative_keep_all_when_absent": true, + "hard_negatives_per_positive": 20, + "loss_reduction": "sum", + "max_gold_per_query": 64, + "max_negative_queries_per_batch": 64, + "min_pool_per_query": 8, + "minimum_hard_negatives": 16, + "multihead_pair_compat_heads": 8, + "negative_query_ratio": 1.0, + "overlap_policy": "flat", + "pair_dim": 128, + "pair_temperature": 1.0, + "pool_boundary_top_k": 32, + "pool_size": 192, + "proposal_loss_weight": 0.3, + "query_attention_layers": 0, + "query_conditioned_inside_weight": true, + "record_anchor_proposal_threshold": 0.2, + "record_anchor_threshold": 0.5, + "record_dim": 128, + "record_field_threshold": 0.5, + "record_instance_queries": 32, + "record_loss_weight": 1.0, + "record_temperature": 1.0, + "relation_argument_proposal_threshold": 0.2, + "relation_biaffine_content": true, + "relation_heads_per_type": 32, + "relation_loss_weight": 1.0, + "relation_pair_cap": 64, + "relation_tails_per_type": 32, + "relation_temperature": 1.0, + "rerank_listwise_weight": 0.3, + "reranker_endpoint_compat": true, + "rotary_base": 10000.0, + "soft_iou_anneal_steps": 20000, + "soft_iou_aux_weight": 0.2, + "start_top_k": 24, + "starts_per_end": 12, + "training_candidate_budget": 192, + "use_inside_evidence": true, + "vectorized_pair_elements": 16777216 + }, + "config_version": 3, + "max_len": 4096, + "model_name": "microsoft/deberta-v3-base", + "model_type": "extractor", + "token_pooling": "first", + "transformers_version": "5.8.0" +} diff --git a/convert-explicit-coreml.py b/convert-explicit-coreml.py new file mode 100644 index 0000000000000000000000000000000000000000..a4e069d6ea3297ce77b8d36f027f8a64cce709da --- /dev/null +++ b/convert-explicit-coreml.py @@ -0,0 +1,170 @@ +"""Export GLiNER2.5 base trained explicit-span scorer for attributes and enums.""" + +import argparse +import json +from pathlib import Path + +import coremltools as ct +import numpy as np +import torch +from gliner2 import AutoExtractor, Schema +from huggingface_hub import snapshot_download + +from extraction_export import ExtractionExplicitSpanExport, ExtractionFeaturesExport, coreml_trace_patches +from preprocessing import prepare_extraction + +MODEL_ID = "fastino/gliner2.5-base-v1" +MODEL_REVISION = "1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +INPUT_NAMES = ( + "text_states", + "text_mask", + "query_states", + "query_mask", + "boundary_states", + "start_logits", + "end_logits", + "inside_prefix", + "inside_prefix_mean", + "span_indices", + "span_mask", +) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--output-dir", default="build/extraction") + parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32") + parser.add_argument("--length", type=int, default=128) + parser.add_argument("--max-words", type=int, default=64) + parser.add_argument("--max-queries", type=int, default=8) + parser.add_argument("--max-spans", type=int, default=64) + args = parser.parse_args() + torch.set_num_threads(4) + source = snapshot_download( + MODEL_ID, + revision=MODEL_REVISION, + allow_patterns=[ + "config.json", + "encoder_config/*", + "model.safetensors", + "tokenizer.json", + "tokenizer_config.json", + ], + ) + native = AutoExtractor.from_pretrained(source, map_location="cpu").eval() + text = "Alice founded Acme in Toronto in 2020." + schema = Schema().entities(["person", "organization", "location"]) + arrays, batch = prepare_extraction(native.processor, text, schema, args.length, args.max_words, args.max_queries) + tensors = tuple(torch.from_numpy(value) for value in arrays.values()) + with torch.no_grad(), coreml_trace_patches(): + features = ExtractionFeaturesExport(native).eval()(*tensors) + with torch.no_grad(): + core = native._encode_core(batch) + candidates = native.boundary_head( + core["text_states"], core["text_mask"], core["query_states"], core["query_mask"] + ).candidates + query_count = core["query_states"].shape[1] + indices = torch.zeros(1, args.max_queries, args.max_spans, 2, dtype=torch.int32) + mask = torch.zeros(1, args.max_queries, args.max_spans, dtype=torch.float32) + count = min(args.max_spans, candidates.indices.shape[2]) + indices[:, :query_count, :count] = candidates.indices[:, :query_count, :count].int() + mask[:, :query_count, :count] = candidates.valid_mask[:, :query_count, :count].float() + wrapper = ExtractionExplicitSpanExport(native).eval() + arguments = ( + features[0], + tensors[3], + features[1], + tensors[5], + features[2], + features[4], + features[5], + features[6], + features[7], + indices, + mask, + ) + with torch.no_grad(): + reference = wrapper(*arguments) + native_reference = native.boundary_head.score_explicit_spans( + core["text_states"], + core["text_mask"], + core["query_states"], + core["query_mask"], + indices[:, :query_count].long(), + mask[:, :query_count].bool(), + ) + wrapper_error = float( + (reference[:, :query_count][mask[:, :query_count].bool()] - native_reference[mask[:, :query_count].bool()]) + .abs() + .max() + ) + traced = torch.jit.trace(wrapper, arguments, check_trace=False) + if wrapper_error > 1e-4: + raise RuntimeError(f"Explicit span wrapper differs from native: {wrapper_error}") + precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32 + converted = ct.convert( + traced, + convert_to="mlprogram", + minimum_deployment_target=ct.target.iOS17, + compute_precision=precision, + compute_units=ct.ComputeUnit.CPU_ONLY, + inputs=[ + ct.TensorType(name=name, shape=tuple(value.shape), dtype=np.int32 if name == "span_indices" else np.float32) + for name, value in zip(INPUT_NAMES, arguments) + ], + outputs=[ct.TensorType(name="span_logits", dtype=np.float32)], + ) + converted.short_description = "GLiNER2.5 base trained explicit-span extraction scorer" + converted.author = "Fastino (original); Fluid Inference (Core ML conversion)" + converted.license = "Apache-2.0" + converted.user_defined_metadata.update( + { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "stage": "trained explicit-span proposal and reranker", + "word_capacity": str(args.max_words), + "query_capacity": str(args.max_queries), + "span_capacity": str(args.max_spans), + } + ) + out = Path(args.output_dir) + out.mkdir(parents=True, exist_ok=True) + suffix = f"{args.precision}_W{args.max_words}_Q{args.max_queries}_S{args.max_spans}" + package = out / f"gliner2_base_explicit_{suffix}.mlpackage" + converted.save(str(package)) + runtime = ct.models.MLModel(str(package), compute_units=ct.ComputeUnit.CPU_ONLY) + predicted = runtime.predict( + { + name: value.detach().numpy().astype(np.int32 if name == "span_indices" else np.float32) + for name, value in zip(INPUT_NAMES, arguments) + } + )["span_logits"] + runtime_error = float( + np.max( + np.abs( + predicted[:, :query_count][mask[:, :query_count].bool().numpy()] + - reference.numpy()[:, :query_count][mask[:, :query_count].bool().numpy()] + ) + ) + ) + if not np.isfinite(runtime_error): + raise RuntimeError("Explicit-span scorer produced non-finite logits") + report = { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "precision": args.precision, + "fixture": text, + "valid_spans": int(mask.sum()), + "wrapper_max_absolute_error": wrapper_error, + "coreml_max_absolute_error": runtime_error, + "package": str(package), + "package_bytes": sum(file.stat().st_size for file in package.rglob("*") if file.is_file()), + "coremltools": ct.__version__, + "torch": torch.__version__, + } + (out / f"explicit-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps(report, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/convert-extraction-coreml.py b/convert-extraction-coreml.py new file mode 100644 index 0000000000000000000000000000000000000000..be81b62a17acc74b7b296327b4254637e4163632 --- /dev/null +++ b/convert-extraction-coreml.py @@ -0,0 +1,217 @@ +"""Export the trained GLiNER2.5 base boundary extraction stages to Core ML.""" + +import argparse +import json +import shutil +from pathlib import Path + +import coremltools as ct +import numpy as np +import torch +from gliner2 import AutoExtractor, Schema +from huggingface_hub import snapshot_download + +from convert_extraction_names import FEATURE_NAMES, SCORE_INPUT_NAMES +from extraction_export import ExtractionFeaturesExport, ExtractionScoreExport, coreml_trace_patches +from extraction_pool import select_candidates +from preprocessing import prepare_extraction + +MODEL_ID = "fastino/gliner2.5-base-v1" +MODEL_REVISION = "1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +FIXTURE_TEXT = "Alice founded Acme in Toronto in 2020." +FIXTURE_SCHEMA = Schema().entities(["person", "organization", "location"]) + + +def package_bytes(path: Path) -> int: + return sum(file.stat().st_size for file in path.rglob("*") if file.is_file()) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--output-dir", default="build/extraction") + parser.add_argument("--length", type=int, default=128, help="Subword capacity") + parser.add_argument("--max-words", type=int, default=64) + parser.add_argument("--max-queries", type=int, default=8) + parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32") + args = parser.parse_args() + torch.set_num_threads(4) + source = snapshot_download( + MODEL_ID, + revision=MODEL_REVISION, + allow_patterns=[ + "config.json", + "encoder_config/*", + "model.safetensors", + "tokenizer.json", + "tokenizer_config.json", + ], + ) + native = AutoExtractor.from_pretrained(source, map_location="cpu").eval() + arrays, batch = prepare_extraction( + native.processor, FIXTURE_TEXT, FIXTURE_SCHEMA, args.length, args.max_words, args.max_queries + ) + tensors = tuple(torch.from_numpy(value) for value in arrays.values()) + features_wrapper = ExtractionFeaturesExport(native).eval() + with torch.no_grad(), coreml_trace_patches(): + features_reference = features_wrapper(*tensors) + traced_features = torch.jit.trace(features_wrapper, tensors, check_trace=False) + with torch.no_grad(): + native_core = native._encode_core(batch) + valid_words = native_core["text_states"].shape[1] + valid_queries = native_core["query_states"].shape[1] + routing_error = max( + float((features_reference[0][:, :valid_words] - native_core["text_states"]).abs().max()), + float((features_reference[1][:, :valid_queries] - native_core["query_states"]).abs().max()), + ) + if routing_error > 1e-4: + raise RuntimeError(f"Traced routing differs from native: {routing_error}") + + precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32 + input_names = tuple(arrays) + features_model = ct.convert( + traced_features, + convert_to="mlprogram", + minimum_deployment_target=ct.target.iOS17, + compute_precision=precision, + compute_units=ct.ComputeUnit.CPU_ONLY, + inputs=[ct.TensorType(name=name, shape=arrays[name].shape, dtype=arrays[name].dtype) for name in input_names], + outputs=[ct.TensorType(name=name, dtype=np.float32) for name in FEATURE_NAMES], + ) + features_model.short_description = "GLiNER2.5 base trained boundary extraction features" + features_model.author = "Fastino (original); Fluid Inference (Core ML conversion)" + features_model.license = "Apache-2.0" + features_model.user_defined_metadata.update( + { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "stage": "extraction features and trained boundary heads", + "subword_capacity": str(args.length), + "word_capacity": str(args.max_words), + "query_capacity": str(args.max_queries), + } + ) + out = Path(args.output_dir) + out.mkdir(parents=True, exist_ok=True) + shutil.copy2(Path(source) / "config.json", out / "config.json") + tokenizer_dir = out / "tokenizer" + tokenizer_dir.mkdir(exist_ok=True) + for name in ("tokenizer.json", "tokenizer_config.json"): + shutil.copy2(Path(source) / name, tokenizer_dir / name) + suffix = f"{args.precision}_L{args.length}_W{args.max_words}_Q{args.max_queries}" + features_path = out / f"gliner2_base_extraction_features_{suffix}.mlpackage" + if features_path.exists(): + shutil.rmtree(features_path) + features_model.save(str(features_path)) + print(f"Saved {features_path}", flush=True) + + head = native.boundary_head + pooled = select_candidates( + features_reference[8], + features_reference[9], + features_reference[3].bool(), + tensors[5].bool(), + features_reference[4], + features_reference[5], + boundary_top_k=head.shared_pool_builder.pool_boundary_top_k, + pool_size=head.shared_pool_builder.pool_size, + min_pool_per_query=head.shared_pool_builder.min_pool_per_query, + ) + score_tensors = ( + features_reference[0], + tensors[3], + features_reference[1], + tensors[5], + features_reference[2], + features_reference[4], + features_reference[5], + features_reference[6], + features_reference[7], + pooled.indices.int(), + pooled.mask.float(), + pooled.compat_logits, + ) + scorer_wrapper = ExtractionScoreExport(native).eval() + with torch.no_grad(): + scores_reference = scorer_wrapper(*score_tensors) + traced_scores = torch.jit.trace(scorer_wrapper, score_tensors, check_trace=False) + scorer_model = ct.convert( + traced_scores, + convert_to="mlprogram", + minimum_deployment_target=ct.target.iOS17, + compute_precision=precision, + compute_units=ct.ComputeUnit.CPU_ONLY, + inputs=[ + ct.TensorType( + name=name, shape=tuple(value.shape), dtype=np.int32 if name == "candidate_indices" else np.float32 + ) + for name, value in zip(SCORE_INPUT_NAMES, score_tensors) + ], + outputs=[ + ct.TensorType(name="pair_logits", dtype=np.float32), + ct.TensorType(name="candidate_states", dtype=np.float32), + ], + ) + scorer_model.short_description = "GLiNER2.5 base trained shared-pool extraction scorer" + scorer_model.author = "Fastino (original); Fluid Inference (Core ML conversion)" + scorer_model.license = "Apache-2.0" + scorer_model.user_defined_metadata.update( + { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "stage": "trained extraction candidate scorer", + "candidate_capacity": str(head.shared_pool_builder.pool_size), + } + ) + scorer_path = out / f"gliner2_base_extraction_scorer_{suffix}.mlpackage" + if scorer_path.exists(): + shutil.rmtree(scorer_path) + scorer_model.save(str(scorer_path)) + print(f"Saved {scorer_path}", flush=True) + + # The first runtime check uses the same selected real fixture as the trace. + runtime_features = ct.models.MLModel(str(features_path), compute_units=ct.ComputeUnit.CPU_ONLY) + predicted_features = runtime_features.predict(arrays) + errors = { + name: float(np.max(np.abs(np.asarray(predicted_features[name]) - reference.detach().numpy()))) + for name, reference in zip(FEATURE_NAMES, features_reference) + } + if any(not np.isfinite(value) for value in errors.values()): + raise RuntimeError("Extraction features contain non-finite values") + runtime_scorer = ct.models.MLModel(str(scorer_path), compute_units=ct.ComputeUnit.CPU_ONLY) + score_arrays = { + name: value.detach().numpy().astype(np.int32 if name == "candidate_indices" else np.float32) + for name, value in zip(SCORE_INPUT_NAMES, score_tensors) + } + predicted_scores = runtime_scorer.predict(score_arrays) + errors["pair_logits"] = float(np.max(np.abs(predicted_scores["pair_logits"] - scores_reference[0].numpy()))) + errors["candidate_states"] = float( + np.max(np.abs(predicted_scores["candidate_states"] - scores_reference[1].numpy())) + ) + if any(not np.isfinite(value) for value in errors.values()): + raise RuntimeError("Extraction scorer contains non-finite values") + report = { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "precision": args.precision, + "fixture": FIXTURE_TEXT, + "shape": { + "subwords": args.length, + "words": args.max_words, + "queries": args.max_queries, + "candidates": head.shared_pool_builder.pool_size, + }, + "routing_max_absolute_error": routing_error, + "runtime_max_absolute_errors": errors, + "packages": { + "features": {"path": str(features_path), "bytes": package_bytes(features_path)}, + "scorer": {"path": str(scorer_path), "bytes": package_bytes(scorer_path)}, + }, + "coremltools": ct.__version__, + "torch": torch.__version__, + } + (out / f"conversion-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps(report, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/convert-record-coreml.py b/convert-record-coreml.py new file mode 100644 index 0000000000000000000000000000000000000000..94b6f07410f3daa07d5328665a5d004d6e2b2f28 --- /dev/null +++ b/convert-record-coreml.py @@ -0,0 +1,222 @@ +"""Export GLiNER2.5 base trained record assignment and anchorless heads.""" + +import argparse +import json +import shutil +from pathlib import Path + +import coremltools as ct +import numpy as np +import torch +from gliner2 import AutoExtractor, Schema +from gliner2.training.trainer import ExtractorCollator +from huggingface_hub import snapshot_download + +from extraction_export import ExtractionRecordAnchorlessExport, ExtractionRecordAssignmentExport + +MODEL_ID = "fastino/gliner2.5-base-v1" +MODEL_REVISION = "1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +FIXTURE_TEXT = "Alice works at Acme. Bob works at Beta." + + +def record_fixture(native, mode): + schema = Schema() + builder = schema.structure("employment", mode=mode, anchor="person" if mode == "natural" else None) + builder.field("person", dtype="str") + builder.field("company", dtype="str") + batch = ExtractorCollator(native.processor, is_training=False, max_len=None, architecture="boundary")( + [(FIXTURE_TEXT, schema.build())] + ) + with torch.no_grad(): + core = native._encode_core(batch) + candidates = native.boundary_head( + core["text_states"], core["text_mask"], core["query_states"], core["query_mask"] + ).candidates + spec = next(iter(batch.record_specs[0].values())) + group = native.record_decoder.forward_group(spec, core["query_states"][0], candidates, 0) + field_states = [ + candidates.candidate_states[0, query_id][candidates.valid_mask[0, query_id]] + for query_id in group.field_query_ids + ] + queries = core["query_states"][0][group.field_query_ids] + return group, spec, field_states, queries + + +def pad_first(value, size: int): + if value.shape[0] > size: + raise ValueError(f"Real record fixture exceeds bucket capacity {size}") + result = value.new_zeros((size, *value.shape[1:])) + result[: value.shape[0]] = value + return result + + +def package_bytes(path): + return sum(file.stat().st_size for file in path.rglob("*") if file.is_file()) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--output-dir", default="build/extraction") + parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32") + parser.add_argument("--max-fields", type=int, default=8) + parser.add_argument("--max-candidates", type=int, default=192) + parser.add_argument("--max-instances", type=int, default=1536) + args = parser.parse_args() + torch.set_num_threads(4) + source = snapshot_download( + MODEL_ID, + revision=MODEL_REVISION, + allow_patterns=[ + "config.json", + "encoder_config/*", + "model.safetensors", + "tokenizer.json", + "tokenizer_config.json", + ], + ) + native = AutoExtractor.from_pretrained(source, map_location="cpu").eval() + head = native.record_decoder + if args.max_instances < args.max_fields * args.max_candidates: + raise ValueError("Instance bucket must hold all latent field candidates") + + group, spec, field_states, queries = record_fixture(native, "natural") + anchor_index = group.field_query_ids.index(spec.anchor_query_id) + instances = field_states[anchor_index] + hidden = instances.shape[-1] + field_candidates = torch.zeros(args.max_fields, args.max_candidates, hidden) + for field_index, states in enumerate(field_states): + if states.shape[0] > args.max_candidates: + raise ValueError("Record candidate count exceeds bucket") + field_candidates[field_index, : states.shape[0]] = states + assignment_args = (pad_first(instances, args.max_instances), pad_first(queries, args.max_fields), field_candidates) + assignment_wrapper = ExtractionRecordAssignmentExport(native).eval() + with torch.no_grad(): + assignment_reference = assignment_wrapper(*assignment_args) + for field_index, expected in enumerate(group.assign_logits): + actual = assignment_reference[0][: instances.shape[0], field_index, : expected.shape[1]] + if not torch.allclose(actual, expected, atol=1e-4): + raise RuntimeError("Record assignment wrapper differs from native") + assignment_trace = torch.jit.trace(assignment_wrapper, assignment_args, check_trace=False) + + precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32 + assignment_model = ct.convert( + assignment_trace, + convert_to="mlprogram", + minimum_deployment_target=ct.target.iOS17, + compute_precision=precision, + compute_units=ct.ComputeUnit.CPU_ONLY, + inputs=[ + ct.TensorType(name=name, shape=tuple(value.shape), dtype=np.float32) + for name, value in zip(("instance_states", "field_queries", "field_candidate_states"), assignment_args) + ], + outputs=[ + ct.TensorType(name="assignment_logits", dtype=np.float32), + ct.TensorType(name="object_logits", dtype=np.float32), + ct.TensorType(name="latent_seed_logits", dtype=np.float32), + ], + ) + assignment_model.short_description = "GLiNER2.5 base trained record assignment and object heads" + assignment_model.author = "Fastino (original); Fluid Inference (Core ML conversion)" + assignment_model.license = "Apache-2.0" + assignment_model.user_defined_metadata.update( + { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "stage": "trained record assignment, object and latent seed heads", + "field_capacity": str(args.max_fields), + "candidate_capacity": str(args.max_candidates), + "instance_capacity": str(args.max_instances), + } + ) + out = Path(args.output_dir) + out.mkdir(parents=True, exist_ok=True) + suffix = f"{args.precision}_F{args.max_fields}_C{args.max_candidates}_I{args.max_instances}" + assignment_path = out / f"gliner2_base_record_assignment_{suffix}.mlpackage" + if assignment_path.exists(): + shutil.rmtree(assignment_path) + assignment_model.save(str(assignment_path)) + assignment_runtime = ct.models.MLModel(str(assignment_path), compute_units=ct.ComputeUnit.CPU_ONLY) + assignment_prediction = assignment_runtime.predict( + { + name: value.numpy().astype(np.float32) + for name, value in zip(("instance_states", "field_queries", "field_candidate_states"), assignment_args) + } + ) + assignment_errors = { + name: float(np.max(np.abs(assignment_prediction[name] - expected.numpy()))) + for name, expected in zip(("assignment_logits", "object_logits", "latent_seed_logits"), assignment_reference) + } + + _, _, anchorless_field_states, _ = record_fixture(native, "anchorless") + context = torch.cat(anchorless_field_states, 0) + context_size = args.max_fields * args.max_candidates + context_states = pad_first(context, context_size) + context_mask = torch.zeros(context_size, dtype=torch.float32) + context_mask[: context.shape[0]] = 1.0 + anchorless_wrapper = ExtractionRecordAnchorlessExport(native).eval() + with torch.no_grad(): + anchorless_reference = anchorless_wrapper(context_states, context_mask) + native_states = head._anchorless_states(anchorless_field_states) + wrapper_error = float((anchorless_reference - native_states).abs().max()) + if wrapper_error > 1e-4: + raise RuntimeError(f"Anchorless wrapper differs from native: {wrapper_error}") + anchorless_trace = torch.jit.trace(anchorless_wrapper, (context_states, context_mask), check_trace=False) + anchorless_model = ct.convert( + anchorless_trace, + convert_to="mlprogram", + minimum_deployment_target=ct.target.iOS17, + compute_precision=precision, + compute_units=ct.ComputeUnit.CPU_ONLY, + inputs=[ + ct.TensorType(name="context_states", shape=tuple(context_states.shape), dtype=np.float32), + ct.TensorType(name="context_mask", shape=tuple(context_mask.shape), dtype=np.float32), + ], + outputs=[ct.TensorType(name="instance_states", dtype=np.float32)], + ) + anchorless_model.short_description = "GLiNER2.5 base trained anchorless record instance head" + anchorless_model.author = "Fastino (original); Fluid Inference (Core ML conversion)" + anchorless_model.license = "Apache-2.0" + anchorless_model.user_defined_metadata.update( + { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "stage": "trained anchorless record instance head", + "context_capacity": str(context_size), + } + ) + anchorless_path = out / f"gliner2_base_record_anchorless_{suffix}.mlpackage" + if anchorless_path.exists(): + shutil.rmtree(anchorless_path) + anchorless_model.save(str(anchorless_path)) + anchorless_runtime = ct.models.MLModel(str(anchorless_path), compute_units=ct.ComputeUnit.CPU_ONLY) + anchorless_prediction = anchorless_runtime.predict( + { + "context_states": context_states.numpy().astype(np.float32), + "context_mask": context_mask.numpy().astype(np.float32), + } + )["instance_states"] + anchorless_error = float(np.max(np.abs(anchorless_prediction - anchorless_reference.numpy()))) + if not all(np.isfinite(value) for value in (*assignment_errors.values(), anchorless_error)): + raise RuntimeError("Record head produced non-finite values") + report = { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "precision": args.precision, + "fixture": FIXTURE_TEXT, + "shape": {"fields": args.max_fields, "candidates": args.max_candidates, "instances": args.max_instances}, + "assignment_max_absolute_errors": assignment_errors, + "anchorless_wrapper_max_absolute_error": wrapper_error, + "anchorless_coreml_max_absolute_error": anchorless_error, + "packages": { + "assignment": {"path": str(assignment_path), "bytes": package_bytes(assignment_path)}, + "anchorless": {"path": str(anchorless_path), "bytes": package_bytes(anchorless_path)}, + }, + "coremltools": ct.__version__, + "torch": torch.__version__, + } + (out / f"record-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps(report, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/convert-relation-coreml.py b/convert-relation-coreml.py new file mode 100644 index 0000000000000000000000000000000000000000..3ab95531366ab0b085ef08c19946d90eff427ed9 --- /dev/null +++ b/convert-relation-coreml.py @@ -0,0 +1,165 @@ +"""Export the trained GLiNER2.5 base sparse relation scorer to Core ML.""" + +import argparse +import json +from pathlib import Path + +import coremltools as ct +import numpy as np +import torch +from gliner2 import AutoExtractor, Schema +from gliner2.models.base import QueryLayout +from gliner2.training.trainer import ExtractorCollator +from huggingface_hub import snapshot_download + +from extraction_export import ExtractionRelationExport + +MODEL_ID = "fastino/gliner2.5-base-v1" +MODEL_REVISION = "1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +INPUT_NAMES = ( + "text_states", + "text_length", + "relation_states", + "batch_index", + "relation_index", + "head_start", + "head_end", + "tail_start", + "tail_end", + "pair_mask", +) + + +def pad(value, size: int, fill=0): + if value.shape[0] > size: + raise ValueError(f"Relation fixture exceeds capacity {size}") + output = value.new_full((size, *value.shape[1:]), fill) + output[: value.shape[0]] = value + return output + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--output-dir", default="build/extraction") + parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32") + parser.add_argument("--max-words", type=int, default=64) + parser.add_argument("--max-relations", type=int, default=4) + args = parser.parse_args() + torch.set_num_threads(4) + source = snapshot_download( + MODEL_ID, + revision=MODEL_REVISION, + allow_patterns=[ + "config.json", + "encoder_config/*", + "model.safetensors", + "tokenizer.json", + "tokenizer_config.json", + ], + ) + native = AutoExtractor.from_pretrained(source, map_location="cpu").eval() + text = "Alice founded Acme in Toronto." + schema = Schema().relations(["founded"]) + batch = ExtractorCollator(native.processor, is_training=False, max_len=None, architecture="boundary")( + [(text, schema.build())] + ) + with torch.no_grad(): + core = native._encode_core(batch) + output = native.boundary_head(core["text_states"], core["text_mask"], core["query_states"], core["query_mask"]) + sample = native._single_sample_candidates(output.candidates, 0) + relation_specs = core["rel_specs"][0] + pairs = native.relation_pair_generator.generate_batched( + sample, [QueryLayout(queries=())], [[entry["spec"] for entry in relation_specs]], compact=False + ) + relation_states = torch.stack([entry["query_state"] for entry in relation_specs]).unsqueeze(0) + native_scores = native.relation_scorer(core["text_states"], relation_states, sample, pairs) + pair_cap = args.max_relations * native.boundary_settings.relation_pair_cap + text_states = torch.zeros(1, args.max_words, core["text_states"].shape[-1]) + text_states[:, : core["text_states"].shape[1]] = core["text_states"] + relation_padded = torch.zeros(1, args.max_relations, relation_states.shape[-1]) + relation_padded[:, : relation_states.shape[1]] = relation_states + arguments = ( + text_states, + torch.tensor([core["text_states"].shape[1]], dtype=torch.int32), + relation_padded, + pad(pairs.batch_index.int(), pair_cap), + pad(pairs.relation_index.int(), pair_cap), + pad(pairs.head_start.int(), pair_cap), + pad(pairs.head_end.int(), pair_cap), + pad(pairs.tail_start.int(), pair_cap), + pad(pairs.tail_end.int(), pair_cap), + pad(pairs.pair_mask.float(), pair_cap), + ) + wrapper = ExtractionRelationExport(native).eval() + with torch.no_grad(): + reference = wrapper(*arguments) + wrapper_error = float((reference[: len(pairs)] - native_scores).abs().max()) + traced = torch.jit.trace(wrapper, arguments, check_trace=False) + if wrapper_error > 1e-4: + raise RuntimeError(f"Relation wrapper differs from native: {wrapper_error}") + precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32 + converted = ct.convert( + traced, + convert_to="mlprogram", + minimum_deployment_target=ct.target.iOS17, + compute_precision=precision, + compute_units=ct.ComputeUnit.CPU_ONLY, + inputs=[ + ct.TensorType( + name=name, + shape=tuple(value.shape), + dtype=np.float32 if name in ("text_states", "relation_states", "pair_mask") else np.int32, + ) + for name, value in zip(INPUT_NAMES, arguments) + ], + outputs=[ct.TensorType(name="relation_logits", dtype=np.float32)], + ) + converted.short_description = "GLiNER2.5 base trained sparse relation scoring head" + converted.author = "Fastino (original); Fluid Inference (Core ML conversion)" + converted.license = "Apache-2.0" + converted.user_defined_metadata.update( + { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "stage": "trained relation scorer", + "word_capacity": str(args.max_words), + "relation_capacity": str(args.max_relations), + "pair_capacity": str(pair_cap), + } + ) + out = Path(args.output_dir) + out.mkdir(parents=True, exist_ok=True) + suffix = f"{args.precision}_W{args.max_words}_R{args.max_relations}_P{pair_cap}" + package = out / f"gliner2_base_relation_{suffix}.mlpackage" + converted.save(str(package)) + model = ct.models.MLModel(str(package), compute_units=ct.ComputeUnit.CPU_ONLY) + prediction = model.predict( + { + name: value.numpy().astype( + np.float32 if name in ("text_states", "relation_states", "pair_mask") else np.int32 + ) + for name, value in zip(INPUT_NAMES, arguments) + } + )["relation_logits"] + runtime_error = float(np.max(np.abs(prediction[: len(pairs)] - reference.numpy()[: len(pairs)]))) + if not np.isfinite(runtime_error): + raise RuntimeError("Relation scorer produced non-finite values") + report = { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "precision": args.precision, + "fixture": text, + "valid_pairs": int(pairs.pair_mask.sum()), + "wrapper_max_absolute_error": wrapper_error, + "coreml_max_absolute_error": runtime_error, + "package": str(package), + "package_bytes": sum(file.stat().st_size for file in package.rglob("*") if file.is_file()), + "coremltools": ct.__version__, + "torch": torch.__version__, + } + (out / f"relation-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps(report, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/convert_extraction_names.py b/convert_extraction_names.py new file mode 100644 index 0000000000000000000000000000000000000000..1e199efd1299a3e6f0985b056dec87b2f5b31efd --- /dev/null +++ b/convert_extraction_names.py @@ -0,0 +1,31 @@ +"""Ordered tensor contract shared by extraction conversion and verification.""" + +FEATURE_NAMES = ( + "text_states", + "query_states", + "boundary_states", + "boundary_mask", + "start_logits", + "end_logits", + "inside_prefix", + "inside_prefix_mean", + "pool_start_projection", + "pool_end_projection", + "null_logits", + "count_log_rates", + "classification_logits", +) +SCORE_INPUT_NAMES = ( + "text_states", + "text_mask", + "query_states", + "query_mask", + "boundary_states", + "start_logits", + "end_logits", + "inside_prefix", + "inside_prefix_mean", + "candidate_indices", + "candidate_mask", + "candidate_compatibility", +) diff --git a/extraction-assets.lock.json b/extraction-assets.lock.json new file mode 100644 index 0000000000000000000000000000000000000000..f123bc9fa444744b201881d31492e76f7610c6b2 --- /dev/null +++ b/extraction-assets.lock.json @@ -0,0 +1,275 @@ +{ + "source_model": "fastino/gliner2.5-base-v1", + "source_revision": "1a8bc24e00dc7300b9017c81d63e3dcdabb26596", + "package_files": [ + { + "package": "gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage", + "bytes": 614010, + "files": [ + { + "path": "gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 61009, + "sha256": "833cfc75c7b45d95f423b00770569ad00707aad4d7e993d9288aacd6392901be" + }, + { + "path": "gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 552384, + "sha256": "b8f472bf52874cf6fbe92e08333cb1976236fe179a186efeba049f1814ca2f6f" + }, + { + "path": "gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "2f83e83160221b29ce2f9156518bfa6bdaefd94c7c1eabff8de06131374b41d6" + } + ] + }, + { + "package": "gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage", + "bytes": 1152468, + "files": [ + { + "path": "gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 48811, + "sha256": "6226a1f7b91ceb1b2528d5eac17fa7b30db7c6e921ebc4a8b1b7ad4b8dca9096" + }, + { + "path": "gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 1103040, + "sha256": "7dded10597f6d93d30f9842d963e6a1ed503a7e1ed53d6f713ee22da38101e1f" + }, + { + "path": "gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "18d42e32e177e558362500e951fdde1310afce5a11435e7a108264c184e42d54" + } + ] + }, + { + "package": "gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage", + "bytes": 390998448, + "files": [ + { + "path": "gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 1046087, + "sha256": "81a989bd9c9791ca2cb2b10c9456795a3f98cb2a41132ad894ec6aae3bee8c58" + }, + { + "path": "gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 389951744, + "sha256": "f89a14b93e79bc0c5f03ae1d2afdecabc12d0ba0a6551107efb60d19e2fef913" + }, + { + "path": "gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "5b86a3a9736fb9849f4cf6bcae5fd8038332c2ac4106f5ed5fc23a72c4b77ed4" + } + ] + }, + { + "package": "gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage", + "bytes": 780899033, + "files": [ + { + "path": "gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 1012396, + "sha256": "952aba39fb31af06ae240e1c03ce0a9fe065c2e5997d4f455fb0df2055059103" + }, + { + "path": "gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 779886020, + "sha256": "3edc72e18c141125c4231b7a3b0d7d0105135271f7deff924818951e6f354ef0" + }, + { + "path": "gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "5d9751b4961ccdb017b0e8a01e73920ced48c101a9845164a19741802dc43f8c" + } + ] + }, + { + "package": "gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage", + "bytes": 903154, + "files": [ + { + "path": "gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 42953, + "sha256": "a28d35b1697ca5877e4c37be3fc73636a6795c7a7fb589aadee053ecbc06d496" + }, + { + "path": "gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 859584, + "sha256": "63e7a5f12ea123730faf9248e3d5245d8861cda7bdc678a29b6ed10c1c0450a3" + }, + { + "path": "gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "25563e15f2ec7a511142b626c74007b9816de4fd3729395c2a40ca6da5e6a866" + } + ] + }, + { + "package": "gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage", + "bytes": 1750631, + "files": [ + { + "path": "gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 32574, + "sha256": "dc02f86228a4c95872de1d6dab5d5a1ca2d89c464ba1df0eb1f4333cc4f517f9" + }, + { + "path": "gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 1717440, + "sha256": "ab96965841dd7802bb22d3a4d65ac2ef27e4df4daf4be1a49a65d8bc3c87f595" + }, + { + "path": "gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "753acbe0781d89b5b157da08ce5d75d21f61b0c4f86a9ceafa21d3274df54259" + } + ] + }, + { + "package": "gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage", + "bytes": 1443015, + "files": [ + { + "path": "gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 6558, + "sha256": "3ae7e7e6540577572bc1ae82d334d9294dd2bfb2d270cdf5166edcb9ce487495" + }, + { + "path": "gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 1435840, + "sha256": "73f5d6812653173c71dcd666e9477dc275fae42290bb0bfa5aef00e1c721a7f5" + }, + { + "path": "gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "4b6d679bf502518e30cdc5e5a65154a6c37552937301ba9146c4a2057d13f7dd" + } + ] + }, + { + "package": "gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage", + "bytes": 2876843, + "files": [ + { + "path": "gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 4994, + "sha256": "fa573f67c4b36f80f02668d377abd244956d1afba07be7d55866080b8188b6ef" + }, + { + "path": "gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 2871232, + "sha256": "f48cb084768846394746b4a39d189a47be19392708075f854af3dfe8076db4cd" + }, + { + "path": "gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "f646f499b1c601271b51c1420dc21ffc68fc983e2a65d4377a2094a094992385" + } + ] + }, + { + "package": "gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage", + "bytes": 605158, + "files": [ + { + "path": "gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 9981, + "sha256": "5ff5970e8851aeae902aac01e4c4586d453b0f39797f342d9a9b2a2791d7e605" + }, + { + "path": "gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 594560, + "sha256": "d59545d5a9df722194e38a0a3ee03c6a5f95c58fa9608f4b2f3f6e3f56d0136e" + }, + { + "path": "gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "365d8c13d2a88c83072fa7ec640919f817ad87b245c10caf1445cfde6e0dd040" + } + ] + }, + { + "package": "gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage", + "bytes": 1196559, + "files": [ + { + "path": "gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 7462, + "sha256": "63cad042b4d8487a0e6f57eec23a6a69f8687c33720c5213b8d52c1dd2169572" + }, + { + "path": "gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 1188480, + "sha256": "5f870796d2a8e0539b20964ebcd80cae65677ed11d4b36b12df9a21d158181c1" + }, + { + "path": "gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "991014787fc73007fc398cfe7c77537d241c730ca8363697f2e70478389b95f8" + } + ] + }, + { + "package": "gliner2_base_relation_fp16_W64_R4_P256.mlpackage", + "bytes": 11848707, + "files": [ + { + "path": "gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 32410, + "sha256": "e6562f5850272ffe0b38363da5a85a70492e170e547f58b008021b6a8aa375ec" + }, + { + "path": "gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 11815680, + "sha256": "1577561a8686c9657ec38df97ff701cbc7b588da3efae6404e33622ab1cc77e5" + }, + { + "path": "gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "a79be16e25f1d7b5a261a8a2f1a5cd7dea41fc2d18e755338bab5dabf024662c" + } + ] + }, + { + "package": "gliner2_base_relation_fp32_W64_R4_P256.mlpackage", + "bytes": 23661032, + "files": [ + { + "path": "gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel", + "bytes": 29823, + "sha256": "5970ae822c86e3f11ef86341034824661fa10184730953ac4bcdb7086e87a6ea" + }, + { + "path": "gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin", + "bytes": 23630592, + "sha256": "1dd7c8e640688002c174080710cc5ba3fa32c5dc399b52ef0bf7f4c5ef8d01cb" + }, + { + "path": "gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Manifest.json", + "bytes": 617, + "sha256": "22d5d1c3db4225665d32ae55f8266c9c3ff6980bfe7c60db655529b8dfefbba9" + } + ] + } + ], + "runtime_files": [ + { + "path": "config.json", + "bytes": 3150, + "sha256": "0eb92d00584d613aab32b2178f84a85176b62c87ae3689ce9084e83f6eba64d1" + }, + { + "path": "tokenizer/tokenizer_config.json", + "bytes": 645, + "sha256": "0bf3ea0873234bd9bfdd3853c440395009ac6365a925b91654daed5396d655e1" + }, + { + "path": "tokenizer/tokenizer.json", + "bytes": 8341713, + "sha256": "cbc8ae6037812709c9c26f2a160f8dc48b0440bcb79c8141804259ae2d6adac3" + } + ] +} diff --git a/extraction_export.py b/extraction_export.py new file mode 100644 index 0000000000000000000000000000000000000000..8f52f4e7091b60ac0fb1bdce9eb503d9110c734b --- /dev/null +++ b/extraction_export.py @@ -0,0 +1,408 @@ +"""Core ML graph wrappers for GLiNER2.5's trained boundary extraction path.""" + +import math +from contextlib import contextmanager + +import torch +from coremltools.converters.mil import Builder as mb +from coremltools.converters.mil.frontend.torch.ops import _get_inputs +from coremltools.converters.mil.frontend.torch.torch_op_registry import register_torch_op +from coremltools.converters.mil.mil import types +from gliner2.models.boundary import encoding, heads +from gliner2.models.boundary.pool import PooledCandidates +from gliner2.models.boundary.proposal import BoundaryProposals +from transformers.models.deberta_v2 import modeling_deberta_v2 + +from export_model import coreml_safe_attention_forward + + +@register_torch_op(override=True) +def clamp_min(context, node): + """Preserve the tensor dtype when TorchScript supplied a Python scalar.""" + x, y = _get_inputs(context, node, expected=2) + if x.dtype != y.dtype: + y = mb.cast(x=y, dtype=types.builtin_to_string(x.dtype)) + context.add(mb.maximum(x=x, y=y, name=node.name)) + + +@register_torch_op(torch_alias=["clip"], override=True) +def clamp(context, node): + """Avoid promoting integer span indices to float for an absent bound.""" + inputs = _get_inputs(context, node, expected=[1, 2, 3]) + x = inputs[0] + lower = inputs[1] if len(inputs) > 1 and inputs[1] is not None else None + upper = inputs[2] if len(inputs) > 2 and inputs[2] is not None else None + result = x + for bound, op in ((upper, mb.minimum), (lower, mb.maximum)): + if bound is None: + continue + if bound.dtype != x.dtype: + bound = mb.cast(x=bound, dtype=types.builtin_to_string(x.dtype)) + result = op(x=result, y=bound) + context.add(mb.identity(x=result, name=node.name)) + + +def shift_left(text_states, bos_state): + """Functional equivalent of the upstream in-place BOS placement.""" + bos = bos_state.to(text_states.dtype).view(1, 1, -1) + return torch.cat((bos.expand(text_states.shape[0], 1, -1), text_states), 1) + + +def shift_right(text_states, text_lengths, eos_state): + """Functional equivalent of the upstream in-place EOS placement.""" + batch, length, hidden = text_states.shape + eos = eos_state.to(text_states.dtype).view(1, 1, hidden) + right = torch.cat((text_states, eos.expand(batch, 1, hidden)), 1) + positions = torch.arange(length + 1, device=text_states.device).view(1, length + 1, 1) + return torch.where( + positions == text_lengths.view(batch, 1, 1), + eos.expand(batch, length + 1, hidden), + right, + ) + + +def safe_boundary_attention(self, states, mask): + """Explicit scaled attention with the same finite masked result as upstream.""" + batch, length, dim = states.shape + qkv = self.qkv_projection(self.norm(states)).view(batch, length, 3, self.num_heads, self.head_dim) + query, key, value = qkv.permute(2, 0, 3, 1, 4) + allowed = mask.view(batch, 1, 1, length).expand(batch, 1, length, length) + if self.window > 0: + positions = torch.arange(length, device=states.device) + local = (positions.view(length, 1) - positions.view(1, length)).abs() <= self.window + allowed = allowed & local.view(1, 1, length, length) + diagonal = torch.eye(length, dtype=torch.bool, device=states.device).view(1, 1, length, length) + allowed = (allowed.float() + diagonal.float()) > 0.5 + scores = torch.matmul(query, key.transpose(-1, -2)) / math.sqrt(self.head_dim) + scores = scores.masked_fill(~allowed, -1e4) + attended = torch.matmul(torch.softmax(scores, dim=-1), value) + attended = attended.transpose(1, 2).reshape(batch, length, dim) + return (states + self.dropout(self.output_projection(attended))) * mask.unsqueeze(-1).to(states.dtype) + + +def safe_query_head(self, boundary, boundary_mask, text, text_mask, query, query_mask): + """Upstream marginals with a dtype-safe count clamp for Core ML.""" + scale = 1.0 / math.sqrt(self.boundary_dim) + start = ( + torch.einsum( + "bld,bqd->bql", + self.dropout(self.start_boundary_projection(boundary)), + self.start_query_projection(query), + ) + * scale + ) + end = ( + torch.einsum( + "bld,bqd->bql", + self.dropout(self.end_boundary_projection(boundary)), + self.end_query_projection(query), + ) + * scale + ) + inside = ( + torch.einsum( + "bld,bqd->bql", + self.dropout(self.inside_text_projection(text)), + self.inside_query_projection(query), + ) + * scale + ) + boundary_keep = boundary_mask.unsqueeze(1) & query_mask.unsqueeze(-1) + text_keep = text_mask.unsqueeze(1) & query_mask.unsqueeze(-1) + start = heads._masked_fill_min(start, boundary_keep) + end = heads._masked_fill_min(end, boundary_keep) + inside = heads._masked_fill_min(inside, text_keep) + inside_for_prefix = inside.masked_fill(~text_keep, 0.0).float() + count = torch.clamp(text_keep.sum(-1, keepdim=True).float(), min=1.0) + mean = (inside_for_prefix.sum(-1, keepdim=True) / count).detach() + centered = (inside_for_prefix - mean) * text_keep.to(inside_for_prefix.dtype) + zeros = torch.zeros(centered.shape[0], centered.shape[1], 1, dtype=torch.float32, device=text.device) + prefix = torch.cat((zeros, centered.cumsum(dim=-1)), dim=-1) + return heads.BoundaryMarginals(start, end, inside, prefix, mean) + + +@contextmanager +def coreml_trace_patches(): + """Apply and restore mathematically equivalent trace-safe operations.""" + saved = ( + modeling_deberta_v2.scaled_size_sqrt, + modeling_deberta_v2.build_rpos, + modeling_deberta_v2.DisentangledSelfAttention.forward, + encoding.shift_left_with_bos, + encoding.shift_right_with_eos, + encoding.BoundaryAttentionBlock.forward, + heads.BoundaryQueryHead.forward, + ) + + def static_scale(query_layer, scale_factor): + value = math.sqrt(float(query_layer.shape[-1] * scale_factor)) + return torch.tensor(value, dtype=torch.float32, device=query_layer.device) + + modeling_deberta_v2.scaled_size_sqrt = static_scale + modeling_deberta_v2.build_rpos = lambda query, key, relative_pos, buckets, max_pos: relative_pos + modeling_deberta_v2.DisentangledSelfAttention.forward = coreml_safe_attention_forward + encoding.shift_left_with_bos = shift_left + encoding.shift_right_with_eos = shift_right + encoding.BoundaryAttentionBlock.forward = safe_boundary_attention + heads.BoundaryQueryHead.forward = safe_query_head + try: + yield + finally: + ( + modeling_deberta_v2.scaled_size_sqrt, + modeling_deberta_v2.build_rpos, + modeling_deberta_v2.DisentangledSelfAttention.forward, + encoding.shift_left_with_bos, + encoding.shift_right_with_eos, + encoding.BoundaryAttentionBlock.forward, + heads.BoundaryQueryHead.forward, + ) = saved + + +class ExtractionFeaturesExport(torch.nn.Module): + """Trained encoder, boundary marginals, pool projections and null/count heads.""" + + def __init__(self, native): + super().__init__() + self.encoder = native.encoder + self.head = native.boundary_head + self.classifier = native.classifier + + def forward( + self, + input_ids, + attention_mask, + text_indices, + text_mask, + query_indices, + query_mask, + cls_indices, + cls_mask, + ): + hidden = self.encoder(input_ids=input_ids.long(), attention_mask=attention_mask.long()).last_hidden_state + text_idx = text_indices.long().unsqueeze(-1).expand(-1, -1, hidden.shape[-1]) + query_idx = query_indices.long().unsqueeze(-1).expand(-1, -1, hidden.shape[-1]) + text = hidden.gather(1, text_idx) * text_mask.unsqueeze(-1) + query = hidden.gather(1, query_idx) * query_mask.unsqueeze(-1) + cls_idx = cls_indices.long().unsqueeze(-1).expand(-1, -1, hidden.shape[-1]) + classification_states = hidden.gather(1, cls_idx) + classification_logits = self.classifier(classification_states).squeeze(-1) + classification_logits = torch.where( + cls_mask > 0.5, classification_logits, torch.full_like(classification_logits, -1e4) + ) + tm, qm = text_mask > 0.5, query_mask > 0.5 + encoded = self.head.boundary_encoder(text, tm) + marginal = self.head.boundary_query_head(encoded.states, encoded.mask, text, tm, query, qm) + return ( + text, + query, + encoded.states, + encoded.mask.float(), + marginal.start_logits, + marginal.end_logits, + marginal.inside_prefix, + marginal.inside_prefix_mean, + self.head.shared_pool_builder.start_projection(encoded.states), + self.head.shared_pool_builder.end_projection(encoded.states), + self.head.null_projection(query).squeeze(-1), + self.head.count_head(query).squeeze(-1), + classification_logits, + ) + + +class ExtractionScoreExport(torch.nn.Module): + """Trained shared-pool reranker and record candidate state projection.""" + + def __init__(self, native): + super().__init__() + self.scorer = native.boundary_head.shared_pool_scorer + self.candidate_encoder = native.boundary_head.candidate_encoder + + def forward( + self, + text, + text_mask, + query, + query_mask, + boundary, + starts, + ends, + inside, + inside_mean, + indices, + pool_mask, + compatibility, + ): + tm, qm = text_mask > 0.5, query_mask > 0.5 + pooled = PooledCandidates(indices.long(), pool_mask > 0.5, None, None, compatibility) + score, _ = self.scorer( + boundary, + query, + qm, + pooled, + starts, + ends, + inside, + tm.sum(-1).long(), + text, + tm, + inside_prefix_mean=inside_mean, + ) + index = indices.long() + start_states = boundary.gather(1, index[..., 0].unsqueeze(-1).expand(-1, -1, boundary.shape[-1])) + end_states = boundary.gather(1, index[..., 1].unsqueeze(-1).expand(-1, -1, boundary.shape[-1])) + candidate_states = self.candidate_encoder(torch.cat((start_states, end_states), -1)) + candidate_states = candidate_states * pool_mask.unsqueeze(-1) + return score.transpose(1, 2), candidate_states + + +class ExtractionRelationExport(torch.nn.Module): + """The trained sparse relation scorer with tensor-only pair routing.""" + + def __init__(self, native): + super().__init__() + self.scorer = native.relation_scorer + + def forward( + self, + text, + text_length, + relation, + batch_index, + relation_index, + head_start, + head_end, + tail_start, + tail_end, + pair_mask, + ): + scorer = self.scorer + length = text.shape[1] + batch_valid = (batch_index >= 0) & (batch_index < text.shape[0]) + relation_valid = (relation_index >= 0) & (relation_index < relation.shape[1]) + valid = batch_valid & relation_valid & (pair_mask > 0.5) + batch = batch_index.long().clamp(0, text.shape[0] - 1) + rel_index = relation_index.long().clamp(0, relation.shape[1] - 1) + + def gather(position): + return text[batch, position.long().clamp(0, length - 1)] + + h_start = gather(head_start) + h_end = gather(head_end - 1) + t_start = gather(tail_start) + t_end = gather(tail_end - 1) + rel = relation[batch, rel_index] + delta = (tail_start - head_start).to(text.dtype) + order = torch.sign(delta).unsqueeze(-1) + distance = (delta.abs() / text_length.float().clamp_min(1.0)).unsqueeze(-1) + features = torch.cat((h_start, h_end, t_start, t_end, rel, order, distance), -1) + score = scorer.mlp(features).squeeze(-1) + if scorer.use_biaffine_content: + prefix = torch.cat( + (text.new_zeros(text.shape[0], 1, scorer.hidden_size), text.float().cumsum(1).to(text.dtype)), + dim=1, + ) + + def pool(start, end): + total = prefix[batch, end.long().clamp(0, length)] - prefix[batch, start.long().clamp(0, length)] + width = (end - start).clamp_min(1).unsqueeze(-1).to(total.dtype) + return total / width + + head_content = scorer.head_content_projection(pool(head_start, head_end)) + tail_content = scorer.tail_content_projection(pool(tail_start, tail_end)) + gate = torch.sigmoid(scorer.relation_content_gate(rel)) + biaffine = (head_content * gate * tail_content).sum(-1) / (scorer.hidden_size**0.5) + linear = scorer.content_linear(torch.cat((head_content, tail_content, rel), -1)).squeeze(-1) + score = score + biaffine + linear + return score.masked_fill(~valid, 0.0) + + +class ExtractionRecordAssignmentExport(torch.nn.Module): + """All trained natural/latent/anchorless object and field assignment layers.""" + + def __init__(self, native): + super().__init__() + self.head = native.record_decoder + + def forward(self, instances, field_queries, field_candidates): + head = self.head + instance_projection = head.inst_proj(instances) + field_projection = head.field_proj(field_queries) + query = instance_projection.unsqueeze(1) + field_projection.unsqueeze(0) + null_scores = torch.einsum("ifd,d->if", query, head.null_embed) + candidate_scores = torch.einsum("ifd,fcd->ifc", query, head.cand_proj(field_candidates)) + assignment = torch.cat((null_scores.unsqueeze(-1), candidate_scores), -1) + object_scores = head.object_head(instances).squeeze(-1) + latent_scores = head.latent_seed_head(instances).squeeze(-1) + return assignment, object_scores, latent_scores + + +class ExtractionRecordAnchorlessExport(torch.nn.Module): + """Trained learned-instance and contextual attention path for records.""" + + def __init__(self, native): + super().__init__() + self.head = native.record_decoder + + def forward(self, context_states, context_mask): + head = self.head + instances = head.instance_embed + query = head.q_proj(instances) + key = head.k_proj(context_states) + value = head.v_proj(context_states) + attention = torch.matmul(query, key.transpose(-1, -2)) / math.sqrt(head.record_dim) + attention = attention.masked_fill(context_mask.unsqueeze(0) < 0.5, -1e4) + pooled = torch.matmul(torch.softmax(attention, -1), value) + return instances + pooled * (context_mask.sum() > 0).to(pooled.dtype) + + +class ExtractionExplicitSpanExport(torch.nn.Module): + """Trained proposal prior and reranker for forced attribute/enum spans.""" + + def __init__(self, native): + super().__init__() + self.proposer = native.boundary_head.boundary_proposer + self.scorer = native.boundary_head.pair_scorer + + def forward( + self, + text, + text_mask, + query, + query_mask, + boundary, + starts, + ends, + inside, + inside_mean, + indices, + valid_mask, + ): + tm, qm = text_mask > 0.5, query_mask > 0.5 + idx = indices.long() + legal = ( + (idx[..., 0] >= 0) + & (idx[..., 1] > idx[..., 0]) + & (idx[..., 1] <= tm.sum(-1).view(-1, 1, 1)) + & qm.unsqueeze(-1) + & (valid_mask > 0.5) + ) + compatibility = self.proposer.score_explicit_pairs(boundary, query, idx, legal) + proposals = BoundaryProposals( + indices=idx, + logits=None, + valid_mask=legal, + compat_logits=compatibility, + ) + return self.scorer( + boundary, + query, + proposals, + starts, + ends, + inside, + tm.sum(-1).long(), + text, + tm, + inside_prefix_mean=inside_mean, + ) diff --git a/extraction_pool.py b/extraction_pool.py new file mode 100644 index 0000000000000000000000000000000000000000..58632239a0fdcf8115f99c6e0f3cc0c5ca622d8a --- /dev/null +++ b/extraction_pool.py @@ -0,0 +1,117 @@ +"""Weight-free GLiNER2 boundary candidate selection between Core ML stages. + +The learned start/end projections are outputs of the first Core ML stage. This +module preserves GLiNER2 2.0.0's stable ranking and deduplication on the host. +""" + +import math + +import torch +from gliner2.models.boundary.constants import MASK_LOGIT +from gliner2.models.boundary.indexing import gather_rows +from gliner2.models.boundary.pool import PooledCandidates, _deduplicate_pool +from gliner2.models.boundary.proposal import select_top_boundaries + + +def select_candidates( + start_projection: torch.Tensor, + end_projection: torch.Tensor, + boundary_mask: torch.Tensor, + query_mask: torch.Tensor, + start_logits: torch.Tensor, + end_logits: torch.Tensor, + *, + boundary_top_k: int, + pool_size: int, + min_pool_per_query: int, +) -> PooledCandidates: + """Select the native shared pool using already projected Core ML states.""" + batch, n_boundaries, dim = start_projection.shape + n_queries = query_mask.shape[1] + if end_projection.shape != start_projection.shape: + raise ValueError("Start and end projections must have the same shape") + if start_logits.shape != (batch, n_queries, n_boundaries): + raise ValueError("Start logits do not match boundary and query dimensions") + if end_logits.shape != start_logits.shape: + raise ValueError("End logits do not match start logits") + if boundary_mask.shape != (batch, n_boundaries) or query_mask.shape != (batch, n_queries): + raise ValueError("Boundary or query mask has an unexpected shape") + + floor = torch.full_like(start_logits, MASK_LOGIT) + valid_boundary = boundary_mask.unsqueeze(1) & query_mask.unsqueeze(-1) + union_start = torch.where(valid_boundary, start_logits, floor).amax(1) + union_end = torch.where(valid_boundary, end_logits, floor).amax(1) + union_valid = boundary_mask & query_mask.any(-1, keepdim=True) + _, starts, starts_valid = select_top_boundaries(union_start.unsqueeze(1), union_valid.unsqueeze(1), boundary_top_k) + _, ends, ends_valid = select_top_boundaries(union_end.unsqueeze(1), union_valid.unsqueeze(1), boundary_top_k) + starts, ends = starts[:, 0], ends[:, 0] + starts_valid, ends_valid = starts_valid[:, 0], ends_valid[:, 0] + n_starts, n_ends = starts.shape[1], ends.shape[1] + pair_start = starts.unsqueeze(-1).expand(batch, n_starts, n_ends).reshape(batch, -1) + pair_end = ends.unsqueeze(1).expand(batch, n_starts, n_ends).reshape(batch, -1) + pair_valid = ( + starts_valid.unsqueeze(-1) & ends_valid.unsqueeze(1) & (ends.unsqueeze(1) > starts.unsqueeze(-1)) + ).reshape(batch, -1) + + selected_start = gather_rows(start_projection, pair_start) + selected_end = gather_rows(end_projection, pair_end) + compatibility = (selected_start * selected_end).sum(-1) / math.sqrt(dim) + union_pair_score = ( + compatibility + + union_start.gather(1, pair_start.clamp(0, n_boundaries - 1)) + + union_end.gather(1, pair_end.clamp(0, n_boundaries - 1)) + ) + + quota = min(min_pool_per_query, pair_start.shape[-1]) + quota_keys = pair_start.new_zeros((batch, 0)) + quota_scores = union_pair_score.new_zeros((batch, 0)) + quota_valid = pair_valid.new_zeros((batch, 0)) + if quota: + start_idx = pair_start.clamp(0, start_logits.shape[2] - 1).unsqueeze(1).expand(batch, n_queries, -1) + end_idx = pair_end.clamp(0, end_logits.shape[2] - 1).unsqueeze(1).expand(batch, n_queries, -1) + per_query = start_logits.gather(2, start_idx) + end_logits.gather(2, end_idx) + compatibility.unsqueeze(1) + per_query_valid = pair_valid.unsqueeze(1) & query_mask.unsqueeze(-1) + ranked = torch.argsort( + per_query.masked_fill(~per_query_valid, MASK_LOGIT), + dim=-1, + descending=True, + stable=True, + )[..., :quota] + quota_start = start_idx.gather(-1, ranked) + quota_end = end_idx.gather(-1, ranked) + quota_valid = per_query_valid.gather(-1, ranked).reshape(batch, -1) + quota_keys = (quota_start * n_boundaries + quota_end).reshape(batch, -1) + rank_bonus = torch.arange(quota, 0, -1, device=start_projection.device, dtype=union_pair_score.dtype) + quota_scores = ( + union_pair_score.new_full((batch, n_queries, quota), -MASK_LOGIT * 0.5) + rank_bonus.view(1, 1, quota) + ).reshape(batch, -1) + + global_keys = pair_start * n_boundaries + pair_end + all_keys = torch.cat((quota_keys, global_keys), -1) + all_scores = torch.cat((quota_scores, union_pair_score.detach()), -1) + all_valid = torch.cat((quota_valid, pair_valid), -1) + selected_keys, selected_valid = _deduplicate_pool(all_keys, all_scores, all_valid, pool_size, n_boundaries) + selected_keys = torch.where(selected_valid, selected_keys, torch.zeros_like(selected_keys)) + selected_s = torch.div(selected_keys, n_boundaries, rounding_mode="floor") + selected_e = selected_keys - selected_s * n_boundaries + indices = torch.stack((selected_s, selected_e), -1) + indices = torch.where(selected_valid.unsqueeze(-1), indices, torch.zeros_like(indices)) + + gathered_start = gather_rows(start_projection, selected_s) + gathered_end = gather_rows(end_projection, selected_e) + selected_compat = (gathered_start * gathered_end).sum(-1) / math.sqrt(dim) + selected_score = ( + selected_compat + + union_start.gather(1, selected_s.clamp(0, n_boundaries - 1)) + + union_end.gather(1, selected_e.clamp(0, n_boundaries - 1)) + ) + selected_score = selected_score.masked_fill(~selected_valid, MASK_LOGIT) + selected_compat = torch.where(selected_valid, selected_compat, torch.zeros_like(selected_compat)) + return PooledCandidates( + indices=indices, + mask=selected_valid, + proposal_logits=selected_score, + gold_mask=None, + compat_logits=selected_compat, + stats=None, + ) diff --git a/extraction_runtime.py b/extraction_runtime.py new file mode 100644 index 0000000000000000000000000000000000000000..7b904fedf9fabfa529059e052e5df49e9cd8dc0f --- /dev/null +++ b/extraction_runtime.py @@ -0,0 +1,525 @@ +"""GLiNER2.5 base extraction runtime using only Core ML trained weights.""" + +import json +from contextvars import ContextVar +from pathlib import Path + +import coremltools as ct +import numpy as np +import torch +from gliner2.configuration import BoundaryHeadSettings +from gliner2.models.boundary.engine import BoundaryExtractor +from gliner2.models.boundary.records import RecordGroupOutput +from gliner2.models.boundary.relations import ( + RelationProposalSettings, + RelationTypeSpec, + TypedRelationPairGenerator, +) +from gliner2.models.outputs import CandidateTensorBatch, ExtractorOutput + +from convert_extraction_names import FEATURE_NAMES, SCORE_INPUT_NAMES +from extraction_pool import select_candidates +from preprocessing import load_processor, prepare_extraction + +MODEL_PREFIX = "gliner2_base" + + +def to_tensor(value): + return torch.from_numpy(np.asarray(value).copy()) + + +def padded(value, size): + if value.shape[0] > size: + raise ValueError(f"Request exceeds Core ML bucket capacity {size}") + result = value.new_zeros((size, *value.shape[1:])) + result[: value.shape[0]] = value + return result + + +class CoreMLBoundaryHead(torch.nn.Module): + """Provide native decoder tensors from the converted extraction stages.""" + + def __init__(self, context: ContextVar, explicit_model, max_queries: int, max_spans: int): + super().__init__() + self.context = context + self.explicit_model = explicit_model + self.max_queries = max_queries + self.max_spans = max_spans + + def forward(self, text, text_mask, query, query_mask, return_candidates=True): + state = self.context.get() + features = state["features"] + full = state["candidates"] + count = query.shape[1] + selected = CandidateTensorBatch( + indices=full.indices[:, :count], + proposal_logits=full.proposal_logits[:, :count], + pair_logits=full.pair_logits[:, :count], + valid_mask=full.valid_mask[:, :count], + query_mask=full.query_mask[:, :count], + candidate_states=full.candidate_states[:, :count], + ) + return ExtractorOutput( + candidates=selected if return_candidates else None, + start_logits=features["start_logits"][:, :count], + end_logits=features["end_logits"][:, :count], + null_logits=features["null_logits"][:, :count], + count_log_rates=features["count_log_rates"][:, :count], + batch_size=1, + ) + + def score_explicit_spans(self, text, text_mask, query, query_mask, indices, valid_mask=None): + state = self.context.get() + features = state["features"] + full_queries = features["query_states"][0] + active_queries = int(state["arrays"]["query_mask"].sum()) + query_count = query.shape[1] + span_count = indices.shape[2] + if query_count > self.max_queries or span_count > self.max_spans: + raise ValueError("Explicit-span request exceeds Core ML bucket capacity") + selected = [] + for row in query[0]: + equal = torch.isclose(full_queries[:active_queries], row, atol=1e-6, rtol=0).all(-1) + matches = equal.nonzero(as_tuple=False).flatten() + if matches.numel() != 1: + raise ValueError("Explicit-span query cannot be mapped to the encoded schema") + selected.append(int(matches[0])) + selection = torch.tensor(selected, dtype=torch.long) + query_states = padded(query[0], self.max_queries).unsqueeze(0) + query_mask_padded = padded(query_mask[0].float(), self.max_queries).unsqueeze(0) + + def selected_feature(name): + source = features[name][0].index_select(0, selection) + return padded(source, self.max_queries).unsqueeze(0) + + span_indices = torch.zeros(1, self.max_queries, self.max_spans, 2, dtype=torch.int32) + span_mask = torch.zeros(1, self.max_queries, self.max_spans, dtype=torch.float32) + span_indices[:, :query_count, :span_count] = indices.int() + span_mask[:, :query_count, :span_count] = valid_mask.float() if valid_mask is not None else 1.0 + values = ( + text, + text_mask.float(), + query_states, + query_mask_padded, + features["boundary_states"], + selected_feature("start_logits"), + selected_feature("end_logits"), + selected_feature("inside_prefix"), + selected_feature("inside_prefix_mean"), + span_indices, + span_mask, + ) + names = ( + "text_states", + "text_mask", + "query_states", + "query_mask", + "boundary_states", + "start_logits", + "end_logits", + "inside_prefix", + "inside_prefix_mean", + "span_indices", + "span_mask", + ) + output = self.explicit_model.predict( + { + name: value.numpy().astype(np.int32 if name == "span_indices" else np.float32) + for name, value in zip(names, values) + } + )["span_logits"] + return to_tensor(output)[:, :query_count, :span_count] + + +class CoreMLRelationScorer(torch.nn.Module): + """Call the trained Core ML relation graph after native pair selection.""" + + def __init__(self, context: ContextVar, model, max_words: int, max_relations: int, pair_cap: int): + super().__init__() + self.context = context + self.model = model + self.max_words = max_words + self.max_relations = max_relations + self.pair_cap = pair_cap + + def forward(self, text, relation, candidates, pairs): + count = len(pairs) + if text.shape[1] != self.max_words or relation.shape[1] > self.max_relations or count > self.pair_cap: + raise ValueError("Relation request exceeds Core ML bucket capacity") + relation_states = torch.zeros(1, self.max_relations, relation.shape[-1], dtype=relation.dtype) + relation_states[:, : relation.shape[1]] = relation + state = self.context.get() + text_length = int(state["arrays"]["text_mask"].sum()) + fields = ( + text, + torch.tensor([text_length], dtype=torch.int32), + relation_states, + padded(pairs.batch_index.int(), self.pair_cap), + padded(pairs.relation_index.int(), self.pair_cap), + padded(pairs.head_start.int(), self.pair_cap), + padded(pairs.head_end.int(), self.pair_cap), + padded(pairs.tail_start.int(), self.pair_cap), + padded(pairs.tail_end.int(), self.pair_cap), + padded(pairs.pair_mask.float(), self.pair_cap), + ) + names = ( + "text_states", + "text_length", + "relation_states", + "batch_index", + "relation_index", + "head_start", + "head_end", + "tail_start", + "tail_end", + "pair_mask", + ) + output = self.model.predict( + { + name: value.numpy().astype( + np.float32 if name in ("text_states", "relation_states", "pair_mask") else np.int32 + ) + for name, value in zip(names, fields) + } + )["relation_logits"] + return to_tensor(output)[:count] + + +class CoreMLRecordHead(torch.nn.Module): + """Keep GLiNER2's instance ordering while running learned layers in Core ML.""" + + def __init__(self, assignment_model, anchorless_model, max_fields=8, max_candidates=192, max_instances=1536): + super().__init__() + self.assignment_model = assignment_model + self.anchorless_model = anchorless_model + self.max_fields = max_fields + self.max_candidates = max_candidates + self.max_instances = max_instances + + def forward_group(self, spec, query_states, candidates, sample_index): + field_specs = list(spec.fields) + field_query_ids = [field.query_id for field in field_specs] + if len(field_specs) > self.max_fields: + raise ValueError("Record has more fields than the Core ML bucket") + query_count = query_states.shape[0] + if any(query_id < 0 or query_id >= query_count for query_id in field_query_ids): + raise ValueError("Record field query is outside the encoded schema") + field_states, field_spans, field_logits, field_masks = [], [], [], [] + for query_id in field_query_ids: + mask = candidates.valid_mask[sample_index, query_id] + if int(mask.sum()) > self.max_candidates: + raise ValueError("Record candidate count exceeds Core ML bucket") + field_states.append(candidates.candidate_states[sample_index, query_id][mask]) + field_spans.append(candidates.indices[sample_index, query_id][mask]) + field_logits.append(candidates.pair_logits[sample_index, query_id][mask]) + field_masks.append(torch.ones(int(mask.sum()), dtype=torch.bool)) + + instance_seed = [] + instance_spans = [] + if spec.mode == "natural": + anchor = field_query_ids.index(spec.anchor_query_id) + instances = field_states[anchor] + for index, span in enumerate(field_spans[anchor]): + instance_seed.append((anchor, index)) + instance_spans.append((int(span[0]), int(span[1]))) + elif spec.mode == "latent": + instances = ( + torch.cat(field_states, 0) if field_states else query_states.new_zeros((0, query_states.shape[-1])) + ) + for field_index, spans in enumerate(field_spans): + for index, span in enumerate(spans): + instance_seed.append((field_index, index)) + instance_spans.append((int(span[0]), int(span[1]))) + else: + context_states = ( + torch.cat(field_states, 0) if field_states else query_states.new_zeros((0, query_states.shape[-1])) + ) + context_capacity = self.max_fields * self.max_candidates + context_mask = torch.zeros(context_capacity, dtype=torch.float32) + context_mask[: context_states.shape[0]] = 1.0 + output = self.anchorless_model.predict( + { + "context_states": padded(context_states, context_capacity).numpy().astype(np.float32), + "context_mask": context_mask.numpy(), + } + )["instance_states"] + instances = to_tensor(output) + instance_seed = [None] * instances.shape[0] + instance_spans = [None] * instances.shape[0] + + count = instances.shape[0] + if count > self.max_instances: + raise ValueError("Record instance count exceeds Core ML bucket") + hidden = query_states.shape[-1] + candidate_states = torch.zeros(self.max_fields, self.max_candidates, hidden) + for index, states in enumerate(field_states): + candidate_states[index, : states.shape[0]] = states + fields = ( + padded(instances, self.max_instances), + padded(query_states[field_query_ids], self.max_fields), + candidate_states, + ) + predicted = self.assignment_model.predict( + { + name: value.numpy().astype(np.float32) + for name, value in zip(("instance_states", "field_queries", "field_candidate_states"), fields) + } + ) + assignment = to_tensor(predicted["assignment_logits"]) + assignment_by_field = [ + assignment[:count, field_index, : 1 + states.shape[0]] for field_index, states in enumerate(field_states) + ] + if spec.mode == "natural": + object_logits = field_logits[field_query_ids.index(spec.anchor_query_id)] + elif spec.mode == "latent": + object_logits = to_tensor(predicted["latent_seed_logits"])[:count] + else: + object_logits = to_tensor(predicted["object_logits"])[:count] + return RecordGroupOutput( + spec=spec, + object_logits=object_logits, + assign_logits=assignment_by_field, + field_query_ids=field_query_ids, + field_specs=field_specs, + field_spans=field_spans, + field_cand_mask=field_masks, + field_cand_logits=field_logits, + instance_seed=instance_seed, + instance_spans=instance_spans, + ) + + +class CoreMLBoundaryExtractor(BoundaryExtractor): + """Native GLiNER2 schema/decoder with all trained extraction heads in Core ML.""" + + def __init__(self, model_dir: str, precision: str = "fp32", compute_units=ct.ComputeUnit.CPU_ONLY): + if precision not in ("fp16", "fp32"): + raise ValueError("precision must be fp16 or fp32") + torch.nn.Module.__init__(self) + folder = Path(model_dir) + config = json.loads((folder / "config.json").read_text()) + self.boundary_settings = BoundaryHeadSettings(**config["boundary_head"]) + self.processor = load_processor(str(folder / "tokenizer")) + self.enable_records = self.boundary_settings.enable_records + self.enable_relations = self.boundary_settings.enable_relations + self.strict_extraction = True + self.length, self.max_words, self.max_queries = 128, 64, 8 + self._context = ContextVar("gliner2_coreml_extraction_context") + suffix = f"{precision}_L128_W64_Q8" + self.features_model = ct.models.MLModel( + str(folder / f"{MODEL_PREFIX}_extraction_features_{suffix}.mlpackage"), compute_units=compute_units + ) + self.scorer_model = ct.models.MLModel( + str(folder / f"{MODEL_PREFIX}_extraction_scorer_{suffix}.mlpackage"), compute_units=compute_units + ) + explicit = ct.models.MLModel( + str(folder / f"{MODEL_PREFIX}_explicit_{precision}_W64_Q8_S64.mlpackage"), compute_units=compute_units + ) + relation = ct.models.MLModel( + str(folder / f"{MODEL_PREFIX}_relation_{precision}_W64_R4_P256.mlpackage"), compute_units=compute_units + ) + assignment = ct.models.MLModel( + str(folder / f"{MODEL_PREFIX}_record_assignment_{precision}_F8_C192_I1536.mlpackage"), + compute_units=compute_units, + ) + anchorless = ct.models.MLModel( + str(folder / f"{MODEL_PREFIX}_record_anchorless_{precision}_F8_C192_I1536.mlpackage"), + compute_units=compute_units, + ) + self.boundary_head = CoreMLBoundaryHead(self._context, explicit, self.max_queries, 64) + self.relation_scorer = CoreMLRelationScorer(self._context, relation, self.max_words, 4, 256) + self.record_decoder = CoreMLRecordHead(assignment, anchorless) + self.relation_pair_generator = TypedRelationPairGenerator( + RelationProposalSettings( + heads_per_relation=self.boundary_settings.relation_heads_per_type, + tails_per_relation=self.boundary_settings.relation_tails_per_type, + pair_cap=self.boundary_settings.relation_pair_cap, + argument_threshold=self.boundary_settings.relation_argument_proposal_threshold, + ) + ) + + def _encode_core(self, batch): + features = self._context.get()["features"] + query_states = features["query_states"] + text_states = features["text_states"] + query_mask = to_tensor(self._context.get()["arrays"]["query_mask"]).bool() + query_count = int(query_mask.sum()) + query_states = query_states[:, :query_count] + query_mask = query_mask[:, :query_count] + text_mask = to_tensor(self._context.get()["arrays"]["text_mask"]).bool() + ext_specs, cls_specs, rel_specs, word_offsets = [], [], [], [] + for sample_index in range(len(batch)): + specs = [ + { + "group_index": item.task_index, + "field_index": item.role_index, + "task_type": item.task_type, + "task_name": item.task_name, + "field_name": item.role_name, + } + for item in batch.query_layouts[sample_index].queries + ] + ext_specs.append(specs) + classifications = [] + choice_offset = 0 + for group_index in range(batch.schema_counts[sample_index]): + if batch.task_types[sample_index][group_index] != "classifications": + continue + count = max(len(batch.schema_special_indices[sample_index][group_index]) - 1, 0) + schema_tokens = batch.schema_tokens_list[sample_index][group_index] + if count: + classifications.append( + { + "group_index": group_index, + "task_name": schema_tokens[2], + "schema_tokens": schema_tokens, + "group_embs": features["classification_logits"][ + sample_index, choice_offset : choice_offset + count + ], + } + ) + choice_offset += count + cls_specs.append(classifications) + word_offsets.append( + max(int(batch.text_word_counts[sample_index]) - len(batch.start_mappings[sample_index]), 0) + ) + groups = {} + for query_id, spec in enumerate(specs): + groups.setdefault(spec["group_index"], []).append(query_id) + relations = [] + for group_index, role_ids in groups.items(): + if batch.task_types[sample_index][group_index] != "relations" or len(role_ids) < 2: + continue + head_id, tail_id = role_ids[:2] + role_states = query_states[sample_index, [head_id, tail_id]] + state = ( + torch.cat((role_states[0], role_states[1]), -1) + if self.boundary_settings.directional_relation_states + else role_states.mean(0) + ) + relations.append( + { + "group_index": group_index, + "relation_type": specs[head_id]["task_name"], + "spec": RelationTypeSpec( + specs[head_id]["task_name"], head_query_ids=(head_id,), tail_query_ids=(tail_id,) + ), + "query_state": state, + } + ) + rel_specs.append(relations) + return { + "text_states": text_states, + "text_mask": text_mask, + "text_lengths": text_mask.sum(-1).long(), + "query_states": query_states, + "query_mask": query_mask, + "ext_specs": ext_specs, + "cls_specs": cls_specs, + "rel_specs": rel_specs, + "word_offsets": word_offsets, + } + + def _extract_classification_result(self, results, schema_name, schema, embs, schema_tokens, temperature=1.0): + cls_config = self._resolve_classification_config(schema_tokens[2], schema.get("classifications", [])) + if cls_config is None: + return + if temperature <= 0: + raise ValueError("Classification temperature must be positive") + logits = embs / temperature + activation = cls_config.get("class_act", "auto") + multi_label = cls_config.get("multi_label", False) + if activation == "sigmoid" or (activation != "softmax" and multi_label): + probabilities = torch.sigmoid(logits) + else: + probabilities = torch.softmax(logits, dim=-1) + labels = cls_config["labels"] + if multi_label: + threshold = cls_config.get("cls_threshold", 0.5) + chosen = [ + (labels[index], float(probabilities[index])) + for index in range(len(labels)) + if float(probabilities[index]) >= threshold + ] + if not chosen: + best = int(probabilities.argmax()) + chosen = [(labels[best], float(probabilities[best]))] + results[cls_config["task"]] = chosen + return + best = int(probabilities.argmax()) + results[cls_config["task"]] = (labels[best], float(probabilities[best])) + + def extract( + self, + text: str, + schema, + threshold: float = 0.5, + format_results: bool = True, + include_confidence: bool = False, + include_spans: bool = False, + overlap_policy=None, + ): + schema_dicts, metadata_list = self._build_schema_dicts_and_metadata([schema]) + if overlap_policy is not None: + metadata_list[0]["_overlap_policy"] = self._resolved_overlap_policy(overlap_policy) + arrays, batch = prepare_extraction( + self.processor, text, schema_dicts[0], self.length, self.max_words, self.max_queries + ) + predicted = self.features_model.predict(arrays) + features = {name: to_tensor(predicted[name]) for name in FEATURE_NAMES} + query_mask = to_tensor(arrays["query_mask"]).bool() + candidates = None + if bool(query_mask.any()): + head = self.boundary_settings + pool = select_candidates( + features["pool_start_projection"], + features["pool_end_projection"], + features["boundary_mask"].bool(), + query_mask, + features["start_logits"], + features["end_logits"], + boundary_top_k=head.pool_boundary_top_k, + pool_size=head.pool_size, + min_pool_per_query=head.min_pool_per_query, + ) + values = ( + features["text_states"], + to_tensor(arrays["text_mask"]), + features["query_states"], + to_tensor(arrays["query_mask"]), + features["boundary_states"], + features["start_logits"], + features["end_logits"], + features["inside_prefix"], + features["inside_prefix_mean"], + pool.indices.int(), + pool.mask.float(), + pool.compat_logits, + ) + scored = self.scorer_model.predict( + { + name: value.numpy().astype(np.int32 if name == "candidate_indices" else np.float32) + for name, value in zip(SCORE_INPUT_NAMES, values) + } + ) + candidates = CandidateTensorBatch( + indices=pool.indices.unsqueeze(1).expand(1, self.max_queries, -1, 2), + proposal_logits=pool.proposal_logits.unsqueeze(1).expand(1, self.max_queries, -1), + pair_logits=to_tensor(scored["pair_logits"]), + valid_mask=pool.mask.unsqueeze(1).expand(1, self.max_queries, -1), + query_mask=query_mask, + candidate_states=to_tensor(scored["candidate_states"]).unsqueeze(1).expand(1, self.max_queries, -1, -1), + ) + token = self._context.set({"arrays": arrays, "features": features, "candidates": candidates}) + try: + raw = self._extract_from_batch(batch, threshold, metadata_list, include_confidence, include_spans)[0] + if format_results: + return self.format_results( + raw, + include_confidence, + metadata_list[0].get("relation_order", []), + metadata_list[0].get("classification_tasks", []), + ) + return raw + finally: + self._context.reset(token) diff --git a/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..923170ac0fcf203802f68bcc803fe36caa598118 --- /dev/null +++ b/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:833cfc75c7b45d95f423b00770569ad00707aad4d7e993d9288aacd6392901be +size 61009 diff --git a/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..dfb0739d29df4a39cfdbe7a43028a3f0f0293a8f --- /dev/null +++ b/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8f472bf52874cf6fbe92e08333cb1976236fe179a186efeba049f1814ca2f6f +size 552384 diff --git a/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Manifest.json b/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..3135184d81be7cd2739d6c0e1bc4a21d40a33090 --- /dev/null +++ b/gliner2_base_explicit_fp16_W64_Q8_S64.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "D726DDD0-C257-44BB-B2AB-E6F00C6422DA": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "EA5D631C-0A1C-45BC-BA64-A5F37269708E": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "EA5D631C-0A1C-45BC-BA64-A5F37269708E" +} diff --git a/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..9415a168180f40a440d6374cd073d41d7bfc3e0e --- /dev/null +++ b/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6226a1f7b91ceb1b2528d5eac17fa7b30db7c6e921ebc4a8b1b7ad4b8dca9096 +size 48811 diff --git a/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..afab03915cbdf6758063515b30c696e6d8bf6720 --- /dev/null +++ b/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7dded10597f6d93d30f9842d963e6a1ed503a7e1ed53d6f713ee22da38101e1f +size 1103040 diff --git a/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Manifest.json b/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..4cc4aa64e51652f21ddbe2e05f1a51a99e3f86bd --- /dev/null +++ b/gliner2_base_explicit_fp32_W64_Q8_S64.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "7CEC5B51-AEEB-4F06-BB27-342E29DF9350": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "AD3570EB-1FD4-4674-9D29-2A3B6B2083D7": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "7CEC5B51-AEEB-4F06-BB27-342E29DF9350" +} diff --git a/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..2b53c5a57bdffa730a3e308d6ef65fc3a5b33fa0 --- /dev/null +++ b/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81a989bd9c9791ca2cb2b10c9456795a3f98cb2a41132ad894ec6aae3bee8c58 +size 1046087 diff --git a/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..4aca6aa68dbbc9a72eb79c172d03260c67d4f842 --- /dev/null +++ b/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f89a14b93e79bc0c5f03ae1d2afdecabc12d0ba0a6551107efb60d19e2fef913 +size 389951744 diff --git a/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Manifest.json b/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..be6a4f56097a77e8bc11dde1ab308b930e7fca14 --- /dev/null +++ b/gliner2_base_extraction_features_fp16_L128_W64_Q8.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "CE6FE6C9-1B05-44D1-86BE-D05BA3A90F48": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "FABF84D1-9ACF-40E8-AD85-5BFDFA7D831E": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "FABF84D1-9ACF-40E8-AD85-5BFDFA7D831E" +} diff --git a/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..a37bf755240ad154f3e4ee4efe954aa07671efc5 --- /dev/null +++ b/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:952aba39fb31af06ae240e1c03ce0a9fe065c2e5997d4f455fb0df2055059103 +size 1012396 diff --git a/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..4417a4eb5623c968bb83504da9d8873ba362b866 --- /dev/null +++ b/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3edc72e18c141125c4231b7a3b0d7d0105135271f7deff924818951e6f354ef0 +size 779886020 diff --git a/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Manifest.json b/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..533080cf4680a7d8a29c5b214d43f54e4c5e78aa --- /dev/null +++ b/gliner2_base_extraction_features_fp32_L128_W64_Q8.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "27195BA1-5D5E-4583-8356-AEF31C964DF5": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "B9ECF184-78AE-4CED-A667-63FF8B2342CE": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "B9ECF184-78AE-4CED-A667-63FF8B2342CE" +} diff --git a/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..b82ecd8ce5aa6d5f3a685b7ae4f2b762ff2819eb --- /dev/null +++ b/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a28d35b1697ca5877e4c37be3fc73636a6795c7a7fb589aadee053ecbc06d496 +size 42953 diff --git a/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..8e88abf2a71f90a81be4c139262a97422b556fab --- /dev/null +++ b/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63e7a5f12ea123730faf9248e3d5245d8861cda7bdc678a29b6ed10c1c0450a3 +size 859584 diff --git a/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Manifest.json b/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..856d9c55dd556c8eae4233d58f4c39f1858541fb --- /dev/null +++ b/gliner2_base_extraction_scorer_fp16_L128_W64_Q8.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "3FDDF70C-07D4-4EC0-B0F5-C918F277B00E": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "44AFC159-B1AA-41A7-B5E7-7A63C4CADFB4": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "3FDDF70C-07D4-4EC0-B0F5-C918F277B00E" +} diff --git a/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..86fc02689c56348959af5aeafdd7a91724db4384 --- /dev/null +++ b/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc02f86228a4c95872de1d6dab5d5a1ca2d89c464ba1df0eb1f4333cc4f517f9 +size 32574 diff --git a/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..90090a20c79e36adfd44f7b8a6adf1a0af67a2bc --- /dev/null +++ b/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab96965841dd7802bb22d3a4d65ac2ef27e4df4daf4be1a49a65d8bc3c87f595 +size 1717440 diff --git a/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Manifest.json b/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..cf5b4ef4d2576761f83439b8c5604f7769f13012 --- /dev/null +++ b/gliner2_base_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "39A0473F-D89D-43AB-A591-4D885B89F208": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "403A1040-2367-428B-A9F2-2E969F0B9FDC": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "403A1040-2367-428B-A9F2-2E969F0B9FDC" +} diff --git a/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..85e4c19c2db5c45f5c6fef0d139e3f08079cc277 --- /dev/null +++ b/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ae7e7e6540577572bc1ae82d334d9294dd2bfb2d270cdf5166edcb9ce487495 +size 6558 diff --git a/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..7170e57e8fd43103e5f81c9a2eea6dea6d2971e1 --- /dev/null +++ b/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:73f5d6812653173c71dcd666e9477dc275fae42290bb0bfa5aef00e1c721a7f5 +size 1435840 diff --git a/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Manifest.json b/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..4a4527d04dae7ae978f5178341acd3809af2ff84 --- /dev/null +++ b/gliner2_base_record_anchorless_fp16_F8_C192_I1536.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "299C8B61-4089-4454-B109-2D7002032011": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "63E77F60-B2F1-4E33-9505-B84C84607AC7": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "63E77F60-B2F1-4E33-9505-B84C84607AC7" +} diff --git a/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..a2cef7eca60da840129fbdb4999a0704c07e84e6 --- /dev/null +++ b/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa573f67c4b36f80f02668d377abd244956d1afba07be7d55866080b8188b6ef +size 4994 diff --git a/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..a73ce3c944f843d684c5470d16e88e6c1e810b4e --- /dev/null +++ b/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f48cb084768846394746b4a39d189a47be19392708075f854af3dfe8076db4cd +size 2871232 diff --git a/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Manifest.json b/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..08de4f73e951a4c558c66e0a50645a5b633e4a2a --- /dev/null +++ b/gliner2_base_record_anchorless_fp32_F8_C192_I1536.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "05BE39C5-5E8D-408A-802C-C14032032B68": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "8059378D-AEA5-4997-8297-940F2F714C91": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "8059378D-AEA5-4997-8297-940F2F714C91" +} diff --git a/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..c8f07dc497e477e0c1df8ae7a4bba7c67fe52a33 --- /dev/null +++ b/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ff5970e8851aeae902aac01e4c4586d453b0f39797f342d9a9b2a2791d7e605 +size 9981 diff --git a/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..9edc99994551e4117fa647c391d73703c73bc67a --- /dev/null +++ b/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d59545d5a9df722194e38a0a3ee03c6a5f95c58fa9608f4b2f3f6e3f56d0136e +size 594560 diff --git a/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Manifest.json b/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..43eac248f4eac65129117f39fa1e6327403aca3a --- /dev/null +++ b/gliner2_base_record_assignment_fp16_F8_C192_I1536.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "68A464FA-A24A-46B9-9356-80E6888F5D9D": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "A795C3EE-066E-4321-A479-969549184B3F": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "68A464FA-A24A-46B9-9356-80E6888F5D9D" +} diff --git a/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..7697a777369c556fd486ff7aa0936f56c859b8dd --- /dev/null +++ b/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63cad042b4d8487a0e6f57eec23a6a69f8687c33720c5213b8d52c1dd2169572 +size 7462 diff --git a/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..1d7338be029cea4bc7c7deb4910a2f700265beb3 --- /dev/null +++ b/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f870796d2a8e0539b20964ebcd80cae65677ed11d4b36b12df9a21d158181c1 +size 1188480 diff --git a/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Manifest.json b/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..3a9d1d6e119607621059f08a3b79c3b886ca7e94 --- /dev/null +++ b/gliner2_base_record_assignment_fp32_F8_C192_I1536.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "832C4AAB-6488-45A5-B0EB-42E172A71A5B": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "E0009128-6F7C-4DFF-8EE5-473660E36248": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "E0009128-6F7C-4DFF-8EE5-473660E36248" +} diff --git a/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..706076bb576c9ed7bbc073862c780e0fe6cfe19b --- /dev/null +++ b/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e6562f5850272ffe0b38363da5a85a70492e170e547f58b008021b6a8aa375ec +size 32410 diff --git a/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..5aa6e867cd58555c3d53b0b44d2aa369100c2bf4 --- /dev/null +++ b/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1577561a8686c9657ec38df97ff701cbc7b588da3efae6404e33622ab1cc77e5 +size 11815680 diff --git a/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Manifest.json b/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..02785554b9b04f5b33347057e11513883c6dc70f --- /dev/null +++ b/gliner2_base_relation_fp16_W64_R4_P256.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "21433269-C580-4889-9FA2-579489ECCCB1": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "6A4D1B00-0FCB-4AA0-B64F-43CF443BCA7C": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "6A4D1B00-0FCB-4AA0-B64F-43CF443BCA7C" +} diff --git a/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel b/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..3b09aa2401a29e34f72b366a5088490c3e90b524 --- /dev/null +++ b/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5970ae822c86e3f11ef86341034824661fa10184730953ac4bcdb7086e87a6ea +size 29823 diff --git a/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..b56e951e533519979dcaac07bdfb214d048e9d0d --- /dev/null +++ b/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1dd7c8e640688002c174080710cc5ba3fa32c5dc399b52ef0bf7f4c5ef8d01cb +size 23630592 diff --git a/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Manifest.json b/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..a8f8c0c72b7493c401d02aa1aed172bf148433e6 --- /dev/null +++ b/gliner2_base_relation_fp32_W64_R4_P256.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "1BDDDB06-D37B-4A40-BEAB-8A1B63278068": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "ACF6B577-6E60-42EF-B900-AA17259DF0D8": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "1BDDDB06-D37B-4A40-BEAB-8A1B63278068" +} diff --git a/preprocessing.py b/preprocessing.py index ed6744c8ffd0df122d5bc5db823beafac9444e89..b8674df27be40dc15a7b01afee3963c87250f6dd 100644 --- a/preprocessing.py +++ b/preprocessing.py @@ -1,4 +1,5 @@ """Native GLiNER2 schema preprocessing for a fixed Core ML bucket.""" + import numpy as np from gliner2 import Schema from gliner2.models.base import load_extractor_tokenizer @@ -10,11 +11,13 @@ def load_processor(tokenizer_dir: str): """Load only the tokenizer and schema formatter needed by the Core ML model.""" return SchemaTransformer(tokenizer=load_extractor_tokenizer(tokenizer_dir), token_pooling="first") + def native_batch(native, text: str, task: str, labels: list[str], length: int): schema = Schema().classification(task, labels) collator = ExtractorCollator(native.processor, is_training=False, max_len=length, architecture=native.architecture) return collator([(text, schema.build())]) + def prepare_classification(native, text: str, task: str, labels: list[str], length: int, max_options: int): return prepare_with_processor(native.processor, text, task, labels, length, max_options) @@ -41,3 +44,46 @@ def prepare_with_processor(processor, text: str, task: str, labels: list[str], l "marker_indices": indices.astype(np.int32), "marker_mask": mask.astype(np.float32), } + + +def prepare_extraction( + processor, + text: str, + schema, + length: int, + max_words: int, + max_queries: int, + max_choices: int = 8, +): + """Prepare an extractive schema without allowing upstream word truncation.""" + if min(length, max_words, max_queries, max_choices) < 1: + raise ValueError("Extraction bucket dimensions must all be positive") + built_schema = schema.build() if hasattr(schema, "build") else schema + collator = ExtractorCollator(processor, is_training=False, max_len=None, architecture="boundary") + batch = collator([(text, built_schema)]) + if batch.input_ids.shape[1] > length: + raise ValueError(f"Schema and text require {batch.input_ids.shape[1]} subwords; bucket holds {length}") + if batch.text_word_indices.shape[1] > max_words: + raise ValueError(f"Text requires {batch.text_word_indices.shape[1]} words; bucket holds {max_words}") + if batch.query_marker_indices.shape[1] > max_queries: + raise ValueError(f"Schema requires {batch.query_marker_indices.shape[1]} queries; bucket holds {max_queries}") + if batch.cls_marker_indices.shape[1] > max_choices: + raise ValueError(f"Schema requires {batch.cls_marker_indices.shape[1]} choices; bucket holds {max_choices}") + if batch.query_marker_indices.shape[1] == 0 and batch.cls_marker_indices.shape[1] == 0: + raise ValueError("Schema has no extraction or classification queries") + + def padded(values, width, fill=0): + array = values.numpy() + return np.pad(array, ((0, 0), (0, width - array.shape[1])), constant_values=fill) + + arrays = { + "input_ids": padded(batch.input_ids, length, processor.tokenizer.pad_token_id).astype(np.int32), + "attention_mask": padded(batch.attention_mask, length).astype(np.int32), + "text_indices": padded(batch.text_word_indices, max_words).astype(np.int32), + "text_mask": padded(batch.text_word_mask, max_words).astype(np.float32), + "query_indices": padded(batch.query_marker_indices, max_queries).astype(np.int32), + "query_mask": padded(batch.query_marker_mask, max_queries).astype(np.float32), + "cls_indices": padded(batch.cls_marker_indices, max_choices).astype(np.int32), + "cls_mask": padded(batch.cls_marker_mask, max_choices).astype(np.float32), + } + return arrays, batch diff --git a/pyproject.toml b/pyproject.toml index 08126b66849ba41e3b1c88e6a4e8b229aa67db82..9c9dc662b1e4db3add3f312ae1be9567f3668ccd 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "gliner2-base-coreml" version = "0.1.0" -description = "Pinned GLiNER 2.5 small decision classifier Core ML export" +description = "Pinned GLiNER 2.5 base Core ML classification and extraction export" requires-python = ">=3.12,<3.13" dependencies = [ "coremltools==9.0", @@ -9,6 +9,7 @@ dependencies = [ "huggingface-hub>=0.34,<1", "numpy<2.3", "protobuf>=5,<7", + "psutil>=7,<8", "sentencepiece>=0.2,<0.3", "torch==2.7.0", "transformers==4.57.6", @@ -16,6 +17,7 @@ dependencies = [ [dependency-groups] dev = ["pytest>=8.4", "ruff>=0.13"] +compression = ["scikit-learn==1.5.1"] [tool.pytest.ini_options] testpaths = ["tests"] diff --git a/reports/extraction-validation.json b/reports/extraction-validation.json new file mode 100644 index 0000000000000000000000000000000000000000..b386de15a261e3c0bdf9df137edae6d78fcc1f9d --- /dev/null +++ b/reports/extraction-validation.json @@ -0,0 +1,89 @@ +{ + "purpose": "selected local Core ML extraction parity and latency, not a Decision Index score", + "source_model": "fastino/gliner2.5-base-v1", + "source_revision": "1a8bc24e00dc7300b9017c81d63e3dcdabb26596", + "runtime": { + "coremltools": "9.0", + "gliner2": "2.0.0", + "mac": "M5 Pro, 24 GB, macOS 27.0" + }, + "bucket": "L128/W64/Q8/K8; candidate C192, explicit S64, relation R4/P256, records F8/C192/I1536", + "fixtures": { + "fp32": { + "matched_structures": 11, + "fixture_count": 11, + "failed_cases": [], + "maximum_confidence_error": 1.1324882507324219e-06 + }, + "fp16": { + "matched_structures": 11, + "fixture_count": 11, + "failed_cases": [], + "maximum_confidence_error": 0.16530340909957886 + } + }, + "latency": { + "fp16_all": { + "p50_ms": 8.981791470432654, + "p95_ms": 16.104084032122046, + "warmup": 20, + "iterations": 200, + "shape": "L128/W64/Q8/C192" + }, + "fp16_cpu_and_gpu": { + "p50_ms": 8.275083499029279, + "p95_ms": 13.875250006094575, + "warmup": 20, + "iterations": 200, + "shape": "L128/W64/Q8/C192" + }, + "fp16_cpu_and_neural_engine": { + "p50_ms": 10.255208006128669, + "p95_ms": 14.293458021711558, + "warmup": 20, + "iterations": 200, + "shape": "L128/W64/Q8/C192" + }, + "fp16_cpu_only": { + "p50_ms": 21.39054099097848, + "p95_ms": 33.066082978621125, + "warmup": 20, + "iterations": 200, + "shape": "L128/W64/Q8/C192" + }, + "fp32_all": { + "p50_ms": 9.933604509569705, + "p95_ms": 10.4843340232037, + "warmup": 20, + "iterations": 200, + "shape": "L128/W64/Q8/C192" + } + }, + "feature_compute_plan": { + "fp16-cpu_and_neural_engine": { + "cpu_percent": 46.87, + "gpu_percent": 0.0, + "ane_percent": 53.13 + } + }, + "compression": { + "attempt": "LUT8 per-tensor k-means on FP16 feature package using scikit-learn 1.5.1", + "source_bytes": 390998448, + "compressed_bytes": 196248206, + "matched_structures": 10, + "fixture_count": 11, + "failed_cases": [ + "record_latent" + ], + "release": false + }, + "published_extraction_precisions": [ + "fp16", + "fp32" + ], + "limitations": [ + "fixed shape; requests over capacity fail", + "small selected real-text fixtures only; no full Decision Index score", + "current pinned source revision is not established as the historical evaluation checkpoint" + ] +} diff --git a/reports/extraction-verify-fp16.json b/reports/extraction-verify-fp16.json new file mode 100644 index 0000000000000000000000000000000000000000..ab0cd3d6d81793fd49cfdc52ef71b94f4e7eb55b --- /dev/null +++ b/reports/extraction-verify-fp16.json @@ -0,0 +1,650 @@ +{ + "source_model": "fastino/gliner2.5-base-v1", + "source_revision": "1a8bc24e00dc7300b9017c81d63e3dcdabb26596", + "precision": "fp16", + "selected_manifest": "eleven fixed real-text schema fixtures, not a Decision Index score", + "matched": 11, + "total": 11, + "cases": [ + { + "name": "entities", + "text": "Alice founded Acme in Toronto in 2020.", + "native": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9975394010543823, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9973369240760803, + "start": 14, + "end": 18 + } + ], + "location": [ + { + "text": "Toronto", + "confidence": 0.9967356324195862, + "start": 22, + "end": 29 + } + ] + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9974491000175476, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9972744584083557, + "start": 14, + "end": 18 + } + ], + "location": [ + { + "text": "Toronto", + "confidence": 0.9965165853500366, + "start": 22, + "end": 29 + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 0.00021904706954956055 + }, + { + "name": "entities_multi", + "text": "Marie Curie was born in Warsaw and worked in Paris.", + "native": { + "entities": { + "person": [ + { + "text": "Marie Curie", + "confidence": 0.9981662631034851, + "start": 0, + "end": 11 + } + ], + "city": [ + { + "text": "Warsaw", + "confidence": 0.9975911378860474, + "start": 24, + "end": 30 + }, + { + "text": "Paris", + "confidence": 0.9972521662712097, + "start": 45, + "end": 50 + } + ] + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Marie Curie", + "confidence": 0.9981030225753784, + "start": 0, + "end": 11 + } + ], + "city": [ + { + "text": "Warsaw", + "confidence": 0.9975560903549194, + "start": 24, + "end": 30 + }, + { + "text": "Paris", + "confidence": 0.9972099661827087, + "start": 45, + "end": 50 + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 6.324052810668945e-05 + }, + { + "name": "relations", + "text": "Alice founded Acme in Toronto.", + "native": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Alice", + "start": 0, + "end": 5, + "confidence": 0.9709571003913879 + }, + "tail": { + "text": "Acme", + "start": 14, + "end": 18, + "confidence": 0.9709571003913879 + } + } + ] + } + }, + "coreml": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Alice", + "start": 0, + "end": 5, + "confidence": 0.9704086184501648 + }, + "tail": { + "text": "Acme", + "start": 14, + "end": 18, + "confidence": 0.9704086184501648 + } + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0005484819412231445 + }, + { + "name": "relations_two", + "text": "Steve Jobs founded Apple.", + "native": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Steve Jobs", + "start": 0, + "end": 10, + "confidence": 0.9652665853500366 + }, + "tail": { + "text": "Apple", + "start": 19, + "end": 24, + "confidence": 0.9652665853500366 + } + } + ] + } + }, + "coreml": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Steve Jobs", + "start": 0, + "end": 10, + "confidence": 0.9654463529586792 + }, + "tail": { + "text": "Apple", + "start": 19, + "end": 24, + "confidence": 0.9654463529586792 + } + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 0.00017976760864257812 + }, + { + "name": "record_natural", + "text": "Alice works at Acme. Bob works at Beta.", + "native": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.9950059056282043, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8282835483551025, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Bob", + "confidence": 0.986412763595581, + "start": 21, + "end": 24 + }, + "company": { + "text": "Beta", + "confidence": 0.8703667521476746, + "start": 34, + "end": 38 + } + } + ] + }, + "coreml": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.9949199557304382, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8305941224098206, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Bob", + "confidence": 0.9862046241760254, + "start": 21, + "end": 24 + }, + "company": { + "text": "Beta", + "confidence": 0.8689380288124084, + "start": 34, + "end": 38 + } + } + ] + }, + "structure_match": true, + "maximum_confidence_error": 0.0023105740547180176 + }, + { + "name": "record_latent", + "text": "Alice works at Acme. Bob works at Beta.", + "native": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.8226537108421326, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7471229434013367, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9520154595375061, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.9656922221183777, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.9382809400558472, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9312703013420105, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7089357972145081, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.5311785936355591, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.8056825995445251, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8749582767486572, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9526177644729614, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7116475105285645, + "start": 15, + "end": 19 + } + } + ] + }, + "coreml": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.8165527582168579, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7521946430206299, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9497627019882202, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.9669764637947083, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.936168372631073, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9278878569602966, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7086160182952881, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.5185309648513794, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9497627019882202, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7164170145988464, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.7975567579269409, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8769509196281433, + "start": 15, + "end": 19 + } + } + ] + }, + "structure_match": true, + "maximum_confidence_error": 0.16530340909957886 + }, + { + "name": "record_anchorless", + "text": "Alice works at Acme. Bob works at Beta.", + "native": {}, + "coreml": {}, + "structure_match": true, + "maximum_confidence_error": null + }, + { + "name": "attributes", + "text": "Alice founded Acme.", + "native": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9964190721511841, + "start": 0, + "end": 5, + "role": { + "label": "founder", + "confidence": 0.999975323677063 + } + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9967321157455444, + "start": 14, + "end": 18, + "role": { + "label": "founder", + "confidence": 0.9873692393302917 + } + } + ] + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9964063763618469, + "start": 0, + "end": 5, + "role": { + "label": "founder", + "confidence": 0.999972939491272 + } + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9966885447502136, + "start": 14, + "end": 18, + "role": { + "label": "founder", + "confidence": 0.9858275651931763 + } + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0015416741371154785 + }, + { + "name": "mixed_classification", + "text": "Alice founded Acme.", + "native": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9987560510635376, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9988266825675964, + "start": 14, + "end": 18 + } + ] + }, + "sentiment": { + "label": "positive", + "confidence": 0.9995654225349426 + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9987695813179016, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.998798131942749, + "start": 14, + "end": 18 + } + ] + }, + "sentiment": { + "label": "positive", + "confidence": 0.9995417594909668 + } + }, + "structure_match": true, + "maximum_confidence_error": 2.855062484741211e-05 + }, + { + "name": "classification_only", + "text": "The rocket launched successfully.", + "native": { + "topic": { + "label": "science", + "confidence": 0.9931463003158569 + } + }, + "coreml": { + "topic": { + "label": "science", + "confidence": 0.9928845167160034 + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0002617835998535156 + }, + { + "name": "enum_choice", + "text": "The item is electronics.", + "native": { + "product": [ + { + "category": { + "text": "electronics", + "confidence": 0.9938459992408752 + } + } + ] + }, + "coreml": { + "product": [ + { + "category": { + "text": "electronics", + "confidence": 0.9936116337776184 + } + } + ] + }, + "structure_match": true, + "maximum_confidence_error": 0.00023436546325683594 + } + ] +} diff --git a/reports/extraction-verify-fp32.json b/reports/extraction-verify-fp32.json new file mode 100644 index 0000000000000000000000000000000000000000..8b8b5706fae065aaedf9e57c964ebb829a99ee57 --- /dev/null +++ b/reports/extraction-verify-fp32.json @@ -0,0 +1,650 @@ +{ + "source_model": "fastino/gliner2.5-base-v1", + "source_revision": "1a8bc24e00dc7300b9017c81d63e3dcdabb26596", + "precision": "fp32", + "selected_manifest": "eleven fixed real-text schema fixtures, not a Decision Index score", + "matched": 11, + "total": 11, + "cases": [ + { + "name": "entities", + "text": "Alice founded Acme in Toronto in 2020.", + "native": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9975394010543823, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9973369240760803, + "start": 14, + "end": 18 + } + ], + "location": [ + { + "text": "Toronto", + "confidence": 0.9967356324195862, + "start": 22, + "end": 29 + } + ] + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9975394010543823, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9973369240760803, + "start": 14, + "end": 18 + } + ], + "location": [ + { + "text": "Toronto", + "confidence": 0.9967356324195862, + "start": 22, + "end": 29 + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0 + }, + { + "name": "entities_multi", + "text": "Marie Curie was born in Warsaw and worked in Paris.", + "native": { + "entities": { + "person": [ + { + "text": "Marie Curie", + "confidence": 0.9981662631034851, + "start": 0, + "end": 11 + } + ], + "city": [ + { + "text": "Warsaw", + "confidence": 0.9975911378860474, + "start": 24, + "end": 30 + }, + { + "text": "Paris", + "confidence": 0.9972521662712097, + "start": 45, + "end": 50 + } + ] + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Marie Curie", + "confidence": 0.9981662631034851, + "start": 0, + "end": 11 + } + ], + "city": [ + { + "text": "Warsaw", + "confidence": 0.9975911378860474, + "start": 24, + "end": 30 + }, + { + "text": "Paris", + "confidence": 0.9972521662712097, + "start": 45, + "end": 50 + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0 + }, + { + "name": "relations", + "text": "Alice founded Acme in Toronto.", + "native": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Alice", + "start": 0, + "end": 5, + "confidence": 0.9709571003913879 + }, + "tail": { + "text": "Acme", + "start": 14, + "end": 18, + "confidence": 0.9709571003913879 + } + } + ] + } + }, + "coreml": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Alice", + "start": 0, + "end": 5, + "confidence": 0.9709572196006775 + }, + "tail": { + "text": "Acme", + "start": 14, + "end": 18, + "confidence": 0.9709572196006775 + } + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 1.1920928955078125e-07 + }, + { + "name": "relations_two", + "text": "Steve Jobs founded Apple.", + "native": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Steve Jobs", + "start": 0, + "end": 10, + "confidence": 0.9652665853500366 + }, + "tail": { + "text": "Apple", + "start": 19, + "end": 24, + "confidence": 0.9652665853500366 + } + } + ] + } + }, + "coreml": { + "relation_extraction": { + "founded": [ + { + "head": { + "text": "Steve Jobs", + "start": 0, + "end": 10, + "confidence": 0.9652665853500366 + }, + "tail": { + "text": "Apple", + "start": 19, + "end": 24, + "confidence": 0.9652665853500366 + } + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0 + }, + { + "name": "record_natural", + "text": "Alice works at Acme. Bob works at Beta.", + "native": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.9950059056282043, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8282835483551025, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Bob", + "confidence": 0.986412763595581, + "start": 21, + "end": 24 + }, + "company": { + "text": "Beta", + "confidence": 0.8703667521476746, + "start": 34, + "end": 38 + } + } + ] + }, + "coreml": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.9950060248374939, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8282831311225891, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Bob", + "confidence": 0.986412763595581, + "start": 21, + "end": 24 + }, + "company": { + "text": "Beta", + "confidence": 0.8703671097755432, + "start": 34, + "end": 38 + } + } + ] + }, + "structure_match": true, + "maximum_confidence_error": 4.172325134277344e-07 + }, + { + "name": "record_latent", + "text": "Alice works at Acme. Bob works at Beta.", + "native": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.8226537108421326, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7471229434013367, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9520154595375061, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.9656922221183777, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.9382809400558472, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9312703013420105, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7089357972145081, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.5311785936355591, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.8056825995445251, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8749582767486572, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9526177644729614, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7116475105285645, + "start": 15, + "end": 19 + } + } + ] + }, + "coreml": { + "employment": [ + { + "person": { + "text": "Alice", + "confidence": 0.822653591632843, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7471233010292053, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.952015221118927, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.9656922221183777, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.9382812976837158, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9312704801559448, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7089352607727051, + "start": 15, + "end": 19 + } + }, + { + "person": null, + "company": { + "text": "Beta", + "confidence": 0.5311788320541382, + "start": 34, + "end": 38 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.8056837320327759, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.8749585151672363, + "start": 15, + "end": 19 + } + }, + { + "person": { + "text": "Alice", + "confidence": 0.9526176452636719, + "start": 0, + "end": 5 + }, + "company": { + "text": "Acme", + "confidence": 0.7116484045982361, + "start": 15, + "end": 19 + } + } + ] + }, + "structure_match": true, + "maximum_confidence_error": 1.1324882507324219e-06 + }, + { + "name": "record_anchorless", + "text": "Alice works at Acme. Bob works at Beta.", + "native": {}, + "coreml": {}, + "structure_match": true, + "maximum_confidence_error": null + }, + { + "name": "attributes", + "text": "Alice founded Acme.", + "native": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9964190721511841, + "start": 0, + "end": 5, + "role": { + "label": "founder", + "confidence": 0.999975323677063 + } + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9967321157455444, + "start": 14, + "end": 18, + "role": { + "label": "founder", + "confidence": 0.9873692393302917 + } + } + ] + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9964190721511841, + "start": 0, + "end": 5, + "role": { + "label": "founder", + "confidence": 0.999975323677063 + } + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9967321157455444, + "start": 14, + "end": 18, + "role": { + "label": "founder", + "confidence": 0.9873693585395813 + } + } + ] + } + }, + "structure_match": true, + "maximum_confidence_error": 1.1920928955078125e-07 + }, + { + "name": "mixed_classification", + "text": "Alice founded Acme.", + "native": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9987560510635376, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9988266825675964, + "start": 14, + "end": 18 + } + ] + }, + "sentiment": { + "label": "positive", + "confidence": 0.9995654225349426 + } + }, + "coreml": { + "entities": { + "person": [ + { + "text": "Alice", + "confidence": 0.9987560510635376, + "start": 0, + "end": 5 + } + ], + "organization": [ + { + "text": "Acme", + "confidence": 0.9988266825675964, + "start": 14, + "end": 18 + } + ] + }, + "sentiment": { + "label": "positive", + "confidence": 0.9995654225349426 + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0 + }, + { + "name": "classification_only", + "text": "The rocket launched successfully.", + "native": { + "topic": { + "label": "science", + "confidence": 0.9931463003158569 + } + }, + "coreml": { + "topic": { + "label": "science", + "confidence": 0.9931463003158569 + } + }, + "structure_match": true, + "maximum_confidence_error": 0.0 + }, + { + "name": "enum_choice", + "text": "The item is electronics.", + "native": { + "product": [ + { + "category": { + "text": "electronics", + "confidence": 0.9938459992408752 + } + } + ] + }, + "coreml": { + "product": [ + { + "category": { + "text": "electronics", + "confidence": 0.9938459992408752 + } + } + ] + }, + "structure_match": true, + "maximum_confidence_error": 0.0 + } + ] +} diff --git a/tests/test_extraction_export.py b/tests/test_extraction_export.py new file mode 100644 index 0000000000000000000000000000000000000000..7c27cbcb31ef2ec9ba0755e408e4ece167afe02d --- /dev/null +++ b/tests/test_extraction_export.py @@ -0,0 +1,67 @@ +"""Real-checkpoint parity for trained extraction graph wrappers.""" + +from pathlib import Path + +import torch +from gliner2 import AutoExtractor, Schema +from gliner2.training.trainer import ExtractorCollator + +from extraction_export import ExtractionFeaturesExport, ExtractionScoreExport, coreml_trace_patches +from extraction_pool import select_candidates +from preprocessing import prepare_extraction + +SOURCE = ( + Path.home() + / ".cache/huggingface/hub/models--fastino--gliner2.5-base-v1/snapshots/1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +) + + +def test_export_wrappers_match_native_entity_scores(): + torch.set_num_threads(4) + native = AutoExtractor.from_pretrained(str(SOURCE), map_location="cpu").eval() + text = "Alice founded Acme in Toronto in 2020." + schema = Schema().entities(["person", "organization", "location"]) + batch = ExtractorCollator(native.processor, is_training=False, max_len=128, architecture=native.architecture)( + [(text, schema.build())] + ) + arrays, _ = prepare_extraction(native.processor, text, schema, 128, 64, 8) + arguments = tuple(torch.from_numpy(value) for value in arrays.values()) + with torch.no_grad(): + core = native._encode_core(batch) + expected = native.boundary_head( + core["text_states"], core["text_mask"], core["query_states"], core["query_mask"] + ) + with coreml_trace_patches(): + features = ExtractionFeaturesExport(native).eval()(*arguments) + assert torch.allclose(features[0][:, : core["text_states"].shape[1]], core["text_states"], atol=1e-5) + assert torch.allclose(features[1][:, : core["query_states"].shape[1]], core["query_states"], atol=1e-5) + pool = select_candidates( + features[8], + features[9], + features[3].bool(), + arguments[5].bool(), + features[4], + features[5], + boundary_top_k=native.boundary_head.shared_pool_builder.pool_boundary_top_k, + pool_size=native.boundary_head.shared_pool_builder.pool_size, + min_pool_per_query=native.boundary_head.shared_pool_builder.min_pool_per_query, + ) + assert torch.equal( + pool.indices.unsqueeze(1).expand_as(expected.candidates.indices), expected.candidates.indices + ) + scores, candidate_states = ExtractionScoreExport(native).eval()( + features[0], + arguments[3], + features[1], + arguments[5], + features[2], + features[4], + features[5], + features[6], + features[7], + pool.indices.int(), + pool.mask.float(), + pool.compat_logits, + ) + assert torch.allclose(scores[:, : core["query_states"].shape[1]], expected.candidates.pair_logits, atol=1e-4) + assert torch.allclose(candidate_states.unsqueeze(1), expected.candidates.candidate_states[:, :1], atol=1e-4) diff --git a/tests/test_extraction_pool.py b/tests/test_extraction_pool.py new file mode 100644 index 0000000000000000000000000000000000000000..e745ae201c766c951d8e6cfe030bf7046f6ae731 --- /dev/null +++ b/tests/test_extraction_pool.py @@ -0,0 +1,62 @@ +"""Real-checkpoint parity for the weight-free candidate selection stage.""" + +from pathlib import Path + +import torch +from gliner2 import AutoExtractor, Schema +from gliner2.training.trainer import ExtractorCollator + +from extraction_pool import select_candidates + +SOURCE = ( + Path.home() + / ".cache/huggingface/hub/models--fastino--gliner2.5-base-v1/snapshots/1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +) + + +def test_candidate_pool_matches_native_with_real_weights(): + torch.set_num_threads(4) + native = AutoExtractor.from_pretrained(str(SOURCE), map_location="cpu").eval() + fixtures = [ + ("Alice founded Acme in Toronto in 2020.", ["person", "organization", "location"]), + ("Apple acquired Beats for three billion dollars.", ["company", "product", "money"]), + ] + for text, labels in fixtures: + schema = Schema().entities(labels) + batch = ExtractorCollator(native.processor, is_training=False, max_len=128, architecture=native.architecture)( + [(text, schema.build())] + ) + with torch.no_grad(): + core = native._encode_core(batch) + head = native.boundary_head + encoded = head.boundary_encoder(core["text_states"], core["text_mask"]) + marginals = head.boundary_query_head( + encoded.states, + encoded.mask, + core["text_states"], + core["text_mask"], + core["query_states"], + core["query_mask"], + ) + expected = head.shared_pool_builder( + encoded.states, + encoded.mask, + core["query_mask"], + marginals.start_logits, + marginals.end_logits, + ) + actual = select_candidates( + head.shared_pool_builder.start_projection(encoded.states), + head.shared_pool_builder.end_projection(encoded.states), + encoded.mask, + core["query_mask"], + marginals.start_logits, + marginals.end_logits, + boundary_top_k=head.shared_pool_builder.pool_boundary_top_k, + pool_size=head.shared_pool_builder.pool_size, + min_pool_per_query=head.shared_pool_builder.min_pool_per_query, + ) + assert torch.equal(actual.indices, expected.indices) + assert torch.equal(actual.mask, expected.mask) + assert torch.allclose(actual.compat_logits, expected.compat_logits, atol=1e-5) + assert torch.allclose(actual.proposal_logits, expected.proposal_logits, atol=1e-5) diff --git a/tests/test_extraction_preprocessing.py b/tests/test_extraction_preprocessing.py new file mode 100644 index 0000000000000000000000000000000000000000..aa5ca74f0f2e8e4a0da47583f553d4dcca3d2675 --- /dev/null +++ b/tests/test_extraction_preprocessing.py @@ -0,0 +1,43 @@ +"""Capacity checks for the real pinned GLiNER2 tokenizer and schema.""" + +from pathlib import Path + +import pytest +from gliner2 import Schema + +from preprocessing import load_processor, prepare_extraction + +SOURCE = ( + Path.home() + / ".cache/huggingface/hub/models--fastino--gliner2.5-base-v1/snapshots/1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +) + + +def test_extraction_keeps_all_queries_and_offsets(): + processor = load_processor(str(SOURCE)) + text = "Alice founded Acme in Toronto in 2020." + arrays, batch = prepare_extraction( + processor, text, Schema().entities(["person", "organization", "location"]), 128, 64, 8 + ) + assert int(arrays["query_mask"].sum()) == 3 + assert int(arrays["text_mask"].sum()) == len(batch.start_mappings[0]) + assert text[batch.start_mappings[0][0] : batch.end_mappings[0][0]] == "Alice" + + +@pytest.mark.parametrize("bucket", [(10, 64, 8), (128, 2, 8), (128, 64, 2)]) +def test_extraction_rejects_capacity_exceeded(bucket): + processor = load_processor(str(SOURCE)) + with pytest.raises(ValueError, match="bucket holds"): + prepare_extraction( + processor, + "Alice founded Acme in Toronto in 2020.", + Schema().entities(["person", "organization", "location"]), + *bucket, + ) + + +def test_extraction_rejects_classification_choice_capacity(): + processor = load_processor(str(SOURCE)) + choices = [chr(ord("a") + index) for index in range(9)] + with pytest.raises(ValueError, match="choices; bucket holds 8"): + prepare_extraction(processor, "A short report.", Schema().classification("topic", choices), 128, 64, 8) diff --git a/tests/test_extraction_record.py b/tests/test_extraction_record.py new file mode 100644 index 0000000000000000000000000000000000000000..efdc9d20f7f734760cc2321924911da182add89d --- /dev/null +++ b/tests/test_extraction_record.py @@ -0,0 +1,66 @@ +"""Real-checkpoint parity of all three native record formation modes.""" + +from pathlib import Path + +import torch +from gliner2 import AutoExtractor, Schema +from gliner2.training.trainer import ExtractorCollator + +from extraction_export import ExtractionRecordAnchorlessExport, ExtractionRecordAssignmentExport + +SOURCE = ( + Path.home() + / ".cache/huggingface/hub/models--fastino--gliner2.5-base-v1/snapshots/1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +) + + +def test_record_heads_match_native_natural_latent_and_anchorless(): + torch.set_num_threads(4) + native = AutoExtractor.from_pretrained(str(SOURCE), map_location="cpu").eval() + text = "Alice works at Acme. Bob works at Beta." + for mode in ("natural", "latent", "anchorless"): + schema = Schema() + builder = schema.structure("employment", mode=mode, anchor="person" if mode == "natural" else None) + builder.field("person", dtype="str") + builder.field("company", dtype="str") + batch = ExtractorCollator(native.processor, is_training=False, max_len=None, architecture="boundary")( + [(text, schema.build())] + ) + with torch.no_grad(): + core = native._encode_core(batch) + candidates = native.boundary_head( + core["text_states"], core["text_mask"], core["query_states"], core["query_mask"] + ).candidates + spec = next(iter(batch.record_specs[0].values())) + group = native.record_decoder.forward_group(spec, core["query_states"][0], candidates, 0) + field_states = [] + for query_id in group.field_query_ids: + keep = candidates.valid_mask[0, query_id] + field_states.append(candidates.candidate_states[0, query_id][keep]) + field_queries = core["query_states"][0][group.field_query_ids] + max_candidates = max(states.shape[0] for states in field_states) + padded_fields = torch.stack( + [ + torch.nn.functional.pad(states, (0, 0, 0, max_candidates - states.shape[0])) + for states in field_states + ] + ) + if mode == "natural": + anchor_index = group.field_query_ids.index(spec.anchor_query_id) + instances = field_states[anchor_index] + elif mode == "latent": + instances = torch.cat(field_states, 0) + else: + instances = native.record_decoder._anchorless_states(field_states) + context = torch.cat(field_states, 0) + predicted = ExtractionRecordAnchorlessExport(native).eval()(context, torch.ones(context.shape[0])) + assert torch.allclose(predicted, instances, atol=1e-5) + assignment, object_scores, latent_scores = ExtractionRecordAssignmentExport(native).eval()( + instances, field_queries, padded_fields + ) + for field_index, expected in enumerate(group.assign_logits): + assert torch.allclose(assignment[:, field_index, : expected.shape[1]], expected, atol=1e-5) + if mode == "anchorless": + assert torch.allclose(object_scores, group.object_logits, atol=1e-5) + if mode == "latent": + assert torch.allclose(latent_scores, group.object_logits, atol=1e-5) diff --git a/uv.lock b/uv.lock index c58855c6660ad216b8132cf99fe5c29867f084f1..abb7498db8b9a3c0f3ad1b567958349970278bdd 100644 --- a/uv.lock +++ b/uv.lock @@ -101,6 +101,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/cc/61/d01fc49b8dea277640b55a9e15960dbca9fdc8c9fde18e572d39c59f4019/charset_normalizer-3.5.1-py3-none-any.whl", hash = "sha256:6df0ec430f9a831772c23ca5a224cba36517a58a84bb32c32bb59a9fa67c47f6", size = 68658, upload-time = "2026-08-15T08:20:43.306Z" }, ] +[[package]] +name = "cloudpickle" +version = "3.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/27/fb/576f067976d320f5f0114a8d9fa1215425441bb35627b1993e5afd8111e5/cloudpickle-3.1.2.tar.gz", hash = "sha256:7fda9eb655c9c230dab534f1983763de5835249750e85fbcef43aaa30a9a2414", size = 22330, upload-time = "2025-11-03T09:25:26.604Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/88/39/799be3f2f0f38cc727ee3b4f1445fe6d5e4133064ec2e4115069418a5bb6/cloudpickle-3.1.2-py3-none-any.whl", hash = "sha256:9acb47f6afd73f60dc1df93bb801b472f05ff42fa6c84167d25cb206be1fbf4a", size = 22228, upload-time = "2025-11-03T09:25:25.534Z" }, +] + [[package]] name = "colorama" version = "0.4.6" @@ -183,12 +192,16 @@ dependencies = [ { name = "huggingface-hub" }, { name = "numpy" }, { name = "protobuf" }, + { name = "psutil" }, { name = "sentencepiece" }, { name = "torch" }, { name = "transformers" }, ] [package.dev-dependencies] +compression = [ + { name = "scikit-learn" }, +] dev = [ { name = "pytest" }, { name = "ruff" }, @@ -201,12 +214,14 @@ requires-dist = [ { name = "huggingface-hub", specifier = ">=0.34,<1" }, { name = "numpy", specifier = "<2.3" }, { name = "protobuf", specifier = ">=5,<7" }, + { name = "psutil", specifier = ">=7,<8" }, { name = "sentencepiece", specifier = ">=0.2,<0.3" }, { name = "torch", specifier = "==2.7.0" }, { name = "transformers", specifier = "==4.57.6" }, ] [package.metadata.requires-dev] +compression = [{ name = "scikit-learn", specifier = "==1.5.1" }] dev = [ { name = "pytest", specifier = ">=8.4" }, { name = "ruff", specifier = ">=0.13" }, @@ -277,6 +292,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, ] +[[package]] +name = "joblib" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cloudpickle" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/d5/1d/537ab090f302b838943a1b56497dd53059b9a9b46a074936470173a2e207/joblib-1.6.0.tar.gz", hash = "sha256:2ccc96785b12046c08fd6d55839c12857831b54a3c1673ffadd2f04bfc4eda03", size = 327903, upload-time = "2026-08-31T09:39:04.122Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/53/84099323c2ec4be98d935f63c033ac4151ee83836ca1050ede3b3aadf155/joblib-1.6.0-py3-none-any.whl", hash = "sha256:3dbbf9f6e4b592a2357b854608e980fe6390d131d7a82f011a377ef2ebef7aba", size = 306115, upload-time = "2026-08-31T09:39:02.298Z" }, +] + [[package]] name = "markupsafe" version = "3.0.3" @@ -723,6 +750,46 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/8d/fc/7eedc3510d97878876e32774eebbeb61c43f148a96e915c84229a3e967aa/safetensors-0.8.0-cp310-abi3-win_arm64.whl", hash = "sha256:f7838e5135a406ad3e02efdcb8cf2e5397d368b0154537c4fec682dbc544d452", size = 340500, upload-time = "2026-06-09T07:52:26.745Z" }, ] +[[package]] +name = "scikit-learn" +version = "1.5.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "joblib" }, + { name = "numpy" }, + { name = "scipy" }, + { name = "threadpoolctl" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/92/72/2961b9874a9ddf2b0f95f329d4e67f67c3301c1d88ba5e239ff25661bb85/scikit_learn-1.5.1.tar.gz", hash = "sha256:0ea5d40c0e3951df445721927448755d3fe1d80833b0b7308ebff5d2a45e6414", size = 6958368, upload-time = "2024-07-03T09:12:21.925Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b1/8d/cf392a56e24627093a467642c8b9263052372131359b570df29aaf4811ab/scikit_learn-1.5.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:5944ce1faada31c55fb2ba20a5346b88e36811aab504ccafb9f0339e9f780395", size = 12102404, upload-time = "2024-07-03T09:11:46.261Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2c/734fc9269bdb6768905ac41b82d75264b26925b1e462f4ebf45fe4f17646/scikit_learn-1.5.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:0828673c5b520e879f2af6a9e99eee0eefea69a2188be1ca68a6121b809055c1", size = 11037398, upload-time = "2024-07-03T09:11:49.783Z" }, + { url = "https://files.pythonhosted.org/packages/d3/a9/15774b178bcd1cde1c470adbdb554e1504dce7c302e02ff736c90d65e014/scikit_learn-1.5.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:508907e5f81390e16d754e8815f7497e52139162fd69c4fdbd2dfa5d6cc88915", size = 12089887, upload-time = "2024-07-03T09:11:53.134Z" }, + { url = "https://files.pythonhosted.org/packages/8a/5d/047cde25131eef3a38d03317fa7d25d6f60ce6e8ccfd24ac88b3e309fc00/scikit_learn-1.5.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:97625f217c5c0c5d0505fa2af28ae424bd37949bb2f16ace3ff5f2f81fb4498b", size = 13079093, upload-time = "2024-07-03T09:11:55.93Z" }, + { url = "https://files.pythonhosted.org/packages/cb/be/dec2a8d31d133034a8ec51ae68ac564ec9bde1c78a64551f1438c3690b9e/scikit_learn-1.5.1-cp312-cp312-win_amd64.whl", hash = "sha256:da3f404e9e284d2b0a157e1b56b6566a34eb2798205cba35a211df3296ab7a74", size = 10945350, upload-time = "2024-07-03T09:12:02.494Z" }, +] + +[[package]] +name = "scipy" +version = "1.18.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/7e/74/66de6258867beb2ef08f35f9f2ac017a52cacd5081714d239ff1a442d458/scipy-1.18.1.tar.gz", hash = "sha256:52c4b7422442aba924d03ad4019852b08a92e64ea187b933135687bfe2747307", size = 30781235, upload-time = "2026-08-21T23:28:50.599Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/f7/240c110c08693826b4513a52f5717d62ec7c7af72f2920821247c03b17b3/scipy-1.18.1-cp312-cp312-macosx_10_15_x86_64.whl", hash = "sha256:457fd7a2a8edeb044ab6ffbc0aa03ff6cd18491356e5e0c834d76ce621b916d1", size = 31111061, upload-time = "2026-08-21T23:23:44.522Z" }, + { url = "https://files.pythonhosted.org/packages/05/4a/78c6285577c375e7cf27277ea8ee6961224327f1e1a0c44af5f17f23635c/scipy-1.18.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:e708533e8b2ae2497d65346538a7dcc92814410b25b81432eac66de0f2af8265", size = 28733332, upload-time = "2026-08-21T23:23:50.015Z" }, + { url = "https://files.pythonhosted.org/packages/a5/f6/a5b82f8abbe14d134691b8b903696f701d25a081353a29dc655c364d9e62/scipy-1.18.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:7bbf207c4453ce1ad2e00b17313852b33310b83090c2311bdaf97f93c0380d12", size = 20475078, upload-time = "2026-08-21T23:23:54.138Z" }, + { url = "https://files.pythonhosted.org/packages/23/22/0858a0bbd6b3e825ceb8cd9baf9eaf3b2f2b1d77727eb6be40500bcdc92f/scipy-1.18.1-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:78c0665edead396b1abb4897c41a5c1d9bf090c8a637a4c20a61678e0a264e66", size = 23108904, upload-time = "2026-08-21T23:23:57.824Z" }, + { url = "https://files.pythonhosted.org/packages/75/9a/2e71719f31eaefe0e3a1706c4a1ded94e664bfd95ffca2b219a671faee01/scipy-1.18.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c085faa2cfa879c5141df483f836f4d691045a078224a670fa570fa01612d89", size = 34025113, upload-time = "2026-08-21T23:24:02.209Z" }, + { url = "https://files.pythonhosted.org/packages/df/64/ff35eb9e54894cf471ff4716abd3c81eb0a0626869217ce3e6ba4ccf17d7/scipy-1.18.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f55fa87b6c612ecd6b058f167c53231b1d14e412efe361d3d6e38b3631c73218", size = 35344199, upload-time = "2026-08-21T23:24:07.844Z" }, + { url = "https://files.pythonhosted.org/packages/d3/af/c5538be1792f7034c12c7db6ee67cace58253c7b87b122d68253eaf5de89/scipy-1.18.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c35d74ce0e193ff740c2f2be2ac913ddc232fe6c1ff40b26cfecb9c670c63314", size = 35639587, upload-time = "2026-08-21T23:24:13.05Z" }, + { url = "https://files.pythonhosted.org/packages/91/4c/075e4f66471bac101141ac739e9e135549be1bae584571bd03a530c056e1/scipy-1.18.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:d2924a03db38dc2e848bca2fe9f077dafb891480b91a00a0963a8cf86dfc31c1", size = 37480330, upload-time = "2026-08-21T23:24:19.608Z" }, + { url = "https://files.pythonhosted.org/packages/39/e7/979fd14e75008623df31ba70d6bb144700f68feadcea042021c06a05bf82/scipy-1.18.1-cp312-cp312-win_amd64.whl", hash = "sha256:5e4d44984abc0020154ea81b247adeddcc3ac5527b975ff798bd1ba0adc513c2", size = 36658278, upload-time = "2026-08-21T23:24:25.463Z" }, + { url = "https://files.pythonhosted.org/packages/c7/0b/e1525354ff9d7d5feb6d1b31af6d14072e5c91e9607b421fa1ec889660b3/scipy-1.18.1-cp312-cp312-win_arm64.whl", hash = "sha256:d65d448389b8436493abcf629cc94ad0cf32aecaf06e1acca1de53cc795f2f12", size = 24400588, upload-time = "2026-08-21T23:24:30.579Z" }, +] + [[package]] name = "sentencepiece" version = "0.2.2" @@ -759,6 +826,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" }, ] +[[package]] +name = "threadpoolctl" +version = "3.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/00/dc/6c58154c1c65f758ea979e7139cb76993a9cfc662d14e9be3c4a667cfb77/threadpoolctl-3.7.0.tar.gz", hash = "sha256:61348cfb77d53b9242e0017029244b559b810c142ced65b4e21eeca1843959a7", size = 31961, upload-time = "2026-09-15T15:46:20.263Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/43/3f/f88a53f60a472b46f4023f56d204dd7de33d34c5d2acbfa0d70a674e639e/threadpoolctl-3.7.0-py3-none-any.whl", hash = "sha256:cd8b60b5641b45c67bbf73c64c843235fc2d8a480c87389f52f5dbee893b86be", size = 26362, upload-time = "2026-09-15T15:46:19.168Z" }, +] + [[package]] name = "tokenizers" version = "0.22.2" diff --git a/verify-extraction.py b/verify-extraction.py new file mode 100644 index 0000000000000000000000000000000000000000..8f736946f921f9db648069c3e2135ee7bf039872 --- /dev/null +++ b/verify-extraction.py @@ -0,0 +1,168 @@ +"""Small real-fixture native/Core ML parity check for entity extraction.""" + +import argparse +import json +from pathlib import Path + +import coremltools as ct +import numpy as np +import torch +from gliner2 import AutoExtractor, Schema +from gliner2.models.boundary.model import _group_scored_candidates +from gliner2.models.outputs import CandidateTensorBatch +from huggingface_hub import snapshot_download + +from convert_extraction_names import FEATURE_NAMES, SCORE_INPUT_NAMES +from extraction_pool import select_candidates +from preprocessing import prepare_extraction + +MODEL_ID = "fastino/gliner2.5-base-v1" +MODEL_REVISION = "1a8bc24e00dc7300b9017c81d63e3dcdabb26596" +FIXTURES = [ + ("Alice founded Acme in Toronto in 2020.", ["person", "organization", "location"]), + ("Apple acquired Beats for three billion dollars.", ["company", "product", "money"]), + ("Marie Curie was born in Warsaw and worked in Paris.", ["person", "city"]), +] + + +def coreml_entity_result(native, features_model, scorer_model, text, schema, length, words, queries): + arrays, batch = prepare_extraction(native.processor, text, schema, length, words, queries) + features = features_model.predict(arrays) + tensor = {name: torch.from_numpy(np.asarray(features[name]).copy()) for name in FEATURE_NAMES} + text_mask = torch.from_numpy(arrays["text_mask"]).bool() + query_mask = torch.from_numpy(arrays["query_mask"]).bool() + head = native.boundary_head + pool = select_candidates( + tensor["pool_start_projection"], + tensor["pool_end_projection"], + tensor["boundary_mask"].bool(), + query_mask, + tensor["start_logits"], + tensor["end_logits"], + boundary_top_k=head.shared_pool_builder.pool_boundary_top_k, + pool_size=head.shared_pool_builder.pool_size, + min_pool_per_query=head.shared_pool_builder.min_pool_per_query, + ) + score_values = ( + tensor["text_states"], + text_mask.float(), + tensor["query_states"], + query_mask.float(), + tensor["boundary_states"], + tensor["start_logits"], + tensor["end_logits"], + tensor["inside_prefix"], + tensor["inside_prefix_mean"], + pool.indices.int(), + pool.mask.float(), + pool.compat_logits, + ) + score_arrays = { + name: value.numpy().astype(np.int32 if name == "candidate_indices" else np.float32) + for name, value in zip(SCORE_INPUT_NAMES, score_values) + } + scores = scorer_model.predict(score_arrays) + candidate_batch = CandidateTensorBatch( + indices=pool.indices.unsqueeze(1).expand(1, queries, -1, 2), + proposal_logits=pool.proposal_logits.unsqueeze(1).expand(1, queries, -1), + pair_logits=torch.from_numpy(np.asarray(scores["pair_logits"]).copy()), + valid_mask=pool.mask.unsqueeze(1).expand(1, queries, -1), + query_mask=query_mask, + ) + _, metadata = native._build_schema_dicts_and_metadata([schema]) + with torch.no_grad(): + core = native._encode_core(batch) + native_output = native._extract_from_batch(batch, 0.5, metadata, True, True)[0] + probs = torch.sigmoid(candidate_batch.pair_logits / native.boundary_settings.pair_temperature) + grouped = _group_scored_candidates( + candidate_batch, + threshold=0.5, + probabilities=probs, + count_log_rates=tensor["count_log_rates"], + adaptive_threshold=native.boundary_settings.adaptive_threshold, + ) + extracted = native._decode_entities( + 0, + core, + core["ext_specs"][0], + grouped[0], + metadata[0], + torch.sigmoid(tensor["null_logits"])[0], + "flat", + core["word_offsets"][0], + batch.start_mappings[0], + batch.end_mappings[0], + text, + len(batch.start_mappings[0]), + True, + True, + ) + coreml_output = {"entities": [extracted]} if extracted else {} + return native_output, coreml_output, pool, candidate_batch + + +def stripped_spans(result): + output = {} + for name, entities in result.get("entities", [{}])[0].items(): + output[name] = [(item["text"], item["start"], item["end"]) for item in entities] + return output + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--model-dir", required=True) + parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32") + parser.add_argument("--length", type=int, default=128) + parser.add_argument("--max-words", type=int, default=64) + parser.add_argument("--max-queries", type=int, default=8) + args = parser.parse_args() + torch.set_num_threads(4) + source = snapshot_download(MODEL_ID, revision=MODEL_REVISION) + native = AutoExtractor.from_pretrained(str(source), map_location="cpu").eval() + suffix = f"{args.precision}_L{args.length}_W{args.max_words}_Q{args.max_queries}" + folder = Path(args.model_dir) + features = ct.models.MLModel( + str(folder / f"gliner2_base_extraction_features_{suffix}.mlpackage"), compute_units=ct.ComputeUnit.CPU_ONLY + ) + scorer = ct.models.MLModel( + str(folder / f"gliner2_base_extraction_scorer_{suffix}.mlpackage"), compute_units=ct.ComputeUnit.CPU_ONLY + ) + cases = [] + for text, labels in FIXTURES: + reference, actual, _, _ = coreml_entity_result( + native, + features, + scorer, + text, + Schema().entities(labels), + args.length, + args.max_words, + args.max_queries, + ) + expected_spans = stripped_spans(reference) + actual_spans = stripped_spans(actual) + cases.append( + { + "text": text, + "native_spans": expected_spans, + "coreml_spans": actual_spans, + "span_match": expected_spans == actual_spans, + } + ) + report = { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "precision": args.precision, + "selected_manifest": "three fixed real-text entity fixtures", + "cases": cases, + "span_matches": sum(case["span_match"] for case in cases), + } + path = folder / f"verify-entities-{suffix}.json" + path.write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps(report, indent=2)) + if report["span_matches"] != len(cases): + raise SystemExit(1) + + +if __name__ == "__main__": + main() diff --git a/verify-full-extraction.py b/verify-full-extraction.py new file mode 100644 index 0000000000000000000000000000000000000000..8290c028d1f785d71a287f78a193832eeeeaf06a --- /dev/null +++ b/verify-full-extraction.py @@ -0,0 +1,132 @@ +"""Selected end-to-end native/Core ML checks across GLiNER2 schema paths.""" + +import argparse +import json +from pathlib import Path + +import torch +from gliner2 import AttributeGroup, AutoExtractor, Schema +from huggingface_hub import snapshot_download + +from extraction_runtime import CoreMLBoundaryExtractor + +MODEL_ID = "fastino/gliner2.5-base-v1" +MODEL_REVISION = "1a8bc24e00dc7300b9017c81d63e3dcdabb26596" + + +def structure(mode): + schema = Schema() + builder = schema.structure("employment", mode=mode, anchor="person" if mode == "natural" else None) + builder.field("person", dtype="str") + builder.field("company", dtype="str") + return schema + + +def fixtures(): + attribute = Schema().entities(["person", "organization"]) + attribute.entity_attributes({"role": AttributeGroup(labels=["founder", "employee"])}) + mixed = Schema().entities(["person", "organization"]) + mixed.classification("sentiment", ["positive", "negative"]) + choice = Schema() + choice.structure("product").field("category", dtype="str", choices=["electronics", "clothing"]) + return [ + ( + "entities", + "Alice founded Acme in Toronto in 2020.", + Schema().entities(["person", "organization", "location"]), + ), + ( + "entities_multi", + "Marie Curie was born in Warsaw and worked in Paris.", + Schema().entities(["person", "city"]), + ), + ("relations", "Alice founded Acme in Toronto.", Schema().relations(["founded"])), + ("relations_two", "Steve Jobs founded Apple.", Schema().relations(["founded"])), + ("record_natural", "Alice works at Acme. Bob works at Beta.", structure("natural")), + ("record_latent", "Alice works at Acme. Bob works at Beta.", structure("latent")), + ("record_anchorless", "Alice works at Acme. Bob works at Beta.", structure("anchorless")), + ("attributes", "Alice founded Acme.", attribute), + ("mixed_classification", "Alice founded Acme.", mixed), + ( + "classification_only", + "The rocket launched successfully.", + Schema().classification("topic", ["science", "sports", "politics"]), + ), + ("enum_choice", "The item is electronics.", choice), + ] + + +def without_confidence(value): + if isinstance(value, dict): + return {key: without_confidence(item) for key, item in value.items() if key != "confidence"} + if isinstance(value, list): + return [without_confidence(item) for item in value] + return value + + +def confidence_errors(reference, actual): + if isinstance(reference, dict) and isinstance(actual, dict): + errors = [] + for key, value in reference.items(): + if key == "confidence" and key in actual: + errors.append(abs(float(value) - float(actual[key]))) + elif key in actual: + errors.extend(confidence_errors(value, actual[key])) + return errors + if isinstance(reference, list) and isinstance(actual, list): + return [error for a, b in zip(reference, actual) for error in confidence_errors(a, b)] + return [] + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--model-dir", required=True) + parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32") + parser.add_argument("--allow-mismatch", action="store_true") + args = parser.parse_args() + torch.set_num_threads(4) + source = snapshot_download(MODEL_ID, revision=MODEL_REVISION) + native = AutoExtractor.from_pretrained(str(source), map_location="cpu").eval() + runtime = CoreMLBoundaryExtractor(args.model_dir, precision=args.precision) + cases = [] + for name, text, schema in fixtures(): + expected = native.extract(text, schema, include_confidence=True, include_spans=True) + actual = runtime.extract(text, schema, include_confidence=True, include_spans=True) + errors = confidence_errors(expected, actual) + cases.append( + { + "name": name, + "text": text, + "native": expected, + "coreml": actual, + "structure_match": without_confidence(expected) == without_confidence(actual), + "maximum_confidence_error": max(errors, default=None), + } + ) + result = { + "source_model": MODEL_ID, + "source_revision": MODEL_REVISION, + "precision": args.precision, + "selected_manifest": "eleven fixed real-text schema fixtures, not a Decision Index score", + "matched": sum(case["structure_match"] for case in cases), + "total": len(cases), + "cases": cases, + } + path = Path(args.model_dir) / f"verify-full-{args.precision}.json" + path.write_text(json.dumps(result, indent=2, ensure_ascii=False) + "\n") + print( + json.dumps( + { + "matched": result["matched"], + "total": result["total"], + "failed": [case["name"] for case in cases if not case["structure_match"]], + }, + indent=2, + ) + ) + if result["matched"] != result["total"] and not args.allow_mismatch: + raise SystemExit(1) + + +if __name__ == "__main__": + main()