mlboydaisuke/Holo2-4B-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).

This model has no row on DeviceMark, the on-device LLM leaderboard.

Holo2-4B — Core AI (on-device, iPhone) · GUI-grounding VLM

Hcompany/Holo2-4B converted to Apple Core AI for on-device inference, served by the CoreAIChat app.

Holo2 is H Company's computer-use / GUI-grounding vision-language model: given a screenshot and an instruction ("click the submit button"), it predicts the click coordinates / locates the UI element. Built on the Qwen3-VL-4B backbone, so it rides the Core AI zoo's existing Qwen3-VL pipeline. The zoo's first GUI-grounding / computer-use model.

Use it

New to Core AI? Start with CoreAIKit 0.7.1. Follow its requirements and first-run steps for qwen3-0.6b, then open the same release's ChatDemo. The README records the tested OS/SDK and download size; model and device coverage is stated per example.

▶️ Run it (source) — the VLChat runner (GUI + CLI, one app for every vision-language model in the catalog):

git clone --branch 0.7.1 --depth 1 https://github.com/john-rocky/coreai-kit
export DEVELOPER_DIR=/Applications/Xcode-27.0.0-RC.app/Contents/Developer
open -a /Applications/Xcode-27.0.0-RC.app coreai-kit/Examples/VLChat/VLChat.xcodeproj
# → Run, then pick "Holo2 4B" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/VLChat
swift run -c release vlchat-cli --model holo2-4b --image screenshot.png --prompt "Localize an element on the GUI image according to my instructions and output a click position as Click(x, y) with x num pixels from the left edge and y num pixels from the top edge. Instruction: click the Submit button."

Use Xcode build 27A266a from the release's .xcode-pin; adjust the app path if your installation is named differently.

💻 Build with it — complete; the glue is kit API, copy-paste runs:

import CoreAIKit
import FoundationModels

let vlm = try await KitVisionModel(catalog: "holo2-4b")
let session = LanguageModelSession(model: vlm)
let image = try ImageFile.load(imageURL)  // any image file → CGImage + EXIF orientation
let reply = try await session.respond(to: Prompt {
    prompt
    Attachment(image.cgImage, orientation: image.orientation)
})
// reply.content: "Click(x, y)" in 0-1000-normalized coordinates for a grounding prompt,
// or a plain answer for a normal question - all generated on-device

The take-home is Examples/VLChat/Sources/QuickStart.swift — this exact code as one typed function, no UI; the CLI is an argument shell over it, and the GUI drives the same KitVisionModel(catalog:) behind a LanguageModelSession. Holo2 is a GUI-grounding model: feed a screenshot and H Company's localization prompt (see the card's grounding section) and it returns Click(x, y) in 0-1000-normalized coordinates — multiply by imageSize / 1000 for pixels. It also answers free-form questions like its Qwen3-VL base.

Integration checklist

• SPM: https://github.com/john-rocky/coreai-kit (exact 0.7.1) → product CoreAIKit
• Info.plist: NSPhotoLibraryUsageDescription — only if you use PhotosPicker
• Entitlements (iOS): com.apple.developer.kernel.increased-memory-limit
• First run downloads the model — ~5,484 MB (Mac) / ~5,484 MB (iPhone) — then it loads from the local cache (Application Support; progress via the downloadProgress callback)
• Measure in Release — Debug is ~3× slower on per-token host work

Contents (gpu-pipelined/)

• holo2_4b_decode_int8lin_s1/ — the decode bundle (static query=1, per-block-32 int8 linear body; rides Apple's coreai-pipelined GPU engine, specializes on-device — no AOT needed). ~4.4 GB.
• holo2_4b_vision/ — the fixed-grid vision encoder .aimodel (fp16): patches [784,1536] -> (image_embeds [196,2560], deepstack [3,196,2560]). Run once per image. ~0.8 GB.

Parity (vs fp32 HF oracle, Core AI GPU engine)

• Vision: image-embeds cos 0.999983, deepstack cos 0.999989.
• Decoder (int8lin): teacher-forced S=1 sweep 4/4, 16/16 decode steps token-exact, HF-seeded decode match. All PASS.

Use

Install CoreAIChat, pick Holo2 4B, attach a screenshot, and ask where an element is / what to click — it grounds the instruction to the image.

License

Apache-2.0, inherited from the base model Hcompany/Holo2-4B. See LICENSE.

---

More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.