card: Neural Engine bundle section — 6-bit, why, speed, first-load cost, GSM8K 200

README.md

@@ -81,22 +81,44 @@ conversation history; `streamResponse(to:)` yields tokens as they decode.
 - Measure in Release — Debug is ~3× slower on per-token host work
 <!-- gen-cards:use-it end -->
 
-## Neural Engine bundle (iOS static export, AOT h18p) — 2026-09-15
+## Neural Engine bundle (iOS static export, AOT h18p) — 2026-09-16
 
-`ios-ane-h18p/` is Apple's stock `coreai.llm.export --platform iOS` static export of this checkpoint
-(k-means **4-bit palettized, group 32** (Apple's iOS default preset); embeddings int8; static graphs `prompt_opt`/`extend` ×
+`ios-ane-h18p/` is Apple's stock `coreai.llm.export --platform iOS` static export of this checkpoint at
+**6-bit k-means palettization, group 8** (the shape of Apple's own iOS preset for Qwen3-1.7B;
+`conversion/minicpm5_pal6_g8.yaml` in the zoo; embeddings int8; static graphs `prompt_opt`/`extend` ×
 contexts {256, 512, 1024, 2048, 4096} × query {8, 16, 64}), AOT-compiled with
 `xcrun coreai-build compile --platform iOS --preferred-compute neural-engine --architecture h18p`
-(31/31 ANE regions, 1.4 GB). It loads through Apple's `EngineFactory` → `StaticShapeEngine` unchanged
+(31/31 ANE regions, 2.5 GB). It loads through Apple's `EngineFactory` → `StaticShapeEngine` unchanged
 (iPhone 17-class devices). `ios-static/` is the same export before AOT — the portable IR; compile it
-for another chip yourself. 
-
-Gated on an iPhone 17 Pro (iOS 27.0) against the fp32 HF oracle: teacher-forced single-step sweep 24/24 + 8/8 + 16/16 and free-run greedy token-exact including the stop — transcript in the zoo,
-`models/minicpm5-2b/gate-minicpm5-2b-ane-device.json`. Speed on the same phone (Apple llm-benchmark
-method, 512-token prompt, 1024 generated, 5 trials, two back-to-back runs): decode **48.0 / 38.2 tok/s**, prefill 1858 / 1494 tok/s; footprint 2.0 GB. The Neural Engine is the power-efficient lane; these are
-not a GPU comparison (different protocol from the `int8/` row above).
-
-Same-day interleaved A/B on the same phone (one app embedding both bundles, 128-token prompt, 256 generated, 5 trials, ANE-GPU-ANE-GPU): Neural Engine 4-bit **55.4 / 51.5 tok/s** decode (1954 / 1793 prefill) vs this repo's `int8/` GPU bundle **22.8 / 20.6** (894 / 715) — 2.4× decode, but the ANE bundle also reads half the weight bytes per token, so this is what-ships vs what-ships, not equal precision; GPU cold specialization 18.4 s vs ANE AOT load 0.2 s.
+for another chip yourself.
+
+Why 6-bit: the 2026-09-15 bundle was Apple's iOS default (4-bit, group 32). It passed the three short
+gate prompts but diverged from fp32 at the first token of a 109-token free-form answer (and got the
+physics wrong). At 6-bit the same phone matches fp32 token-for-token on the three short prompts (24/24,
+8/8 including the stop, 16/16) and for 33 tokens of the long answer, then diverges on a step where fp32
+itself is nearly tied (top-2 margin 0.17); 5 of its 109 teacher-forced steps differ — transcript
+`models/minicpm5-2b/gate-minicpm5-2b-ane-6bit-device.json` in the zoo. An 8-bit shape is exact in an
+fp32 simulation of the recipe, but the 2B at 8 bits (2.16 GB of weights) loads on the ANE and never
+returns its first token, and 4-bit group 8 does not compile for the ANE at all — zoo
+`knowledge/ane-vs-gpu-iphone-2026-09.md` §9.
+
+Speed on an iPhone 17 Pro (iOS 27.0), same day, engine released and idle until the thermal state was
+back at `fair`, then 60 s of 128-token-prompt / 256-token trials: **38.5 tok/s decode** (flat over the
+minute), prefill 1679 tok/s, footprint 2.6 GB. In the same run the `int8/` GPU bundle decoded 23.8 tok/s;
+the previous 4-bit ANE bundle did ~55 (`models/minicpm5-2b/bench-iphone-ane-6bit-vs-gpu-2026-09-16.json`).
+**The first launch on a phone builds the ANE programs: about 23 minutes** (0.2 s afterwards; the cache
+lives in the app's container and is invalidated by an iOS update).
+
+Task accuracy, GSM8K test (first 200 questions, 0-shot CoT, greedy, no-think, max 640 new tokens; same
+prompt and scoring as the litertlm-convert evals) — `models/minicpm5-2b/gsm8k-200-2026-09-17.json` in the zoo:
+
+| model | correct / 200 |
+|---|---:|
+| fp32 checkpoint (bf16 on a Mac) | 172 (86.0 %) |
+| **`ios-ane-h18p/` 6-bit, on the iPhone 17 Pro** | **173 (86.5 %)** |
+| the same 6-bit recipe applied to the fp32 weights (Mac) | 172 (86.0 %) |
+| the `int8/` recipe applied to the fp32 weights (Mac) | 171 (85.5 %) |
+| the replaced 4-bit bundle, on the iPhone 17 Pro | 131 (65.5 %) |
 
 ## Measured