msluszniak commited on
Commit
28277b0
·
verified ·
1 Parent(s): 177f558

Pin Core ML fp16 to CPU_AND_NE (6.0x on s, 2.8x on x)

Browse files

Pin compute_unit=CPU_AND_NE for the Core ML fp16 builds.

ExecuTorch defaults to ComputeUnit.ALL, which here splits the graph across GPU
and ANE and pays the transition cost. Pinning the ANE removes the GPU from
consideration. Measured on an iPhone 16 (ET 1.4.1, 10 warmup + 100 timed runs,
median of two interleaved repetitions, published build vs this one):

fast_sam s 52.45 ms -> 8.70 ms 6.0x
fast_sam x 108.85 ms -> 38.90 ms 2.8x

This is NOT a general rule and was not applied elsewhere. The same panel found
ALL already optimal for efficientnet, ssdlite, fcn, selfie and yolo26; CPU_AND_GPU
worse than ALL almost everywhere; lraspp identical at 5.90 vs 5.90 ms once
measured properly; and deeplab unchanged (0.98-1.06x).

Weights, contract and precision are unchanged; only the compile specs differ.

s/coreml/fast_sam_s_coreml_fp16.pte CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:845666c1d376d9c007490a05cacdf5db47f55a8662cb049b2bb0916e0f6e5f5e
3
- size 24218564
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b00b01db4c7c8f7f554a6e3d68e7ea344289c44700b8d989ff1d7335b8edff7b
3
+ size 24219702