msluszniak commited on
Commit
e9d0267
·
verified ·
1 Parent(s): f3cedaa

Rebuild the Core ML fp16 build with compute_unit=CPU_AND_GPU

Browse files

This architecture is pathologically slow on the ANE. Measured on an iPhone 16
(ExecuTorch 1.4.1, 3 warmup + 15 timed runs, median, mean over the four styles):

fp16 ALL (previous) 209.4 ms ANE-eligible, and the slowest
fp16 CPU_ONLY 109.7 ms
fp16 CPU_AND_GPU 114.2 ms <- this build
fp32 ALL 152.8 ms dropped at v0.10.0

ExecuTorch defaults to ComputeUnit.ALL, under which fp16 is ANE-eligible and
fp32 is not (the ANE is fp16-only). That made fp32 look faster than fp16 and
briefly cast doubt on its removal; the comparison was measuring placement, not
precision. Excluding the ANE makes fp16 the fastest build at half of fp32's size.

CPU_AND_GPU rather than CPU_ONLY despite CPU_ONLY's slightly better mean:
CPU_ONLY ranges 99.6-119.9 ms across the four styles while CPU_AND_GPU holds
~114 ms on all four, so it should degrade less under CPU contention.

Weights, contract and precision are unchanged; only the compile specs differ.

coreml/style_transfer_candy_coreml_fp16.pte CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:529b53bfef717367a5467dbaf5e960dfbab4c1b039945bfed2f2d0c65c6a98cc
3
- size 3792956
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6886d832b7c3247195cde85c65e5c7611060bf5b14a6c2b9245ebadf568692a
3
+ size 3791869