Rebuild the Core ML fp16 build with compute_unit=CPU_AND_GPU
Browse filesThis architecture is pathologically slow on the ANE. Measured on an iPhone 16
(ExecuTorch 1.4.1, 3 warmup + 15 timed runs, median, mean over the four styles):
fp16 ALL (previous) 209.4 ms ANE-eligible, and the slowest
fp16 CPU_ONLY 109.7 ms
fp16 CPU_AND_GPU 114.2 ms <- this build
fp32 ALL 152.8 ms dropped at v0.10.0
ExecuTorch defaults to ComputeUnit.ALL, under which fp16 is ANE-eligible and
fp32 is not (the ANE is fp16-only). That made fp32 look faster than fp16 and
briefly cast doubt on its removal; the comparison was measuring placement, not
precision. Excluding the ANE makes fp16 the fastest build at half of fp32's size.
CPU_AND_GPU rather than CPU_ONLY despite CPU_ONLY's slightly better mean:
CPU_ONLY ranges 99.6-119.9 ms across the four styles while CPU_AND_GPU holds
~114 ms on all four, so it should degrade less under CPU contention.
Weights, contract and precision are unchanged; only the compile specs differ.
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e6886d832b7c3247195cde85c65e5c7611060bf5b14a6c2b9245ebadf568692a
|
| 3 |
+
size 3791869
|