msluszniak commited on
Commit
ad13e14
·
verified ·
1 Parent(s): eb4f497

PP-OCRv6: XNNPACK detect is now static int8

Browse files

Previously fp32 on the belief that static-activation int8 was unstable across
dynamic input sizes. Re-measured: correlation 0.956-0.994 against eager fp32
from 640 to 1280 square, and on-device the int8 build returns the same
detections ~15% faster end to end at 23.9 MB instead of 31.1 MB. The SVTR
recognizer stays fp32.

.gitattributes CHANGED
@@ -39,3 +39,4 @@ PP-OCRv6_xnnpack.pte filter=lfs diff=lfs merge=lfs -text
39
  coreml/pp_ocrv6_coreml_int8.pte filter=lfs diff=lfs merge=lfs -text
40
  vulkan/pp_ocrv6_vulkan_fp16.pte filter=lfs diff=lfs merge=lfs -text
41
  xnnpack/pp_ocrv6_xnnpack_fp32.pte filter=lfs diff=lfs merge=lfs -text
 
 
39
  coreml/pp_ocrv6_coreml_int8.pte filter=lfs diff=lfs merge=lfs -text
40
  vulkan/pp_ocrv6_vulkan_fp16.pte filter=lfs diff=lfs merge=lfs -text
41
  xnnpack/pp_ocrv6_xnnpack_fp32.pte filter=lfs diff=lfs merge=lfs -text
42
+ xnnpack/pp_ocrv6_xnnpack_int8.pte filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -62,7 +62,7 @@ whatever width it picks instead of inferring a ratio.
62
 
63
  | backend | target | detect | recognize | warm latency (detect @960² / recognize) |
64
  |---|---|---|---|---|
65
- | `xnnpack` | CPU | fp32, true-dynamic | fp32, true-dynamic | ~574 ms / ~28 ms (Galaxy S24) |
66
  | `coreml` | Apple ANE | weight-only int8, enumerated | weight-only int8, enumerated | ~12–15 ms / ~2 ms (Apple M-series ANE) |
67
  | `vulkan` | Android GPU | fp16, true-dynamic (resize) | fp32 on **XNNPACK** (mixed-delegate) | ~73 ms / ~27 ms (Galaxy S24, Xclipse 940) |
68
 
@@ -70,9 +70,12 @@ whatever width it picks instead of inferring a ratio.
70
  > (XNNPACK) — the 18 709-token vocab head is not Vulkan-safe, and int8 SVTR is lossy, so the
71
  > recognizer stays fp32 on CPU for correctness.
72
 
73
- > **Why fp32 on CPU?** Static-activation int8 quantization is not stable across dynamic input
74
- > sizes for the detector (measured broken at ≥960px including in the old per-bucket builds);
75
- > fp32 is bit-exact at every shape.
 
 
 
76
 
77
  ## CoreML notes (iOS)
78
 
 
62
 
63
  | backend | target | detect | recognize | warm latency (detect @960² / recognize) |
64
  |---|---|---|---|---|
65
+ | `xnnpack` | CPU | static int8, true-dynamic | fp32, true-dynamic | ~28 ms recognize (Galaxy S24) |
66
  | `coreml` | Apple ANE | weight-only int8, enumerated | weight-only int8, enumerated | ~12–15 ms / ~2 ms (Apple M-series ANE) |
67
  | `vulkan` | Android GPU | fp16, true-dynamic (resize) | fp32 on **XNNPACK** (mixed-delegate) | ~73 ms / ~27 ms (Galaxy S24, Xclipse 940) |
68
 
 
70
  > (XNNPACK) — the 18 709-token vocab head is not Vulkan-safe, and int8 SVTR is lossy, so the
71
  > recognizer stays fp32 on CPU for correctness.
72
 
73
+ > **Why int8 detect but fp32 recognize?** The detector is static-activation int8, calibrated on
74
+ > real pages at 1280. It was previously shipped as fp32 on the belief that int8 was unstable
75
+ > across dynamic input sizes; re-measured in 2026-08 that no longer holds (correlation 0.956 to
76
+ > 0.994 against eager fp32 from 640 to 1280 square, best at 1280), and on-device the int8 build
77
+ > returns the same detections about 15% faster end to end at 23.9 MB instead of 31.1 MB. The
78
+ > SVTR recognizer stays fp32: int8 is lossy on its attention stack.
79
 
80
  ## CoreML notes (iOS)
81
 
xnnpack/config.json CHANGED
@@ -10,11 +10,11 @@
10
  "license": "apache-2.0",
11
  "variants": [
12
  {
13
- "file": "pp_ocrv6_xnnpack_fp32.pte",
14
- "precision": "fp32",
15
- "quantized": false,
16
  "default": true,
17
- "size_bytes": 31066200,
18
  "methods": {
19
  "detect": {
20
  "inputs": [
 
10
  "license": "apache-2.0",
11
  "variants": [
12
  {
13
+ "file": "pp_ocrv6_xnnpack_int8.pte",
14
+ "precision": "int8",
15
+ "quantized": true,
16
  "default": true,
17
+ "size_bytes": 23890904,
18
  "methods": {
19
  "detect": {
20
  "inputs": [
xnnpack/pp_ocrv6_xnnpack_int8.pte ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:726d414858606c890e3909ce89d639e6bc70898b155f7042722fd13c3c2ea913
3
+ size 23890904