benito47 commited on
Commit
cfdeaac
·
1 Parent(s): 0fd55a0

fixed the readmes

Browse files
Files changed (1) hide show
  1. README.md +48 -18
README.md CHANGED
@@ -13,31 +13,61 @@ exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML and Vulkan ba
13
  If you'd like to run these models in your own ExecuTorch runtime, refer to the
14
  [official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
15
 
16
- PP-OCRv6 is the **primary** OCR pipeline — smallest and fastest. It ships as **one fused `.pte`** per backend, using **static bucketed
17
- methods**. The `.pte` is a pure tensor→tensor function; all pre/post-processing (resize,
18
- normalize, DBNet box decode, perspective crop, CTC decode) is the client's job and is driven by
19
- `config.json`. One model covers **all languages** (18 709-entry multilingual charset).
 
20
 
21
- ## Backends
 
 
 
 
 
 
 
 
 
 
22
 
23
- | backend | target | detect | recognize |
24
- |---|---|---|---|
25
- | `xnnpack` | CPU | int8 (DBNet) | fp32 (SVTR) |
26
- | `coreml` | Apple ANE | weight-only int8 | weight-only int8 |
27
- | `vulkan` | Android GPU | fp16 (GPU) | fp32 on **XNNPACK** (mixed-delegate) |
 
 
 
 
 
 
 
 
 
 
28
 
29
- > **Vulkan is mixed-delegate**: DBNet detects on the GPU, but the SVTR recognizer runs on the
30
- > CPU (XNNPACK). int8 SVTR is lossy on the 18 709-token vocab — so the recognizer stays fp32 on CPU for correctness.
 
 
 
31
 
32
- ## Buckets
 
 
33
 
34
- `is_bucketed()` reports `[detect sides ; recognize widths]`:
 
 
35
 
36
- - **detect** (square sides ÷32): `640, 960, 1280` → `detect_640 / detect_960 / detect_1280`
37
- (+ a `1280×640` portrait method `detect_640x1280`)
38
- - **recognize** (widths ÷8, height 48): `160, 320, 480, 640` → `recognize_160 … recognize_640`
39
 
40
- Detector input is RGB, normalized `(x/255 0.5)/0.5`.
 
 
 
 
41
 
42
  ## Compatibility
43
 
 
13
  If you'd like to run these models in your own ExecuTorch runtime, refer to the
14
  [official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
15
 
16
+ PP-OCRv6 is the **primary** OCR pipeline — smallest and fastest. It ships as **one fused
17
+ `.pte`** per backend with a single **dynamic** `detect` and `recognize` method each (no
18
+ per-size method buckets). The `.pte` is a pure tensor→tensor function; all pre/post-processing
19
+ (resize, normalize, DBNet box decode, perspective crop, CTC decode) is the client's job and is
20
+ driven by `config.json`. One model covers **all languages** (18 709-entry multilingual charset).
21
 
22
+ ## Methods & I/O contract
23
+
24
+ | method | input | output |
25
+ |---|---|---|
26
+ | `detect` (DBNet) | `[1,3,H,W]` f32 RGB, **ImageNet-normalized by the client**: `(x/255 − mean)/std`, `mean=[0.485,0.456,0.406]`, `std=[0.229,0.224,0.225]` | `[1,1,H,W]` probability map (sigmoid baked) |
27
+ | `recognize` (SVTR) | `[1,3,48,W]` f32 RGB, client-normalized `(x/255 − 0.5)/0.5` | `[1,W/8,18709+1]` probs (softmax baked); `charset[i]` → logit `i+1`, blank = 0 |
28
+
29
+ **Nothing is baked for input normalization** — the client normalizes before calling.
30
+ Note the two methods use *different* norms (ImageNet for detect, `0.5/0.5` for recognize).
31
+
32
+ ## Shape discovery (companion methods)
33
 
34
+ Every method carries exactly **one** no-arg discovery companion:
35
+
36
+ - `get_dynamic_dims_<method>` dynamic method. Returns one `int32 [rank, 3]` tensor per
37
+ tensor input; each row is `[min, max, step]` (static dims are `[n, n, 1]`). Any conforming
38
+ shape is valid.
39
+ - `get_enum_shapes_<method>` — enumerated method. Returns one `int32 [N, rank]` tensor per
40
+ tensor input; each row is a complete legal shape (cross-dimension coupling is exact —
41
+ a listed `1280×640` does **not** imply `640×1280`). Snap inputs to the nearest row.
42
+
43
+ | backend | `detect` | `recognize` |
44
+ |---|---|---|
45
+ | `xnnpack`, `vulkan` | `get_dynamic_dims_detect` → H, W ∈ `[640, 1280]` step 32 | `get_dynamic_dims_recognize` → W ∈ `[160, 1280]` step 8 (H fixed 48) |
46
+ | `coreml` | `get_enum_shapes_detect` → `640²`, `960²`, `1280²`, `1280×640` | `get_enum_shapes_recognize` → widths `160, 320, 480, 640, 1280` |
47
+
48
+ ## Backends
49
 
50
+ | backend | target | detect | recognize | warm latency (detect @960² / recognize) |
51
+ |---|---|---|---|---|
52
+ | `xnnpack` | CPU | fp32, true-dynamic | fp32, true-dynamic | ~574 ms / ~28 ms (Galaxy S24) |
53
+ | `coreml` | Apple ANE | weight-only int8, enumerated | weight-only int8, enumerated | ~12–15 ms / ~2 ms (Apple M-series ANE) |
54
+ | `vulkan` | Android GPU | fp16, true-dynamic (resize) | fp32 on **XNNPACK** (mixed-delegate) | ~73 ms / ~27 ms (Galaxy S24, Xclipse 940) |
55
 
56
+ > **Vulkan is mixed-delegate**: DBNet detects on the GPU, the SVTR recognizer runs on the CPU
57
+ > (XNNPACK) — the 18 709-token vocab head is not Vulkan-safe, and int8 SVTR is lossy, so the
58
+ > recognizer stays fp32 on CPU for correctness.
59
 
60
+ > **Why fp32 on CPU?** Static-activation int8 quantization is not stable across dynamic input
61
+ > sizes for the detector (measured broken at ≥960px — including in the old per-bucket builds);
62
+ > fp32 is bit-exact at every shape.
63
 
64
+ ## CoreML notes (iOS)
 
 
65
 
66
+ - The CoreML `.pte` is a **multifunction** Core ML model (`detect` + `recognize` share one
67
+ precompiled `.mlmodelc`). Requires **iOS 18+** and an ExecuTorch runtime ≥ 1.3 (multifunction
68
+ loading via `functionName`).
69
+ - First-ever load on a device triggers a one-time per-shape ANE specialization (OS-cached
70
+ afterwards) — warm each model once after install.
71
 
72
  ## Compatibility
73