fixed the readmes
Browse files
README.md
CHANGED
|
@@ -13,31 +13,61 @@ exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML and Vulkan ba
|
|
| 13 |
If you'd like to run these models in your own ExecuTorch runtime, refer to the
|
| 14 |
[official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
|
| 15 |
|
| 16 |
-
PP-OCRv6 is the **primary** OCR pipeline — smallest and fastest. It ships as **one fused
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
|
|
|
| 20 |
|
| 21 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
-
|
|
|
|
|
|
|
| 33 |
|
| 34 |
-
|
|
|
|
|
|
|
| 35 |
|
| 36 |
-
|
| 37 |
-
(+ a `1280×640` portrait method `detect_640x1280`)
|
| 38 |
-
- **recognize** (widths ÷8, height 48): `160, 320, 480, 640` → `recognize_160 … recognize_640`
|
| 39 |
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
## Compatibility
|
| 43 |
|
|
|
|
| 13 |
If you'd like to run these models in your own ExecuTorch runtime, refer to the
|
| 14 |
[official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
|
| 15 |
|
| 16 |
+
PP-OCRv6 is the **primary** OCR pipeline — smallest and fastest. It ships as **one fused
|
| 17 |
+
`.pte`** per backend with a single **dynamic** `detect` and `recognize` method each (no
|
| 18 |
+
per-size method buckets). The `.pte` is a pure tensor→tensor function; all pre/post-processing
|
| 19 |
+
(resize, normalize, DBNet box decode, perspective crop, CTC decode) is the client's job and is
|
| 20 |
+
driven by `config.json`. One model covers **all languages** (18 709-entry multilingual charset).
|
| 21 |
|
| 22 |
+
## Methods & I/O contract
|
| 23 |
+
|
| 24 |
+
| method | input | output |
|
| 25 |
+
|---|---|---|
|
| 26 |
+
| `detect` (DBNet) | `[1,3,H,W]` f32 RGB, **ImageNet-normalized by the client**: `(x/255 − mean)/std`, `mean=[0.485,0.456,0.406]`, `std=[0.229,0.224,0.225]` | `[1,1,H,W]` probability map (sigmoid baked) |
|
| 27 |
+
| `recognize` (SVTR) | `[1,3,48,W]` f32 RGB, client-normalized `(x/255 − 0.5)/0.5` | `[1,W/8,18709+1]` probs (softmax baked); `charset[i]` → logit `i+1`, blank = 0 |
|
| 28 |
+
|
| 29 |
+
**Nothing is baked for input normalization** — the client normalizes before calling.
|
| 30 |
+
Note the two methods use *different* norms (ImageNet for detect, `0.5/0.5` for recognize).
|
| 31 |
+
|
| 32 |
+
## Shape discovery (companion methods)
|
| 33 |
|
| 34 |
+
Every method carries exactly **one** no-arg discovery companion:
|
| 35 |
+
|
| 36 |
+
- `get_dynamic_dims_<method>` — dynamic method. Returns one `int32 [rank, 3]` tensor per
|
| 37 |
+
tensor input; each row is `[min, max, step]` (static dims are `[n, n, 1]`). Any conforming
|
| 38 |
+
shape is valid.
|
| 39 |
+
- `get_enum_shapes_<method>` — enumerated method. Returns one `int32 [N, rank]` tensor per
|
| 40 |
+
tensor input; each row is a complete legal shape (cross-dimension coupling is exact —
|
| 41 |
+
a listed `1280×640` does **not** imply `640×1280`). Snap inputs to the nearest row.
|
| 42 |
+
|
| 43 |
+
| backend | `detect` | `recognize` |
|
| 44 |
+
|---|---|---|
|
| 45 |
+
| `xnnpack`, `vulkan` | `get_dynamic_dims_detect` → H, W ∈ `[640, 1280]` step 32 | `get_dynamic_dims_recognize` → W ∈ `[160, 1280]` step 8 (H fixed 48) |
|
| 46 |
+
| `coreml` | `get_enum_shapes_detect` → `640²`, `960²`, `1280²`, `1280×640` | `get_enum_shapes_recognize` → widths `160, 320, 480, 640, 1280` |
|
| 47 |
+
|
| 48 |
+
## Backends
|
| 49 |
|
| 50 |
+
| backend | target | detect | recognize | warm latency (detect @960² / recognize) |
|
| 51 |
+
|---|---|---|---|---|
|
| 52 |
+
| `xnnpack` | CPU | fp32, true-dynamic | fp32, true-dynamic | ~574 ms / ~28 ms (Galaxy S24) |
|
| 53 |
+
| `coreml` | Apple ANE | weight-only int8, enumerated | weight-only int8, enumerated | ~12–15 ms / ~2 ms (Apple M-series ANE) |
|
| 54 |
+
| `vulkan` | Android GPU | fp16, true-dynamic (resize) | fp32 on **XNNPACK** (mixed-delegate) | ~73 ms / ~27 ms (Galaxy S24, Xclipse 940) |
|
| 55 |
|
| 56 |
+
> **Vulkan is mixed-delegate**: DBNet detects on the GPU, the SVTR recognizer runs on the CPU
|
| 57 |
+
> (XNNPACK) — the 18 709-token vocab head is not Vulkan-safe, and int8 SVTR is lossy, so the
|
| 58 |
+
> recognizer stays fp32 on CPU for correctness.
|
| 59 |
|
| 60 |
+
> **Why fp32 on CPU?** Static-activation int8 quantization is not stable across dynamic input
|
| 61 |
+
> sizes for the detector (measured broken at ≥960px — including in the old per-bucket builds);
|
| 62 |
+
> fp32 is bit-exact at every shape.
|
| 63 |
|
| 64 |
+
## CoreML notes (iOS)
|
|
|
|
|
|
|
| 65 |
|
| 66 |
+
- The CoreML `.pte` is a **multifunction** Core ML model (`detect` + `recognize` share one
|
| 67 |
+
precompiled `.mlmodelc`). Requires **iOS 18+** and an ExecuTorch runtime ≥ 1.3 (multifunction
|
| 68 |
+
loading via `functionName`).
|
| 69 |
+
- First-ever load on a device triggers a one-time per-shape ANE specialization (OS-cached
|
| 70 |
+
afterwards) — warm each model once after install.
|
| 71 |
|
| 72 |
## Compatibility
|
| 73 |
|