Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,64 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
|
| 5 |
+
# Introduction
|
| 6 |
+
This repository hosts [PaddleOCR PP-DocLayoutV3](https://huggingface.co/PaddlePaddle/PP-DocLayoutV3_safetensors),
|
| 7 |
+
an RT-DETR-based **document layout detector** (~33M params), for the
|
| 8 |
+
[React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch) library,
|
| 9 |
+
exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML, Vulkan). It finds and
|
| 10 |
+
classifies document regions — titles, paragraphs, tables, figures, formulas, headers/footers,
|
| 11 |
+
etc. — and is a companion to
|
| 12 |
+
[`react-native-executorch-paddleocr`](https://huggingface.co/software-mansion/react-native-executorch-PP-OCRv6).
|
| 13 |
+
|
| 14 |
+
If you'd like to run these models in your own ExecuTorch runtime, refer to the
|
| 15 |
+
[official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
|
| 16 |
+
|
| 17 |
+
The `.pte` is a pure tensor→tensor function; all pre/post-processing (resize, normalize, score
|
| 18 |
+
threshold, box convert) is the client's job.
|
| 19 |
+
|
| 20 |
+
## Output contract
|
| 21 |
+
|
| 22 |
+
A single static method `forward`, fixed input (no buckets):
|
| 23 |
+
|
| 24 |
+
```
|
| 25 |
+
in [1, 3, 800, 800] # RGB, ImageNet-normalized (x/255 - mean)/std
|
| 26 |
+
out logits [1, 300, 25] # 25 layout classes, per query (apply sigmoid)
|
| 27 |
+
pred_boxes [1, 300, 4] # (cx, cy, w, h), normalized [0,1]
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
PP-DocLayoutV3 is a **DETR set-prediction** model → **no NMS**. Post-processing is just:
|
| 31 |
+
`score = sigmoid(logits)`, keep queries above a threshold, convert `(cx,cy,w,h) → (x1,y1,x2,y2)`,
|
| 32 |
+
scale to image size. Class names are in `labels.json` (index → label).
|
| 33 |
+
|
| 34 |
+
### Classes (25)
|
| 35 |
+
|
| 36 |
+
`abstract, algorithm, aside_text, chart, content, formula, doc_title, figure_title, footer,
|
| 37 |
+
footnote, formula_number, header, image, number, paragraph_title, reference, reference_content,
|
| 38 |
+
seal, table, text, vision_footnote` (some indices map to the same display label; use
|
| 39 |
+
`labels.json` as the authoritative index→label map).
|
| 40 |
+
|
| 41 |
+
## Backends, sizes & latency (warm)
|
| 42 |
+
|
| 43 |
+
| backend | target | precision | size | latency |
|
| 44 |
+
|---|---|---|---|---|
|
| 45 |
+
| `xnnpack` | CPU | fp32 | 132 MB | ~2.0 s (S24) |
|
| 46 |
+
| `coreml` | Apple ANE | fp16 | 91 MB | ANE fp16 |
|
| 47 |
+
| `vulkan` | Android GPU | fp16 (mixed-delegate) | **66 MB** | **~0.86 s (S24)** |
|
| 48 |
+
|
| 49 |
+
> **Vulkan is the recommended Android backend** — ~2.4× faster than XNNPACK and half the size.
|
| 50 |
+
> It's mixed-delegate: most of RT-DETR runs fp16 on the GPU, while the box-head matmuls run on
|
| 51 |
+
> XNNPACK (they delegate as `addmm`→`linear`). XNNPACK stays fp32 because RT-DETR's deformable
|
| 52 |
+
> attention feeds non-contiguous tensors that int8/portable paths mis-handle.
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
## Compatibility
|
| 57 |
+
|
| 58 |
+
If you intend to use these models outside of React Native ExecuTorch, make sure your runtime is
|
| 59 |
+
compatible with the **ExecuTorch** version used to export the `.pte` files. For more details, see
|
| 60 |
+
the compatibility note in the
|
| 61 |
+
[ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
|
| 62 |
+
If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
|
| 63 |
+
runtime used behind the scenes.
|
| 64 |
+
|