benito47 commited on
Commit
fe5a411
·
verified ·
1 Parent(s): 434a065

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -0
README.md CHANGED
@@ -1,3 +1,64 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ # Introduction
6
+ This repository hosts [PaddleOCR PP-DocLayoutV3](https://huggingface.co/PaddlePaddle/PP-DocLayoutV3_safetensors),
7
+ an RT-DETR-based **document layout detector** (~33M params), for the
8
+ [React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch) library,
9
+ exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML, Vulkan). It finds and
10
+ classifies document regions — titles, paragraphs, tables, figures, formulas, headers/footers,
11
+ etc. — and is a companion to
12
+ [`react-native-executorch-paddleocr`](https://huggingface.co/software-mansion/react-native-executorch-PP-OCRv6).
13
+
14
+ If you'd like to run these models in your own ExecuTorch runtime, refer to the
15
+ [official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
16
+
17
+ The `.pte` is a pure tensor→tensor function; all pre/post-processing (resize, normalize, score
18
+ threshold, box convert) is the client's job.
19
+
20
+ ## Output contract
21
+
22
+ A single static method `forward`, fixed input (no buckets):
23
+
24
+ ```
25
+ in [1, 3, 800, 800] # RGB, ImageNet-normalized (x/255 - mean)/std
26
+ out logits [1, 300, 25] # 25 layout classes, per query (apply sigmoid)
27
+ pred_boxes [1, 300, 4] # (cx, cy, w, h), normalized [0,1]
28
+ ```
29
+
30
+ PP-DocLayoutV3 is a **DETR set-prediction** model → **no NMS**. Post-processing is just:
31
+ `score = sigmoid(logits)`, keep queries above a threshold, convert `(cx,cy,w,h) → (x1,y1,x2,y2)`,
32
+ scale to image size. Class names are in `labels.json` (index → label).
33
+
34
+ ### Classes (25)
35
+
36
+ `abstract, algorithm, aside_text, chart, content, formula, doc_title, figure_title, footer,
37
+ footnote, formula_number, header, image, number, paragraph_title, reference, reference_content,
38
+ seal, table, text, vision_footnote` (some indices map to the same display label; use
39
+ `labels.json` as the authoritative index→label map).
40
+
41
+ ## Backends, sizes & latency (warm)
42
+
43
+ | backend | target | precision | size | latency |
44
+ |---|---|---|---|---|
45
+ | `xnnpack` | CPU | fp32 | 132 MB | ~2.0 s (S24) |
46
+ | `coreml` | Apple ANE | fp16 | 91 MB | ANE fp16 |
47
+ | `vulkan` | Android GPU | fp16 (mixed-delegate) | **66 MB** | **~0.86 s (S24)** |
48
+
49
+ > **Vulkan is the recommended Android backend** — ~2.4× faster than XNNPACK and half the size.
50
+ > It's mixed-delegate: most of RT-DETR runs fp16 on the GPU, while the box-head matmuls run on
51
+ > XNNPACK (they delegate as `addmm`→`linear`). XNNPACK stays fp32 because RT-DETR's deformable
52
+ > attention feeds non-contiguous tensors that int8/portable paths mis-handle.
53
+
54
+
55
+
56
+ ## Compatibility
57
+
58
+ If you intend to use these models outside of React Native ExecuTorch, make sure your runtime is
59
+ compatible with the **ExecuTorch** version used to export the `.pte` files. For more details, see
60
+ the compatibility note in the
61
+ [ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
62
+ If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
63
+ runtime used behind the scenes.
64
+