docs: add Windows ONNX Runtime usage (CPU / NPU / GPU) with WinML CLI
#21
by xieofxie - opened
README.md
CHANGED
|
@@ -17,4 +17,19 @@ The Table Transformer is equivalent to [DETR](https://huggingface.co/docs/transf
|
|
| 17 |
|
| 18 |
## Usage
|
| 19 |
|
| 20 |
-
You can use the raw model for detecting tables in documents. See the [documentation](https://huggingface.co/docs/transformers/main/en/model_doc/table-transformer) for more info.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
## Usage
|
| 19 |
|
| 20 |
+
You can use the raw model for detecting tables in documents. See the [documentation](https://huggingface.co/docs/transformers/main/en/model_doc/table-transformer) for more info.
|
| 21 |
+
|
| 22 |
+
### Run as ONNX (CPU / NPU / GPU)
|
| 23 |
+
|
| 24 |
+
Detect tables ~14× faster on a Windows NPU at half the model size, with mAP within 1% of the original PyTorch checkpoint — by exporting this model to ONNX. You can also export to ONNX to run on CPU or GPU.
|
| 25 |
+
|
| 26 |
+
Benchmarked on an Intel Core Ultra 7 258V (PubTables-1M validation, 1000 samples):
|
| 27 |
+
|
| 28 |
+
| Model | Device | Precision | mAP | mean latency (ms) | p50 latency (ms) | Size (MB) |
|
| 29 |
+
|---------|--------------|-------------|--------|-------------------|------------------|-----------|
|
| 30 |
+
| PyTorch | CPU | fp32 | 0.9887 | 620.9 | 600.3 | 115 |
|
| 31 |
+
| ONNX | OpenVINO NPU | w8a16 (QDQ) | 0.9822 | 44.1 | 41.6 | 58 |
|
| 32 |
+
|
| 33 |
+
- **How to convert** — Export and quantize with [Microsoft's WinML CLI](https://github.com/microsoft/winml-cli). The NPU build is QDQ-quantized to w8a16; fp32 builds for CPU and GPU are also supported. End-to-end build, evaluation, and a Python inference example: [examples/microsoft-table-transformer-detection](https://github.com/microsoft/winml-cli/blob/main/examples/microsoft-table-transformer-detection/README.md).
|
| 34 |
+
- **How to run on Windows** — Use [Windows ML](https://learn.microsoft.com/en-us/windows/ai/new-windows-ml/overview), which manages execution providers for NPU / GPU / CPU and routes ONNX inference to the right backend automatically.
|
| 35 |
+
- **How to run on other platforms** — Use [ONNX Runtime](https://onnxruntime.ai/docs/) with the execution provider of your choice (OpenVINO, QNN, DirectML, CUDA, CPU, etc.).
|