Image-Text-to-Text
Transformers
Safetensors
English
Chinese
multilingual
qwen3_5_moe
ocr
pdf
document-parsing
document-understanding
layout-analysis
table-recognition
chart-parsing
formula-recognition
chemical-formula
markdown
vision-language
infinity-parser
infinity_parser2
conversational
Eval Results
Instructions to use infly/Infinity-Parser2-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use infly/Infinity-Parser2-Pro with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="infly/Infinity-Parser2-Pro") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("infly/Infinity-Parser2-Pro") model = AutoModelForMultimodalLM.from_pretrained("infly/Infinity-Parser2-Pro", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use infly/Infinity-Parser2-Pro with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "infly/Infinity-Parser2-Pro" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "infly/Infinity-Parser2-Pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/infly/Infinity-Parser2-Pro
- SGLang
How to use infly/Infinity-Parser2-Pro with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "infly/Infinity-Parser2-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "infly/Infinity-Parser2-Pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "infly/Infinity-Parser2-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "infly/Infinity-Parser2-Pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use infly/Infinity-Parser2-Pro with Docker Model Runner:
docker model run hf.co/infly/Infinity-Parser2-Pro
Update README.md
Browse files
README.md
CHANGED
|
@@ -33,7 +33,7 @@ datasets:
|
|
| 33 |
<p align="center">
|
| 34 |
💻 <a href="https://github.com/infly-ai/INF-MLLM">Github</a> |
|
| 35 |
📊 <a href="https://huggingface.co/datasets/infly/Infinity-Doc2-5M">Dataset</a> |
|
| 36 |
-
📄 <a
|
| 37 |
🚀 <a href="https://huggingface.co/spaces/infly/Infinity-Parser2-Demo">Demo</a>
|
| 38 |
</p>
|
| 39 |
|
|
@@ -47,13 +47,13 @@ datasets:
|
|
| 47 |
|
| 48 |
## Introduction
|
| 49 |
|
| 50 |
-
We are excited to release Infinity-Parser2, our latest flagship document understanding model. We offer two distinct variants to address diverse deployment constraints: Infinity-Parser2-Pro, optimized for maximum accuracy in precision-critical tasks, achieves state-of-the-art results on olmOCR-Bench (87.6%) and ParseBench (74.3%), surpassing frontier models including DeepSeek-OCR-2, PaddleOCR-VL-1.5, and
|
| 51 |
|
| 52 |
### Key Features
|
| 53 |
|
| 54 |
- **Upgraded Data Engine**: We have comprehensively enhanced our synthetic data engine to support both fixed-layout and flexible-layout document formats. By curating nearly 5 million diverse document parsing samples across a wide range of layouts, combined with a dynamic adaptive sampling strategy, we ensure highly balanced and robust multi-task learning across various document types.
|
| 55 |
- **Multi-Task Reinforcement Learning**: We designed a novel verifiable reward system to support Joint Reinforcement Learning (RL), enabling seamless and simultaneous co-optimization of multiple complex tasks, including document parsing, element parsing, chart parsing, chemical formula parsing, document vqa, and general multimodal understanding.
|
| 56 |
-
- **Breakthrough Parsing Performance**: Infinity-Parser2-Pro substantially outperforms our previous 7B model, achieving 87.6% on olmOCR-Bench and 74.3% on ParseBench, surpassing frontier models such as DeepSeek-OCR-2, PaddleOCR-VL, and
|
| 57 |
- **Inference Acceleration**: Infinity-Parser2-Flash delivers significantly higher efficiency than Infinity-Parser-7B, with inference throughput increased by 3.68x (from 441 to 1,624 tokens/sec), reducing both deployment latency and costs.
|
| 58 |
|
| 59 |
## Performance
|
|
@@ -74,7 +74,7 @@ We are excited to release Infinity-Parser2, our latest flagship document underst
|
|
| 74 |
<th>Infinity-Parser2-Flash</th>
|
| 75 |
<th>PaddleOCR-VL-1.5</th>
|
| 76 |
<th>DeepSeek-OCR-2</th>
|
| 77 |
-
<th>
|
| 78 |
<th>Gemini-3-Pro</th>
|
| 79 |
</tr>
|
| 80 |
</thead>
|
|
@@ -86,7 +86,7 @@ We are excited to release Infinity-Parser2, our latest flagship document underst
|
|
| 86 |
<td>olmOCR-bench</td>
|
| 87 |
<td><b>87.6</b></td>
|
| 88 |
<td>86.0</td>
|
| 89 |
-
<td>
|
| 90 |
<td>76.3</td>
|
| 91 |
<td>75.2</td>
|
| 92 |
<td>-</td>
|
|
@@ -95,7 +95,7 @@ We are excited to release Infinity-Parser2, our latest flagship document underst
|
|
| 95 |
<td>ParseBench</td>
|
| 96 |
<td><b>74.3</b></td>
|
| 97 |
<td>72.2</td>
|
| 98 |
-
<td>
|
| 99 |
<td>41.2</td>
|
| 100 |
<td>45.9</td>
|
| 101 |
<td>69.1‡</td>
|
|
@@ -144,9 +144,9 @@ We are excited to release Infinity-Parser2, our latest flagship document underst
|
|
| 144 |
</tr>
|
| 145 |
<tr>
|
| 146 |
<td>OmniDocBench-v1.5-TextBlock</td>
|
| 147 |
-
<td>
|
| 148 |
-
<td>
|
| 149 |
-
<td>
|
| 150 |
<td>84.13*</td>
|
| 151 |
<td>86.00</td>
|
| 152 |
<td>-</td>
|
|
@@ -174,10 +174,10 @@ We are excited to release Infinity-Parser2, our latest flagship document underst
|
|
| 174 |
</tr>
|
| 175 |
<tr>
|
| 176 |
<td>Chart2Table</td>
|
| 177 |
-
<td>
|
| 178 |
<td>80.49</td>
|
| 179 |
-
<td>
|
| 180 |
-
<td>
|
| 181 |
<td>-</td>
|
| 182 |
<td>-</td>
|
| 183 |
</tr>
|
|
@@ -195,10 +195,10 @@ We are excited to release Infinity-Parser2, our latest flagship document underst
|
|
| 195 |
</tr>
|
| 196 |
<tr>
|
| 197 |
<td>CoSyn_Chemical</td>
|
| 198 |
-
<td><b>
|
| 199 |
-
<td>
|
| 200 |
<td>-</td>
|
| 201 |
-
<td>
|
| 202 |
<td>-</td>
|
| 203 |
<td>-</td>
|
| 204 |
</tr>
|
|
@@ -292,7 +292,7 @@ We are excited to release Infinity-Parser2, our latest flagship document underst
|
|
| 292 |
</tbody>
|
| 293 |
</table>
|
| 294 |
|
| 295 |
-
Note: '*' denotes results evaluated using our internal evaluation tools.
|
| 296 |
|
| 297 |
## Quick Start
|
| 298 |
|
|
@@ -577,6 +577,20 @@ Infinity-Parser2 has several known limitations to consider. It primarily support
|
|
| 577 |
|
| 578 |
We would like to thank [Qwen3.5](https://github.com/QwenLM/Qwen3.5), [ms-swift](https://github.com/modelscope/ms-swift), [VeRL](https://github.com/verl-project/verl), [lmms-eval](https://github.com/EvolvingLMMs-Lab/lmms-eval), [olmocr](https://huggingface.co/datasets/allenai/olmOCR-bench), [PaddleOCR-VL](https://github.com/PaddlePaddle/PaddleOCR), [MinerU](https://github.com/opendatalab/MinerU), [dots.ocr](https://github.com/rednote-hilab/dots.ocr), [Chandra-OCR-2](https://github.com/datalab-to/chandra) for providing dataset, code and models.
|
| 579 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 580 |
# License
|
| 581 |
|
| 582 |
This model is licensed under apache-2.0.
|
|
|
|
| 33 |
<p align="center">
|
| 34 |
💻 <a href="https://github.com/infly-ai/INF-MLLM">Github</a> |
|
| 35 |
📊 <a href="https://huggingface.co/datasets/infly/Infinity-Doc2-5M">Dataset</a> |
|
| 36 |
+
📄 <a href="https://arxiv.org/pdf/2607.07836v1">Paper</a> |
|
| 37 |
🚀 <a href="https://huggingface.co/spaces/infly/Infinity-Parser2-Demo">Demo</a>
|
| 38 |
</p>
|
| 39 |
|
|
|
|
| 47 |
|
| 48 |
## Introduction
|
| 49 |
|
| 50 |
+
We are excited to release Infinity-Parser2, our latest flagship document understanding model. We offer two distinct variants to address diverse deployment constraints: Infinity-Parser2-Pro, optimized for maximum accuracy in precision-critical tasks, achieves state-of-the-art results on olmOCR-Bench (87.6%) and ParseBench (74.3%), surpassing frontier models including DeepSeek-OCR-2, PaddleOCR-VL-1.5, and MinerU2.5. Infinity-Parser2-Flash, engineered for low-latency inference, delivers a 3.68x speedup over our previous Infinity-Parser-7B model. With significant upgrades to both our data engine and multi-task reinforcement learning approach, the model consolidates robust multi-modal parsing capabilities into a unified architecture, unlocking brand-new zero-shot capabilities across a wide range of real-world business scenarios.
|
| 51 |
|
| 52 |
### Key Features
|
| 53 |
|
| 54 |
- **Upgraded Data Engine**: We have comprehensively enhanced our synthetic data engine to support both fixed-layout and flexible-layout document formats. By curating nearly 5 million diverse document parsing samples across a wide range of layouts, combined with a dynamic adaptive sampling strategy, we ensure highly balanced and robust multi-task learning across various document types.
|
| 55 |
- **Multi-Task Reinforcement Learning**: We designed a novel verifiable reward system to support Joint Reinforcement Learning (RL), enabling seamless and simultaneous co-optimization of multiple complex tasks, including document parsing, element parsing, chart parsing, chemical formula parsing, document vqa, and general multimodal understanding.
|
| 56 |
+
- **Breakthrough Parsing Performance**: Infinity-Parser2-Pro substantially outperforms our previous 7B model, achieving 87.6% on olmOCR-Bench and 74.3% on ParseBench, surpassing frontier models such as DeepSeek-OCR-2, PaddleOCR-VL, and MinerU2.5.
|
| 57 |
- **Inference Acceleration**: Infinity-Parser2-Flash delivers significantly higher efficiency than Infinity-Parser-7B, with inference throughput increased by 3.68x (from 441 to 1,624 tokens/sec), reducing both deployment latency and costs.
|
| 58 |
|
| 59 |
## Performance
|
|
|
|
| 74 |
<th>Infinity-Parser2-Flash</th>
|
| 75 |
<th>PaddleOCR-VL-1.5</th>
|
| 76 |
<th>DeepSeek-OCR-2</th>
|
| 77 |
+
<th>MinerU2.5</th>
|
| 78 |
<th>Gemini-3-Pro</th>
|
| 79 |
</tr>
|
| 80 |
</thead>
|
|
|
|
| 86 |
<td>olmOCR-bench</td>
|
| 87 |
<td><b>87.6</b></td>
|
| 88 |
<td>86.0</td>
|
| 89 |
+
<td>78.5</td>
|
| 90 |
<td>76.3</td>
|
| 91 |
<td>75.2</td>
|
| 92 |
<td>-</td>
|
|
|
|
| 95 |
<td>ParseBench</td>
|
| 96 |
<td><b>74.3</b></td>
|
| 97 |
<td>72.2</td>
|
| 98 |
+
<td>66.0</td>
|
| 99 |
<td>41.2</td>
|
| 100 |
<td>45.9</td>
|
| 101 |
<td>69.1‡</td>
|
|
|
|
| 144 |
</tr>
|
| 145 |
<tr>
|
| 146 |
<td>OmniDocBench-v1.5-TextBlock</td>
|
| 147 |
+
<td><b>95.05</b></td>
|
| 148 |
+
<td>94.31</td>
|
| 149 |
+
<td>94.97*</td>
|
| 150 |
<td>84.13*</td>
|
| 151 |
<td>86.00</td>
|
| 152 |
<td>-</td>
|
|
|
|
| 174 |
</tr>
|
| 175 |
<tr>
|
| 176 |
<td>Chart2Table</td>
|
| 177 |
+
<td><b>86.5</b></td>
|
| 178 |
<td>80.49</td>
|
| 179 |
+
<td>86.2*</td>
|
| 180 |
+
<td>49.7*</td>
|
| 181 |
<td>-</td>
|
| 182 |
<td>-</td>
|
| 183 |
</tr>
|
|
|
|
| 195 |
</tr>
|
| 196 |
<tr>
|
| 197 |
<td>CoSyn_Chemical</td>
|
| 198 |
+
<td><b>73.19</b></td>
|
| 199 |
+
<td>63.34</td>
|
| 200 |
<td>-</td>
|
| 201 |
+
<td>47.02*</td>
|
| 202 |
<td>-</td>
|
| 203 |
<td>-</td>
|
| 204 |
</tr>
|
|
|
|
| 292 |
</tbody>
|
| 293 |
</table>
|
| 294 |
|
| 295 |
+
Note: '*' denotes results evaluated using our internal evaluation tools. '‡' denotes results from the Gemini-3.1-Pro.
|
| 296 |
|
| 297 |
## Quick Start
|
| 298 |
|
|
|
|
| 577 |
|
| 578 |
We would like to thank [Qwen3.5](https://github.com/QwenLM/Qwen3.5), [ms-swift](https://github.com/modelscope/ms-swift), [VeRL](https://github.com/verl-project/verl), [lmms-eval](https://github.com/EvolvingLMMs-Lab/lmms-eval), [olmocr](https://huggingface.co/datasets/allenai/olmOCR-bench), [PaddleOCR-VL](https://github.com/PaddlePaddle/PaddleOCR), [MinerU](https://github.com/opendatalab/MinerU), [dots.ocr](https://github.com/rednote-hilab/dots.ocr), [Chandra-OCR-2](https://github.com/datalab-to/chandra) for providing dataset, code and models.
|
| 579 |
|
| 580 |
+
## Citation
|
| 581 |
+
|
| 582 |
+
```
|
| 583 |
+
@misc{huang2026infinityparser2technicalreport,
|
| 584 |
+
title={Infinity-Parser2 Technical Report},
|
| 585 |
+
author={Zuming Huang and Jun Huang and Kexuan Ren and Baode Wang and Weizhen Li and Jianming Feng and Yu Wang and Yichen Yao and Shijun Lin and Yige Tang and Cheng Peng and Weidi Xu and Wei Chu and Yinghui Xu and Yuan Qi},
|
| 586 |
+
year={2026},
|
| 587 |
+
eprint={2607.07836},
|
| 588 |
+
archivePrefix={arXiv},
|
| 589 |
+
primaryClass={cs.AI},
|
| 590 |
+
url={https://arxiv.org/abs/2607.07836},
|
| 591 |
+
}
|
| 592 |
+
```
|
| 593 |
+
|
| 594 |
# License
|
| 595 |
|
| 596 |
This model is licensed under apache-2.0.
|