README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen2.5-VL-7B-Instruct
tags:
- multimodal
- reasoning
- vision-r1
- qwen2.5-vl
- chain-of-thought
---

# Vision-R1-CI-7B

Vision-R1-CI-7B is a multimodal reasoning model that serves as the **Cold-start Initialization (CI)** checkpoint for the **Vision-R1** project. It is introduced in the paper [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://arxiv.org/abs/2503.06749).

- **GitHub Repository:** [Osilly/Vision-R1](https://github.com/Osilly/Vision-R1)
- **Paper:** [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://arxiv.org/abs/2503.06749)
- **Base Model:** [Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)

## Model Description

Vision-R1-CI (Cold-start Initialized) is a 7B parameter multimodal large language model (MLLM) developed to bridge the gap between standard vision-language tasks and complex reasoning. It was obtained by fine-tuning the **Qwen2.5-VL-7B-Instruct** base model on the **Vision-R1-cold** dataset—a 200K high-quality multimodal Chain-of-Thought (CoT) dataset constructed by leveraging DeepSeek-R1 and existing MLLMs through modality bridging and data filtering.

This model acts as the critical starting point for subsequent Reinforcement Learning (RL) using Group Relative Policy Optimization (GRPO) and the Progressive Thinking Suppression Training (PTST) strategy, which enables the emergence of "Aha moments" and self-reflective reasoning in multimodal contexts.

## Performance

The Vision-R1 series demonstrates strong performance across various math-centric multimodal benchmarks. Vision-R1-7B (the version after RL training) achieves significant improvements:

| Model                      | MathVista | MathVerse | MM-Math | DynaMath | AVG. |
| -------------------------- | --------- | --------- | ------- | -------- | ---- |
| Qwen2.5-VL-7B              | 68.1      | 46.7      | 34.1    | 50.7     | 47.9 |
| **Vision-R1-7B (Ours)**    | 73.5      | 52.4      | 40.2    | 56.3     | 53.8 |

## Citation

If you find this model useful in your research, please cite the following paper:

```bibtex
@article{huang2025visionr1,
  title={Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models},
  author={Huang, Wenxuan and Jia, Bohan and Zhai, Zijie and Cao, Shaosheng and Ye, Zheyu and Zhao, Fei and Hu, Yao and Lin, Shaohui},
  journal={arXiv preprint arXiv:2503.06749},
  year={2025}
}
```