| --- |
| base_model: Qwen/Qwen3-VL-8B-Instruct |
| library_name: transformers |
| license: other |
| pipeline_tag: image-text-to-text |
| tags: |
| - reward-model |
| - image-editing |
| - FIRM |
| - llama-factory |
| - generated_from_trainer |
| model-index: |
| - name: FIRM-Edit-8B |
| results: [] |
| --- |
| |
| # FIRM-Edit-8B |
|
|
| [**Project Page**](https://firm-reward.github.io/) | [**Paper**](https://arxiv.org/abs/2603.12247) | [**GitHub**](https://github.com/VisionXLab/FIRM-Reward) |
|
|
| **FIRM-Edit-8B** is a robust reward model (critic) designed for faithful image editing. It is a fine-tuned version of [Qwen/Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) on the **FIRM-Edit-370K** dataset. The model is part of the **FIRM (Faithful Image Reward Modeling)** framework, which provides accurate and reliable guidance for visual reinforcement learning pipelines. |
|
|
| ## Model Description |
|
|
| Conventional reward models used for image editing often suffer from hallucinations and assign noisy scores, misguiding the optimization process. FIRM-Edit-8B addresses these issues by evaluating edits through two competing objectives: |
| 1. **Execution**: Adherence to the editing instruction. |
| 2. **Consistency**: Preservation of original content in unedited regions. |
|
|
| By formulating a "Consistency-Modulated Execution" (CME) reward strategy, this model acts as a stable critic that mitigates hallucinations and helps establish a new standard for fidelity in image editing. |
|
|
| ## Intended Uses & Limitations |
|
|
| - **Reward Modeling**: To be used as a reward signal in Reinforcement Learning (RL) pipelines for image editing. |
| - **Evaluation**: To serve as a metric for benchmarking the performance of image editing models. |
|
|
| ## Training procedure |
|
|
| The model was fine-tuned using the [LLaMA Factory](https://github.com/hiyouga/LLaMA-Factory) framework. |
|
|
| ### Training hyperparameters |
|
|
| The following hyperparameters were used during training: |
| - learning_rate: 1e-05 |
| - train_batch_size: 10 |
| - eval_batch_size: 2 |
| - seed: 42 |
| - gradient_accumulation_steps: 2 |
| - lr_scheduler_type: cosine |
| - lr_scheduler_warmup_ratio: 0.1 |
| - num_epochs: 1.0 |
| |
| ### Training results |
| |
| | Training Loss | Epoch | Step | Validation Loss | |
| |:-------------:|:------:|:----:|:---------------:| |
| | 0.591 | 0.2182 | 500 | 0.5827 | |
| | 0.5605 | 0.4364 | 1000 | 0.5460 | |
| | 0.5252 | 0.6546 | 1500 | 0.5199 | |
| | 0.5075 | 0.8728 | 2000 | 0.5055 | |
| |
| ## Usage |
| |
| To use the model as a reward server for RL training, you can use the script provided in the official repository: |
| |
| ```bash |
| # Launch the reward server |
| python editing/reward_server/reward_server_qwen3_vl_8b_sft.py |
| ``` |
| |
| ## Citation |
| |
| If you find this work useful, please cite: |
| |
| ```bibtex |
| @article{zhao2026trust, |
| title={Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation}, |
| author={Zhao, Xiangyu and Zhang, Peiyuan and Lin, Junming and Liang, Tianhao and Duan, Yuchen and Ding, Shengyuan and Tian, Changyao and Zang, Yuhang and Yan, Junchi and Yang, Xue}, |
| journal={arXiv preprint arXiv:2603.12247}, |
| year={2026} |
| } |
| ``` |
