Hugging Face's logo

ImDim
/
RotVL-4B

Image-Text-to-Text
Transformers
Safetensors
qwen3_vl
vision-language
multimodal
android
gui
software-testing
rotation-bug
qwen3-vl
vllm
conversational
Model card Files
xet
 Community
ImDim commited on
Commit
8bb44c2
·
verified ·
1 Parent(s): cf225a0

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +170 -0
README.md ADDED
@@ -0,0 +1,170 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ pipeline_tag: image-text-to-text
4
+ base_model:
5
+ - Qwen/Qwen3-VL-2B-Instruct
6
+ - Qwen/Qwen3-VL-4B-Instruct
7
+ - Qwen/Qwen3-VL-8B-Instruct
8
+ tags:
9
+ - vision-language
10
+ - multimodal
11
+ - android
12
+ - gui
13
+ - software-testing
14
+ - rotation-bug
15
+ - qwen3-vl
16
+ - vllm
17
+ ---
18
+ # RotVL
19
+ RotVL is a rotation-aware vision-language model fine-tuned for cross-orientation state-equivalence checking and for detecting, classifying, and localizing GUI defects caused by screen rotation in Android applications. It accompanies the paper *“RotDroid: Cross-Orientation State Equivalence Testing for Detecting GUI Rotation Bugs in Android Apps”*, accepted at the 37th IEEE International Symposium on Software Reliability Engineering (ISSRE 2026).
20
+
21
+ RotDroid supplies RotVL with two screenshots representing corresponding portrait and landscape states. RotVL determines whether a rotation-induced GUI defect is present and, when requested, returns the defective orientation, defect type, and bounding box.
22
+
23
+ ## Released checkpoints
24
+ The three released checkpoints are:
25
+ | Checkpoint | Base model |
26
+ | ------------------------------------------------------ | -------------------- |
27
+ | [ImDim/RotVL-2B](https://huggingface.co/ImDim/RotVL-2B) | Qwen3-VL-2B-Instruct |
28
+ | [ImDim/RotVL-4B](https://huggingface.co/ImDim/RotVL-4B) | Qwen3-VL-4B-Instruct |
29
+ | [ImDim/RotVL-8B](https://huggingface.co/ImDim/RotVL-8B) | Qwen3-VL-8B-Instruct |
30
+
31
+ Each RotVL checkpoint is obtained by fine-tuning the corresponding Qwen3-VL-Instruct model on the [RotBench dataset](https://huggingface.co/datasets/ImDim/RotBench). It accepts two Android GUI screenshots and a text instruction, and produces JSON output for bug detection or bug classification/localization.
32
+
33
+ ## Evaluation on RotBench
34
+ The models are evaluated on RotBench test splits for bug detection, bug classification, defective-orientation identification, and bounding-box localization. Accuracy (Acc.), precision (Prec.), recall (Rec.), and F1 are percentages. For coordinate localization, a lower center-point distance is better, and an area ratio closer to 1 is better.
35
+
36
+ ### Bug detection (%)
37
+ | Model | Acc. | Prec. | Rec. | F1 |
38
+ | ------------------ | --------------: | --------------: | --------------: | --------------: |
39
+ | Qwen3-VL-2B | 53.24 | 62.82 | 53.24 | 42.48 |
40
+ | Qwen3-VL-4B | 61.47 | 61.54 | 61.47 | 61.41 |
41
+ | Qwen3-VL-8B | 66.47 | 67.65 | 66.47 | 65.90 |
42
+ | Qwen3-VL-32B | 58.82 | 59.73 | 58.82 | 57.84 |
43
+ | Qwen3-VL-235B | 60.00 | 60.72 | 60.00 | 59.32 |
44
+ | GPT-5.2 | 70.88 | 71.01 | 70.88 | 70.84 |
45
+ | RotVL-2B | 71.76 | 74.21 | 71.76 | 71.03 |
46
+ | RotVL-4B | 76.47 | 76.50 | 76.47 | 76.46 |
47
+ | **RotVL-8B** | **85.29** | **85.37** | **85.29** | **85.29** |
48
+
49
+ ### Bug classification (%)
50
+ | Model | Acc. | Prec. | Rec. | F1 |
51
+ | ------------------ | --------------: | --------------: | --------------: | --------------: |
52
+ | Qwen3-VL-2B | 20.59 | 8.76 | 20.59 | 9.56 |
53
+ | Qwen3-VL-4B | 21.47 | 20.54 | 21.47 | 13.03 |
54
+ | Qwen3-VL-8B | 30.59 | 52.49 | 30.59 | 26.02 |
55
+ | Qwen3-VL-32B | 30.00 | 65.02 | 30.00 | 25.89 |
56
+ | Qwen3-VL-235B | 37.06 | 46.05 | 37.06 | 33.77 |
57
+ | GPT-5.2 | 49.71 | 65.70 | 49.71 | 49.95 |
58
+ | RotVL-2B | 48.53 | 59.51 | 48.53 | 42.97 |
59
+ | RotVL-4B | 44.41 | 65.37 | 44.41 | 39.54 |
60
+ | **RotVL-8B** | **62.65** | **66.71** | **62.65** | **62.68** |
61
+
62
+ ### Bug localization: orientation (%)
63
+ | Model | Acc. | Prec. | Rec. | F1 |
64
+ | ------------------ | --------------: | --------------: | --------------: | --------------: |
65
+ | Qwen3-VL-2B | 54.12 | 54.41 | 54.12 | 53.34 |
66
+ | Qwen3-VL-4B | 52.06 | 75.53 | 52.06 | 37.75 |
67
+ | Qwen3-VL-8B | 54.41 | 69.03 | 54.41 | 43.58 |
68
+ | Qwen3-VL-32B | 53.82 | 75.99 | 53.82 | 41.31 |
69
+ | Qwen3-VL-235B | 56.18 | 74.48 | 56.18 | 46.10 |
70
+ | GPT-5.2 | 68.82 | 70.61 | 68.82 | 68.13 |
71
+ | RotVL-2B | 57.35 | 68.95 | 57.35 | 49.56 |
72
+ | RotVL-4B | 71.47 | 77.38 | 71.47 | 69.84 |
73
+ | **RotVL-8B** | **80.29** | **82.36** | **80.29** | **79.97** |
74
+
75
+ ### Bug localization: coordinates
76
+ | Model | Center-point distance (px) | Area ratio |
77
+ | ------------------ | -------------------------: | -------------: |
78
+ | Qwen3-VL-2B | 466 | 73.29 |
79
+ | Qwen3-VL-4B | 499 | 53.72 |
80
+ | Qwen3-VL-8B | 435 | 38.45 |
81
+ | Qwen3-VL-32B | 478 | 30.07 |
82
+ | Qwen3-VL-235B | 459 | 45.06 |
83
+ | GPT-5.2 | 276 | 9.76 |
84
+ | RotVL-2B | 237 | 4.87 |
85
+ | RotVL-4B | 263 | 16.62 |
86
+ | **RotVL-8B** | **198** | **3.17** |
87
+
88
+ RotVL-8B achieves the best result in every evaluated dimension. Its improvements over the strongest baseline are statistically significant for bug detection, bug classification, bug localization error (`p < 0.001`).
89
+
90
+ ## Evaluation on natural bugs
91
+
92
+ Bug detection generalization is also evaluated using 100 runtime cross-orientation screenshot pairs sampled from 44 real-world applications with developer-confirmed rotation bugs.
93
+
94
+ | Model | Accuracy | Precision | Recall | F1 |
95
+ | ------------------ | --------------: | --------------: | --------------: | --------------: |
96
+ | Qwen3-VL-8B | 63.00 | 70.22 | 63.00 | 66.41 |
97
+ | GPT-5.2 | 65.00 | 59.21 | 65.00 | 61.97 |
98
+ | **RotVL-8B** | **70.00** | **74.81** | **70.00** | **72.32** |
99
+
100
+ RotVL-8B performs best on this natural bug set, indicating that its gains are not limited to the synthetic bug patterns in RotBench.
101
+
102
+ ## Download
103
+ Choose one of `2B`, `4B`, or `8B` and download the corresponding checkpoint with the Hugging Face CLI. The following example downloads RotVL-8B:
104
+ ```bash
105
+ MODEL_SIZE=8B
106
+ hf download "ImDim/RotVL-${MODEL_SIZE}" \
107
+ --local-dir "RotVL-${MODEL_SIZE}"
108
+ ```
109
+
110
+ ## Serve with vLLM
111
+ The tested model-server setup uses Linux, NVIDIA V100 GPUs, CUDA 12.2 or later, and vLLM 0.11.1. Start the selected checkpoint as an OpenAI-compatible service:
112
+ ```bash
113
+ MODEL_SIZE=8B
114
+ CUDA_VISIBLE_DEVICES=0 VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
115
+ vllm serve "RotVL-${MODEL_SIZE}" \
116
+ --served-model-name "RotVL-${MODEL_SIZE}" \
117
+ --limit-mm-per-prompt '{"image": 2}' \
118
+ --host 0.0.0.0 \
119
+ --port 8000 \
120
+ --dtype float16 \
121
+ --api-key xxx \
122
+ --max-model-len 32768 \
123
+ --tensor-parallel-size 1
124
+ ```
125
+
126
+ The value passed to `--served-model-name` must match the model name used by the client.
127
+
128
+ ## Output formats
129
+ ### Bug detection
130
+ RotVL is prompted to return a strict JSON object:
131
+ ```json
132
+ {"bug": true}
133
+ ```
134
+
135
+ ### Bug classification and localization
136
+ For a defective pair, RotVL is prompted to return:
137
+ ```json
138
+ {
139
+ "type": "layout-overlap",
140
+ "image": "portrait",
141
+ "bbox_2d": [100, 200, 400, 500]
142
+ }
143
+ ```
144
+ - `type` is one of `layout-overlap`, `layout-clip`, `layout-miss`, `direction-mismatch`, and `state-loseinput`.
145
+ - `image` is `portrait` or `landscape`.
146
+ - `bbox_2d` is `[x_min, y_min, x_max, y_max]` in the defective image's pixel coordinates.
147
+
148
+ ## Intended uses
149
+ RotVL is intended for:
150
+ - use as the visual defect detector in RotDroid;
151
+ - research on Android GUI rotation-bug detection;
152
+ - evaluation on RotBench.
153
+
154
+ ## Use with RotDroid
155
+ For integration details and instructions on running RotDroid, see the [RotDroid source repository](https://github.com/ImDiM/RotDroid).
156
+
157
+ ## Related resources
158
+ - RotDroid source code: https://github.com/ImDiM/RotDroid
159
+ - RotBench dataset: https://huggingface.co/datasets/ImDim/RotBench
160
+ - Full artifact: https://doi.org/10.5281/zenodo.21897206
161
+
162
+ ## Citation
163
+ ```bibtex
164
+ @inproceedings{qin2026rotdroid,
165
+ title = {RotDroid: Cross-Orientation State Equivalence Testing for Detecting GUI Rotation Bugs in Android Apps},
166
+ author = {Qin, Mengdi and Jiang, Bo},
167
+ booktitle = {Proceedings of the 37th IEEE International Symposium on Software Reliability Engineering (ISSRE)},
168
+ year = {2026}
169
+ }
170
+ ```