RotVL

RotVL is a rotation-aware vision-language model fine-tuned for cross-orientation state-equivalence checking and for detecting, classifying, and localizing GUI defects caused by screen rotation in Android applications. It accompanies the paper “RotDroid: Cross-Orientation State Equivalence Testing for Detecting GUI Rotation Bugs in Android Apps”, accepted at the 37th IEEE International Symposium on Software Reliability Engineering (ISSRE 2026).

RotDroid supplies RotVL with two screenshots representing corresponding portrait and landscape states. RotVL determines whether a rotation-induced GUI defect is present and, when requested, returns the defective orientation, defect type, and bounding box.

Released checkpoints

The three released checkpoints are:

Checkpoint	Base model
ImDim/RotVL-2B	Qwen3-VL-2B-Instruct
ImDim/RotVL-4B	Qwen3-VL-4B-Instruct
ImDim/RotVL-8B	Qwen3-VL-8B-Instruct

Each RotVL checkpoint is obtained by fine-tuning the corresponding Qwen3-VL-Instruct model on the RotBench dataset. It accepts two Android GUI screenshots and a text instruction, and produces JSON output for bug detection or bug classification/localization.

Evaluation on RotBench

The models are evaluated on RotBench test splits for bug detection, bug classification, defective-orientation identification, and bounding-box localization. Accuracy (Acc.), precision (Prec.), recall (Rec.), and F1 are percentages. For coordinate localization, a lower center-point distance is better, and an area ratio closer to 1 is better.

Bug detection (%)

Model	Acc.	Prec.	Rec.	F1
Qwen3-VL-2B	53.24	62.82	53.24	42.48
Qwen3-VL-4B	61.47	61.54	61.47	61.41
Qwen3-VL-8B	66.47	67.65	66.47	65.90
Qwen3-VL-32B	58.82	59.73	58.82	57.84
Qwen3-VL-235B	60.00	60.72	60.00	59.32
GPT-5.2	70.88	71.01	70.88	70.84
RotVL-2B	71.76	74.21	71.76	71.03
RotVL-4B	76.47	76.50	76.47	76.46
RotVL-8B	85.29	85.37	85.29	85.29

Bug classification (%)

Model	Acc.	Prec.	Rec.	F1
Qwen3-VL-2B	20.59	8.76	20.59	9.56
Qwen3-VL-4B	21.47	20.54	21.47	13.03
Qwen3-VL-8B	30.59	52.49	30.59	26.02
Qwen3-VL-32B	30.00	65.02	30.00	25.89
Qwen3-VL-235B	37.06	46.05	37.06	33.77
GPT-5.2	49.71	65.70	49.71	49.95
RotVL-2B	48.53	59.51	48.53	42.97
RotVL-4B	44.41	65.37	44.41	39.54
RotVL-8B	62.65	66.71	62.65	62.68

Bug localization: orientation (%)

Model	Acc.	Prec.	Rec.	F1
Qwen3-VL-2B	54.12	54.41	54.12	53.34
Qwen3-VL-4B	52.06	75.53	52.06	37.75
Qwen3-VL-8B	54.41	69.03	54.41	43.58
Qwen3-VL-32B	53.82	75.99	53.82	41.31
Qwen3-VL-235B	56.18	74.48	56.18	46.10
GPT-5.2	68.82	70.61	68.82	68.13
RotVL-2B	57.35	68.95	57.35	49.56
RotVL-4B	71.47	77.38	71.47	69.84
RotVL-8B	80.29	82.36	80.29	79.97

Bug localization: coordinates

Model	Center-point distance (px)	Area ratio
Qwen3-VL-2B	466	73.29
Qwen3-VL-4B	499	53.72
Qwen3-VL-8B	435	38.45
Qwen3-VL-32B	478	30.07
Qwen3-VL-235B	459	45.06
GPT-5.2	276	9.76
RotVL-2B	237	4.87
RotVL-4B	263	16.62
RotVL-8B	198	3.17

RotVL-8B achieves the best result in every evaluated dimension. Its improvements over the strongest baseline are statistically significant for bug detection, bug classification, bug localization error (p < 0.001).

Evaluation on natural bugs

Bug detection generalization is also evaluated using 100 runtime cross-orientation screenshot pairs sampled from 44 real-world applications with developer-confirmed rotation bugs.

Model	Accuracy	Precision	Recall	F1
Qwen3-VL-8B	63.00	70.22	63.00	66.41
GPT-5.2	65.00	59.21	65.00	61.97
RotVL-8B	70.00	74.81	70.00	72.32

RotVL-8B performs best on this natural bug set, indicating that its gains are not limited to the synthetic bug patterns in RotBench.

Download

Choose one of 2B, 4B, or 8B and download the corresponding checkpoint with the Hugging Face CLI. The following example downloads RotVL-8B:

MODEL_SIZE=8B
hf download "ImDim/RotVL-${MODEL_SIZE}" \
  --local-dir "RotVL-${MODEL_SIZE}"

Serve with vLLM

The tested model-server setup uses Linux, NVIDIA V100 GPUs, CUDA 12.2 or later, and vLLM 0.11.1. Start the selected checkpoint as an OpenAI-compatible service:

MODEL_SIZE=8B
CUDA_VISIBLE_DEVICES=0 VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
vllm serve "RotVL-${MODEL_SIZE}" \
  --served-model-name "RotVL-${MODEL_SIZE}" \
  --limit-mm-per-prompt '{"image": 2}' \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype float16 \
  --api-key xxx \
  --max-model-len 32768 \
  --tensor-parallel-size 1

The value passed to --served-model-name must match the model name used by the client.

Output formats

Bug detection

RotVL is prompted to return a strict JSON object:

{"bug": true}

Bug classification and localization

For a defective pair, RotVL is prompted to return:

{
  "type": "layout-overlap",
  "image": "portrait",
  "bbox_2d": [100, 200, 400, 500]
}

• type is one of layout-overlap, layout-clip, layout-miss, direction-mismatch, and state-loseinput.
• image is portrait or landscape.
• bbox_2d is [x_min, y_min, x_max, y_max] in the defective image's pixel coordinates.

Intended uses

RotVL is intended for:

• use as the visual defect detector in RotDroid;
• research on Android GUI rotation-bug detection;
• evaluation on RotBench.

Use with RotDroid

For integration details and instructions on running RotDroid, see the RotDroid source repository.

Related resources

• RotDroid source code: https://github.com/ImDiM/RotDroid
• RotBench dataset: https://huggingface.co/datasets/ImDim/RotBench
• Full artifact: https://doi.org/10.5281/zenodo.21897206

Citation

@inproceedings{qin2026rotdroid,
  title     = {RotDroid: Cross-Orientation State Equivalence Testing for Detecting GUI Rotation Bugs in Android Apps},
  author    = {Qin, Mengdi and Jiang, Bo},
  booktitle = {Proceedings of the 37th IEEE International Symposium on Software Reliability Engineering (ISSRE)},
  year      = {2026}
}