--- library_name: transformers pipeline_tag: image-text-to-text base_model: - Qwen/Qwen3-VL-2B-Instruct - Qwen/Qwen3-VL-4B-Instruct - Qwen/Qwen3-VL-8B-Instruct tags: - vision-language - multimodal - android - gui - software-testing - rotation-bug - qwen3-vl - vllm --- # RotVL RotVL is a rotation-aware vision-language model fine-tuned for cross-orientation state-equivalence checking and for detecting, classifying, and localizing GUI defects caused by screen rotation in Android applications. It accompanies the paper *“RotDroid: Cross-Orientation State Equivalence Testing for Detecting GUI Rotation Bugs in Android Apps”*, accepted at the 37th IEEE International Symposium on Software Reliability Engineering (ISSRE 2026). RotDroid supplies RotVL with two screenshots representing corresponding portrait and landscape states. RotVL determines whether a rotation-induced GUI defect is present and, when requested, returns the defective orientation, defect type, and bounding box. ## Released checkpoints The three released checkpoints are: | Checkpoint | Base model | | ------------------------------------------------------ | -------------------- | | [ImDim/RotVL-2B](https://huggingface.co/ImDim/RotVL-2B) | Qwen3-VL-2B-Instruct | | [ImDim/RotVL-4B](https://huggingface.co/ImDim/RotVL-4B) | Qwen3-VL-4B-Instruct | | [ImDim/RotVL-8B](https://huggingface.co/ImDim/RotVL-8B) | Qwen3-VL-8B-Instruct | Each RotVL checkpoint is obtained by fine-tuning the corresponding Qwen3-VL-Instruct model on the [RotBench dataset](https://huggingface.co/datasets/ImDim/RotBench). It accepts two Android GUI screenshots and a text instruction, and produces JSON output for bug detection or bug classification/localization. ## Evaluation on RotBench The models are evaluated on RotBench test splits for bug detection, bug classification, defective-orientation identification, and bounding-box localization. Accuracy (Acc.), precision (Prec.), recall (Rec.), and F1 are percentages. For coordinate localization, a lower center-point distance is better, and an area ratio closer to 1 is better. ### Bug detection (%) | Model | Acc. | Prec. | Rec. | F1 | | ------------------ | --------------: | --------------: | --------------: | --------------: | | Qwen3-VL-2B | 53.24 | 62.82 | 53.24 | 42.48 | | Qwen3-VL-4B | 61.47 | 61.54 | 61.47 | 61.41 | | Qwen3-VL-8B | 66.47 | 67.65 | 66.47 | 65.90 | | Qwen3-VL-32B | 58.82 | 59.73 | 58.82 | 57.84 | | Qwen3-VL-235B | 60.00 | 60.72 | 60.00 | 59.32 | | GPT-5.2 | 70.88 | 71.01 | 70.88 | 70.84 | | RotVL-2B | 71.76 | 74.21 | 71.76 | 71.03 | | RotVL-4B | 76.47 | 76.50 | 76.47 | 76.46 | | **RotVL-8B** | **85.29** | **85.37** | **85.29** | **85.29** | ### Bug classification (%) | Model | Acc. | Prec. | Rec. | F1 | | ------------------ | --------------: | --------------: | --------------: | --------------: | | Qwen3-VL-2B | 20.59 | 8.76 | 20.59 | 9.56 | | Qwen3-VL-4B | 21.47 | 20.54 | 21.47 | 13.03 | | Qwen3-VL-8B | 30.59 | 52.49 | 30.59 | 26.02 | | Qwen3-VL-32B | 30.00 | 65.02 | 30.00 | 25.89 | | Qwen3-VL-235B | 37.06 | 46.05 | 37.06 | 33.77 | | GPT-5.2 | 49.71 | 65.70 | 49.71 | 49.95 | | RotVL-2B | 48.53 | 59.51 | 48.53 | 42.97 | | RotVL-4B | 44.41 | 65.37 | 44.41 | 39.54 | | **RotVL-8B** | **62.65** | **66.71** | **62.65** | **62.68** | ### Bug localization: orientation (%) | Model | Acc. | Prec. | Rec. | F1 | | ------------------ | --------------: | --------------: | --------------: | --------------: | | Qwen3-VL-2B | 54.12 | 54.41 | 54.12 | 53.34 | | Qwen3-VL-4B | 52.06 | 75.53 | 52.06 | 37.75 | | Qwen3-VL-8B | 54.41 | 69.03 | 54.41 | 43.58 | | Qwen3-VL-32B | 53.82 | 75.99 | 53.82 | 41.31 | | Qwen3-VL-235B | 56.18 | 74.48 | 56.18 | 46.10 | | GPT-5.2 | 68.82 | 70.61 | 68.82 | 68.13 | | RotVL-2B | 57.35 | 68.95 | 57.35 | 49.56 | | RotVL-4B | 71.47 | 77.38 | 71.47 | 69.84 | | **RotVL-8B** | **80.29** | **82.36** | **80.29** | **79.97** | ### Bug localization: coordinates | Model | Center-point distance (px) | Area ratio | | ------------------ | -------------------------: | -------------: | | Qwen3-VL-2B | 466 | 73.29 | | Qwen3-VL-4B | 499 | 53.72 | | Qwen3-VL-8B | 435 | 38.45 | | Qwen3-VL-32B | 478 | 30.07 | | Qwen3-VL-235B | 459 | 45.06 | | GPT-5.2 | 276 | 9.76 | | RotVL-2B | 237 | 4.87 | | RotVL-4B | 263 | 16.62 | | **RotVL-8B** | **198** | **3.17** | RotVL-8B achieves the best result in every evaluated dimension. Its improvements over the strongest baseline are statistically significant for bug detection, bug classification, bug localization error (`p < 0.001`). ## Evaluation on natural bugs Bug detection generalization is also evaluated using 100 runtime cross-orientation screenshot pairs sampled from 44 real-world applications with developer-confirmed rotation bugs. | Model | Accuracy | Precision | Recall | F1 | | ------------------ | --------------: | --------------: | --------------: | --------------: | | Qwen3-VL-8B | 63.00 | 70.22 | 63.00 | 66.41 | | GPT-5.2 | 65.00 | 59.21 | 65.00 | 61.97 | | **RotVL-8B** | **70.00** | **74.81** | **70.00** | **72.32** | RotVL-8B performs best on this natural bug set, indicating that its gains are not limited to the synthetic bug patterns in RotBench. ## Download Choose one of `2B`, `4B`, or `8B` and download the corresponding checkpoint with the Hugging Face CLI. The following example downloads RotVL-8B: ```bash MODEL_SIZE=8B hf download "ImDim/RotVL-${MODEL_SIZE}" \ --local-dir "RotVL-${MODEL_SIZE}" ``` ## Serve with vLLM The tested model-server setup uses Linux, NVIDIA V100 GPUs, CUDA 12.2 or later, and vLLM 0.11.1. Start the selected checkpoint as an OpenAI-compatible service: ```bash MODEL_SIZE=8B CUDA_VISIBLE_DEVICES=0 VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \ vllm serve "RotVL-${MODEL_SIZE}" \ --served-model-name "RotVL-${MODEL_SIZE}" \ --limit-mm-per-prompt '{"image": 2}' \ --host 0.0.0.0 \ --port 8000 \ --dtype float16 \ --api-key xxx \ --max-model-len 32768 \ --tensor-parallel-size 1 ``` The value passed to `--served-model-name` must match the model name used by the client. ## Output formats ### Bug detection RotVL is prompted to return a strict JSON object: ```json {"bug": true} ``` ### Bug classification and localization For a defective pair, RotVL is prompted to return: ```json { "type": "layout-overlap", "image": "portrait", "bbox_2d": [100, 200, 400, 500] } ``` - `type` is one of `layout-overlap`, `layout-clip`, `layout-miss`, `direction-mismatch`, and `state-loseinput`. - `image` is `portrait` or `landscape`. - `bbox_2d` is `[x_min, y_min, x_max, y_max]` in the defective image's pixel coordinates. ## Intended uses RotVL is intended for: - use as the visual defect detector in RotDroid; - research on Android GUI rotation-bug detection; - evaluation on RotBench. ## Use with RotDroid For integration details and instructions on running RotDroid, see the [RotDroid source repository](https://github.com/ImDiM/RotDroid). ## Related resources - RotDroid source code: https://github.com/ImDiM/RotDroid - RotBench dataset: https://huggingface.co/datasets/ImDim/RotBench - Full artifact: https://doi.org/10.5281/zenodo.21897206 ## Citation ```bibtex @inproceedings{qin2026rotdroid, title = {RotDroid: Cross-Orientation State Equivalence Testing for Detecting GUI Rotation Bugs in Android Apps}, author = {Qin, Mengdi and Jiang, Bo}, booktitle = {Proceedings of the 37th IEEE International Symposium on Software Reliability Engineering (ISSRE)}, year = {2026} } ```