We propose UI-R1, the first framework to explore how rule-based RL can enhance the reasoning capabilities of multimodal large language models (MLLMs) for GUI action prediction tasks.

Experimental results demonstrate that our proposed UI-R1-3B achieves significant improvements over the base model (i.e. Qwen2.5-VL-3B) on both in-domain (ID) and out-of-domain (OOD) tasks, with average accuracy gains of 22.1% on ScreenSpot, 6.0% on ScreenSpot-Pro, and 12.7% on AndroidControl. Furthermore, UI-R1-3B delivers competitive performance compared to larger models (e.g., OS-Atlas-7B) trained via supervised fine-tuning (SFT) on 76K samples.

Thinking is not needed for GUI grounding.

Inspired by concurrent works studying efficient LRM, we realize efficient reasoning by RFT training. UI-R1-3B-E's training consists of two steps:

1. DAST (Difficulty-Adaptive Slow-Thinking): Add difficulty-adaptive length reward to make reasoning from slow to fast.
2. Nothinking: Not output reasoning process.

Note: UI-R1-3B (v2) and UI-R1-3B-E both train on larger dataset (2K grounding data in GUI-R1-3K) compared to UI-R1-3B (v1).

1. Our UI-R1-3B-E achieves SOTA with least answer tokens in 3B/7B Open-source methods, demonstrating GUI grounding needs no reasoning.

• Performance on 7B may be opposite.
• Performance on Planning may be opposite. The author predicts that Fast Grounding, Slow Planning.
• The checkpoints of UI-R1-3B-E will be released soon.
• The updated paper will come soon.
• The efficient training code will come soon. (in src/script/train_e.sh)

conda create -n ui-r1 python=3.10
conda activate ui-r1
bash setup.sh

Our training mobile data is a subset from AndroidControl and ScreenSpot.

You can also prepare your training or inference data like:

images/:
	image1.png
	image2.png

test.json:
[
	{
	"img_filename": "image1.png",
 "bbox": [
 825,
 72,
 1673,
 149
 ],
 "instruction": "search bar"
 },
 {
	"img_filename": "image2.png",
 "bbox": [
 123,
 732,
 334,
 812
 ],
 "instruction": "check weather"
 }
]

where bbox : [x1,y1,x2,y2] is the coordinate of the left top and the right bottom of the ground truth bbox

We provide an example here

cd evaluation/
bash test.sh

Please fill the MODEL_PATH, IMG_PATH, TEST_JSON with your real checkpoint path and data path.

cd src/script/
bash train.sh
# efficient training
bash train_e.sh

• 2025年11月08日: Our paper was accepted by AAAI-2026.
• 2025年05月14日: We update the paper with UI-R1-E-3B.
• 2025年05月12日: We release the checkpoints of the UI-R1-E-3B model.
• 2025年05月12日: We fix the bug of scales when batch_size > 1.
• 2025年05月11日: We release the efficient training code of the UI-R1-E-3B model.
• 2025年04月02日: We release the datasets of the UI-R1-3B (v1) model.
• 2025年03月30日: We release the checkpoints of the UI-R1-3B (v1) model.
• 2025年03月30日: We release the UI-R1 repository.
• 2025年03月27日: We release our paper.

If you find this project useful, welcome to cite us.

@article{lu2025ui,
 title={UI-R1: Enhancing Action Prediction of GUI Agents by Reinforcement Learning},
 author={Lu, Zhengxi and Chai, Yuxiang and Guo, Yaxuan and Yin, Xi and Liu, Liang and Wang, Hao and Xiong, Guanjing and Li, Hongsheng},
 journal={arXiv preprint arXiv:2503.21620},
 year={2025}
}

We sincerely thank projects R1-V, Open-R1, and Open-r1-multimodal, VLM-R1 for providing their open-source resources.