Unified GUI Element Detection Dataset (Rico + WebUI, YOLO format)
This dataset combines two publicly available GUI datasets — Rico (UIST 2017) and WebUI (CHI 2023) — into a unified cross-platform benchmark for GUI element detection, formatted for training YOLO-based object detection models. Dataset composition: <t
This dataset combines two publicly available GUI datasets — Rico (UIST 2017) and WebUI (CHI 2023) — into a unified cross-platform benchmark for GUI element detection, formatted for training YOLO-based object detection models.
Dataset composition:
| Source | Platform | Images | Annotations |
|---|---|---|---|
| Rico | Android mobile | 63,160 | 1,075,398 |
| WebUI | Web (6 viewports) | 41,970 | 2,242,576 |
| Combined | Cross-platform | 105,130 | 3,317,974 |
Unified class taxonomy (12 classes): Button, Text, Image, Icon, Input, Link, Checkbox, Toggle, Toolbar, Navigation, Modal, Tab
Annotation format: YOLO (normalized class_id cx cy w h per line)
Dataset split: 80% train / 10% val / 10% test
Original sources:
- Rico: Deka et al., “Rico: A Mobile App Dataset for Building Data-Driven Design Applications”, UIST 2017
- WebUI: Wu et al., “WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics”, CHI 2023 (Best Paper)
The pipeline used to produce this dataset (download, annotation conversion, class mapping, and merge scripts) is publicly available at: https://github.com/Rodrigomgs24/Dissertacao_GUI_Detection
Produced as part of: Master’s Dissertation — Detection of Graphical User Interface Elements Using Deep Learning, ISEP, 2026.
Please cite the original datasets:
,
author=Deka, Biplab and Huang, Zifeng and Franzen, Chad and Hibschman, Joshua and Afergan, Daniel and Li, Yang and Nichols, Jeffrey and Kumar, Ranjitha,
booktitle=Proceedings of the 30th annual ACM symposium on user interface software and technology,
pages=845–854,
year=2017
}
,
author=Jason Wu and Siyan Wang and Siman Shen and Yi-Hao Peng and Jeffrey Nichols and Jeffrey Bigham,
journal=ACM Conference on Human Factors in Computing Systems (CHI),
year=2023
}
📤 Share this page
Found this useful? Share it with your network.
Files are hosted on the source repository. Click download to access the full dataset.