基于YOLO的移动应用自动测试

Mobile Application Automatic Testing Based on YOLO

现有的GUI自动测试技术通常需要借助被测设备的底层界面框架来获取信息,难以实现跨平台。针对这种现象,文章提出了基于视觉的移动应用自动测试技术,使用YOLOv5模型检测界面中可执行控件,对不同测试输入下的状态合并方法进行研究,采用交并比与平均表面距离相结合的方式识别界面中相似状态。经过实验证明,该方法状态识别效果良好,为基于视觉特征的GUI自动化测试提供一定的参考。

The existing GUI automatic testing technology usually relies on the underlying interface framework of the detected equipment to obtain information,which is hard to realize cross-platform.In response to the phenomenon,this paper proposes a vision-based mobile application automatic testing technology,uses YOLOv5 model to detect the executable widgets in the interface,researches the state merging method under different test inputs,and combines the average intersection union ratio with the average surface distance to identify similar states in the interface.Experiments prove that the method has a good effect on identifying status and provides a certain reference for the automatic GUI testing based on visual feature.