面向移动应用自动化测试的同构用户界面视觉判断方法

Visual judgment approach of isomorphic GUI for automated mobile app testing

当前快速增长的移动应用程序迫切需求自动化测试技术以保证其质量。移动应用的自动化测试与其图形用户界面(GUI)的识别与判断紧密相关,但移动应用却通常存在大量的样式与内容有差异而结构与功能相类似的同构GUI。在自动化测试中,同构GUI容易引发应用状态空间的爆炸问题,进而导致测试的低效或失败。针对传统自动化识别同构GUI的局限性,提出一种基于视觉特征信息的GUI相似度判断方法。通过目标检测技术识别GUI组件元素进而抽象出GUI结构框架;采用卷积自编码器提取出GUI结构视觉特征;对比GUI视觉特征的相似度完成同构GUI判断。经过实验验证,所提方法能够屏蔽GUI的样式、内容等影响,从而更精确地完成同构GUI识别,优化自动化测试效率。

Currently,the rapid growth of mobile apps requires automated testing technology to ensure their quality.Automated testing of mobile apps is usually closely related to the recognition and judgment of their graphical user interface(GUI),but there usually are numerous isomorphic GUIs with different styles and contents,and similar structure and function in mobile apps.In automatic testing,isomorphic GUI is easy to cause the issue of state space explosion,which leads to low efficiency or failure of testing.In view of the limitations of traditional automatic recog⁃nition of isomorphic GUI,this paper presents a GUI similarity judgment approach based on visual feature informa⁃tion.Firstly,the GUI component elements are identified by object detection,and then the GUI skeleton is abstract⁃ed.Secondly,the visual features of the GUI skeleton are extracted by a convolutional autoencoder.Finally,the iso⁃morphic GUI judgment is completed by comparing the similarity of GUI visual features.The experimental results show that the proposed approach can effectively shield the influence of GUI style and content,complete the isomor⁃phic GUI recognition more accurately and optimize the efficiency of automated mobile app testing.