In software development process, the last step is usually the Graphic User In- terface（GUI） test, which is part of the final user experience （UE） test. Traditionally, there exist some GUI test tools in the market,...In software development process, the last step is usually the Graphic User In- terface（GUI） test, which is part of the final user experience （UE） test. Traditionally, there exist some GUI test tools in the market, such as Abbot Java GUI Test Framework and Pounder, in which testers pre-configure in the script all desired actions and instructions for the computer, nonetheless requiring too much of invariance of GUI environment; and they require reconfiguration in case of GUI changes, therefore still to be done mostly manually and hard for non-programmer testers to. Consequently, we proposed GUI tests by image recognition to automate the last process; we managed to innovate upon current algorithms such as SIFT and Random Fern, from which we develop the new algorithm scheme retrieving most efficient feature and dispelling inefficient part of each algorithm. Computers then apply the algorithm, to search for target patterns themselves and take subsequent actions such as manual mouse, keyboard and screen I/O automatically to test the GUI without any manual instructions. Test results showed that the proposed approach can accelerate GU! test largely compared to current benchmarks.展开更多
基金supported by the National Natural Science Foundation of China(Nos.61572316,61133009)National Hightech R&D Program of China(863 Program)(Grant No.2015AA015904)+3 种基金the Science and Technology Commission of Shanghai Municipality Program(No.13511505000)the Interdisciplinary Program of Shanghai Jiao Tong University(No.14JCY10)a grant from the Research Grants Council of Hong Kong(Project No.:28200215)a grant from The Education University of Hong Kong(Project No:FLASS/DRF/ECR-7)
文摘In software development process, the last step is usually the Graphic User In- terface（GUI） test, which is part of the final user experience （UE） test. Traditionally, there exist some GUI test tools in the market, such as Abbot Java GUI Test Framework and Pounder, in which testers pre-configure in the script all desired actions and instructions for the computer, nonetheless requiring too much of invariance of GUI environment; and they require reconfiguration in case of GUI changes, therefore still to be done mostly manually and hard for non-programmer testers to. Consequently, we proposed GUI tests by image recognition to automate the last process; we managed to innovate upon current algorithms such as SIFT and Random Fern, from which we develop the new algorithm scheme retrieving most efficient feature and dispelling inefficient part of each algorithm. Computers then apply the algorithm, to search for target patterns themselves and take subsequent actions such as manual mouse, keyboard and screen I/O automatically to test the GUI without any manual instructions. Test results showed that the proposed approach can accelerate GU! test largely compared to current benchmarks.
