一种面向UI手稿识别的数据集制作方法

An Efficient Approach to Creating Hand-Drawn Dataset for UI Manuscript Recognition

UI手稿识别是图像目标检测技术在软件工程领域的重要应用。由于UI手稿图像与自然图像有着较大的差异,而且主要依靠人工绘制,所以制作用于深度学习模型训练的UI手稿数据集往往比较困难,耗费大量人力。针对此问题,通过对UI手稿数据集的制作流程进行优化改进,提出了一种UI手稿数据集快速制作方法UIsketcher。在UIsketcher方法中,用户只需要完成一些基础UI组件的绘制,不需要任何框选标注,即可自动生成用于深度学习模型训练的数据集。与传统方法进行对比实验,结果表明:用户只需要绘制相对于传统方法25%的组件数量,即可得到相似的训练效果;若绘制传统方法75%的组件数量,训练效果将更好,可达到比传统方法更高的准确率。

UI manuscript recognition is one of the important applications of image object detection in the area of software engineering.Due to the significant difference between UI manuscript images and natural images where UI manuscript images usually need to be drawn manually,it is difficult to build UI manuscript dataset for deep learning because of the dependency on tremendous manual efforts.To address the issue,in this study an approach called UIsketcher was proposed to efficiently generate UI manuscript dataset based on optimizing the current workflow.In UIsketcher,users should just draw some basic elements without labeling,and then the dataset could be automatically generated for training deep learning model.According to the experiment with UIsketcher,only 25%drawing workload of the traditional methods could get the similar training results.If the workload was 75%,the final accuracy was even better than that of traditional methods.