结合双维注意力与特征融合的GUI代码生成方法

GUI code generation method combining two-dimensional attention and feature fusion

针对传统GUI代码生成方法在准确率和性能上的不足,提出了一种新颖的结合双维注意力与特征融合的GUI代码生成方法。该方法首先利用视觉特征网络提取图像特征、DSL编码网络编码上下文向量;然后利用双维注意力模块对图像特征与上下文向量进行高效筛选,得到空间、通道两个维度上的带权图像特征;最后利用特征融合单元将带权图像特征与上下文向量进行对齐,使得图像特征同时包含预测代码所需的视觉信息与语义信息,协助模型更加快速、准确地解码生成目标代码。实验结果表明笔者方法代码生成准确率和BLEU分数表现优于多个对比模型和方法。

Aiming at the shortcomings of traditional GUI code generation methods in accuracy and performance,a novel GUI code generation method combining two-dimensional attention and feature fusion is proposed.Firstly,the method uses visual feature network to extract image features and coding context vector of DSL coding network.Then,the image features and context vectors are filtered efficiently through the two-dimensional attention module,and the weighted image features in the spatial dimension and channel dimensions are obtained.Finally,the feature fusion unit is used to align the weighted image features with the context vector,so that the image features contain both visual information and semantic information required by the prediction code.It can be helpful for the model to decode and generate the target code more quickly and accurately.The experimental results show that the code generation accuracy and BLEU score of the author s method are better than many comparison models and methods.