图像与问题双引导注意力机制视觉问答算法

Visual Question Answering Algorithm based on Dual-Guided Attention Mechanism of Image and Question

针对视觉问答任务中问题特征与图像特征缺乏交互推理关系的问题,提出了图像与问题双引导注意力机制视觉问答算法。模型结构主要由问题特征注意力模块、图像特征注意力模块、问题与图像双引导注意力模块、特征融合模块4部分构成。该算法先针对问题特征和图像特征分别使用自我注意力机制实现特征的自我加强,然后引入图像与问题双引导注意力机制,最后使用线性分类器分类输出。在VQA V2.0数据集上实验验证,该算法表现出较好的性能,准确率达到70.98%。

In order to deal with the problem of lack of interactive reasoning relationship between question features and image features in visual question answering(VQA)tasks,a visual question answering algorithm based on image and question dual-guided attention(DGA)mechanism is proposed.The model structure is mainly composed of four parts:question feature attention module,image feature attention module,question and image dual-guided attention module,and feature fusion module.Firstly,the question features and image features are self-reinforced by self-attention mechanism respectively,then the image and question dual-guided attention mechanism is introduced,and finally the output is classified by linear classifier.The algorithm proposed is validated on VQA V2.0 dataset and indicates good performance with an accuracy of 70.98%.