基于注意力反馈机制的深度图像标注模型

Depth image caption model based on attention feedback mechanism

针对图像标注任务提出了一种基于注意力反馈机制的深度图像标注模型。该模型采用编码器-解码器框架;编码器采用VGG-16的网络结构,以提取图像的特征信息;在解码器部分设计了一种堆叠方式自上而下的处理注意力信息,使网络的每一层都可以获得额外的特征信息。然后从生成的标注语句中提取特征,将关注特征和图像的关注区域结合,增强和图像关注区域的匹配性,使生成的标注语句近似真实语境。在Flickr8k、Flickr30k和MSCOCO等数据集进行实验,实验结果显示,所提出模型的识别率比经典图像识别模型高5%～9%。

A depth image caption model based on attention feedback mechanism is proposed for image caption tasks. The model uses the encoder-decoder framework. The encoder adopts the VGG-16 network structure to extract the feature information of images. A stacking method is designed in the decoder part to handle the attention information from top to bottom, so that additional feature information is available for each layer of network. Then, the feature is extracted from the generated annotation statement, and the attention feature is combined with the attention area of the image to enhance the matching with the image attention area, so that the generated annotation statement approximates the real context. Experiments were carried out on data sets such as Flickr8 k, Flickr30 k and MSCOCO. The experimental results show that the recognition rate of the proposed model is 5%~9% higher than that of the classical image recognition model.