基于多头图注意力网络与图模型的多标签图像分类

Multi-label Image Classification Based on Multi-head Graph Attention Network and Graph Model

多标签图像分类是多标签数据分类问题中的研究热点.针对目前多标签图像分类方法只学习图像的视觉表示特征,忽略了图像标签之间的相关信息以及标签语义与图像特征的对应关系等问题,提出了一种基于多头图注意力网络与图模型的多标签图像分类模型(ML-M-GAT).该模型利用标签共现关系与标签属性信息构建图模型,使用多头注意力机制学习标签的注意力权重,并利用标签权重将标签语义特征与图像特征进行融合,从而将标签相关性与标签语义信息融入到多标签图像分类模型中.为验证本文所提模型的有效性,在公开数据集VOC-2007和COCO-2014上进行实验,实验结果表明, ML-M-GAT模型在两个数据集上的平均均值精度(mAP)分别为94%和82.2%,均优于CNN-RNN、ResNet101、MLIR、MIC-FLC模型,比ResNet101模型分别提高了4.2%和3.9%.因此,本文所提的ML-M-GAT模型能够利用图像标签信息提高多标签图像分类性能.

Multi-label image classification is a research hotspot in multi-label data classification.The existing multi-label image classification methods only learn the visual representation features of images and ignore the relevant information between image labels and the correspondence between label semantics and image features.In order to solve these problems,a multi-label image classification model based on a multi-head graph attention network and graph model(MLM-GAT)is proposed.By using label co-occurrence and attribute information,the model builds a graph model,and it employs the multi-head attention mechanism to learn the attention weight of the label.In addition,the model utilizes label weights to fuse label semantic features and image features,so as to integrate label correlation and label semantic information into the multi-label image classification model.In order to verify the effectiveness of the proposed model,experiments are carried out on the public datasets VOC-2007 and COCO-2014,and the experimental results show that the average mean accuracy(mAP)of the ML-M-GAT model on the two datasets is 94%and 82.2%,respectively,which are better than that of CNN-RNN,ResNet101,MLIR,and MIC-FLC models and are 4.2%and 3.9%higher than that of ResNet101 models,respectively.Therefore,the proposed model can improve the performance of multi-label image classification by using image label information.