基于多模态图卷积神经网络的行人重识别方法

Person re-identification method based on multi-modal graph convolutional neural network

针对行人重识别中行人文本属性信息未被充分利用以及文本属性之间语义联系未被挖掘的问题,提出一种基于多模态的图卷积神经网络(GCN)行人重识别方法。首先使用深度卷积神经网络(DCNN)学习行人文本属性与行人图像特征;然后借助GCN有效的关系挖掘能力,将文本属性特征与图像特征作为GCN的输入,通过图卷积运算来传递文本属性节点间的语义信息,从而学习文本属性间隐含的语义联系信息,并将该语义信息融入图像特征中;最后GCN输出鲁棒的行人特征。该多模态的行人重识别方法在Market-1501数据集上获得了87.6%的平均精度均值(mAP)和95.1%的Rank-1准确度;在DukeMTMC-reID数据集上获得了77.3%的mAP和88.4%的Rank-1准确度,验证了所提方法的有效性。

Aiming at the problems that person textual attribute information is not fully utilized and the semantic relationships among the textual attributes are not mined in person re-identification,a person re-identification method based on multi-modal Graph Convolutional neural Network(GCN)was proposed.Firstly,Deep Convolutional Neural Network(DCNN)was used to learn person textual attributes and person image features.Then,with the help of the effective relationship mining ability of GCN,the textual attribute features and image features were treated as the input of GCN,and the semantic information of the textual attribute nodes was transferred through the graph convolution operation,so as to learn the implicit semantic relationship information among the textual attributes and incorporate this semantic information into image features.Finally,the robust person features were output by GCN.The multi-modal person re-identification method achieves the mean Average Precision(mAP)of 87.6% and the Rank-1 accuracy of 95.1% on Market-1501 dataset,and achieves the mAP of 77.3% and the Rank-1 accuracy of 88.4%on DukeMTMC-reID dataset,which verify the effectiveness of the proposed method.