四流输入引导的特征互补可见光-红外行人重识别

Quadrupl-stream input-guided feature complementary visible-infrared person re-identification

目前可见光-红外行人重识别研究侧重于通过注意力机制提取模态共享显著性特征来最小化模态差异。然而,这类方法仅关注行人最显著特征,无法充分利用模态信息。针对此问题,本文提出了一种四流输入引导的特征互补网络(QFCNet)。首先在模态特定特征提取阶段设计了四流特征提取和融合模块,通过增加两流输入,缓解模态间颜色差异,丰富模态的语义信息,进一步促进多维特征融合;其次设计了一个次显著特征互补模块,通过反转操作补充全局特征中被注意力机制忽略的行人细节信息,强化行人鉴别性特征。在SYSU-MM01,Reg DB两个公开数据集上的实验数据表明了此方法的先进性,其中在SYSU-MM01的全搜索模式中rank-1和m AP值达到了76.12%和71.51%。

Current visible-infrared person re-identification research focuses on extracting modal shared saliency features through the attention mechanism to minimize modal differences.However,these methods only focus on the most salient features of pedestrians,and cannot make full use of modal information.To solve this problem,a quadrupl-stream input-guided feature complementary network(QFCNet)is proposed in this paper.Firstly,a quadrupl-stream feature extraction and fusion module is designed in the mode-specific feature extraction stage.By adding two data enhancement inputs,the color differences between modalities are alleviated,the semantic information of the modalities is enriched and the multi-dimensional feature fusion is further promoted.Secondly,a sub-salient feature complementation module is designed to supplement the pedestrian detail information ignored by the attention mechanism in the global feature through the inversion operation,to strengthen the pedestrian discriminative features.The experimental results on two public datasets SYSU-MM01 and RegDB show the superiority of this method.In the full search mode of SYSU-MM01,the rank-1 and mAP values reach 76.12%and 71.51%,respectively.