跨模态身份不一致的交叉分类素描行人重识别

Cross-Classification-Based Sketch Person Re-Identification in Inconsistent Cross-Modal Identity Scenes

素描行人重识别旨在从可见光行人图像库中查找与给定素描行人图像身份相同的图像。虽然已经有较多的跨模态检索算法可以应用于该类问题,但这些算法的背景设定较为单一,没有考虑到训练集中部分身份的行人仅有一个模态样本,即跨模态身份不一致,这极大限制了算法在实际场景下的应用。为此,提出了基于交叉分类的素描行人重识别网络。该网络包括交叉分类和基于距离的身份信息对齐两部分。其中,交叉分类利用单一模态数据训练的分类器引导编码器从另一模态提取到模态不变的信息。而基于距离的身份信息对齐能够将同身份不同模态间的特征距离减小,同时抑制跨模态身份不一致的影响,进而强化了特征的判别性和鲁棒性。为验证跨模态身份不一致时模型的性能表现,基于Matket-1501数据集生成了新的素描行人重识别数据集S-Market1501,并在该数据集上将Rank-1指标提升了11.0个百分点。同时模型在公开数据集Sketch Re-ID上Rank-1指标达到了60%,所设计的数据集将开源在“https://github.com/huangdaichui/Sketch_dataset”。

Sketch person reidentification aims to identify images with identities similar to those of sketched person images located in an RGB image gallery.Although several crossmodal retrieval algorithms can be adopted for this purpose,the background settings of such algorithms are relatively simple and fail to consider that certain identities have only one modal sample in the training set,that is,the crossmodal identity is inconsistent.This significantly limits the application of such algorithms in practical scenarios.In this paper,a sketch reidentification network based on crossclassification is proposed.The network consists of two parts:crossclassification and identity information alignment based on distance.Among these,crossclassification guides the encoder to extract modalinvariant information from one modal using constraints of the classifier trained using other modal data.The alignment of identity information based on distance can reduce the feature distance between different modals of the same identity,suppress the influence of crossmodal identity inconsistencies,and strengthen the discrimination and robustness of features.To verify the performance of the reidentification network when the crossmodal identity is inconsistent,a new sketch reidentification dataset is generated based on Market1501.The Rank1 is improved by 11.0 percentage points on this dataset.Simultaneously,the model also achieves a Rank1 of 60%on the public dataset Sketch ReID.The dataset used in this study is an opensource dataset available on“https://github.com/huangdaichui/Sketch_dataset”.