集成空间注意力和姿态估计的遮挡行人再辨识

Integrated Spatial Attention and Pose Estimation for Occluded Person Re-Identification

自然场景下监控设备所拍摄的行人图片总是存在被各种障碍物遮挡的情况,因此遮挡是行人再辨识面临的一个很大的挑战.针对遮挡问题,提出了一种集成空间注意力和姿态估计(spatial attention and pose estimation, SAPE)的遮挡行人再辨识模型.为了同时兼顾全局特征和局部特征,实现特征的多细粒度表示,构建了多任务网络.通过空间注意力机制将感兴趣区域锚定到图像中未遮挡的空间语义信息,从全局结构模式中挖掘有助于再辨识的视觉知识;然后结合分块匹配的思想,将残差网络提取到的特征图水平均匀分割成若干块,通过局部特征的匹配增加辨识的细粒度;在此基础之上,改进姿态估计器去提取图像中行人的关键点信息,并与卷积神经网络抽取的特征图相融合,然后设置阈值去除掉遮挡区域,得到辨识性强的特征,以消除遮挡对再辨识结果的影响.在Occluded-DukeMTMC, Occluded-REID, Partial-REID这3个数据集上验证了SAPE模型的有效性,实验结果表明提出的针对遮挡的模型具有良好的效果.

Since the pedestrian images taken by the monitoring equipment in natural scenes are always occluded by various obstacles, occlusions is a great challenge for person re-identification. For the above problems, a spatial attention and pose estimation(SAPE) is proposed. In order to give consideration to both global and local features, a multi-task network is constructed to realize multi-granularity representation of features. By means of spatial attention mechanism, the region of interest is directed to the spatial semantic information in the image, and the visual knowledge which is helpful for re-identification is mined from the global structural pattern. Then, combined with the idea of part matching, the feature map extracted from the residual network is evenly divided into several parts horizontally, and the identification granularity is increased by matching the local features. On this basis, the key information of pedestrians in the image extracted by the improved pose estimator is fused with the feature map extracted by the convolutional neural network, and the threshold is set to remove the occlusion area, and the features with strong identification are obtained, so as to eliminate the influence of occlusion on the re-identification results. We verify the effectiveness of the SAPE model on three datasets of Occluded-DukeMTMC, Occluded-REID and Partial-REID. The experimental results show that SAPE has achieved good experimental results.