基于特征融合的RGBT双模态孪生跟踪网络

RGBT dual-modal Siamese tracking network with feature fusion

热红外成像技术被广泛地应用于军事、遥感和安防等领域中的目标跟踪,但热红外图像对对比度较低、目标模糊等跟踪场景效果一般。因此,将热红外图像与可见光图像进行融合提高跟踪性能具有重要意义。与基于可见光或热红外图像的单模态跟踪算法相比,基于可见光/热红外(RGB/Thermal,RGBT)图像的双模态跟踪算法对光照变化、云雾遮挡具有更强的鲁棒性。提出了一种基于特征融合的RGBT双模态孪生跟踪网络架构。该网络将双模态图像中提取的深度特征进行融合,提高目标外观特征的判别力。该网络可以利用训练数据进行端到端的离线训练。公开数据集RGBT234上的实验结果表明,所提出的RGBT双模态孪生特征融合跟踪网络能够实现复杂场景下鲁棒持续的目标跟踪。

Infrared imaging technology has been widely used for object tracking in military,remote sensing,security and other fields.However,thermal infrared images generally suffer from low contrast and blurry targets.Therefore,it has great importance of fusing infrared images with visible images.Compared with single-modal RGB trackers,dual-modal RGBT(RGB/Thermal infrared)trackers are more robust to illumination variation and fog.In this paper,a RGBT dual-modal siamese tracking network with feature fusion was proposed.Convolutional features extracted from the visible image and infrared image were fused to improve the appearance feature discrimination.The network can use the training data for end-to-end off-line training.Experimental results on the public RGBT234 dataset demonstrate that our tracker achieves robust and persistent tracking in complex scenarios.