基于双孪生网络的可见光热红外目标实时鲁棒跟踪

Real-time robust RGB-T target tracking based on dual Siamesenetwork

针对可见光与热红外图像融合跟踪中采用孪生网络架构进行跟踪时鲁棒性较差的问题,提出了一种基于双孪生网络特征融合的可见光热红外(RGB-T)目标实时跟踪方法。首先,采用两个孪生网络分别对可见光和红外图像的模板分支和搜索分支进行特征提取,得到两种模态特征层;然后,利用自注意力特征增强模块(SFEM)对两种模态的特征进行增强,并使用双模特征融合(DMFF)模块对模板分支和搜索分支增强后的特征分别进行融合;最后,将融合后的模板分支和搜索分支进行特定任务的互相关操作,并通过分类和回归分支得到目标位置,从而完成跟踪。在灰度热红外目标跟踪数据集(GTOT)上的测试结果表明,本文方法的精确率(PR)为91.8%,成功率(SR)为78.1%,运行速度为60 f/s。与其他RGB-T融合跟踪方法相比,本文方法能够在保持实时处理速度的同时具备较高的鲁棒性。

Aiming at the problem of poor robustness in visible light and thermal infrared image fusion tracking using Siamese network architecture,a real-time RGB-Thermal(RGB-T)infrared target tracking method based on feature fusion of dual Siamese network was proposed.Firstly,dual Siamese network was used to extract the features of template branch and the search branch of the visible light and infrared images,and two different modalities feature layers were obtained.Secondly,the self-attention feature enhancement module(SFEM)was used to enhance the features of the two modalities,and the dual-modal feature fusion(DMFF)module was used to fuse the features of template branch and search branch respectively.Finally,the template branch and search branch were used for cross-correlation operation,and the target position was obtained by classification and regression branches,so as to complete target tracking.The test results on the grayscale thermal infrared target tracking dataset(GTOT)show that the precision rate(PR)of the proposed method is 91.8%,the success rate(SR)is 78.1%,and the running speed is 60 f/s.It shows that,compared with other RGB-T fusion tracking methods,the proposed method has higher robustness while maintaining real-time processing speed.