可逆多分支的双模态自适应融合目标跟踪算法

Enhanced target tracking algorithm with reversible multi branch bimodal adaptive fusion

由于可见光和红外图像具有很强的互补性,越来越多的关注集中在通过这两种模态的联合信息进行跟踪。然而,在现有的跟踪算法中,不能有效地学习两者的互补信息并挖掘模态特定特征,这限制了跟踪器的性能。因此,本文提出了一种可逆多分支的双模态自适应融合跟踪网络。首先,设计了一个三分支结构网络,分别用于学习热红外、可见光以及它们的通用特征。这不仅充分利用了两种模态之间的共享信息,还保留了红外和可见光数据之间的差异特性以及丰富的细节信息。此外,还引入了一个模态特征交互模块,以自适应地挖掘模态之间的互补信息并滤除冗余信息。通过在多个公开数据集上进行大量实验证明了该跟踪器的有效性,尤其在面对尺度变化、镜头抖动、遮挡等环境时,表现出卓越的抗干扰能力。

Due to the strong complementarity between visible light and infrared images,more attention has been focused on tracking through the joint information of these two modalities.However,in existing tracking algorithms,hthe inability to effectively learn the complementary information of both and mine modality specific features limits the performance of the tracker.In responseto this issue,a reversible multibranch bimodal adaptive fusion network for tracking is proposed.Firstly,a tri branch structured network is designed for separate learning of thermal infrared,visible light,and their shared characteristics.This design not only maximizes the utilization of shared modal information,but also preserves the differential characteristics between infrared and visible data as well as the rich detail information.Furthermore,an adaptive module for modal feature interaction is introduced to efficiently mine complementary modal information and filter out redundant data.Extensive experiments conducted on multiple public datasets proves the effectiveness of this tracker,particularly showcasing remarkable anti interference capabilities in scenarios involving scale changes,camera shakes,and occlusion.