红外-可见光图像融合的全天候目标追踪方法

All-weather tracking method based on infrared-visible image fusion

目前主流的监控视频处理方式仍然是根据光照条件,单独使用可见光或红外光视频。本文提出双流全卷积孪生网络,旨在利用图像融合技术,将可见光与红外光图像各自的优点互补,辅助目标检测与追踪。在特征提取模块,构建以DenseNet为主干的孪生神经网络。在特征融合与重构阶段,使用全卷神经网络构造全局特征和模板特征。在目标追踪阶段,对全局特征和模板特征进行互相关操作,得到跟踪结果。本文方法充分利用双模态视频的深度信息,有效解决极端天气以及夜间光线不足造成的图像采集模糊问题,增强追踪模型在复杂情景下的鲁棒性。

The mainstream surveillance video processing is still using visible or infrared video alone.In this paper,a Two-Stream Fully Convolutional Siamese Network(TFSiamNet)is proposed to assist target detection and tracking.Firstly,in the feature extraction module,a Siamese network with DenseNet is constructed.Secondly,in the feature fusion and reconstruction stage,the fully convolutional layers are used to construct the global feature and template feature.Finally,in the tracking stage,the crosscorrelation between the global feature and the template feature is carried out to get the tracking results.The proposed method makes full use of the depth information of the dual-mode video,effectively solves the problem of fuzzy image acquisition caused by extreme weather and insufficient light at night,and enhances the robustness of the tracking model under complex scenarios.