基于特征对齐与区域图像质量引导融合的可见光-红外装甲车辆检测方法

Method of Visible-Infrared Armored Vehicle Detection Based on Feature Alignment and Regional Image Quality Guided Fusion

提出了一种基于特征对齐与区域图像质量引导融合的可见光-红外装甲车辆(VTAV)检测方法。该方法将当前先进的单阶段无锚框检测方法 YOLOv8拓展为双流目标检测框架,在双流特征提取网络中嵌入特征对齐模块,解决图像对未对齐导致的位置偏移问题。为充分发挥可见光图像和红外图像的互补性,设计了一种图像质量引导融合策略。该策略对可见图像和红外图像的区域图像质量进行评估,并使用特征质量矩阵引导可见光特征和红外特征的融合。最后,在VTAV图像数据集上的实验结果表明,与仅使用可见光图像的检测方法相比,本文方法的mAP(平均精度)和mAP50(50%交并比下的平均精度)分别提升了1.9%和3.5%。与使用特征相加和特征拼接融合方式的检测方法相比,本文方法的mAP分别提升了1.1%和1%。本文方法能够有效克服复杂地面环境下的多种干扰,能够提升对装甲车辆的检测精度和成功率。

Objective Armored vehicles serve as crucial ground combat equipment, playing an irreplaceable role in urban attacks,defense, beach landings, and various other operations. Hence, researching armored vehicle detection technology in complex ground environments holds significant importance for accurate battlefield perception, situational awareness,precise fire targeting, and seizing battlefield opportunities. Existing image-based detection methods for armored vehicles primarily utilize visible or infrared images. Visible images often struggle to effectively handle interference from similar backgrounds, smoke, dust, and camouflage in complex ground battlefield environments. While infrared images can overcome some limitations of visible images, they often lack sufficient texture and color information. Therefore,integrating visible and infrared images and leveraging their complementary characteristics can enhance feature representation and help elevate the detection capabilities of armored vehicles in complex ground battlefield environments.Methods To address the challenge of detecting armored vehicles in complex land environments, we put forward a visibleinfrared armored vehicle detection method that leverages feature alignment and region-based image quality guided fusion.Firstly, we enhance the YOLOv8 object detection method, a state-of-the-art one-stage anchor-free approach, by incorporating a backbone network for infrared feature extraction. This expansion results in a dual-stream architecture for enhanced performance. During the extraction of infrared features, a feature alignment module is introduced built on deformable convolutional networks. This module effectively aligns infrared features, addressing issues caused by misalignment in images. To fully utilize the complementary nature of visible and infrared images, we design a regional image quality guided fusion module for integrating features from both modalities. The regional image quality guided fusion module assesses the quality of multi-scale visible and infrared features extracted by the dual-stream feature extraction network. It generates quality matrices for both visible and infrared features, which are then processed using the Softmax function to obtain weight matrices. These weight matrices are used to combine the two modal features, resulting in a fusion feature achieved through element-wise addition or channel concatenation. Finally, the fusion feature passes through the Neck and the detection head to produce the detection results.Results and Discussions Experimental verification is conducted using self-built visible-infrared armored vehicle image datasets as well as publicly available FPR-aligned datasets. To simulate position shifts, the infrared image is moved along the x and y axes. The experimental results demonstrate that our feature alignment module exhibits a more pronounced effect with increasing offset(Table 5), effectively mitigating the adverse effects of position offset and enhancing the model's robustness. Furthermore, the regional image quality guided fusion module offers improved assessment of regional image quality, fully leveraging the complementarity of the two modal features and attenuating the impact of disturbed regional image features during cross-modal feature fusion(Fig. 7). In comparison to object detection methods that solely utilize visible images, our method has shown improvements in mAP and mAP50 by 1.9% and 3.5%, respectively(Table 7). Additionally, our method demonstrates enhanced capability in addressing challenges such as smoke shielding,interference from similar ground objects, and slight dust shielding, thereby elevating the level of armored vehicle detection(Fig. 8).Conclusions We propose a visible-infrared detection method for armored vehicles based on feature alignment and regional image quality guided fusion to address challenges such as position deviation and varying importance of visible light features across different spatial locations in complex ground environments. The method integrates a feature alignment module,utilizing feasible variable convolution within a two-stream feature extraction network, to align infrared images and strengthen model robustness against unaligned image pairs. Additionally, we design a regional image quality guided fusion module, leveraging semantic label information to train a network for evaluating regional image quality and using the resulting feature quality matrix to guide the fusion of visible and infrared features. Experimental evaluations are conducted on a self-built visible-infrared armored vehicle image dataset, demonstrating that our proposed method outperforms stateof-the-art object detection methods. By effectively leveraging the complementarity of visible and infrared images, this method significantly improves the accuracy and success rate of armored vehicle detection in complex ground environments.