不同光照下多模态注意力融合的车辆检测

Vehicle Detection of Multi-Modal Attention Fusion Under Different Illumination

针对现有基于单模态车辆检测算法受光照变换所导致的性能下降问题,提出了一种红外和可见光融合的多模态检测方法YOLO-MMF。该方法构建高效的双流特征提取网络,分别提取可见光图像和红外图像的特征,用DenseBlock结构代替了YOLOv5中浅层CSP模块中的瓶颈层,加强对小目标的特征提取能力;采用特征级融合机制,利用离散余弦变换获取高频信息,改善因平均池化使细节信息丢失的现象,并与自注意力机制相结合,使网络可以自发捕捉模态间潜在的互补性,从而显著提高车辆检测的性能。在DroneVehicle数据集上的实验结果证实了该方法的有效性,相比单一模态检测方式,平均检测精度分别提升了14.4个百分点和10.8个百分点,该方法在面对光照变换等复杂情况时具有较好的鲁棒性。

Aiming at the performance degradation of existing single-modal vehicle detection algorithms caused by illumination changes,a multi-modal detection method YOLO-MMF,which combines infrared and visible light,is proposed.This method builds an efficient dual-stream feature extraction network,extracts the features of visible light images and infrared images respectively,replaces the bottleneck layer in the shallow CSP module in YOLOv5 with the DenseBlock structure,and strengthens the feature extraction ability of small targets.This method adopts feature fusion mechanism,uses discrete cosine transform to obtain high-frequency information,improves the loss of detail information due to average pooling,and combines the self-attention mechanism,so that the network can spontaneously capture the potential complementarity between modalities,thereby significantly improving vehicle detection performance.The experimental results on the DroneVehicle dataset confirm the effectiveness of the method,with an average detection accuracy improvement of 14.4 percentage points and 10.8 percentage points respectively,compared to the single-modal detection approach,which is more robust in the face of complex situations such as illumination shifts.