基于双模态图像关联式融合的行人实时检测

Real-Time Pedestrian Detection Based on Dual-Modal Relevant Image Fusion

为解决行人检测任务中低能见度场景下单模态图像漏检率高和现有双模态图像融合检测速度低等问题,提出了一种基于双模态图像关联式融合的轻量级行人检测网络。网络模型基于YOLOv7-Tiny设计,主干网络嵌入关联式融合模块RAMFusion用以提取和聚合双模态图像互补特征;将特征提取部分的1×1卷积替换为带有空间感知能力的坐标卷积;引入Soft-NMS改善结群行人漏检问题;嵌入注意力机制模块来提升模型检测精度。在公开的红外与可见光行人数据集LLVIP上的消融实验表明:与其他融合方法相比,所提方法行人漏检率降低、检测速度显著提高;与YOLOv7-Tiny相比,改进后的模型检测精度提高了2.4%,每秒检测帧数达到124 frame/s,能够满足低能见度行人实时检测需求。

In order to solve the problems of high missing detection rate of singlemodel images and low detection speed of existing dualmodel image fusion in pedestrian detection tasks under low visibility scenes,a lightweight pedestrian detection network based on dualmodel relevant image fusion is proposed.The network model is designed based on YOLOv7-Tiny,and the backbone network is embedded with RAMFusion,which is used to extract and aggregate dualmodel image complementary features.The 1×1 convolution of feature extraction is replaced by coordinate convolution with spatial awareness.SoftNMS is introduced to improve the pedestrian omission in the cluster.The attention mechanism module is embedded to improve the accuracy of model detection.The ablation experiments in public infrared and visible pedestrian dataset LLVIP show that compared with other fusion methods,the missing detection rate of pedestrians is reduced and the detection speed of the proposed method is significantly increased.Compared with YOLOv7-Tiny,the detection accuracy of the improved model is increased by 2.4%,and the detection frames per second is up to 124 frame/s,which can meet the requirements of realtime pedestrian detection in lowvisibility scenes.