多尺度特征融合轻量化夜间红外行人实时检测

Multi-Scale Feature Fusion Lightweight Real-Time Infrared Pedestrian Detection at Night

针对辅助驾驶中夜间小目标红外行人检测精度低、网络模型占用内存空间大、检测速度难以满足实时检测要求等问题,提出了一种轻量化的夜间红外图像行人检测神经网络YOLO-Person。首先提出一种以MobileNetV3轻量化网络为骨干网络,以多尺度融合目标检测层为预测模块的网络模型,以解决网络模型大、推理速度慢的问题,大幅减少了模型计算量,初步实现轻量化;然后通过在网络中添加空间金字塔池化模块与更小感受野的检测层,增强网络输出特征图的表征能力,解决数据集中行人目标尺度大小不均衡的问题,提高模型的红外行人检测精度;最后应用通道剪枝对模型进行剪枝,减少特征图的通道数,获得最终网络模型YOLO-Person。通过Jetson Nano移动开发平台,在夜间红外图像行人数据集上验证YOLO-Person轻量化模型,结果表明:与YOLOv3网络模型相比,提出的YOLO-Person网络模型更适于移动端的夜间红外行人检测,平均检测精度达到了92.2%,检测速度由26frame/s提高到了69frame/s,模型大小也由246MB减少到了11.7MB。

Objective Poor lighting conditions lead to a high accident rate during night driving.In order to reduce the incidence of night traffic accidents,various auxiliary driving technologies such as ultrasonic ranging,millimeter wave radar and visual auxiliary driving are widely used.Infrared thermal imaging technology based on the thermal radiation of object and reflection imaging with certain penetrability is less affected by the weather and light conditions at night.Human targets within the vision field can be accurately captured by infrared thermal imaging technology,which is convenient for pedestrian detection.In addition,the cost of infrared imaging equipment has been decreased in recent years,making it possible to be mounted on vehicles.Therefore,the fusion of infrared thermal imaging technology and pedestrian target detection algorithm based on deep learning is of great research significance and with a broad market application prospective in vehicle auxiliary driving.In this paper,a pedestrian detection model based on night infrared image is proposed for night driving,which can detect pedestrians on the night road in real time.This study can be applied to the field of auxiliary driving for early warning and active braking provided to drivers,reducing the probability of night driving accidents and providing higher security for vehicles and pedestrians.Methods Aiming at the problems of low accuracy in infrared pedestrian detection for small targets at night,large committed memory of network model,and the difficulty of real-time detection in auxiliary driving due to the low model detection speed,a lightweight pedestrian detection neural network called YOLO-Person is proposed for night infrared images.Firstly,the MobileNetV3 lightweight network is used as the backbone network,while the multi-scale fusion target detection layer is used as the prediction module to solve the problem of large model size and slow inference speed,which greatly reduces the amount of model calculation and obtains a preliminary lightweight network model.Furthermore,by adding the spatial pyramid pooling module and the detection layer with smaller receptive field in the network,the representation ability is enhanced to solve the problem of unbalanced pedestrian target scale in the dataset and improve the infrared pedestrian detection accuracy.Finally,channel pruning is used to reduce the number of channels in the feature map,and the final network model YOLO-Person is obtained.The lightweight model YOLO-Person is verified on the pedestrian dataset of night infrared images based on Jetson Nano mobile development platform.Results and Discussions A lightweight model YOLO-Person is proposed for night infrared pedestrian detection(Fig.1).Firstly,MobileNetV3 lightweight network is used as the backbone network,and the multi-scale fusion detection layer is used as the prediction module.Although the accuracy is reduced by 1.2%,the speed is increased by 34 frame/s,and the model size is reduced by 151 MB(Table 1),which indicates that the lightweight of the night infrared pedestrian detection model is preliminarily realized.Secondly,aiming at the problem of unbalanced pedestrian target scale in dataset,spatial pyramid pooling module(Fig.2)and small receptive field detection layer are added in the network,through which the accuracy is improved by 3.3%,the speed is reduced by 23 frame/s,and the model size is increased by 5.1 MB(Table 2).Moreover,the model is pruned(Fig.3)to reduce a large number of redundant channels(Fig.6).When the pruning rate is 95%,the number of model channels,accuracy and model size achieve balance and optimization(Table 3).In addition,the model is fine-tuned to obtain the final lightweight model YOLO-Person,which reaches the accuracy of 92.2%,the speed of 69 frame/s,and the model size of 11.7 MB(Table 4).Finally,the model is deployed on the Jetson Nano mobile development platform to verify the detection effect(Fig.7),and the test results of three networks are compared.The lightweight model YOLO-Person gets the best results:the accuracy of 92.2%,the speed of 12 frame/s,and the model size of 11.7 MB(Table 5).Conclusions A lightweight model YOLO-Person for night infrared pedestrian detection is proposed in this paper.Firstly,MobileNetV3 lightweight network is used as the backbone network,and the multi-scale fusion detection layer is used as the prediction module to achieve the preliminary model lightweight.Secondly,spatial pyramid pooling module and small receptive field detection layer are added to improve the detection accuracy of small targets.Finally,the model parameters are greatly reduced through channel pruning,and the final lightweight model YOLO-Person is obtained.The experimental results show that the detection accuracy and speed of YOLO-Person model reach 92.2%and 69 frame/s,respectively,meeting the requirements of real-time pedestrian detection.The YOLO-Person network model is deployed on the Jetson Nano mobile development platform,where the detection speed of 12 frame/s exceeds that of YOLOv3 and approaches that of YOLOv3-tiny,which further verifies the superiority of the proposed method.By optimizing the network structure and increasing the effective functional network layer,the detection accuracy of the model will be further improved in the future research.