基于CNN的点云图像融合目标检测

Fusing point cloud with image for object detection using convolutional neural networks

针对自动驾驶场景中目标检测存在尺度变化、光照变化和缺少距离信息等问题,提出一种极具鲁棒性的多模态数据融合目标检测方法,其主要思想是利用激光雷达提供的深度信息作为附加的特征来训练卷积神经网络(CNN)。首先利用滑动窗对输入数据进行切分匹配网络输入,然后采用两个CNN特征提取器提取RGB图像和点云深度图的特征,将其级联得到融合后的特征图,送入目标检测网络进行候选框的位置回归与分类,最后进行非极大值抑制(NMS)处理输出检测结果,包含目标的位置、类别、置信度和距离信息。在KITTI数据集上的实验结果表明,本文方法通过多模态数据的优势互补提高了在不同光照场景下的检测鲁棒性,附加滑动窗处理改善了小目标的检测效果。对比其他多种检测方法,本文方法具有检测精度与检测速度上的综合优势。

Addressing on the issues like varying object scale,complicated illumination conditions,and lack of reliable distance information in driverless applications,this paper proposes a multi-modal fusion method for object detection by using convolutional neural networks.The depth map is generated by mapping LiDAR point cloud onto the image plane and taken as input data together with the RGB image.The input data is also processed by the sliding window to reduce information loss.Two feature extracting networks are used to extract features of the image and the depth map respectively.The generated feature maps are fused through a connection layer.The objects are detected by processing the fused feature map through position regression and object classification.Non-maximal suppression is used to optimize the detection results.The experimental results on the KITTI dataset show that the proposed method is robust in various illumination conditions and especially effective on detecting small objects.Compared with other methods,the proposed method exhibits integrated advantages in terms of detection accuracy and speed.