融合图像特征的单阶段点云目标检测

Single-Stage Object Detection with Fusion of Point Cloud and Image Feature

使用图像信息补充三维点云的几何和纹理信息,可以对三维物体进行有效地检测与分类。为了能够更好地将图像特征融入点云,设计了一个端到端的深度神经网络,提出了一个新颖的融合模块PI-Fusion(point cloud and image fusion),使用图像特征以逐点融合的方式来增强点云的语义信息。另外,在点云下采样的过程中,使用距离最远点采样和特征最远点采样的融合采样方式,以在小目标上采样到更多的点。经过融合图像和点云特征的三次下采样之后,通过一个候选点生成层将点移动到目标物体的中心。最后,通过一个单阶段目标检测头,得出分类置信度和回归框。在公开数据集KITTI的实验表明,与3DSSD相比,此方法在简单、中等、困难难度的检测上分别提升了3.37、1.92、1.58个百分点。

3D objects can be effectively detected and classified with the supplement of image information to the geometry and texture information of point cloud.Aiming at the problem of effectively integrating image features into point clouds,an end-to-end deep neural network has been designed.A novel fusion module named PI-Fusion(point cloud and image fusion)is proposed to enhance semantic information of point cloud by using image features in a point-by-point manner.In addition,during the downsampling process,a fusion-sampling strategy is adopted to make use of distance farthest point sampling and feature farthest point sampling for small objects.After three downsampling processes of fused image and point cloud features,points are moved to the center of target object through a candidate point generation layer.Finally,the classification confidence and regression are predicted by the network with a single-stage object detection header.Experimental results on KITTI dataset show that the proposed method has improved 3.37,1.92 and 1.58 percentage points in simple,moderate and hard degree of difficulty on KITTI dataset respectively in comparison with 3DSSD.