融合点云与全景影像的路侧多目标识别

Road side multi-object recognition by integrating point cloud and panoramic image

为了从点云中自动准确识别车辆、垃圾桶与杆状交通设施等目标,本文提出了一种融合点云与全景影像的路侧多目标识别方法,充分利用点云数据中的空间几何信息与全景影像中的语义信息提升目标的识别准确度。首先,对全景影像进行实例分割,获取图像中目标的二维掩码(mask);然后,将激光点云投影生成全景深度图后,以深度图像为媒介获得对应的候选目标点云;最后,对于相机拍摄时产生的视角遮挡与被遮挡问题,通过分析目标在三维空间中的连续性与完整性,对候选点云进行二次聚类,最终完成对目标的分类。试验结果表明,3类目标的准确率分别为96.64%、92.68%及90.74%,证明本文方法可有效识别城市场景中的车辆、垃圾桶与杆状等交通设施。

In order to automatically and accurately identify objects such as vehicles,garbage cans and rod-shaped traffic facilities from point clouds,this paper proposes a roadside multi-object recognition method that integrates point clouds and panoramic images,and makes full use of spatial geometry information in point cloud data and semantic information in panoramic images to improve the accuracy of target recognition.Firstly,the panoramic image is segmented to obtain the two-dimensional mask of the object in the image.Then,the laser point cloud is projected to generate a panoramic depth map,and the corresponding candidate point cloud is obtained using the depth image as the medium.Finally,for the problem of angle occlusion and occlusion caused by camera shooting,through analyzing the continuity and integrity of the target in 3D space,secondary clustering of candidate point clouds is carried out,and finally the classification of the target is completed.Experimental results show that the accuracy rates of the three types of targets are 96.64%,92.68% and 90.74% respectively,which prove that the proposed method can effectively identify vehicles,garbage cans and rod-shaped traffic facilities in urban scenes by integrating semantic information in images with spatial geometry information in point clouds.