融合附加神经网络的激光雷达点云单目标跟踪

Single Object Tracking of LiDAR Point Cloud Combined with Auxiliary Deep Neural Network

已有激光雷达点云单目标跟踪工作对分布稀疏、小规模点云目标的跟踪性能不佳。针对该问题,提出了一种融合附加神经网络的点云单目标跟踪算法。所提算法在网络训练时,利用附加网络执行前景点云分割和中心坐标偏移回归的附加任务,引导骨干网络学习稀疏、小规模目标点云的高鉴别力特征,供后续网络在逐帧点云中完成对目标的定位和与背景的区分;网络推断时,则绕过附加网络,直接由骨干网络提取目标点云特征,在保证跟踪任务准确性的同时满足任务实时性的要求。在KITTI数据集上的测试结果表明:相较于已有工作,所提算法在相同参数设定下平均跟踪成功率提高了0.89个百分点,平均跟踪准确率提高了2.51个百分点;并且通过对参数设定的进一步研究与调整,最终所提算法平均跟踪成功率提高了4.54个百分点,平均跟踪准确率提高了7.83个百分点,且对分布稀疏、点数较少的点云目标有明显的性能提升。

Objective Point cloud can be selected as an ideal data format for tasks such as object classification,detection,segmentation,reconstruction,and tracking in a three-dimensional(3D)scene.In the case of a single object tracking task,the approach of considering point clouds as data outperforms that of selecting two-dimensional(2D)picture or video sequences for two reasons.First,the point cloud can better describe the 3D geometric information of the object in real scenes,such as the position,scale,and posture of the object.Second,different from the passive optical imaging principle of cameras,information is collected using light detection and ranging(LiDAR)following an active imaging approach,which is not prone to be affected by natural light conditions.Therefore,the point cloud can adapt to different conditions involving visual degradation or illumination and is robust to glare,reflections,and shadows.Based on this discussion,the single object tracking of 3D point cloud is a topic worth investigating.Generally,single object tracking tasks aim to use the information in the given initial frame to determine the tracked object and predict the locating bounding box of the object in each subsequent frame.However,existing single object trackers of LiDAR point clouds exhibit a poor tracking performance of sparsely distributed and small-scale point cloud objects.This is mainly attributed to the downscaling operation applied to features extracted from the point cloud,leading to the insufficient application of object’s structural information;this distracts the tracker from performing accurate bounding box predictions of sparsely distributed and small-scale point cloud objects.Methods To address this problem,a single object tracking network combined with auxiliary deep neural network is proposed herein.During the training stage,we attach a modified auxiliary network to the backbone network,which accomplishes two auxiliary tasks:1)foreground point cloud segmentation,which guides the backbone network to focus on pointwise semantic information;2)pointwise center coordinate offset regression,which leads the features to be aware of the intrastructural information of the object.These two tasks are jointly supervised using the backbone network such that the semantic and structural features are naturally stored in the object features extracted using the backbone network.However,during the inference stage,the auxiliary network is bypassed in this process because the trained backbone network is already optimized to be structure aware and detaching the auxiliary network can avoid extra computational cost,which is essential for retaining the real-time performance of the tracker.Moreover,we notice that the latest work follows the same manner as the dataset organization.In particular,the number of input points in the search area point cloud and template point cloud is fixed,irrespective of the class of point cloud data.However,as the KITTI dataset presents,the point cloud of some classes is dense and comprises a large number of points,while the point cloud of other classes suffers from scarce points,providing insufficient and limited object information.A fixed number of input points may be unsuitable for all data classes.Hence,we propose setting different input quantities for each class during both the training and inference stages,which is accomplished without changing the network structure.In general,the structure of different network modules is shown in Fig.1 to Fig.6 respectively.Results and Discussions Both qualitative and quantitative experiments are conducted to prove the superiority of our propositions.Table 1 shows the result of extensive comparisons between our proposed tracker and other two former trackers;our tracker achieves better results in three of four data classes and shows higher mean performance than the other two former trackers.Test results on the KITTI dataset show that our network increases the average tracking success by 0.89 percent and the average tracking accuracy by 2.51 percent under the same parameter settings as the existing work.Second,some specific tracking results of four data classes are depicted in Fig.7--Fig.10.These figures show that our tracker can predict bounding boxes closely similar to the ground truth.Furthermore,we present some results of the tasks processed using the auxiliary network.Concretely,Fig.11--Fig.14 show the results of foreground segmentation task,in which our auxiliary network can accurately segment the surface of the object from the background points.Moreover,details of comparisons on tracking performance with different numbers of the search area point cloud and template point cloud are discussed in Table 2,which prove the efficiency of adjusting a suitable quantity of input points of different data classes.Compared with proposed algorithm,the average tracking success increases by 4.54 percent,and the average tracking accuracy increases by 7.83 percent.In particular,for class Cyclist,the average tracking success increases by 2.38 percent and the average tracking accuracy increases by 2.86 percent;for class Pedestrian,the average tracking success increases by 8.47 percent and the average tracking accuracy increases by 13.71 percent.These findings imply that our tracker achieves improvements in terms of the tracking performance of sparse and small-scale objects.Finally,Table 3 and Table 4 show that we succeed in maintaining a balance between performance and computational costs.Conclusions In summary,the proposed method achieves reasonable results in addressing the problem of tracking sparsely distributed and small-scale point cloud objects as expected,and can be applied to solve other tasks.Inspired by our experiment results,to achieve further improvement,we will seek new approaches on data augmentation and extract more useful clues using the background information for search area updates.