Deep learning based point cloud registration:an overview

Point cloud registration aims to find a rigid transformation for aligning one point cloud to another.Such registration is a fundamental problem in computer vision and robotics,and has been widely used in various applications,including 3D reconstruction,simultaneous localization and mapping,and autonomous driving.Over the last decades,numerous researchers have devoted themselves to tackling this challenging problem.The success of deep learning in high-level vision tasks has recently been extended to different geometric vision tasks.Various types of deep learning based point cloud registration methods have been proposed to exploit different aspects of the problem.However,a comprehensive overview of these approaches remains missing.To this end,in this paper,we summarize the recent progress in this area and present a comprehensive overview regarding deep learning based point cloud registration.We classify the popular approaches into different categories such as correspondences-based and correspondences-free approaches,with effective modules,i.e.,feature extractor,matching,outlier rejection,and motion estimation modules.Furthermore,we discuss the merits and demerits of such approaches in detail.Finally,we provide a systematic and compact framework for currently proposed methods and discuss directions of future research.

ALGORITHM OF PRETREATMENT ON AUTOMOBILE BODY POINT CLOUD

As point cloud of one whole vehicle body has the traits of large geometric dimension, huge data and rigorous reverse precision, one pretreatment algorithm on automobile body point cloud is put forward. The basic idea of the registration algorithm based on the skeleton points is to construct the skeleton points of the whole vehicle model and the mark points of the separate point cloud, to search the mapped relationship between skeleton points and mark points using congruence triangle method and to match the whole vehicle point cloud using the improved iterative closed point （ICP） algorithm. The data reduction algorithm, based on average square root of distance, condenses data by three steps, computing datasets＇ average square root of distance in sampling cube grid, sorting order according to the value computed from the first step, choosing sampling percentage. The accuracy of the two algorithms above is proved by a registration and reduction example of whole vehicle point cloud of a certain light truck.

Fast Estimation of Loader’s Shovel Load Volume by 3D Reconstruction of Material Piles

Fast and accurate measurement of the volume of earthmoving materials is of great signifcance for the real-time evaluation of loader operation efciency and the realization of autonomous operation. Existing methods for volume measurement, such as total station-based methods, cannot measure the volume in real time, while the bucket-based method also has the disadvantage of poor universality. In this study, a fast estimation method for a loader’s shovel load volume by 3D reconstruction of material piles is proposed. First, a dense stereo matching method (QORB–MAPM) was proposed by integrating the improved quadtree ORB algorithm (QORB) and the maximum a posteriori probability model (MAPM), which achieves fast matching of feature points and dense 3D reconstruction of material piles. Second, the 3D point cloud model of the material piles before and after shoveling was registered and segmented to obtain the 3D point cloud model of the shoveling area, and the Alpha-shape algorithm of Delaunay triangulation was used to estimate the volume of the 3D point cloud model. Finally, a shovel loading volume measurement experiment was conducted under loose-soil working conditions. The results show that the shovel loading volume estimation method (QORB–MAPM VE) proposed in this study has higher estimation accuracy and less calculation time in volume estimation and bucket fll factor estimation, and it has signifcant theoretical research and engineering application value.

Point cloud registration for agriculture and forestry crops based on calibration balls using Kinect V2

For the process of point cloud registration,and the problem of inaccurate registration due to errors in correspondence between keypoints.In this paper,a registration method based on calibration balls was proposed,the trunk,branch,and crown were selected as experimental objects,and three calibration balls were randomly placed around the experimental objects to ensure different distances between two ball centers.Using the Kinect V2 depth camera to collect the point cloud of the experimental scene from four different viewpoints,the PassThrough filter algorithm was used for point cloud filtering in each view of the experimental scenes.The Euclidean cluster extraction algorithm was employed for point cloud clustering and segmentation to extract the experimental object and the calibration ball.The random sample consensus(RANSAC)algorithm was applied to fit the point cloud of a ball and calculate the coordinates of the ball center so that the distance between two ball centers under different viewpoints can be obtained by using the coordinates of the ball center.Comparing the distance between the ball centers from different viewpoints to determine the corresponding relationship between the ball centers from different viewpoints,and then using the singular value decomposition(SVD)method,the initial registration matrix was obtained.Finally,Iterative Closest Point(ICP)and its improved algorithm were used for accurate registration.The experimental results showed that the method of point cloud registration based on calibration balls can solve the problem of corresponding error of keypoints,and can register point clouds from different viewpoints of the same object.The registration method was evaluated by using the registration running time and the fitness score.The final registration running time of different experimental objects was not more than 6.5 s.The minimum fitness score of the trunk was approximately 0.0001,the minimum fitness score of the branch was approximately 0.0001,and the minimum fitness score of the crown was approximately 0.0006.

A study of projections for key point based registration of panoramic terrestrial 3D laser scan

This paper surveys state-of-the-art image features and descriptors for the task of 3D scan registration based on panoramic reflectance images.As modern terrestrial laser scanners digitize their environment in a spherical way,the sphere has to be projected to a two-dimensional image.To this end,we evaluate the equirectangular,the cylindrical,the Mercator,the rectilinear,the Pannini,the stereographic,and the z-axis projection.We show that the Mercator and the Pannini projection outperform the other projection methods.

The applications of robust estimation method BaySAC in indoor point cloud processing

Based on Bayesian theory and RANSAC,this paper applies Bayesian Sampling Consensus(BaySAC)method using convergence evaluation of hypothesis models in indoor point cloud processing.We implement a conditional sampling method,BaySAC,to always select the minimum number of required data with the highest inlier probabilities.Because the primitive parameters calculated by the different inlier sets should be convergent,this paper presents a statistical testing algorithm for a candidate model parameter histogram to compute the prior probability of each data point.Moreover,the probability update is implemented using the simplified Bayes’formula.The performances of the BaySAC algorithm with the proposed strategies of the prior probability determination and the RANSAC framework are compared using real data-sets.The experimental results indicate that the more outliers contain the data points,the higher computational efficiency of our proposed algorithm gains compared with RANSAC.The results also indicate that the proposed statistical testing strategy can determine sound prior inlier probability free of the change of hypothesis models.

Iterative-Reweighting-Based Robust Iterative-Closest-Point Method

In point cloud registration applications,noise and poor initial conditions lead to many false matches.False matches significantly degrade registration accuracy and speed.A penalty function is adopted in many robust point-to-point registration methods to suppress the influence of false matches.However,after applying a penalty function,problems cannot be solved in their analytical forms based on the introduction of nonlinearity.Therefore,most existing methods adopt the descending method.In this paper,a novel iterative-reweighting-based method is proposed to overcome the limitations of existing methods.The proposed method iteratively solves the eigenvectors of a four-dimensional matrix,whereas the calculation of the descending method relies on solving an eight-dimensional matrix.Therefore,the proposed method can achieve increased computational efficiency.The proposed method was validated on simulated noise corruption data,and the results reveal that it obtains higher efficiency and precision than existing methods,particularly under very noisy conditions.Experimental results for the KITTI dataset demonstrate that the proposed method can be used in real-time localization processes with high accuracy and good efficiency.

Infrastructure-Based Vehicle Localization System for Indoor Parking Lots Using RGB-D Cameras

Accurate vehicle localization is a key technology for autonomous driving tasks in indoor parking lots,such as automated valet parking.Additionally,infrastructure-based cooperative driving systems have become a means to realizing intelligent driving.In this paper,we propose a novel and practical vehicle localization system using infrastructure-based RGB-D cameras for indoor parking lots.In the proposed system,we design a depth data preprocessing method with both simplicity and efficiency to reduce the computational burden resulting from a large amount of data.Meanwhile,the hardware synchronization for all cameras in the sensor network is not implemented owing to the disadvantage that it is extremely cumbersome and would significantly reduce the scalability of our system in mass deployments.Hence,to address the problem of data distortion accompanying vehicle motion,we propose a vehicle localization method by performing template point cloud registration in distributed depth data.Finally,a complete hardware system was built to verify the feasibility of our solution in a real-world environment.Experiments in an indoor parking lot demonstrated the effectiveness and accuracy of the proposed vehicle localization system,with a maximum root mean squared error of 5 cm at 15Hz compared with the ground truth.

3D reconstruction of human head based on consumer RGB-D sensors

Three-dimensional(3D)reconstruction of a human head with high precision has promising applications in scientific research,product design and other fields.However,it still faces resistance from two factors.One is inaccurate registration caused by symmetrical distribution of head feature points,and the other is economic burden due to highaccuracy sensors.Research on 3D reconstruction with portable consumer RGB-D sensors such as the Microsoft Kinect has been highlighted in recent years.Based on our multi-Kinect system,a precise and low-cost three-dimensional modeling method and its system implementation are introduced in this paper.A registration method for multisource point clouds is provided,which can reduce the fusion differences and reconstruct the head model accurately.In addition,a template-based texture generation algorithm is presented to generate a fine texture.The comparison and analysis of our experiments show that our method can reconstruct a head model in an acceptable time with less memory and better effect.

Study of rapid face modeling technology based on Kinect

This paper improves the algorithm of point cloud filtering and registration in 3D modeling,aiming for smaller sampling error and shorter processing time of point cloud data.Based on collaborative sampling among several Kinect devices,we analyze the deficiency of current filtering algorithm,and use a novel method of point cloud filtering.Meanwhile,we use Fast Point Feature Histogram(FPFH)algorithm for feature extraction and point cloud registration.Compared with the aligning process using Point Feature Histograms(PFH),it only takes 9min when the number of points is about 500,000,shortening the aligning time by 47.1%.To measure the accuracy of the registration,we propose an algorithm which calculates the average distance of the corresponding coincident parts of two point clouds,and we improve the accuracy to an average distance of 0.7mm.In the surface reconstruction section,we adopt Ball Pivoting algorithm for surface reconstruction,obtaining image with higher accuracy in a shorter time.