Point Cloud Classification Using Content-Based Transformer via Clustering in Feature Space

Recently, there have been some attempts of Transformer in 3D point cloud classification. In order to reduce computations, most existing methods focus on local spatial attention,but ignore their content and fail to establish relationships between distant but relevant points. To overcome the limitation of local spatial attention, we propose a point content-based Transformer architecture, called PointConT for short. It exploits the locality of points in the feature space(content-based), which clusters the sampled points with similar features into the same class and computes the self-attention within each class, thus enabling an effective trade-off between capturing long-range dependencies and computational complexity. We further introduce an inception feature aggregator for point cloud classification, which uses parallel structures to aggregate high-frequency and low-frequency information in each branch separately. Extensive experiments show that our PointConT model achieves a remarkable performance on point cloud shape classification. Especially, our method exhibits 90.3% Top-1 accuracy on the hardest setting of ScanObjectN N. Source code of this paper is available at https://github.com/yahuiliu99/PointC onT.

Automatic Extraction Method of 3D Feature Guidelines for Complex Cultural Relic Surfaces Based on Point Cloud

Cultural relics line graphic serves as a crucial form of traditional artifact information documentation,which is a simple and intuitive product with low cost of displaying compared with 3D models.Dimensionality reduction is undoubtedly necessary for line drawings.However,most existing methods for artifact drawing rely on the principles of orthographic projection that always cannot avoid angle occlusion and data overlapping while the surface of cultural relics is complex.Therefore,conformal mapping was introduced as a dimensionality reduction way to compensate for the limitation of orthographic projection.Based on the given criteria for assessing surface complexity,this paper proposed a three-dimensional feature guideline extraction method for complex cultural relic surfaces.A 2D and 3D combined factor that measured the importance of points on describing surface features,vertex weight,was designed.Then the selection threshold for feature guideline extraction was determined based on the differences between vertex weight and shape index distributions.The feasibility and stability were verified through experiments conducted on real cultural relic surface data.Results demonstrated the ability of the method to address the challenges associated with the automatic generation of line drawings for complex surfaces.The extraction method and the obtained results will be useful for line graphic drawing,displaying and propaganda of cultural relics.

A LiDAR Point Clouds Dataset of Ships in a Maritime Environment

For the first time, this article introduces a LiDAR Point Clouds Dataset of Ships composed of both collected and simulated data to address the scarcity of LiDAR data in maritime applications. The collected data are acquired using specialized maritime LiDAR sensors in both inland waterways and wide-open ocean environments. The simulated data is generated by placing a ship in the LiDAR coordinate system and scanning it with a redeveloped Blensor that emulates the operation of a LiDAR sensor equipped with various laser beams. Furthermore,we also render point clouds for foggy and rainy weather conditions. To describe a realistic shipping environment, a dynamic tail wave is modeled by iterating the wave elevation of each point in a time series. Finally, networks serving small objects are migrated to ship applications by feeding our dataset. The positive effect of simulated data is described in object detection experiments, and the negative impact of tail waves as noise is verified in single-object tracking experiments. The Dataset is available at https://github.com/zqy411470859/ship_dataset.

Estimating wood quality attributes from dense airborne LiDAR point clouds

Mapping individual tree quality parameters from high-density LiDAR point clouds is an important step towards improved forest inventories.We present a novel machine learning-based workflow that uses individual tree point clouds from drone laser scanning to predict wood quality indicators in standing trees.Unlike object reconstruction methods,our approach is based on simple metrics computed on vertical slices that summarize information on point distances,angles,and geometric attributes of the space between and around the points.Our models use these slice metrics as predictors and achieve high accuracy for predicting the diameter of the largest branch per log (DLBs) and stem diameter at different heights (DS) from survey-grade drone laser scans.We show that our models are also robust and accurate when tested on suboptimal versions of the data generated by reductions in the number of points or emulations of suboptimal single-tree segmentation scenarios.Our approach provides a simple,clear,and scalable solution that can be adapted to different situations both for research and more operational mapping.

Part-Whole Relational Few-Shot 3D Point Cloud Semantic Segmentation

This paper focuses on the task of few-shot 3D point cloud semantic segmentation.Despite some progress,this task still encounters many issues due to the insufficient samples given,e.g.,incomplete object segmentation and inaccurate semantic discrimination.To tackle these issues,we first leverage part-whole relationships into the task of 3D point cloud semantic segmentation to capture semantic integrity,which is empowered by the dynamic capsule routing with the module of 3D Capsule Networks(CapsNets)in the embedding network.Concretely,the dynamic routing amalgamates geometric information of the 3D point cloud data to construct higher-level feature representations,which capture the relationships between object parts and their wholes.Secondly,we designed a multi-prototype enhancement module to enhance the prototype discriminability.Specifically,the single-prototype enhancement mechanism is expanded to the multi-prototype enhancement version for capturing rich semantics.Besides,the shot-correlation within the category is calculated via the interaction of different samples to enhance the intra-category similarity.Ablation studies prove that the involved part-whole relations and proposed multi-prototype enhancement module help to achieve complete object segmentation and improve semantic discrimination.Moreover,under the integration of these two modules,quantitative and qualitative experiments on two public benchmarks,including S3DIS and ScanNet,indicate the superior performance of the proposed framework on the task of 3D point cloud semantic segmentation,compared to some state-of-the-art methods.

SGT-Net: A Transformer-Based Stratified Graph Convolutional Network for 3D Point Cloud Semantic Segmentation

In recent years,semantic segmentation on 3D point cloud data has attracted much attention.Unlike 2D images where pixels distribute regularly in the image domain,3D point clouds in non-Euclidean space are irregular and inherently sparse.Therefore,it is very difficult to extract long-range contexts and effectively aggregate local features for semantic segmentation in 3D point cloud space.Most current methods either focus on local feature aggregation or long-range context dependency,but fail to directly establish a global-local feature extractor to complete the point cloud semantic segmentation tasks.In this paper,we propose a Transformer-based stratified graph convolutional network(SGT-Net),which enlarges the effective receptive field and builds direct long-range dependency.Specifically,we first propose a novel dense-sparse sampling strategy that provides dense local vertices and sparse long-distance vertices for subsequent graph convolutional network(GCN).Secondly,we propose a multi-key self-attention mechanism based on the Transformer to further weight augmentation for crucial neighboring relationships and enlarge the effective receptive field.In addition,to further improve the efficiency of the network,we propose a similarity measurement module to determine whether the neighborhood near the center point is effective.We demonstrate the validity and superiority of our method on the S3DIS and ShapeNet datasets.Through ablation experiments and segmentation visualization,we verify that the SGT model can improve the performance of the point cloud semantic segmentation.

A Random Fusion of Mix 3D and Polar Mix to Improve Semantic Segmentation Performance in 3D Lidar Point Cloud

This paper focuses on the effective utilization of data augmentation techniques for 3Dlidar point clouds to enhance the performance of neural network models.These point clouds,which represent spatial information through a collection of 3D coordinates,have found wide-ranging applications.Data augmentation has emerged as a potent solution to the challenges posed by limited labeled data and the need to enhance model generalization capabilities.Much of the existing research is devoted to crafting novel data augmentation methods specifically for 3D lidar point clouds.However,there has been a lack of focus on making the most of the numerous existing augmentation techniques.Addressing this deficiency,this research investigates the possibility of combining two fundamental data augmentation strategies.The paper introduces PolarMix andMix3D,two commonly employed augmentation techniques,and presents a new approach,named RandomFusion.Instead of using a fixed or predetermined combination of augmentation methods,RandomFusion randomly chooses one method from a pool of options for each instance or sample.This innovative data augmentation technique randomly augments each point in the point cloud with either PolarMix or Mix3D.The crux of this strategy is the random choice between PolarMix and Mix3Dfor the augmentation of each point within the point cloud data set.The results of the experiments conducted validate the efficacy of the RandomFusion strategy in enhancing the performance of neural network models for 3D lidar point cloud semantic segmentation tasks.This is achieved without compromising computational efficiency.By examining the potential of merging different augmentation techniques,the research contributes significantly to a more comprehensive understanding of how to utilize existing augmentation methods for 3D lidar point clouds.RandomFusion data augmentation technique offers a simple yet effective method to leverage the diversity of augmentation techniques and boost the robustness of models.The insights gained from this research can pave the way for future work aimed at developing more advanced and efficient data augmentation strategies for 3D lidar point cloud analysis.

Three-dimensional(3D)parametric measurements of individual gravels in the Gobi region using point cloud technique

Gobi spans a large area of China,surpassing the combined expanse of mobile dunes and semi-fixed dunes.Its presence significantly influences the movement of sand and dust.However,the complex origins and diverse materials constituting the Gobi result in notable differences in saltation processes across various Gobi surfaces.It is challenging to describe these processes according to a uniform morphology.Therefore,it becomes imperative to articulate surface characteristics through parameters such as the three-dimensional(3D)size and shape of gravel.Collecting morphology information for Gobi gravels is essential for studying its genesis and sand saltation.To enhance the efficiency and information yield of gravel parameter measurements,this study conducted field experiments in the Gobi region across Dunhuang City,Guazhou County,and Yumen City(administrated by Jiuquan City),Gansu Province,China in March 2023.A research framework and methodology for measuring 3D parameters of gravel using point cloud were developed,alongside improved calculation formulas for 3D parameters including gravel grain size,volume,flatness,roundness,sphericity,and equivalent grain size.Leveraging multi-view geometry technology for 3D reconstruction allowed for establishing an optimal data acquisition scheme characterized by high point cloud reconstruction efficiency and clear quality.Additionally,the proposed methodology incorporated point cloud clustering,segmentation,and filtering techniques to isolate individual gravel point clouds.Advanced point cloud algorithms,including the Oriented Bounding Box(OBB),point cloud slicing method,and point cloud triangulation,were then deployed to calculate the 3D parameters of individual gravels.These systematic processes allow precise and detailed characterization of individual gravels.For gravel grain size and volume,the correlation coefficients between point cloud and manual measurements all exceeded 0.9000,confirming the feasibility of the proposed methodology for measuring 3D parameters of individual gravels.The proposed workflow yields accurate calculations of relevant parameters for Gobi gravels,providing essential data support for subsequent studies on Gobi environments.

Optimization of the Use of Spherical Targets for Point Cloud Registration Using Monte Carlo Simulation

Registrations based on the manual placement of spherical targets are still being employed by many professionals in the industry.However,the placement of those targets usually relies solely on personal experience without scientific evidence supported by numerical analysis.This paper presents a comprehensive investigation,based on Monte Carlo simulation,into determining the optimal number and positions for efficient target placement in typical scenes consisting of a pair of facades.It demonstrates new check-up statistical rules and geometrical constraints that can effectively extract and analyze massive simulations of unregistered point clouds and their corresponding registrations.More than 6×10^(7) sets of the registrations were simulated,whereas more than IOO registrations with real data were used to verify the results of simulation.The results indicated that using five spherical targets is the best choice for the registration of a large typical registration site consisting of two vertical facades and a ground,when there is only a box set of spherical targets available.As a result,the users can avoid placing extra targets to achieve insignificant improvements in registration accuracy.The results also suggest that the higher registration accuracy can be obtained when the ratio between the facade-to-target distance and target-to-scanner distance is approximately 3:2.Therefore,the targets should be placed closer to the scanner rather than in the middle between the facades and the scanner,contradicting to the traditional thought. Besides,the results reveal that the accuracy can be increased by setting the largest projected triangular area of the targets to be large.

Research on BIM Model Reshaping Method Based on 3D Point Cloud Recognition

In view of the limitations of traditional measurement methods in the field of building information,such as complex operation,low timeliness and poor accuracy,a new way of combining three-dimensional scanning technology and BIM(Building Information Modeling)model was discussed.Focused on the efficient acquisition of building geometric information using the fast-developing 3D point cloud technology,an improved deep learning-based 3D point cloud recognition method was proposed.The method optimised the network structure based on RandLA-Net to adapt to the large-scale point cloud processing requirements,while the semantic and instance features of the point cloud were integrated to significantly improve the recognition accuracy and provide a precise basis for BIM model remodeling.In addition,a visual BIM model generation system was developed,which systematically transformed the point cloud recognition results into BIM component parameters,automatically constructed BIM models,and promoted the open sharing and secondary development of models.The research results not only effectively promote the automation process of converting 3D point cloud data to refined BIM models,but also provide important technical support for promoting building informatisation and accelerating the construction of smart cities,showing a wide range of application potential and practical value.

S^(2)ANet:Combining local spectral and spatial point grouping for point cloud processing

Background Despite the recent progress in 3D point cloud processing using deep convolutional neural networks,the inability to extract local features remains a challenging problem.In addition,existing methods consider only the spatial domain in the feature extraction process.Methods In this paper,we propose a spectral and spatial aggregation convolutional network(S^(2)ANet),which combines spectral and spatial features for point cloud processing.First,we calculate the local frequency of the point cloud in the spectral domain.Then,we use the local frequency to group points and provide a spectral aggregation convolution module to extract the features of the points grouped by the local frequency.We simultaneously extract the local features in the spatial domain to supplement the final features.Results S^(2)ANet was applied in several point cloud analysis tasks;it achieved stateof-the-art classification accuracies of 93.8%,88.0%,and 83.1%on the ModelNet40,ShapeNetCore,and ScanObjectNN datasets,respectively.For indoor scene segmentation,training and testing were performed on the S3DIS dataset,and the mean intersection over union was 62.4%.Conclusions The proposed S^(2)ANet can effectively capture the local geometric information of point clouds,thereby improving accuracy on various tasks.

Intelligent extraction of road cracks based on vehicle laser point cloud and panoramic sequence images

In light of the limited efficacy of conventional methods for identifying pavement cracks and the absence of comprehensive depth and location data in two-dimensional photographs,this study presents an intelligent strategy for extracting road cracks.This methodology involves the integration of laser point cloud data obtained from a vehicle-mounted system and a panoramic sequence of images.The study employs a vehicle-mounted LiDAR measurement system to acquire laser point cloud and panoramic sequence image data simultaneously.A convolutional neural network is utilized to extract cracks from the panoramic sequence image.The extracted sequence image is then aligned with the laser point cloud,enabling the assignment of RGB information to the vehicle-mounted three dimensional(3D)point cloud and location information to the two dimensional(2D)panoramic image.Additionally,a threshold value is set based on the crack elevation change to extract the aligned roadway point cloud.The three-dimensional data pertaining to the cracks can be acquired.The experimental findings demonstrate that the use of convolutional neural networks has yielded noteworthy outcomes in the extraction of road cracks.The utilization of point cloud and image alignment techniques enables the extraction of precise location data pertaining to road cracks.This approach exhibits superior accuracy when compared to conventional methods.Moreover,it facilitates rapid and accurate identification and localization of road cracks,thereby playing a crucial role in ensuring road maintenance and traffic safety.Consequently,this technique finds extensive application in the domains of intelligent transportation and urbanization development.The technology exhibits significant promise for use in the domains of intelligent transportation and city development.

Building Facade Point Clouds Segmentation Based on Optimal Dual-Scale Feature Descriptors

To address the current issues of inaccurate segmentation and the limited applicability of segmentation methods for building facades in point clouds, we propose a facade segmentation algorithm based on optimal dual-scale feature descriptors. First, we select the optimal dual-scale descriptors from a range of feature descriptors. Next, we segment the facade according to the threshold value of the chosen optimal dual-scale descriptors. Finally, we use RANSAC (Random Sample Consensus) to fit the segmented surface and optimize the fitting result. Experimental results show that, compared to commonly used facade segmentation algorithms, the proposed method yields more accurate segmentation results, providing a robust data foundation for subsequent 3D model reconstruction of buildings.

Three Dimensional Laser Point Cloud Slicing Method for Calculating Irregular Volume

Volume parameter is the basic content of a spatial body object morphology analysis.However,the challenge lies in the volume calculation of irregular objects.The point cloud slicing method proposed in this study effectively works in calculating the volume of the point cloud of the spatial object obtained through three-dimensional laser scanning(3DLS).In this method,a uniformly spaced sequent slicing process is first conducted in a specific direction on the point cloud of the spatial object obtained through 3DLS.A series of discrete point cloud slices corresponding to the point cloud bodies are then obtained.Subsequently,the outline boundary polygon of the point cloud slicing is searched one by one in accordance with the slicing sequence and areas of the polygon.The point cloud slice is also calculated.Finally,the individual point cloud section volume is calculated through the slicing areas and the adjacent slicing gap.Thus,the total volume of the scanned spatial object can be calculated by summing up the individual volumes.According to the results and analysis of the calculated examples,the slice-based volume-calculating method for the point cloud of irregular objects obtained through 3DLS is correct,concise in process,reliable in results,efficient in calculation methods,and controllable on accuracy.This method comes as a good solution to the volume calculation of irregular objects.

Hawk‐eye‐inspired perception algorithm of stereo vision for obtaining orchard 3D point cloud navigation map

The binocular stereo vision is the lowest cost sensor for obtaining 3D information.Considering the weakness of long‐distance measurement and stability,the improvement of accuracy and stability of stereo vision is urgently required for application of precision agriculture.To address the challenges of stereo vision long‐distance measurement and stable perception without hardware upgrade,inspired by hawk eyes,higher resolution perception and the adaptive HDR(High Dynamic Range)were introduced in this paper.Simulating the function from physiological structure of‘deep fovea’and‘shallow fovea’of hawk eye,the higher resolution reconstruction method in this paper was aimed at ac-curacy improving.Inspired by adjustment of pupils,the adaptive HDR method was proposed for high dynamic range optimisation and stable perception.In various light conditions,compared with default stereo vision,the accuracy of proposed algorithm was improved by 28.0%evaluated by error ratio,and the stability was improved by 26.56%by disparity accuracy.For fixed distance measurement,the maximum improvement was 78.6%by standard deviation.Based on the hawk‐eye‐inspired perception algorithm,the point cloud of orchard was improved both in quality and quantity.The hawk‐eye‐inspired perception algorithm contributed great advance in binocular 3D point cloud recon-struction in orchard navigation map.

Point Cloud Based Semantic Segmentation Method for Unmanned Shuttle Bus

The complexity of application scenarios and the enormous volume of point cloud data make it difficult to quickly and effectively segment the scenario only based on the point cloud.In this paper,to address the semantic segmentation for safety driving of unmanned shuttle buses,an accurate and effective point cloud-based semantic segmentation method is proposed for specified scenarios(such as campus).Firstly,we analyze the characteristic of the shuttle bus scenarios and propose to use ROI selection to reduce the total points in computation,and then propose an improved semantic segmentation model based on Cylinder3D,which improves mean Intersection over Union(mIoU)by 1.3%over the original model on SemanticKITTI data;then,a semantic category division method is proposed for road scenario of shuttle bus and practical application requirements,and then we further simplify the model to improve the efficiency without losing the accuracy.Finally,the nuScenes dataset and the real gathered campus scene data are used to validate and analyze the proposed method.The experimental results on the nuScenes dataset and our data demonstrate that the proposed method performs better than other point cloud semantic segmentation methods in terms of application requirements for unmanned shuttle buses.Which has a higher accuracy(82.73%in mIoU)and a higher computational efficiency(inference speed of 90 ms).

Point cloud upsampling generative adversarial network based on residual multi-scale off-set attention

Background Owing to the limitations of the working principle of three-dimensional(3D) scanning equipment, the point clouds obtained by 3D scanning are usually sparse and unevenly distributed. Method In this paper, we propose a new generative adversarial network(GAN) that extends PU-GAN for upsampling of point clouds. Its core architecture aims to replace the traditional self-attention(SA) module with an implicit Laplacian offset attention(OA) module and to aggregate the adjacency features using a multiscale offset attention(MSOA)module, which adaptively adjusts the receptive field to learn various structural features. Finally, residual links are added to create our residual multiscale offset attention(RMSOA) module, which utilizes multiscale structural relationships to generate finer details. Result The results of several experiments show that our method outperforms existing methods and is highly robust.

Aggregate Point Cloud Geometric Features for Processing

As 3D acquisition technology develops and 3D sensors become increasingly affordable,large quantities of 3D point cloud data are emerging.How to effectively learn and extract the geometric features from these point clouds has become an urgent problem to be solved.The point cloud geometric information is hidden in disordered,unstructured points,making point cloud analysis a very challenging problem.To address this problem,we propose a novel network framework,called Tree Graph Network(TGNet),which can sample,group,and aggregate local geometric features.Specifically,we construct a Tree Graph by explicit rules,which consists of curves extending in all directions in point cloud feature space,and then aggregate the features of the graph through a cross-attention mechanism.In this way,we incorporate more point cloud geometric structure information into the representation of local geometric features,which makes our network perform better.Our model performs well on several basic point clouds processing tasks such as classification,segmentation,and normal estimation,demonstrating the effectiveness and superiority of our network.Furthermore,we provide ablation experiments and visualizations to better understand our network.

Building 3D CityGML models of mining industrial structures using integrated UAV and TLS point clouds

Mining industrial areas with anthropogenic engineering structures are one of the most distinctive features of the real world.3D models of the real world have been increasingly popular with numerous applications,such as digital twins and smart factory management.In this study,3D models of mining engineering structures were built based on the CityGML standard.For collecting spatial data,the two most popular geospatial technologies,namely UAV-SfM and TLS were employed.The accuracy of the UAV survey was at the centimeter level,and it satisfied the absolute positional accuracy requirement of creat-ing all levels of detail(LoD)according to the CityGML standard.Therefore,the UAV-SfM point cloud dataset was used to build LoD 2 models.In addition,the comparison between the UAV-SfM and TLS sub-clouds of facades and roofs indicates that the UAV-SfM and TLS point clouds of these objects are highly consistent,therefore,point clouds with a higher level of detail and accuracy provided by the integration of UAV-SfM and TLS were used to build LoD 3 models.The resulting 3D CityGML models include 39 buildings at LoD 2,and two mine shafts with hoistrooms,headframes,and sheave wheels at LoD3.

CFSA-Net:Efficient Large-Scale Point Cloud Semantic Segmentation Based on Cross-Fusion Self-Attention

Traditional models for semantic segmentation in point clouds primarily focus on smaller scales.However,in real-world applications,point clouds often exhibit larger scales,leading to heavy computational and memory requirements.The key to handling large-scale point clouds lies in leveraging random sampling,which offers higher computational efficiency and lower memory consumption compared to other sampling methods.Nevertheless,the use of random sampling can potentially result in the loss of crucial points during the encoding stage.To address these issues,this paper proposes cross-fusion self-attention network(CFSA-Net),a lightweight and efficient network architecture specifically designed for directly processing large-scale point clouds.At the core of this network is the incorporation of random sampling alongside a local feature extraction module based on cross-fusion self-attention(CFSA).This module effectively integrates long-range contextual dependencies between points by employing hierarchical position encoding(HPC).Furthermore,it enhances the interaction between each point’s coordinates and feature information through cross-fusion self-attention pooling,enabling the acquisition of more comprehensive geometric information.Finally,a residual optimization(RO)structure is introduced to extend the receptive field of individual points by stacking hierarchical position encoding and cross-fusion self-attention pooling,thereby reducing the impact of information loss caused by random sampling.Experimental results on the Stanford Large-Scale 3D Indoor Spaces(S3DIS),Semantic3D,and SemanticKITTI datasets demonstrate the superiority of this algorithm over advanced approaches such as RandLA-Net and KPConv.These findings underscore the excellent performance of CFSA-Net in large-scale 3D semantic segmentation.