Point Cloud Classification Using Content-Based Transformer via Clustering in Feature Space

Recently, there have been some attempts of Transformer in 3D point cloud classification. In order to reduce computations, most existing methods focus on local spatial attention,but ignore their content and fail to establish relationships between distant but relevant points. To overcome the limitation of local spatial attention, we propose a point content-based Transformer architecture, called PointConT for short. It exploits the locality of points in the feature space(content-based), which clusters the sampled points with similar features into the same class and computes the self-attention within each class, thus enabling an effective trade-off between capturing long-range dependencies and computational complexity. We further introduce an inception feature aggregator for point cloud classification, which uses parallel structures to aggregate high-frequency and low-frequency information in each branch separately. Extensive experiments show that our PointConT model achieves a remarkable performance on point cloud shape classification. Especially, our method exhibits 90.3% Top-1 accuracy on the hardest setting of ScanObjectN N. Source code of this paper is available at https://github.com/yahuiliu99/PointC onT.

Three-dimensional(3D)parametric measurements of individual gravels in the Gobi region using point cloud technique

Gobi spans a large area of China,surpassing the combined expanse of mobile dunes and semi-fixed dunes.Its presence significantly influences the movement of sand and dust.However,the complex origins and diverse materials constituting the Gobi result in notable differences in saltation processes across various Gobi surfaces.It is challenging to describe these processes according to a uniform morphology.Therefore,it becomes imperative to articulate surface characteristics through parameters such as the three-dimensional(3D)size and shape of gravel.Collecting morphology information for Gobi gravels is essential for studying its genesis and sand saltation.To enhance the efficiency and information yield of gravel parameter measurements,this study conducted field experiments in the Gobi region across Dunhuang City,Guazhou County,and Yumen City(administrated by Jiuquan City),Gansu Province,China in March 2023.A research framework and methodology for measuring 3D parameters of gravel using point cloud were developed,alongside improved calculation formulas for 3D parameters including gravel grain size,volume,flatness,roundness,sphericity,and equivalent grain size.Leveraging multi-view geometry technology for 3D reconstruction allowed for establishing an optimal data acquisition scheme characterized by high point cloud reconstruction efficiency and clear quality.Additionally,the proposed methodology incorporated point cloud clustering,segmentation,and filtering techniques to isolate individual gravel point clouds.Advanced point cloud algorithms,including the Oriented Bounding Box(OBB),point cloud slicing method,and point cloud triangulation,were then deployed to calculate the 3D parameters of individual gravels.These systematic processes allow precise and detailed characterization of individual gravels.For gravel grain size and volume,the correlation coefficients between point cloud and manual measurements all exceeded 0.9000,confirming the feasibility of the proposed methodology for measuring 3D parameters of individual gravels.The proposed workflow yields accurate calculations of relevant parameters for Gobi gravels,providing essential data support for subsequent studies on Gobi environments.

A modified method of discontinuity trace mapping using three-dimensional point clouds of rock mass surfaces

This paper presents an automated method for discontinuity trace mapping using three-dimensional point clouds of rock mass surfaces.Specifically,the method consists of five steps:(1)detection of trace feature points by normal tensor voting theory,(2)co ntraction of trace feature points,(3)connection of trace feature points,(4)linearization of trace segments,and(5)connection of trace segments.A sensitivity analysis was then conducted to identify the optimal parameters of the proposed method.Three field cases,a natural rock mass outcrop and two excavated rock tunnel surfaces,were analyzed using the proposed method to evaluate its validity and efficiency.The results show that the proposed method is more efficient and accurate than the traditional trace mapping method,and the efficiency enhancement is more robust as the number of feature points increases.

Point Cloud Classification Network Based on Graph Convolution and Fusion Attention Mechanism

The classification of point cloud data is the key technology of point cloud data information acquisition and 3D reconstruction, which has a wide range of applications. However, the existing point cloud classification methods have some shortcomings when extracting point cloud features, such as insufficient extraction of local information and overlooking the information in other neighborhood features in the point cloud, and not focusing on the point cloud channel information and spatial information. To solve the above problems, a point cloud classification network based on graph convolution and fusion attention mechanism is proposed to achieve more accurate classification results. Firstly, the point cloud is regarded as a node on the graph, the k-nearest neighbor algorithm is used to compose the graph and the information between points is dynamically captured by stacking multiple graph convolution layers;then, with the assistance of 2D experience of attention mechanism, an attention mechanism which has the capability to integrate more attention to point cloud spatial and channel information is introduced to increase the feature information of point cloud, aggregate local useful features and suppress useless features. Through the classification experiments on ModelNet40 dataset, the experimental results show that compared with PointNet network without considering the local feature information of the point cloud, the average classification accuracy of the proposed model has a 4.4% improvement and the overall classification accuracy has a 4.4% improvement. Compared with other networks, the classification accuracy of the proposed model has also been improved.

基于PointCloudTransformer和优化集成学习的三维点云分类

针对三维点云的不规则性和无序性所导致的难于提取特征并进行分类的问题,提出了一种融合深度学习和集成学习的三维点云分类方法。首先,训练深度学习点云分类网络PointCloudTransformer,并使用主干网络提取点云特征,进而训练基分类器,获得基分类器集合;然后,针对集成学习算法设计基分类器选择模型,模型的优化目标为基分类器组合的差异性和平均总体精度。为了降低集成规模,本文基于增强后的白鲸优化算法提出了二元多目标白鲸优化算法,并使用该算法优化基分类器选择模型,获得集成剪枝方案集合;最后,采用多数投票法集成每个基分类器组合在测试集点云特征上的分类结果,获得最优基分类器组合,从而构建基于多目标优化剪枝的集成学习点云分类模型。在点云分类数据集上的实验结果表明,本文方法使用了更小的集成规模,获得了更高的集成精度,能够对多类别三维点云进行准确分类。

Directional Point Net:3D Environmental Classification for Wearable Robots

A subject who wears a suitable robotic device will be able to walk in complex environments with the aid of environmental recognition schemes that provide reliable prior information of the human motion intent.Researchers have utilized 1 D laser signals and 2 D depth images to classify environments,but those approaches can face the problems of self-occlusion.In comparison,3 D point cloud is more appropriate for depicting the environments.This paper proposes a directional PointNet to directly classify the 3 D point cloud.First,an inertial measurement unit(IMU)is used to offset the orientation of point cloud.Then the directional PointNet can accurately classify the daily commuted terrains,including level ground,climbing up stairways,and walking down stairs.A classification accuracy of 98%has been achieved in tests.Moreover,the directional PointNet is more efficient than the previously used PointNet because the T-net,which is utilized to estimate the transformation of the point cloud,is not used in the present approach,and the length of the global feature is optimized.The experimental results demonstrate that the directional PointNet can classify the environments in robust and efficient manner.

Integration system research and development for three-dimensional laser scanning information visualization in goaf

An integration processing system of three-dimensional laser scanning information visualization in goaf was developed. It is provided with multiple functions, such as laser scanning information management for goaf, cloud data de-noising optimization, construction, display and operation of three-dimensional model, model editing, profile generation, calculation of goaf volume and roof area, Boolean calculation among models and interaction with the third party soft ware. Concerning this system with a concise interface, plentiful data input/output interfaces, it is featured with high integration, simple and convenient operations of applications. According to practice, in addition to being well-adapted, this system is favorably reliable and stable.

A state-of-the-art review of automated extraction of rock mass discontinuity characteristics using three-dimensional surface models

In the last two decades,significant research has been conducted in the field of automated extraction of rock mass discontinuity characteristics from three-dimensional(3D)models.This provides several methodologies for acquiring discontinuity measurements from 3D models,such as point clouds generated using laser scanning or photogrammetry.However,even with numerous automated and semiautomated methods presented in the literature,there is not one single method that can automatically characterize discontinuities accurately in a minimum of time.In this paper,we critically review all the existing methods proposed in the literature for the extraction of discontinuity characteristics such as joint sets and orientations,persistence,joint spacing,roughness and block size using point clouds,digital elevation maps,or meshes.As a result of this review,we identify the strengths and drawbacks of each method used for extracting those characteristics.We found that the approaches based on voxels and region growing are superior in extracting joint planes from 3D point clouds.Normal tensor voting with trace growth algorithm is a robust method for measuring joint trace length from 3D meshes.Spacing is estimated by calculating the perpendicular distance between joint planes.Several independent roughness indices are presented to quantify roughness from 3D surface models,but there is a need to incorporate these indices into automated methodologies.There is a lack of efficient algorithms for direct computation of block size from 3D rock mass surface models.

基于知识蒸馏和定位引导的Pointpillars点云检测网络

激光雷达数据由于其几何特性,被广泛应用于三维目标检测任务中。由于点云数据的稀疏性和不规则性,难以实现特征提取的质量和推理速度间的平衡。本文提出一种基于体柱特征编码的三维目标检测算法,以Pointpillars网络为基础,设计Teacher-Student模型框架对回归框尺度进行蒸馏,增加蒸馏损失,优化训练网络模型,提升特征提取的质量。为进一步提高模型检测效果,设计定位引导分类项,增加分类预测和回归预测之间的相关性,提高物体识别准确率。本网络所做改进没有引入额外的网络嵌入。算法在KITTI数据集上的实验结果表明,相比于基准网络,在三维模式下的平均精度值从60.65%提升到了64.69%,鸟瞰图模式下的平均精度值从67.74%提升到70.24%。模型推理速度为45 FPS,在提升检测精度的同时满足了实时性要求。

PCT和PointMLP融合的三维点云分类算法研究

为了提高三维点云数据的分类准确率,对注意力机制的PCT与残差点MLP的PointMLP的融合算法进行研究。首先,加入几何仿射模块对输入点云进行正态分布以解决点云的密度不均匀和几何结构不确定性;其次,在编码模块首先使用注意力机制对上层特征进行全局表达,之后加入残差点MLP模块对注意力模块的输出特征进行深度特征表示以充分提取点云特征;最后是分类模块,完成下游点云分类任务。实验结果表明,改进后的算法在ModelNet40数据集上的分类准确率达到95.6%,相比PCT模型准确率提高了2.4%,且对点云数量的变化具有鲁棒性。

Nearest Neighbor Sampling of Point Sets Using Rays

We propose a new framework for the sampling,compression,and analysis of distributions of point sets and other geometric objects embedded in Euclidean spaces.Our approach involves constructing a tensor called the RaySense sketch,which captures nearest neighbors from the underlying geometry of points along a set of rays.We explore various operations that can be performed on the RaySense sketch,leading to different properties and potential applications.Statistical information about the data set can be extracted from the sketch,independent of the ray set.Line integrals on point sets can be efficiently computed using the sketch.We also present several examples illustrating applications of the proposed strategy in practical scenarios.

An unmanned ground vehicle phenotyping-based method to generate three-dimensional multispectral point clouds for deciphering spatial heterogeneity in plant traits

Fusing three-dimensional(3D)and multispectral(MS)imaging data holds promise for high-throughput and comprehensive plant phenotyping to decipher genome-to-phenome knowledge.Acquiring high-quality 3D MS point clouds(3DMPCs)of plants remains challenging because of poor 3D data quality and limited radiometric calibration methods for plants with a complex canopy structure.Here,we present a novel 3D spatial–spectral data fusion approach to collect high-quality 3DMPCs of plants by integrating the next-best-view planning for adaptive data acquisition and neural reference field(NeREF)for radiometric calibration.This approach was used to acquire 3DMPCs of perilla,tomato,and rapeseed plants with diverse plant architecture and leaf morphological features evaluated by the accuracy of chlorophyll content and equivalent water thickness(EWT)estimation.The results showed that the completeness of plant point clouds collected by this approach was improved by an average of 23.6%compared with the fixed viewpoints alone.The NeREF-based radiometric calibration with the hemispherical reference outperformed the conventional calibration method by reducing the root mean square error(RMSE)of 58.93%for extracted reflectance spectra.The RMSE for chlorophyll content and EWT predictions decreased by 21.25%and 14.13%using partial least squares regression with the generated 3DMPCs.Collectively,our study provides an effective and efficient way to collect high-quality 3DMPCs of plants under natural light conditions,which improves the accuracy and comprehensiveness of phenotyping plant morphological and physiological traits,and thus will facilitate plant biology and genetic studies as well as crop breeding.

Development of vehicle-recognition method on water surfaces using LiDAR data:SPD^(2)(spherically stratified point projection with diameter and distance)

Swarm robot systems are an important application of autonomous unmanned surface vehicles on water surfaces.For monitoring natural environments and conducting security activities within a certain range using a surface vehicle,the swarm robot system is more efficient than the operation of a single object as the former can reduce cost and save time.It is necessary to detect adjacent surface obstacles robustly to operate a cluster of unmanned surface vehicles.For this purpose,a LiDAR(light detection and ranging)sensor is used as it can simultaneously obtain 3D information for all directions,relatively robustly and accurately,irrespective of the surrounding environmental conditions.Although the GPS(global-positioning-system)error range exists,obtaining measurements of the surface-vessel position can still ensure stability during platoon maneuvering.In this study,a three-layer convolutional neural network is applied to classify types of surface vehicles.The aim of this approach is to redefine the sparse 3D point cloud data as 2D image data with a connotative meaning and subsequently utilize this transformed data for object classification purposes.Hence,we have proposed a descriptor that converts the 3D point cloud data into 2D image data.To use this descriptor effectively,it is necessary to perform a clustering operation that separates the point clouds for each object.We developed voxel-based clustering for the point cloud clustering.Furthermore,using the descriptor,3D point cloud data can be converted into a 2D feature image,and the converted 2D image is provided as an input value to the network.We intend to verify the validity of the proposed 3D point cloud feature descriptor by using experimental data in the simulator.Furthermore,we explore the feasibility of real-time object classification within this framework.

基于PointNet++的焊装夹具零件识别

焊装夹具是汽车白车身焊接生产线中重要的组成部分,有效的管理和归纳焊装夹具零件设计数模能够显著提高设计效率。将原始设计数模离散为点云,利用点云数据和PointNet++深度学习网络探讨了一种焊装夹具零件智能分类方法,并对比各模型的分类精度,选取运行效率和精度最高的单尺度分组(SSG)模型完成焊装夹具零件的分类。训练结果表明,该方法在验证集上的准确率为97.5%,型块、连接块、定位销、销座、支座的验证集类内准确率分别为92.5%、97.5%、100%、97.5%和100%。这些结果表明该方法具有较高的识别精度,能够满足焊装夹具零件分类的精度要求。

轻量级实时点云分类网络LightPointNet

点云数据的无序性、稀疏性和有限性等特点给基于深度学习的点云模型分类带来了较大的困难.现有的面向点云的深度学习网络存在模型架构复杂、训练参数较多的问题,难以适用于实时点云识别任务,为此提出一种轻量级实时点云网络——LightPointNet.首先,基于点云模型的特点及轻量级点云分类网络的设计原则,提出面向点云模型分类的深度学习网络原型;然后,通过控制变量法完成网络参数设置的优化,形成最终的点云网络LightPointNet.该网络结构紧凑,仅包含3层卷积, 1层池化和1层全连接,且其参数个数不到0.07M.实验结果表明,在ModelNet40上,相比PointNet,VoxNet和LightNet,LightPointNet分类精度分别提高了0.29%,6.49%和2.59%,参数量减少了98.0%,92.4%和76.6%;在MINST和SHREC15上,该网络拥有良好的普适性;这些结果充分证明了LightPointNet分类性能良好且计算效率高,具有轻量级、实时性优点,可以部署在嵌入式设备中,在物联网和点云实时处理等方面具有广阔的应用前景.

基于SA-PointNetVLAD的点云分类网络

三维点云数据包含着丰富的形状和比例信息,如何有效准确地对点云数据进行分类已经成为了目前计算机视觉领域的研究热点。由于点云在非欧氏空间中的不规则稀疏结构,并且现有的基于深度学习的三维点云分类模型中缺乏对各个点的局部信息和全局信息的有效利用,从而导致较低的分类精度。为了解决上述问题,提出了一种基于SA-PointNetVLAD的点云分类模型框架。该网络采用逐点特征提取和汇集操作来解决无序的点云问题,通过自注意力机制来计算每个点与其他所有点之间的关联,充分挖掘点云的局部区域细粒度特征以及全局信息,采用KNN邻近算法感知点云的局部形状结构,通过VLAD层将每个点的低层几何描述符与视觉单词相关联来间接描述高层特征信息。此外,该网络还引入一个有效的关键点描述符帮助识别整体几何图形。经过实验得出SA-PointNetVLAD模型在公开的ModelNet40数据集上仅使用512个点就可以达到90.9%的精度,显著高于相同条件下的其他方法。

DST-Pointnet++:基于Pointnet++改进的点云分类网络

点云提供了精确的空间位置信息而被广泛应用于环境感知领域。近年来,越来越多的工作尝试直接以点云作为输入进行特征提取,Pointnet[10]和Pointnet++[11]是这个方向的开创者,但Pointnet++没有考虑点云非均匀采样的问题。研究提出了DST-Pointnet++对其进行改进,通过核密度估计和非线性变换从点云中提取出逆密度因子,将其与原点云特征进行加权,得到了具有密度信息的点云特征,提高了边缘点对局部特征的贡献,改善了因点云分布不均造成的问题。经过在公开数据集上测试对比,结果表明DST-Pointnet++具有更好的准确率和鲁棒性。

PointMLP-FD:基于多级自适应下采样的点云分类模型

针对受硬件条件、物体遮挡和背景杂波等客观因素的影响,传感器采集的目标点云具有较强的稀疏性和密度不均匀性,导致分类模型对点云特征的学习效率低、分类泛化能力差的问题,提出了一种基于多级自适应下采样的点云分类模型PointMLP-FD。该模型设计了多个MLP模块作为网络分支,以点云的浅层特征为输入得到每个点云类别维度上的特征表达,之后再根据特征表达进行排序,选择具有更强语义特征的点构成下采样点集。通过过滤背景和与目标相关性低的信息来自适应保留反应目标本质特征的信息。最后分别计算分支网络的损失,与骨干网络并行训练来优化点云特征,减少模型参数。该方法在ScanObjectNN数据集上进行测试,结果表明相较于PointMLP-elite分类精度更高,mAcc提升1%,OA提升0.8%,以更少的参数量接近SOTA模型的性能。

PointNet的点云数据集的破损测试与深度解读

目前人们对二维图像的研究已经取得了非常好的结果,然而随着深度学习的发展,研究正在逐步由二维向三维数据发展,并且应用领域越来越广泛,比如自动驾驶领域的三维场景建模,VR虚拟环境等。对三维数据的研究也逐渐实现了由有序输入到无序输入的过度并且取得了很高的成绩。Point Net则是第一个突破点云数据无序性输入的深度神经网络,值得人们深入的研究和借鉴。但是目前对破损和遮挡的点云数据问题还有待研究。本文着重对Point Net进行了深入研究并对点云数据进行了攻击和测试。测试发现当点云数据dropout约75%以后物体识别准确率显著下降,overlap12. 5%以后准确率也下降了近4个点,值得后续深入的研究和攻克。

Classification of rice seed variety using point cloud data combined with deep learning

Rice variety selection and quality inspection are key links in rice planting.Compared with two-dimensional images,three-dimensional information on rice seeds shows the appearance characteristics of rice seeds more comprehensively and accurately.This study proposed a rice variety classification method using three-dimensional point cloud data of the surface of rice seeds combined with a deep learning network to achieve the rapid and accurate identification of rice varieties.First,a point cloud collection platform was set up with a Raytrix light field camera as the core to collect three-dimensional point cloud data on the surface of rice seeds;then,the collected point cloud was filled,filtered and smoothed;after that,the point cloud segmentation is based on the RANSAC algorithm,and the point cloud downsampling is based on a combination of random sampling algorithm and voxel grid filtering algorithm.Finally,the processed point cloud was input to the improved PointNet network for feature extraction and species classification.The improved PointNet network added a cross-level feature connection structure,made full use of features at different levels,and better extracted the surface structure features of rice seeds.After testing,the improved PointNet model had an average classification accuracy of 89.4%for eight varieties of rice,which was 1.2%higher than that of the PointNet model.The method proposed in this study combined deep learning and point cloud data to achieve the efficient classification of rice varieties.