SGT-Net: A Transformer-Based Stratified Graph Convolutional Network for 3D Point Cloud Semantic Segmentation

In recent years,semantic segmentation on 3D point cloud data has attracted much attention.Unlike 2D images where pixels distribute regularly in the image domain,3D point clouds in non-Euclidean space are irregular and inherently sparse.Therefore,it is very difficult to extract long-range contexts and effectively aggregate local features for semantic segmentation in 3D point cloud space.Most current methods either focus on local feature aggregation or long-range context dependency,but fail to directly establish a global-local feature extractor to complete the point cloud semantic segmentation tasks.In this paper,we propose a Transformer-based stratified graph convolutional network(SGT-Net),which enlarges the effective receptive field and builds direct long-range dependency.Specifically,we first propose a novel dense-sparse sampling strategy that provides dense local vertices and sparse long-distance vertices for subsequent graph convolutional network(GCN).Secondly,we propose a multi-key self-attention mechanism based on the Transformer to further weight augmentation for crucial neighboring relationships and enlarge the effective receptive field.In addition,to further improve the efficiency of the network,we propose a similarity measurement module to determine whether the neighborhood near the center point is effective.We demonstrate the validity and superiority of our method on the S3DIS and ShapeNet datasets.Through ablation experiments and segmentation visualization,we verify that the SGT model can improve the performance of the point cloud semantic segmentation.

A Random Fusion of Mix 3D and Polar Mix to Improve Semantic Segmentation Performance in 3D Lidar Point Cloud

This paper focuses on the effective utilization of data augmentation techniques for 3Dlidar point clouds to enhance the performance of neural network models.These point clouds,which represent spatial information through a collection of 3D coordinates,have found wide-ranging applications.Data augmentation has emerged as a potent solution to the challenges posed by limited labeled data and the need to enhance model generalization capabilities.Much of the existing research is devoted to crafting novel data augmentation methods specifically for 3D lidar point clouds.However,there has been a lack of focus on making the most of the numerous existing augmentation techniques.Addressing this deficiency,this research investigates the possibility of combining two fundamental data augmentation strategies.The paper introduces PolarMix andMix3D,two commonly employed augmentation techniques,and presents a new approach,named RandomFusion.Instead of using a fixed or predetermined combination of augmentation methods,RandomFusion randomly chooses one method from a pool of options for each instance or sample.This innovative data augmentation technique randomly augments each point in the point cloud with either PolarMix or Mix3D.The crux of this strategy is the random choice between PolarMix and Mix3Dfor the augmentation of each point within the point cloud data set.The results of the experiments conducted validate the efficacy of the RandomFusion strategy in enhancing the performance of neural network models for 3D lidar point cloud semantic segmentation tasks.This is achieved without compromising computational efficiency.By examining the potential of merging different augmentation techniques,the research contributes significantly to a more comprehensive understanding of how to utilize existing augmentation methods for 3D lidar point clouds.RandomFusion data augmentation technique offers a simple yet effective method to leverage the diversity of augmentation techniques and boost the robustness of models.The insights gained from this research can pave the way for future work aimed at developing more advanced and efficient data augmentation strategies for 3D lidar point cloud analysis.

Research on BIM Model Reshaping Method Based on 3D Point Cloud Recognition

In view of the limitations of traditional measurement methods in the field of building information,such as complex operation,low timeliness and poor accuracy,a new way of combining three-dimensional scanning technology and BIM(Building Information Modeling)model was discussed.Focused on the efficient acquisition of building geometric information using the fast-developing 3D point cloud technology,an improved deep learning-based 3D point cloud recognition method was proposed.The method optimised the network structure based on RandLA-Net to adapt to the large-scale point cloud processing requirements,while the semantic and instance features of the point cloud were integrated to significantly improve the recognition accuracy and provide a precise basis for BIM model remodeling.In addition,a visual BIM model generation system was developed,which systematically transformed the point cloud recognition results into BIM component parameters,automatically constructed BIM models,and promoted the open sharing and secondary development of models.The research results not only effectively promote the automation process of converting 3D point cloud data to refined BIM models,but also provide important technical support for promoting building informatisation and accelerating the construction of smart cities,showing a wide range of application potential and practical value.

改进的3D-BoNet算法应用于点云实例分割与三维重建

为了更好地利用点云数据重建室内三维模型,本文提出了一种基于3D-BoNet-IAM算法的室内场景三维重建方法。该方法通过改进3D-BoNet算法提高点云数据的实例分割精度。针对点云数据缺失问题,提出了基于平面基元合并优化的拟合平面方法,利用拟合得到的新平面重建建筑表面模型。在S3DIS和ScanNet V2数据集上验证3D-BoNet算法的改进效果。试验结果表明,本文提出的3D-BoNet-IAM算法比原始算法分割精度提高了3.3%;对比本文建模效果与其他建模效果发现,本文方法的建模效果更准确。本文方法能够提高室内点云数据的实例分割精度,同时得到高质量的室内三维模型。

基于无人机3D点云的高陡斜坡岩体结构特征

岩体结构控制着岩质斜坡的变形破坏模式,但传统的结构面信息调查方法存在危险性高、难度大的缺陷。为安全、高效地获取斜坡岩体结构信息,利用无人机摄影技术开展高陡岩质斜坡的测量,构建斜坡岩体的3D点云模型,识别结构面的产状、间距、迹长等岩体结构信息。对湖北秭归卡门子湾滑坡区调查验证,斜坡共发育6组优势结构面,平均间距为0.46~1.01 m,平均迹长为0.82~12.57 m;无人机3D点云获取的岩体结构信息精度满足要求,方法高效、可行;获得的岩体结构信息为斜坡岩体结构模型建立和稳定性评价等工作提供了可靠的数据基础。

基于多分支特征融合的车载激光雷达3D目标检测算法

该文基于多分支特征融合的3D目标检测算法将无序的点云划分为规则的体素,利用体素特征编码模块和卷积神经网络学习体素特征,再将稀疏的3D数据压缩为稠密的二维鸟瞰图,最后通过2D骨干网络的粗糙分支和精细分支对多尺度鸟瞰图特征进行深度融合。该文实现了对多尺度特征的语义信息、纹理信息和上下文信息的聚合,得到了更加精确的原始空间位置信息、物体分类、位置回归和朝向预测,在KITTI数据集上取得优异的平均精度,并在保持一定帧率的同时具有较强的稳健性。

基于FY-3D/MERSI2的内蒙古草地净初级生产力反演

净初级生产力(Net Primary Productivity,NPP)不仅是估算生态系统固碳释氧、衡量陆地碳循环的主要参数,也是评价生态系统健康状况的主要指标。针对目前国产卫星对草地净初级生产力遥感监测应用较少的情况,本文基于FY-3D/MERSI2资料构建一套内蒙古草地净初级生产力反演模型,结合光能利用率模型与生态过程模型,以遥感数据产品和中国气象局陆面数据同化系统(CMA Land Data Assimilation System,CLDAS)资料为驱动,通过较严格的云检测算法得到晴空条件下内蒙古草地NPP。研究中引入分辨率较高的格点化气象数据,提升了反演结果的精细化程度;同时还基于观测数据及MODIS产品构建了内蒙古草地生育期不同月份(5—8月)地上生物量及光合有效辐射吸收比率(Fraction Photosynthetic Active Radiation Absorption Ratio,FPAR)与归一化植被指数(Normalized Differ⁃ence Vegetation Index,NDVI)的多种关系模型,基于FY-3D数据直接估算叶面积指数(Leaf Area Index,LAI)及FPAR等过程参数。将反演的关键生态过程参数与MODIS对应产品对比,发现二者具有较好相关性和空间一致性。最后利用2021年6月18个生态气象观测站牧草观测资料与估算结果进行对比验证,二者具有较好的一致性,相关系数为0.86。本研究利用FY-3D/MERSI2反演的NPP能够完整呈现内蒙古地区植被生产力的普遍状态。

基于深度学习与点云数据的3D检测系统应用研究

为了解决高低差场景中平坦度高的2D视觉检测乏力的问题,基于深度学习和点云数据处理平台,融合3D点云格式图像和深度学习技术,建立微米精度、在线检测、成本可控的3D检测架构机制。采用相移和光栅投影结构光技术的硬件方案获取3D点云原始数据,基于强大的CPU和GPU处理芯片,对经过被测物体调制的光栅图案进行重新编码,并结合标定参数解算3D点云数据。对2D深度学习模型进行升级开发,可对点云数据进行标注、学习训练和检测,并将3D硬件、3D软件和3D算法进行整合。实验结果表明,所提系统有利于3D缺陷检测系统的落地,为智能3D检测设备奠定算法和软件基础。

一种多层多模态融合3D目标检测方法

在自动驾驶感知系统中视觉传感器与激光雷达是关键的信息来源,但在目前的3D目标检测任务中大部分纯点云的网络检测能力都优于图像和激光点云融合的网络,现有的研究将其原因总结为图像与雷达信息的视角错位以及异构特征难以匹配,单阶段融合算法难以充分融合二者的特征.为此,本文提出一种新的多层多模态融合的3D目标检测方法:首先,前融合阶段通过在2D检测框形成的锥视区内对点云进行局部顺序的色彩信息(Red Green Blue,RGB)涂抹编码;然后将编码后点云输入融合了自注意力机制上下文感知的通道扩充PointPillars检测网络;后融合阶段将2D候选框与3D候选框在非极大抑制之前编码为两组稀疏张量,利用相机激光雷达对象候选融合网络得出最终的3D目标检测结果.在KITTI数据集上进行的实验表明,本融合检测方法相较于纯点云网络的基线上有了显著的性能提升,平均mAP提高了6.24%.

3D点云数据处理方法研究进展

点云是理解三维场景的重要形式之一,3D点云在海洋平台逆向建模、海底地形测绘、深水浮式结构系泊系统损伤测量及海底管线可视化等方面都有着重要应用。基于此,文中梳理了点云数据处理方法,将其分为传统处理算法和基于深度学习方法两大类;传统处理算法从滤波、对象识别与分类和配准3方面进行了介绍总结;基于深度学习方法从点云、体素化和多视图3方面进行了介绍总结。对各种算法的优缺点进行了归纳对比,并展望了3D点云处理技术未来的发展趋势与方向。

基于深度学习的自动驾驶场景3D目标检测方法

针对传统PV-RCNN在点云上采样效率低下和采样精度存在偏差等问题,提出了一种基于PV-RCNN改进的3D目标检测方法。更改关键点采样策略,使得有限的关键点可以更加地聚集在proposal区域范围内,更多的编码有效前景点特征来用于后面的proposal refinement,有效产生更具有代表性的关键点。用局部特征聚合的VectorPool聚合模块取代体素集抽象和ROI网格池化模块中的集合抽象,更高效的针对稀疏和不规则点云数据进行编码。在KITTI数据集上对算法验证,结果表明:行人鸟瞰图检测,困难级别检测精度提升较为显著,达到了10.46%,整体帧率提升为33.74%,文中的方法拥有更好的检测性能。

基于轻量型移动测量系统的地外熔岩管3D探测与地表验证

熔岩管广泛存在于地球及地外天体,可为人类未来登陆地外天体提供天然庇护,地外熔岩管探测具有重要意义。然而,熔岩管探测存在许多挑战,已有的地球熔岩管探测方案设备不便携、自动化程度和工作效率低,难以用于地外熔岩管探测。针对上述问题,提出一种基于轻量型移动测量系统的熔岩管3D探测方法,利用穿戴式设备实现高效精细的地图构建和三维形态表达,并通过地球熔岩管探测进行验证。首先利用激光扫描高效获取熔岩管内点云,并基于迭代卡尔曼滤波算法生成熔岩管3D点云实时地图。然后结合地面滤波、管道壁提取、法向量估计等点云智能处理方法,实现熔岩管的三维智能化重建,并作形态学分析。选取海南省海口市的仙人洞及七十二洞作为地外熔岩管的模拟场景进行实验,结果表明,所提方法实现了熔岩管3D实时自主测图,生成的点云地图及3D模型相比于已有研究结果更精准、地貌形态细节信息更多,可更好地满足熔岩管的形态分析需求,有望为地外熔岩管探测提供基础。

基于点云稀疏空间特征聚合激励的单阶段3D目标检测模型

针对目前基于点云的3D目标检测中单阶段体素法存在感受野固定、特征尺度单一,导致模型对点云特征学习不够充分、模型检测效果存在瓶颈等问题,提出了一种可端对端训练的基于体素的单阶段3D目标检测模型.首先,利用多尺度稀疏空间特征聚合模块,聚合点云在不同稀疏空间尺度上的特征,使特征充分保留点云的空间信息;然后,对特征进行分层激励,通过多尺度感受野对特征进行分层学习,强化特征的表达能力,降低噪声信息对检测结果的影响;最后,将特征输入检测头进行候选框的分类和回归.在公开的自动驾驶数据集KITTI上与主流单阶段3D目标检测模型进行了对比实验,包含对3类目标共9个的难度等级目标的检测.所提模型在其中5个等级中的平均准确率有明显提升,尤其对点云稀疏的目标,表现出较好的检测效果.实验结果表明,所提模型能够充分提取点云空间信息并有效地学习点云多尺度特征.

使用中心预测-聚类的3D箱体实例分割方法

随着深度学习技术在工业领域的大量部署,应用于运输、装卸、包装、分拣等环节的自动化系统成为仓储物流行业的研究热点。针对机器人箱体拆垛场景提出一个点云中心预测-聚类网络(center prediction-clustering network,CPCN),对箱体垛进行实例分割,并计算每个箱体的上表面中心坐标。CPCN在传统的语义-实例联合分割结构的基础上,为实例分割分支设计了中心预测模块和中心强化模块。中心预测模块帮助定位实例中心以避免中心点分割错误,中心强化模块令属于同一实例的点在特征空间中向中心聚集,二者有效增强了实例特征的辨识能力。在实例特征处理部分设计的中心-实例聚类方法直接对实例特征进行距离度量来计算实例标签,大幅减少了计算时间。在箱体数据集上进行的实验表明,与现有方法相比CPCN在实例分割任务中的平均精确率最低提高了0.7个百分点,最高提高了17.2个百分点,预测实例中心的准确率达到94.4%,中心偏移量低至13.70 mm,且推理速度快于同类型的联合分割网络,对于箱体拆垛任务更有针对性,具有良好的应用价值。

基于PointCloudTransformer和优化集成学习的三维点云分类

针对三维点云的不规则性和无序性所导致的难于提取特征并进行分类的问题,提出了一种融合深度学习和集成学习的三维点云分类方法。首先,训练深度学习点云分类网络PointCloudTransformer,并使用主干网络提取点云特征,进而训练基分类器,获得基分类器集合;然后,针对集成学习算法设计基分类器选择模型,模型的优化目标为基分类器组合的差异性和平均总体精度。为了降低集成规模,本文基于增强后的白鲸优化算法提出了二元多目标白鲸优化算法,并使用该算法优化基分类器选择模型,获得集成剪枝方案集合;最后,采用多数投票法集成每个基分类器组合在测试集点云特征上的分类结果,获得最优基分类器组合,从而构建基于多目标优化剪枝的集成学习点云分类模型。在点云分类数据集上的实验结果表明,本文方法使用了更小的集成规模,获得了更高的集成精度,能够对多类别三维点云进行准确分类。

基于点云图卷积神经网络的3D目标检测

随着激光雷达传感器的快速发展,目标检测算法从传统的2D检测快速转向3D检测。然而,激光雷达产生的点云是不规则和非结构化的数据,传统的卷积神经网络无法对其进行处理。基于此提出了一种新颖的图卷积神经网络,能够更好地利用数据的几何关系和拓扑结构直接从点云中学习特征以进行3D目标检测。首先将原始激光雷达点云数据进行下采样,再进行固定半径邻域图的构建,随后设计了一个新型的图卷积神经网络对点云进行编码来预测图中每个顶点所属对象的类别和形状。为提升检测准确度,网络中加入了一种校准机制来减少特征在不同维度变化时引入的平移误差,此外还引入了注意力机制,以使用权重来进一步强化输出的顶点特征。在KITTI数据集上进行实验,实验结果表明,此方法能够有效对3D目标进行检测。对比其他多种检测算法,此方法在检测准确度上具有一定的优势。

多尺度特征结合注意力机制的室内3D点云目标检测

为了实现三维点云在室内和工业环境中的实际应用,文章改进了传统的目标检测转换器(Detection Transformer,DeTR)神经网络,并提出了一种基于分层抽象的多层点云特征提取方法;同时,设计了曲面特征提取模块对三维点云进行预处理,增强了点云的附加特征。在公开数据集ScanNet V2和工业室内数据集上对本文方法进行实验验证和评估,该方法在ScanNet V2上的mAP@0.5准确率超过最先进的模型(State-of-the-Art,SOTA)CAGroup3d,达到76.0%;在ScanNet V2上的mAP@0.25准确率超过最先进的模型CAGroup3d,达到62.2%,消融实验进一步验证了所述方法的准确性和高效性。

A LiDAR Point Clouds Dataset of Ships in a Maritime Environment

For the first time, this article introduces a LiDAR Point Clouds Dataset of Ships composed of both collected and simulated data to address the scarcity of LiDAR data in maritime applications. The collected data are acquired using specialized maritime LiDAR sensors in both inland waterways and wide-open ocean environments. The simulated data is generated by placing a ship in the LiDAR coordinate system and scanning it with a redeveloped Blensor that emulates the operation of a LiDAR sensor equipped with various laser beams. Furthermore,we also render point clouds for foggy and rainy weather conditions. To describe a realistic shipping environment, a dynamic tail wave is modeled by iterating the wave elevation of each point in a time series. Finally, networks serving small objects are migrated to ship applications by feeding our dataset. The positive effect of simulated data is described in object detection experiments, and the negative impact of tail waves as noise is verified in single-object tracking experiments. The Dataset is available at https://github.com/zqy411470859/ship_dataset.

3D Ice Shape Description Method Based on BLSOM Neural Network

When checking the ice shape calculation software,its accuracy is judged based on the proximity between the calculated ice shape and the typical test ice shape.Therefore,determining the typical test ice shape becomes the key task of the icing wind tunnel tests.In the icing wind tunnel test of the tail wing model of a large amphibious aircraft,in order to obtain accurate typical test ice shape,the Romer Absolute Scanner is used to obtain the 3D point cloud data of the ice shape on the tail wing model.Then,the batch-learning self-organizing map(BLSOM)neural network is used to obtain the 2D average ice shape along the model direction based on the 3D point cloud data of the ice shape,while its tolerance band is calculated using the probabilistic statistical method.The results show that the combination of 2D average ice shape and its tolerance band can represent the 3D characteristics of the test ice shape effectively,which can be used as the typical test ice shape for comparative analysis with the calculated ice shape.

基于伪文本查询生成及位置感知的弱监督3D视觉定位方法

3D点云视觉定位在自动驾驶、VR/AR等应用中发挥着重要作用。现有大部分点云视觉定位方法依赖对每个目标定位物体的精细人工描述,耗时耗力。为克服视觉语言任务对文本标注的依赖性,现有研究已提出伪文本生成和特征替换方法,在2D领域实现无需文本标注的视觉定位、图像编辑等。在对2D方法研究的基础上,提出了一种自动生成伪文本并实现位置感知的弱监督3D视觉定位方法。在公开数据集ScanRefer、Nr3D/Sr3D上的实验证明了所提方法的有效性和优越性能。