3D scene graph prediction from point clouds

Background In this study,we propose a novel 3D scene graph prediction approach for scene understanding from point clouds.Methods It can automatically organize the entities of a scene in a graph,where objects are nodes and their relationships are modeled as edges.More specifically,we employ the DGCNN to capture the features of objects and their relationships in the scene.A Graph Attention Network(GAT)is introduced to exploit latent features obtained from the initial estimation to further refine the object arrangement in the graph structure.A one loss function modified from cross entropy with a variable weight is proposed to solve the multi-category problem in the prediction of object and predicate.Results Experiments reveal that the proposed approach performs favorably against the state-of-the-art methods in terms of predicate classification and relationship prediction and achieves comparable performance on object classification prediction.Conclusions The 3D scene graph prediction approach can form an abstract description of the scene space from point clouds.

Analysis of Color Landscape Characteristics in“Beautiful Village”of China Based on 3D Real Scene Models

Color,as a significant element of village landscapes,serves various functions such as enhancing aesthetic appeal and attractiveness,conveying emotions and cultural values.To explore the three-dimensional spatial characteristics of color landscapes in beautiful villages,this study conducted a comparative experiment involving eight provinciallevel beautiful villages and eight ordinary villages in Jinzhai County.Landscape pattern indices were used to analyze the color landscape patterns on the facades of these villages,complemented by a quantitative analysis of color attributes using theMunsell color system.The results indicate that(1)Natural landscape colors in beautiful villages are primarily concentrated in the yellow-red to green-yellow interval,while those in ordinary villages are widely distributed in the red to blue-green interval.Artificial landscape colors in beautiful villages aremainly characterized by medium value,with chroma concentrated in the low chroma range.(2)The proportion of color areas for forests,grasslands,and building walls in beautiful villages is higher by 14.76%,2.17%,and 5.16%,respectively,compared to ordinary villages.However,the proportion of yellow exposed areas in ordinary villages ismore than twice that of beautiful villages.(3)The Landscape Shape Index for forests,grasslands,and buildings in beautiful villages is 5.23,8.01,and 8.19,respectively,indicating a higher irregularity in color patches.(4)Ordinary villages exhibit a higher Shannon’s diversity index,indicating amore complex distribution of colors,whereas beautiful villages demonstrate a higher number of connected dominant patches.This study can provide a scientific basis for village color planning and layout.

Perspectives on point cloud-based 3D scene modeling and XR presentation within the cloud-edge-client architecture

With the support of edge computing,the synergy and collaboration among central cloud,edge cloud,and terminal devices form an integrated computing ecosystem known as the cloud-edge-client architecture.This integration unlocks the value of data and computational power,presenting significant opportunities for large-scale 3D scene modeling and XR presentation.In this paper,we explore the perspectives and highlight new challenges in 3D scene modeling and XR presentation based on point cloud within the cloud-edge-client integrated architecture.We also propose a novel cloud-edge-client integrated technology framework and a demonstration of municipal governance application to address these challenges.

A Method for 3D Scene Description and Segmentation in an Object Record

in this poper a novel data-and rule-driven system for 3D scene description and segmentation inan unknown environment is presented.This system generatss hierachies of features that correspond tostructural elements such as boundaries and shape classes of individual object as well as relationshipsbetween objects.It is implemented as an added high-level component to an existing low-level binocularvision system[1]. Based on a pair of matched stereo images produced by that system,3D segmentation is firstperformed to group object boundary data into several edge-sets,each of which is believed to belong to aparticular object.Then gross features of each object are extracted and stored in an object recbrd.The finalstructural description of the scene is accomplished with information in the object record,a set of rules and arule implementor. The System is designed to handle partially occluded objects of different shapes and sizeson the 2D imager.Experimental results have shown its success in computing both object and structurallevel descriptions of common man-made objects.

Structure-aware fusion network for 3D scene understanding

In this paper,we propose a Structure-Aware Fusion Network(SAFNet)for 3D scene understanding.As 2D images present more detailed information while 3D point clouds convey more geometric information,fusing the two complementary data can improve the discriminative ability of the model.Fusion is a very challenging task since 2D and 3D data are essentially different and show different formats.The existing methods first extract 2D multi-view image features and then aggregate them into sparse 3D point clouds and achieve superior performance.However,the existing methods ignore the structural relations between pixels and point clouds and directly fuse the two modals of data without adaptation.To address this,we propose a structural deep metric learning method on pixels and points to explore the relations and further utilize them to adaptively map the images and point clouds into a common canonical space for prediction.Extensive experiments on the widely used ScanNetV2 and S3DIS datasets verify the performance of the proposed SAFNet.

A real 3D scene rendering optimization method based on region of interest and viewing frustum prediction in virtual reality

As an important technology of digital construction,real 3D models can improve the immersion and realism of virtual reality(VR)scenes.The large amount of data for real 3D scenes requires more effective rendering methods,but the current rendering optimization methods have some defects and cannot render real 3D scenes in virtual reality.In this study,the location of the viewing frustum is predicted by a Kalman filter,and eye-tracking equipment is used to recognize the region of interest(ROI)in the scene.Finally,the real 3D model of interest in the predicted frustum is rendered first.The experimental results show that the method of this study can predict the frustrum location approximately 200 ms in advance,the prediction accuracy is approximately 87%,the scene rendering efficiency is improved by 8.3%,and the motion sickness is reduced by approximately 54.5%.These studies help promote the use of real 3D models in virtual reality and ROI recognition methods.In future work,we will further improve the prediction accuracy of viewing frustums in virtual reality and the application of eye tracking in virtual geographic scenes.

Semantic Driven Design Reuse for 3D Scene Modeling

The increasing scale and complexity of 3D scene design work urge an efficient way to understand the design in multi-disciplinary team and exploit the experiences and underlying knowledge in previous works for reuse.However the previous researches lack of concerning on relationship maintaining and design reuse in knowledge level.We propose a novel semantic driven design reuse system,including a property computation algorithm that enables our system to compute the properties while modeling process to maintain the semantic consistency,and a vertex statics based algorithm that enables the system to recognize scene design pattern as universal semantic model for the same type of scenes.With the universal semantic model,the system conducts the modeling process of future design works by suggestions and constraints on operation.The proposed framework empowers the reuse of 3D scene design on both model level and knowledge level.

基于关键视图的文本驱动3D场景编辑方法

基于去噪扩散模型的零样本图像编辑方法取得了瞩目的成就,将之应用于3D场景编辑可实现零样本的文本驱动3D场景编辑。然而,其3D编辑效果容易受扩散模型的3D连续性与过度编辑等问题影响,产生错误的编辑结果。针对这些问题,提出了一种新的文本驱动3D编辑方法,该方法从数据端着手,提出了基于关键视图的数据迭代方法与基于像素点的异常数据掩码模块。关键视图数据可以引导一个3D区域的编辑以减少3D不一致数据的影响,而数据掩码模块则可以过滤掉2D输入数据中的异常点。使用该方法,可以实现生动的照片级文本驱动3D场景编辑效果。实验证明,相较于一些目前先进的文本驱动3D场景编辑方法,可以大大减少3D场景中错误的编辑,实现更加生动的、更具真实感的3D编辑效果。此外,使用该方法生成的编辑结果更具多样性、编辑效率也更高。

3D场景渲染技术——神经辐射场的研究

神经辐射场(NeRF)是一种面向三维隐式空间建模的深度学习模型,在表示和渲染三维场景领域具有重要价值。然而由于神经辐射场算法训练过程复杂、需要大量的计算资源和时间等,其可用性和实用性受到一定限制,如何针对神经辐射场的痛点问题进行优化是当前计算机视觉等领域研究的热点之一。此研究旨在对神经辐射场的优化和应用进行全面综述。首先,在深入解析神经辐射场基本原理的基础上,从渲染质量、计算复杂度、位姿等方面对现阶段神经辐射场的优化情况进行概述;其次,列举神经辐射场应用状况,为未来更高效和实用的算法优化设计提供参考;最后,总结神经辐射场的优势与局限性,并提出未来可能的发展方向,以期发挥神经辐射场在三维渲染、场景合成等方面的巨大潜力。

When a tree model meets texture baking:an approach for quality-preserving lightweight visualization in virtual 3D scene construction

Three-dimensional(3D)high-fidelity surface models play an important role in urban scene construction.However,the data quantity of such models is large and places a tremendous burden on rendering.Many applications must balance the visual quality of the models with the rendering efficiency.The study provides a practical texture baking processing pipeline for generating 3D models to reduce the model complexity and preserve the visually pleasing details.Concretely,we apply a mesh simplification to the original model and use texture baking to create three types of baked textures,namely,a diffuse map,normal map and displacement map.The simplified model with the baked textures has a pleasing visualization effect in a rendering engine.Furthermore,we discuss the influence of various factors in the process on the results,as well as the functional principles and characteristics of the baking textures.The proposed approach is very useful for real-time rendering with limited rendering hardware as no additional memory or computing capacity is required for properly preserving the relief details of the model.Each step in the pipeline is described in detail to facilitate the realization.

基于全局相关性的多尺度3D点云场景流估计

对于场景流估计来说,提取到丰富的全局相关性对于获取精确的特征匹配非常重要。本文提出了一种基于全局相关性的多尺度3D点云场景流估计网络(multi-scale 3D point cloud scene flow based on globalcorrelation,MGCSF),该网络引入了通道亲和性注意力(channel affinity attention,CAA)模块和逐点注意力模块(point-wise attention module,PAM),通过融合不同层次点云的特征信息以捕获全局性的运动趋势和变化,在一定程度上减少了点云特征信息丢失,从而可以更好地计算点云场景流。在关键数据集FlyingThings3D和KITTI上的实验性能均取得了一定的提升。与基线相比,在FlyingThings3D数据集上,三维端点误差(3D end-point-error,EPE3D)降低了13%,三维的严格准确率(3D accuracy strict,ACC3DS)提升了11%,三维的宽松准确率(3Daccuracyrelax,ACC3DR)提升了4.7%,三维异常值(3DOutliers,Outliers3D)降低了10.8%;在KITTI数据集上,全部点上的EPE3D(full EPE3D,EPE3Dfull)降低了10.7%,ACC3DS提升了2.1%,ACC3DR提升了1.7%,Outliers3D降低了5.5%。

基于NeRF的3D场景动画风格迁移及其技术美学价值研究

针对传统3D动画场景建模技术存在操作困难,相关行业对3D模型需求量大、计算机硬件要求较高等诸多问题,本文提出一种改进的神经辐射场(NeRF)技术,通过对拍摄影像进行3D重构,进而完成动画风格迁移,借此形成独特3D动画场景艺术效果的解决方案。作为一种新颖的3D重建方法,NeRF受到计算机图像合成领域广泛关注。本文将NeRF引入3D场景动画制作中,在验证本方案可行性与科学性的同时,也为3D动画场景创建提供新思路,充实技术美学价值。

基于3D实景模型和AHP边坡危险性评价系统的应用:以沿黄公路吴堡–永和段为例

为了有效控制、管理和减少公路边坡地质灾害产生的不利影响,本研究提出了一种基于无人机倾斜摄影三维实景边坡信息提取结合层次分析原理的公路边坡危险性评价新方法,开发了以Cesium为框架的沿黄公路边坡危险性评价三维可视化web平台。利用三维实景模型和计算机算法室内即可提取边坡危险性评价信息(包括路-坡距、坡度、坡高和节理密度等),采用层次分析法对各个边坡的危险性进行评估,依据最终得出的危险性综合评分,将其划分为极低、低、中、高、极高5个危险等级。将所提出的方法应用于沿黄公路边坡危险性评估,共评价边坡656个,其中极低危险边坡0个,低危险边坡23个,中危险边坡405个,高危险边坡210个,极高危险边坡18个。评价结果与实地调查结果吻合,说明提出的评价系统是合理的、有效的。该系统不仅实现地质灾害三维可视化调查和危险性评价,也推进了公路边坡防灾减灾技术数字化、智能化应用的发展。

基于Unity3D的智慧校园平台的开发

文中结合Unity3D游戏开发引擎及测量专业相关知识,将常用的测量数据在Unity3D中实现可视化加载,以实现后续的数据漫游以及对应的开发操作。文中主要研究了测量数据的获取、Unity3D软件的熟悉、场景制作、脚本开发及各种测量模型数据的动态加载。通过应用SuperMap的SDK’s的组件,导入了各种类型的测量模型数据,并实现了漫游的功能。文中实现了测量数据与Unity3D的结合,在Unity3D中开发脚本,对测量数据进行操作及漫游,增强了数据的可视性,提升了用户的体验感,为测量数据的处理提供了新的平台及处理方式。

基于Unity3D和3DSMax的水库虚拟仿真系统设计

随着技术的发展和应用范围的扩大,数字孪生技术已经被广泛应用于诸如能源、制造、交通、医疗、城市规划等领域,成为数字化转型的重要手段之一。介绍了基于Unity3D和3DSMax的水库虚拟仿真系统的设计。该系统旨在利用虚拟现实技术提供沉浸式的漫游体验。设计过程包括场景建模、材质处理和交互功能的实现。以西藏某大坝为例,通过3DSMax软件对实际大坝建筑结构进行建模,创建了一个高度详细的三维模型并添加材质。通过准确的测量数据和参考图纸,精细地重现了大坝的各个细节。通过使用WorldComposer地形建模插件重现了山脉、水体、道路和建筑物,通过Unity3D平台实现飞行、漫游等功能,使用户能够自由地在大坝周围移动和探索。通过细致的场景建模、材质处理和交互功能的实现,用户可以在虚拟环境中自由探索大坝及周围库区的壮丽景观和建筑结构,提供了一种全新的观赏和学习方式。该系统具有广泛的应用前景,可在旅游、教育和工程领域等方面发挥重要作用。

基于人体和场景上下文的多人3D姿态估计

深度歧义是单帧图像多人3D姿态估计面临的重要挑战,提取图像上下文对缓解深度歧义极具潜力.自顶向下方法大多基于人体检测建模关键点关系,人体包围框粒度粗背景噪声占比较大,极易导致关键点偏移或误匹配,还将影响基于人体尺度因子估计绝对深度的可靠性.自底向上的方法直接检出图像中的人体关键点再逐一恢复3D人体姿态.虽然能够显式获取场景上下文,但在相对深度估计方面处于劣势.提出新的双分支网络,自顶向下分支基于关键点区域提议提取人体上下文,自底向上分支基于三维空间提取场景上下文.提出带噪声抑制的人体上下文提取方法,通过建模“关键点区域提议”描述人体目标,建模姿态关联的动态稀疏关键点关系剔除弱连接减少噪声传播.提出从鸟瞰视角提取场景上下文的方法,通过建模图像深度特征并映射鸟瞰平面获得三维空间人体位置布局;设计人体和场景上下文融合网络预测人体绝对深度.在公开数据集MuPoTS-3D和Human3.6M上的实验结果表明:与同类先进模型相比,所提模型HSC-Pose的相对和绝对3D关键点位置精度至少提高2.2%和0.5%;平均根关键点位置误差至少降低4.2 mm.

基于Unity 3D的远距离无线传输三维可视化虚拟仿真场景构建

传统的三维可视化场景往往需要大量的数据传输,尤其是在远距离无线传输时,数据延迟和丢失可能会成为问题。Unity 3D通过优化数据压缩和传输协议,可以提高数据传输的效率和稳定性。因此,文章提出基于Unity 3D的远距离无线传输三维可视化虚拟仿真场景构建。文章通过设计无线传输三维可视化虚拟仿真场景架构,构建远距离无线传输三维可视化虚拟仿真模型,生成Unity 3D传输用户界面,实现了三维可视化虚拟仿真场景的构建。实验结果表明,设计的方法在绕射损耗、大气损耗、反射损耗、自由空间损耗等方面均较低,符合无线传输数据测量要求,提高了远距离无线传输质量。

基于融合采样策略的轻量级RGB-D场景3D目标检测

针对室内RGB-D场景中3D目标检测对复杂背景的适应性较差、难以进行有效采样,以及场景推断时间较长等问题,本文提出一种基于融合采样策略的轻量级RGB-D场景3D目标检测方法。该方法以场景RGB-D数据作为输入,首先通过深度相机投影将其转化为三维点云场景;然后利用一种结合距离最远点采样和特征最远点采样的融合采样策略对场景点云进行采样,有效保留了场景各实例代表点,将所有特征代表点汇集在一起形成场景的特征代表点云;最后在代表点云中利用深度霍夫投票机制投票出场景各物体的中心,并对各物体周围的相关特征进行聚类,从而实现场景的3D目标检测。实验结果表明,与传统方法相比,所提框架的目标检测准确率得到有效提升,同一评估指标下的检测准确率平均提升2.1%,且同一环境下每个场景的推断速度仅需要57 ms,远快于传统方法2倍多,从而保证了室内场景3D目标检测的准确性和高效性。

基于3D数字孪生技术的通信网络可视化方法

为实现对网络通行状态的可视化感知,引进3D数字孪生技术,对通信网络可视化方法展开设计与研究。收集与通信网络相关的数据,包括设备信息、拓扑结构、流量数据、实时监测数据等,进行通信网络运行场景的渲染;根据渲染后通信网络运行场景,掌握邻近节点之间的引力与斥力,以此计算通信网络节点间流量作用力;根据节点之间的流量数据,构建一个流量矩阵,以此实现通信网络可达节点通信强度可视化模型的构建。实验结果表明:设计方法不仅可以实现对通信网络的可视化,还可以实现对网络运行中通信信号强度的精准感知。

城区建筑物3D景观模型建立

为了在 2 D GIS基础上实现城市建筑物的 3D显示 ,提出了一种建立建筑物 3D模型的方法 ,该方法首先采用最大包围盒法将建筑物 3D模型与 2 .5 D DEM集成在一起 ,然后采用了 Meier关系数据结构表示城区建筑物几何模型中的拓扑信息和属性信息 ,并通过 VRML 浏览器来实现城市景观的交互显示 ,从而实现了利用常规 2 D GIS数据建立建筑物 3D景观虚拟模型 .实验结果表明 ,该方法能在 VRML 浏览器中以各种视角 ,在各个位置和按各种缩放比例 ,对建筑物模型进行动态观察 ,并在计算机屏幕上实现立体显示 .根据实验结果和分析可见 ,随着 2 D GIS的发展成熟 ,利用常规 2 D GIS数据来建立建筑物