3D scene graph prediction from point clouds

Background In this study,we propose a novel 3D scene graph prediction approach for scene understanding from point clouds.Methods It can automatically organize the entities of a scene in a graph,where objects are nodes and their relationships are modeled as edges.More specifically,we employ the DGCNN to capture the features of objects and their relationships in the scene.A Graph Attention Network(GAT)is introduced to exploit latent features obtained from the initial estimation to further refine the object arrangement in the graph structure.A one loss function modified from cross entropy with a variable weight is proposed to solve the multi-category problem in the prediction of object and predicate.Results Experiments reveal that the proposed approach performs favorably against the state-of-the-art methods in terms of predicate classification and relationship prediction and achieves comparable performance on object classification prediction.Conclusions The 3D scene graph prediction approach can form an abstract description of the scene space from point clouds.

汽车驾驶模拟器中3D场景的设计和实现

1 引言汽车驾驶模拟器是一种能正确模拟汽车驾驶操作,并能在主要性能上获得与实车相同感觉的汽车驾驶训练仿真教学设备。一个好的模拟器应该有一个立体感强、图形逼真的三维视景。YF三维汽车驾驶模拟器的视景系统采用计算机实时生成技术(称为CIG视景系统),它根据当前汽车的状态实时生成逼真的三维场景,使学员感觉就象驾驶真的汽车一样。

一种改进了的AABB算法在3D场景少儿益智系统中的应用

碰撞检测是三维系统中实现系统角色真实感物理运动的主要算法。通过对碰撞检测AABB算法深入研究,针对碰撞检测AABB算法的运算量大、紧密性差的问题,利用BSP树剖分法和相交叉检测算法对其做了改进。并将其应用于3D场景少儿益智系统中,经实验分析,改进的碰撞检测算法能有效地减少了系统运算次数,提高了系统的运行速度。

基于AutoCAD和3DSMAX的虚拟校园三维场景构建研究

当前的虚拟校园三维场景构建方法是利用osgEarth获取GoogleEarth数据,构建虚拟校园三维地图,然后依据本地地形信息与三维场景模型完成对虚拟校园的三维场景构建;该方法在三维场景构建过程中没有考虑到校园后续建设发展,只凭现在所需进行三维场景构建,导致虚拟校园三维场景杂乱,存在虚拟校园三维场景构建顾虑不周的问题;为此,提出一种基于AutoCAD和3DSMAX的虚拟校园三维场景构建方法;该方法首先利用Kinect的RGB深度摄像头获取校园平面数据、影像数据和贴图资料,采用相机阵列合成算法对所得校园图像进行特征提取,然后依据非局部均值滤波方法对获取的校园图像进行去噪操作,最后利用边缘方向和梯度表示中的结构方向与强度,对校园图像外轮廓的纹理特征进行收敛,依据外轮廓纹理特征收敛实现校园图像内轮廓的准确定位,将校园图像与虚拟校园三维场景构建所需图像进行匹配,由此完成对虚拟校园三维场景的构建;实验结果证明,所提方法是虚拟校园三维场景构建领域革命性的突破,提高了学校的招生率,增加了校园文化历史的展现机会,为该领域的研究发展提供了有效依据。

基于X3D/VRML构建金沙地区古场景

虚拟现实技术已广泛应用于考古领域,为文物和古环境的再现提供了平台。X3D作为VMRL语言的继承版本,是新的3D图形规范。本文以X3D/VRML为工具,结合现有的虚拟现实技术对三维虚拟古场景的开发原理和技术进行了研究。综合各方面专家的研究成果与文物考古成果,以精确的地理位置再现金沙地区三维古场景,并运用X3D的消息映射机制以及Java对VRML的控制实现了交互功能。基于X3D制作的虚拟现实场景文件体积小,方便网上传输;其平台无关性,便于实现交互功能。

RWNeRF:Robust Watermarking Scheme for Neural Radiance Fields Based on Invertible Neural Networks

As neural radiance fields continue to advance in 3D content representation,the copyright issues surrounding 3D models oriented towards implicit representation become increasingly pressing.In response to this challenge,this paper treats the embedding and extraction of neural radiance field watermarks as inverse problems of image transformations and proposes a scheme for protecting neural radiance field copyrights using invertible neural network watermarking.Leveraging 2D image watermarking technology for 3D scene protection,the scheme embeds watermarks within the training images of neural radiance fields through the forward process in invertible neural networks and extracts them from images rendered by neural radiance fields through the reverse process,thereby ensuring copyright protection for both the neural radiance fields and associated 3D scenes.However,challenges such as information loss during rendering processes and deliberate tampering necessitate the design of an image quality enhancement module to increase the scheme’s robustness.This module restores distorted images through neural network processing before watermark extraction.Additionally,embedding watermarks in each training image enables watermark information extraction from multiple viewpoints.Our proposed watermarking method achieves a PSNR(Peak Signal-to-Noise Ratio)value exceeding 37 dB for images containing watermarks and 22 dB for recovered watermarked images,as evaluated on the Lego,Hotdog,and Chair datasets,respectively.These results demonstrate the efficacy of our scheme in enhancing copyright protection.

基于DirectX构建虚拟战场环境研究

虚拟战场的构建是当前军事仿真领域中的一个非常前沿性的课题。随着计算机硬件性能的大幅度提高,使得在PC机上利用底层的图形函数接口构建虚拟战场环境也成为可能,从而可以大大降低成本。探讨了D irectX在三维地形模拟、海洋环境仿真、战场特殊效果、自然现象模拟、武器装备动画、人员动作模拟和战场三维真实感声音生成中的应用。比较全面地介绍了利用D irectX构建虚拟战场环境所涉及到的各种技术。

用Director MX实现虚拟场景漫游

实现虚拟场景漫游的主要技术包括场景建模和程序设计两个方面,利用贴图和建模相结合建立模型可以大大简化模型,利用Director MX导入Shockwave 3D演员,并应用Director的3D行为能够很方便的控制三维模型以实现场景漫游,文章以简单游戏场景漫游为例介绍了这种方法的具体实现。

Generation and Control of Game Virtual Environment

In this paper, we present a framework for the generation and control of an Internet-based 3-dimensional game virtual environment that allows a character to navigate through the environment. Our framework includes 3-dimensional terrain mesh data processing, a map editor, scene processing, collision processing, and walkthrough control. We also define an environment-specific semantic information editor, which can be applied using specific location obtained from the real world. Users can insert text information related to the characters real position in the real world during navigation in the game virtual environment.

Virtual Huanghe River System:Framework and Technology

Virtual Reality provides a new approach for geographical research. In this paper, a framework of the Virtual Huanghe （Yellow） River System was first presented from the view of technology, which included five main modules——data sources, 3D simulation terrain database, 3D simulation model database, 3D simulation implementation and application system. Then the key technoiogies of constructing Virtual Huanghe River System were discussed in detail： 1） OpenGL technology, the 3D graphics developing instrument, was employed in Virtual Huanghe River System to realize the function of dynamic real-time navigation. 2） MO and OpenGL technologies were used to make the mutual response between 3D scene and 2D electronic map available, which made use of the advantages of both 3D scene and 2D electronic map, with the macroscopic view, integrality and conciseness of 2D electronic map combined with the locality, reality and visualization of 3D scene. At the same time the disadvantages of abstract and ambiguity of 2D electronic map and the direction losing of virtual navigation in 3D scene were overcome.

城市3维地理信息服务系统框架研究

数字城市已是当前城市信息化建设的主要目标,本文以数字城市的实质性建设为着眼点,详细阐述了利用3D GIS,WebGIS等技术,建立城市3维地理信息服务系统的原理、方法等。城市3维地理信息服务系统将是全面推进数字城市建设、城市信息资源共享、提升信息资源应用价值的有效途径之一。

Digital Simulation Platform for Satellite Launch Mission Verification

To simulate the satellite launch mission under a general platform which could be used in a full-digital mode as well as in a semi-physical way, is an important way to certify the mission design performance as well as technical feasibilities, and it relates to complex system simulation methods such as multi-disciplinary coupling, multi-language modeling as well as interactive simulation and virtual simulation technologies. This paper introduces the design of a digital simulation platform for satellite launch mission verification.The platform has the advantages of high generality and extensibility, being easy to build up. The Functional Mockup Interface(FMI) Standard is adopted to achieve integration of multi-source models. A WebGL based 3D visual simulation tool is also adopted to implement the virtual display system which could display the rocket launch process and rocket-satellite separation with high fidelity 3D virtual scenes. A configuration tool was developed to map the 3D objects in the visual scene with simulation physical variables for complex rocket flight control mechanisms, which greatly improves the platform's generality and extensibility. Finally the real-time performance had been tested and the YL-1 launch mission was adopted to demonstrate the functions of the platform.The platform will be used to construct a digital twin system for satellite launch missions in the future.

3D indoor scene modeling from RGB-D data:a survey

3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras,there is a growing interest in digitizing real-world indoor 3D scenes. However,modeling indoor3 D scenes remains a challenging problem because of the complex structure of interior objects and poor quality of RGB-D data acquired by consumer-level sensors.Various methods have been proposed to tackle these challenges. In this survey,we provide an overview of recent advances in indoor scene modeling techniques,as well as public datasets and code libraries which can facilitate experiments and evaluation.

SinGRAV: Learning a Generative Radiance Volume from a Single Natural Scene

We present SinGRAV, an attempt to learn a generative radiance volume from multi-view observations of a single natural scene, in stark contrast to existing category-level 3D generative models that learn from images of many object-centric scenes. Inspired by SinGAN, we also learn the internal distribution of the input scene, which necessitates our key designs w.r.t. the scene representation and network architecture. Unlike popular multi-layer perceptrons (MLP)-based architectures, we particularly employ convolutional generators and discriminators, which inherently possess spatial locality bias, to operate over voxelized volumes for learning the internal distribution over a plethora of overlapping regions. On the other hand, localizing the adversarial generators and discriminators over confined areas with limited receptive fields easily leads to highly implausible geometric structures in the spatial. Our remedy is to use spatial inductive bias and joint discrimination on geometric clues in the form of 2D depth maps. This strategy is effective in improving spatial arrangement while incurring negligible additional computational cost. Experimental results demonstrate the ability of SinGRAV in generating plausible and diverse variations from a single scene, the merits of SinGRAV over state-of-the-art generative neural scene models, and the versatility of SinGRAV by its use in a variety of applications. Code and data will be released to facilitate further research.

Deep panoramic depth prediction and completion for indoor scenes

We introduce a novel end-to-end deeplearning solution for rapidly estimating a dense spherical depth map of an indoor environment.Our input is a single equirectangular image registered with a sparse depth map,as provided by a variety of common capture setups.Depth is inferred by an efficient and lightweight single-branch network,which employs a dynamic gating system to process together dense visual data and sparse geometric data.We exploit the characteristics of typical man-made environments to efficiently compress multiresolution features and find short-and long-range relations among scene parts.Furthermore,we introduce a new augmentation strategy to make the model robust to different types of sparsity,including those generated by various structured light sensors and LiDAR setups.The experimental results demonstrate that our method provides interactive performance and outperforms stateof-the-art solutions in computational efficiency,adaptivity to variable depth sparsity patterns,and prediction accuracy for challenging indoor data,even when trained solely on synthetic data without any fine tuning.

HDR-Net-Fusion:Real-time 3D dynamic scene reconstruction with a hierarchical deep reinforcement network

Reconstructing dynamic scenes with commodity depth cameras has many applications in computer graphics,computer vision,and robotics.However,due to the presence of noise and erroneous observations from data capturing devices and the inherently ill-posed nature of non-rigid registration with insufficient information,traditional approaches often produce low-quality geometry with holes,bumps,and misalignments.We propose a novel 3D dynamic reconstruction system,named HDR-Net-Fusion,which learns to simultaneously reconstruct and refine the geometry on the fly with a sparse embedded deformation graph of surfels,using a hierarchical deep reinforcement(HDR)network.The latter comprises two parts:a global HDR-Net which rapidly detects local regions with large geometric errors,and a local HDR-Net serving as a local patch refinement operator to promptly complete and enhance such regions.Training the global HDR-Net is formulated as a novel reinforcement learning problem to implicitly learn the region selection strategy with the goal of improving the overall reconstruction quality.The applicability and efficiency of our approach are demonstrated using a large-scale dynamic reconstruction dataset.Our method can reconstruct geometry with higher quality than traditional methods.

Perspectives on point cloud-based 3D scene modeling and XR presentation within the cloud-edge-client architecture

With the support of edge computing,the synergy and collaboration among central cloud,edge cloud,and terminal devices form an integrated computing ecosystem known as the cloud-edge-client architecture.This integration unlocks the value of data and computational power,presenting significant opportunities for large-scale 3D scene modeling and XR presentation.In this paper,we explore the perspectives and highlight new challenges in 3D scene modeling and XR presentation based on point cloud within the cloud-edge-client integrated architecture.We also propose a novel cloud-edge-client integrated technology framework and a demonstration of municipal governance application to address these challenges.

Structure-aware fusion network for 3D scene understanding

In this paper,we propose a Structure-Aware Fusion Network(SAFNet)for 3D scene understanding.As 2D images present more detailed information while 3D point clouds convey more geometric information,fusing the two complementary data can improve the discriminative ability of the model.Fusion is a very challenging task since 2D and 3D data are essentially different and show different formats.The existing methods first extract 2D multi-view image features and then aggregate them into sparse 3D point clouds and achieve superior performance.However,the existing methods ignore the structural relations between pixels and point clouds and directly fuse the two modals of data without adaptation.To address this,we propose a structural deep metric learning method on pixels and points to explore the relations and further utilize them to adaptively map the images and point clouds into a common canonical space for prediction.Extensive experiments on the widely used ScanNetV2 and S3DIS datasets verify the performance of the proposed SAFNet.

A real 3D scene rendering optimization method based on region of interest and viewing frustum prediction in virtual reality

As an important technology of digital construction,real 3D models can improve the immersion and realism of virtual reality(VR)scenes.The large amount of data for real 3D scenes requires more effective rendering methods,but the current rendering optimization methods have some defects and cannot render real 3D scenes in virtual reality.In this study,the location of the viewing frustum is predicted by a Kalman filter,and eye-tracking equipment is used to recognize the region of interest(ROI)in the scene.Finally,the real 3D model of interest in the predicted frustum is rendered first.The experimental results show that the method of this study can predict the frustrum location approximately 200 ms in advance,the prediction accuracy is approximately 87%,the scene rendering efficiency is improved by 8.3%,and the motion sickness is reduced by approximately 54.5%.These studies help promote the use of real 3D models in virtual reality and ROI recognition methods.In future work,we will further improve the prediction accuracy of viewing frustums in virtual reality and the application of eye tracking in virtual geographic scenes.

A Method for 3D Scene Description and Segmentation in an Object Record

in this poper a novel data-and rule-driven system for 3D scene description and segmentation inan unknown environment is presented.This system generatss hierachies of features that correspond tostructural elements such as boundaries and shape classes of individual object as well as relationshipsbetween objects.It is implemented as an added high-level component to an existing low-level binocularvision system[1]. Based on a pair of matched stereo images produced by that system,3D segmentation is firstperformed to group object boundary data into several edge-sets,each of which is believed to belong to aparticular object.Then gross features of each object are extracted and stored in an object recbrd.The finalstructural description of the scene is accomplished with information in the object record,a set of rules and arule implementor. The System is designed to handle partially occluded objects of different shapes and sizeson the 2D imager.Experimental results have shown its success in computing both object and structurallevel descriptions of common man-made objects.