3D scene graph prediction from point clouds

Background In this study,we propose a novel 3D scene graph prediction approach for scene understanding from point clouds.Methods It can automatically organize the entities of a scene in a graph,where objects are nodes and their relationships are modeled as edges.More specifically,we employ the DGCNN to capture the features of objects and their relationships in the scene.A Graph Attention Network(GAT)is introduced to exploit latent features obtained from the initial estimation to further refine the object arrangement in the graph structure.A one loss function modified from cross entropy with a variable weight is proposed to solve the multi-category problem in the prediction of object and predicate.Results Experiments reveal that the proposed approach performs favorably against the state-of-the-art methods in terms of predicate classification and relationship prediction and achieves comparable performance on object classification prediction.Conclusions The 3D scene graph prediction approach can form an abstract description of the scene space from point clouds.

Constructing Multiple Scene Graphs in Distributed Environment

Scene graph is a infrastructure of the virtual reality system to organize the virtual scene with abstraction, it can provide facility for the rendering engine and should be integrated effectively on demand into a real-time system, where a large quantities of scene objects and resources can be manipulated and managed with high flexibility and reliability. We present a new scheme of multiple scene graphs to accommodate the features of rendering engine and distributed systems. Based upon that, some other functions, e.g. block query, interactive editing, permission management, instance response, ＂redo＂ and ＂undo＂, are implemented to satisfy various requirements. At the same time, our design has compatibility to popular C/S architecture with good concurrent performance. Above all, it is convenient to be used for further development. The results of experiments including responding time demonstrate its good performance.

Traffic Scene Captioning with Multi-Stage Feature Enhancement

Traffic scene captioning technology automatically generates one or more sentences to describe the content of traffic scenes by analyzing the content of the input traffic scene images,ensuring road safety while providing an important decision-making function for sustainable transportation.In order to provide a comprehensive and reasonable description of complex traffic scenes,a traffic scene semantic captioningmodel withmulti-stage feature enhancement is proposed in this paper.In general,the model follows an encoder-decoder structure.First,multilevel granularity visual features are used for feature enhancement during the encoding process,which enables the model to learn more detailed content in the traffic scene image.Second,the scene knowledge graph is applied to the decoding process,and the semantic features provided by the scene knowledge graph are used to enhance the features learned by the decoder again,so that themodel can learn the attributes of objects in the traffic scene and the relationships between objects to generate more reasonable captions.This paper reports extensive experiments on the challenging MS-COCO dataset,evaluated by five standard automatic evaluation metrics,and the results show that the proposed model has improved significantly in all metrics compared with the state-of-the-art methods,especially achieving a score of 129.0 on the CIDEr-D evaluation metric,which also indicates that the proposed model can effectively provide a more reasonable and comprehensive description of the traffic scene.

Dynamic Scene Graph Generation of Point Clouds with Structural Representation Learning

Scene graphs of point clouds help to understand object-level relationships in the 3D space.Most graph generation methods work on 2D structured data,which cannot be used for the 3D unstructured point cloud data.Existing point-cloud-based methods generate the scene graph with an additional graph structure that needs labor-intensive manual annotation.To address these problems,we explore a method to convert the point clouds into structured data and generate graphs without given structures.Specifically,we cluster points with similar augmented features into groups and establish their relationships,resulting in an initial structural representation of the point cloud.Besides,we propose a Dynamic Graph Generation Network(DGGN)to judge the semantic labels of targets of different granularity.It dynamically splits and merges point groups,resulting in a scene graph with high precision.Experiments show that our methods outperform other baseline methods.They output reliable graphs describing the object-level relationships without additional manual labeled data.

Comprehensive Relation Modelling for Image Paragraph Generation

Image paragraph generation aims to generate a long description composed of multiple sentences,which is different from traditional image captioning containing only one sentence.Most of previous methods are dedicated to extracting rich features from image regions,and ignore modelling the visual relationships.In this paper,we propose a novel method to generate a paragraph by modelling visual relationships comprehensively.First,we parse an image into a scene graph,where each node represents a specific object and each edge denotes the relationship between two objects.Second,we enrich the object features by implicitly encoding visual relationships through a graph convolutional network(GCN).We further explore high-order relations between different relation features using another graph convolutional network.In addition,we obtain the linguistic features by projecting the predicted object labels and their relationships into a semantic embedding space.With these features,we present an attention-based topic generation network to select relevant features and produce a set of topic vectors,which are then utilized to generate multiple sentences.We evaluate the proposed method on the Stanford image-paragraph dataset which is currently the only available dataset for image paragraph generation,and our method achieves competitive performance in comparison with other state-of-the-art(SOTA)methods.

基于场景图的实时交互视景系统平台设计

三维场景组织管理中经常采用的空间数据结构--场景图的具体实现,包括场景数据的组织管理,可见性剔除、节点遍历、优化处理、碰撞检测等。在基于场景图的基础上设计并实现了一个实时交互式视景系统开发平台,给出了系统的结构框图、工作流程与信息交互关系,并对视景系统平台的视点控制、场景参数的应用定义文件输出以及场景图像捕捉和视频输出功能进行了详细介绍。平台同时提供了一套基于场景图的可供二次开发的三维图形应用程序接口。

三维模拟驾驶仿真系统的研究与实现

为了解决模拟驾驶仿真系统中开发成本昂贵、缺乏二次开发特性等问题,研究并实现了一种结合3DMAX建模软件、开放源代码的Newton物理引擎和OSG图像渲染引擎的三维模拟驾驶仿真系统。硬件系统采用多传感器融合技术来采集驾驶员模拟驾驶时的操作行为,软件系统根据所采集的操作数据实现对车辆的运动和碰撞反应的控制以及对场景视角的切换。实验结果表明,该方法具有软件开发成本低,场景逼真度高,动态效果好等特点,可以满足对驾驶员的驾驶行为进行记录并分析的需要。

基于二叉树的上下文相关L-系统实现

从底层数据结构的角度讨论了上下文相关L-系统实现中的几个关键问题.结合L-系统的语法,解释了以二叉树为底层数据结构的新系统设计机理,提出了生长方式树和生长状态树的概念,给出了新的数据结构形式下生长状态演化算法的实现.与采用线性数组实现的L系统相比,新系统很好地模拟了L-系统本身的树形逻辑结构,简化了L-系统中上下文搜索的过程,在生长模拟环节有更高的时空效率,且具备更好的可扩展性.以虚拟植物生长为背景,介绍了如何基于新数据结构进行虚拟植物绘制和绘制优化,并通过实例验证了新算法的有效性.

VRML在仿真可视化系统中的应用

该文应用了仿真可视化与仿真计算相分离的思想 ,提出用VRML规范及其浏览器作为仿真可视化系统的支撑平台。作者扩展了script节点的功能 ,使它能够支持网络通信 ,使得VRML节点能够通过网络接收仿真数据并能实时进行场景的渲染。在平台上可以搭建应用于各领域的仿真可视化系统。

基于OSG的分形L-系统三维树木仿真

对现实中的树木结构进行研究,在L系统的基础上对字符集进行扩展,设计了更加丰富的树木L文法描述,提出了比较符合树木生长过程的几何幂衰减规律,并结合图形渲染引擎OSG(open scene graph)实现了三维树木的树干部分的绘制。通过给树木的树干部分添加纹理、绘制树叶并给树叶添加阴影,使得绘制的树木更加真实自然。将绘制的树木作为一个节点加载到虚拟场景中,增强了虚拟场景的真实感。

一种面向场景图系统的刚体模拟应用框架及其实现

场景图系统是在虚拟环境生成领域应用十分广泛的技术,而刚体是虚拟环境中大量存在的一种物体对象类型。借鉴一般场景图的思想,对基于场景图系统的刚体运动建模方法进行了研究,面向通用场景图系统提出了一个实时刚体动力学仿真模拟框架,并针对Vega场景图系统给出了基于开源动力学引擎的模块化实现。

基于OSG的三维数字地球引擎设计与实现

针对三维数字地球引擎的建设需求,结合场景图形管理技术和软件构件化设计思想,进行了面向实际工程应用的三维数字地球引擎框架结构设计和关键技术及解决途径分析。该引擎将三维地理信息、虚拟现实、粒子效果、立体声效、流媒体视频等进行一体化集成实现,便于适应复杂环境的模拟、分析和监视,应用前景广阔。

基于OSG的飞行仿真系统视景平台的研究与开发

分析了飞行仿真系统的特征,并对其应用需求进行分析,在此基础上完成了系统视景平台的规划与设计。采用OpenSenceGraph(OSG)开源场景图形库对视景平台进行开发,针对需求分析实现了飞行仿真系统视景平台各模块的基本功能。

开源BIM理念及其系统构建设想

在回顾与总结BIM发展历程的基础上,综合分析各比较有代表性的BIM学派对BIM技术在项目全过程管理中实现途径的研究与预测,提出了开源BIM理念,阐述了BIM技术从传统工作模式转型升级为开源工作模式的意义与价值;利用开源场景图形进行了开源BIM核心软件——用户平台系统构建,对开源BIM应用进行了初步探索.简述了用户平台系统的系统框架和构建设想,分析了该系统基础数据层、应用模块层、可视交互层等三个层次之间的关系,以及如何在系统构建中实现开源BIM管理的目标.

osgEarth在三维GIS开发中的研究与应用

针对通用三维GIS平台开发提供了一种基于开源跨平台三维GIS渲染引擎osgEarth的技术方案,并对其中大地形三维模型实时生成和加载、LOD分页数据等关键技术展开了研究。分析了osgEarth的特点和数据加载流程,结合实际项目应用介绍了平台方案的实现及效果,对于工程实践有一定的指导意义。

基于osgEarth三维数字地球建设的研究

基于osg Earth开源工程是将地理信息系统和虚拟现实技术结合起来的一个重要应用。以基于osg Earth三维数字地球建设的研究为例,采用LOD与分页、动态调度等技术。研究了三维地形、影像、地物组织、调度及融合方式;设计了地球影像数据获取、坐标纹理校正、earth文件编写、矢量数据栅格化显示;分析了osg Earth中绘制线状矢量数据的模板体技术和该技术所隐藏的场景层次结构。在Visual Studio 2010平台下,使用Open Scene Graph语言进行编程,证明了该技术的可行性,实现了三维数字地球的建设。

无人机飞行控制仿真环境三维视景子系统设计

针对无人机飞行控制仿真环境以数据方式输出仿真结果不利于挖掘隐藏于数据中的本质特性的缺陷,借助虚拟现实技术的相关理论,设计实现基于无人机飞行控制仿真环境的三维视景子系统,实时动态再现无人机飞行运动全过程,为研究无人机飞行运动规律提供直观的事实根据。阐述了无人机飞行控制仿真环境三维视景子系统设计实现的目标,构造了系统结构并提出了三维视景系统的实现框架,对视景系统实现的主要技术进行了着重介绍,在视景系统的具体实现中对视景数据库设计、视景系统开发、视点切换、LynX Prim e和ACF配置文件、数据通信接口的具体实现进行了详细的论述。最后给出了系统的实施效果。

机场数据在合成视景系统中的应用研究

针对合成视景系统显示机场和跑道的需求,提出了一种利用标准机场数据库,在开放绘图架构下的机场合成显示方法。首先分析了AMDB数据的来源、格式与建模方法;然后使用跨平台开源的图形处理库OSG对机场、跑道和周围地形进行了三维视景合成。重点研究了AMDB数据绘图映射和建模方法、机场三维模型与地形的合成显示技术。建立了KSFO机场的合成三维视景并完成了视景的渲染和漫游功能。对该方法设计了仿真环境和仿真用例,通过仿真得到机场和跑道三维视景的实施效果。仿真结果证明了AMDB的建模方法、机场合成与显示技术较好地满足了合成视景系统对显示质量、刷新率、分辨率、视场的需求。

ViWoSG:A distributed scene graph of ultra-massive distributed virtual environments

An ultra-massive distributed virtual environment generally consists of ultra-massive terrain data and a large quantity of objects and their attribute data, such as 2D/3D geometric models, audio/video, images, vectors, characteristics, etc. In this paper, we propose a novel method for constructing distributed scene graphs with high extensibility. This method can support high concurrent interaction of clients and implement various tasks such as editing, querying, accessing and motion controlling. Some application experiments are performed to demonstrate its efficiency and soundness.

锥束CT检测成像仿真系统的研究与实现

设计并实现一个用于CT专业教学和科研的锥束CT检测成像仿真系统.该仿真系统能够模拟对CT设备的控制,生成正投影数据,重建被测物体,并能对重建结果进行三维显示.该系统包括如下关键技术:按真实比例对CT设备建模,实现了锥束CT工作过程的可视化;用场景图方法组织CT场景,简化了场景图形的管理方式;采用GPU加速CT正投影、图像重建和体绘制算法,在一定程度上解决了锥束CT大数据量快速计算问题.实验结果表明,该系统具有较快的运行速度,可满足用户交互操作的需要.