An Improved High Precision 3D Semantic Mapping of Indoor Scenes from RGB-D Images

This paper proposes an improved high-precision 3D semantic mapping method for indoor scenes using RGB-D images.The current semantic mapping algorithms suffer from low semantic annotation accuracy and insufficient real-time performance.To address these issues,we first adopt the Elastic Fusion algorithm to select key frames from indoor environment image sequences captured by the Kinect sensor and construct the indoor environment space model.Then,an indoor RGB-D image semantic segmentation network is proposed,which uses multi-scale feature fusion to quickly and accurately obtain object labeling information at the pixel level of the spatial point cloud model.Finally,Bayesian updating is used to conduct incremental semantic label fusion on the established spatial point cloud model.We also employ dense conditional random fields(CRF)to optimize the 3D semantic map model,resulting in a high-precision spatial semantic map of indoor scenes.Experimental results show that the proposed semantic mapping system can process image sequences collected by RGB-D sensors in real-time and output accurate semantic segmentation results of indoor scene images and the current local spatial semantic map.Finally,it constructs a globally consistent high-precision indoor scenes 3D semantic map.

CSMCCVA:Framework of cross-modal semantic mapping based on cognitive computing of visual and auditory sensations

Cross-modal semantic mapping and cross-media retrieval are key problems of the multimedia search engine.This study analyzes the hierarchy,the functionality,and the structure in the visual and auditory sensations of cognitive system,and establishes a brain-like cross-modal semantic mapping framework based on cognitive computing of visual and auditory sensations.The mechanism of visual-auditory multisensory integration,selective attention in thalamo-cortical,emotional control in limbic system and the memory-enhancing in hippocampal were considered in the framework.Then,the algorithms of cross-modal semantic mapping were given.Experimental results show that the framework can be effectively applied to the cross-modal semantic mapping,and also provides an important significance for brain-like computing of non-von Neumann structure.

An Edge-assisted, Object-oriented Random Forest Approach for Refined Extraction of Tea Plantations Using Multi-temporal Sentinel-2 and High-resolution Gaofen-2 Imagery

As a consumed and influential natural plant beverage,tea is widely planted in subtropical and tropical areas all over the world.Affected by(sub)tropical climate characteristics,the underlying surface of the tea distribution area is extremely complex,with a variety of vegetation types.In addition,tea distribution is scattered and fragmentized in most of China.Therefore,it is difficult to obtain accurate tea information based on coarse resolution remote sensing data and existing feature extraction methods.This study proposed a boundary-enhanced,object-oriented random forest method on the basis of high-resolution GF-2 and multi-temporal Sentinel-2 data.This method uses multispectral indexes,textures,vegetable indices,and variation characteristics of time-series NDVI from the multi-temporal Sentinel-2 imageries to obtain abundant features related to the growth of tea plantations.To reduce feature redundancy and computation time,the feature elimination algorithm based on Mean Decrease Accuracy(MDA)was used to generate the optimal feature set.Considering the serious boundary inconsistency problem caused by the complex and fragmented land cover types,high resolution GF-2 image was segmented based on the MultiResolution Segmentation(MRS)algorithm to assist the segmentation of Sentinel-2,which contributes to delineating meaningful objects and enhancing the reliability of the boundary for tea plantations.Finally,the object-oriented random forest method was utilized to extract the tea information based on the optimal feature combination in the Jingmai Mountain,Yunnan Province.The resulting tea plantation map had high accuracy,with a 95.38%overall accuracy and 0.91 kappa coefficient.We conclude that the proposed method is effective for mapping tea plantations in high heterogeneity mountainous areas and has the potential for mapping tea plantations in large areas.

Visualization Analysis of Multi-Domain Access Control Policy Integration Based on Tree-Maps and Semantic Substrates

The complexity of multi-domain access control policy integration makes it difficult to understand and manage the policy conflict information. The policy information visualization technology can express the logical relation of the complex information intuitively which can effectively improve the management ability of the multi-domain policy integration. Based on the role-based access control model, this paper proposed two policy analyzing methods on the separated domain statistical information of multi-domain policy integration conflicts and the policy element levels of inter-domain and element mapping of cross-domain respectively. In addition, the corresponding visualization tool is developed. We use the tree-maps algorithm to statistically analyze quantity and type of the policy integration conflicts. On that basis, the semantic substrates algorithm is applied to concretely analyze the policy element levels of inter-domain and role and permission mapping of cross-domain. Experimental result shows tree-maps and semantic substrates can effectively analyze the conflicts of multi-domain policy integration and have a good application value.

A new projection method for biological semantic map generation

Low-dimensional representation is a convenient method of obtaining a synthetic view of complex datasets and has been used in various domains for a long time. When the representation is related to words in a document, this kind of representation is also called a semantic map. The two most popular methods are self-organizing maps and generative topographic mapping. The second approach is statistically well-founded but far less computationally efficient than the first. On the other hand, a drawback of self-organizing maps is that they do not project all points, but only map nodes. This paper presents a method of obtaining the projections for all data points complementary to the self-organizing map nodes. The idea is to project points so that their initial distances to some cluster centers are as conserved as possible. The method is tested on an oil flow dataset and then applied to a large protein sequence dataset described by keywords. It has been integrated into an interactive data browser for biological databases.

Building of cognizing semantic map in large-scale semi-unknown environment

The quick response code based artificial labels are applied to provide semantic concepts and relations of surroundings that permit the understanding of complexity and limitations of semantic recognition and scene only with robot's vision.By imitating spatial cognizing mechanism of human,the robot constantly received the information of artificial labels at cognitive-guide points in a wide range of structured environment to achieve the perception of the environment and robot navigation.The immune network algorithm was used to form the environmental awareness mechanism with "distributed representation".The color recognition and SIFT feature matching algorithm were fused to achieve the memory and cognition of scenario tag.Then the cognition-guide-action based cognizing semantic map was built.Along with the continuously abundant map,the robot did no longer need to rely on the artificial label,and it could plan path and navigate freely.Experimental results show that the artificial label designed in this work can improve the cognitive ability of the robot,navigate the robot in the case of semi-unknown environment,and build the cognizing semantic map favorably.

Semantic Web研究综述

近年来 ,SemanticWeb逐渐成为WWW领域的研究热点以及智能化网络服务和应用开发中的关键技术之一。归纳了Se manticWeb技术的研究背景和主要发展历史。在分析了典型的SemanticWeb概念后 ,给出了SemanticWeb的定义。通过讨论SemanticWeb构想的层次框架模型 ,指出了各个层次扮演的角色 ,并着重分析了SemanticWeb的重要研究领域 ,指出了它们在SemanticWeb构架中的核心作用。通过分析SemanticWeb的应用领域和相关开发工具以及面临的问题和挑战 ,指明了SemanticWeb研究和实践的方向。作为总结 ,给出了SemanticWeb领域下一步的研究趋势。

Ontology Based Resolution of Semantic Conflicts in Information Integration

Semantic conflict is the conflict caused by using different ways in heterogeneous systems to express the same entity in reality. This prevents information integration from accomplishing semantic coherence. Since ontology helps to solve semantic problems, this area has become a hot topic in information integration. In this paper, we introduce semantic conflict into information integration of heterogeneous applications. We discuss the origins and categories of the conflict, and present an ontology-based schema mapping approach to eliminate semantic conflicts. Key words ontology - CCSOL - semantic conflict - schema mapping CLC number TP 301 Biography: LU Han (1980-), male, Master candidate, research direction: ontology and information integration.

Smap:基于文献语义的学科知识图景可视化

随着文献爆炸式增长,学科领域不断交叉融合,科研规模扩大和知识体系复杂性日益提升,如何清晰地可视化学科知识图景,进而把握知识结构和研究态势,引起了科技情报人员的广泛关注。本研究基于文档表示学习和流形学习算法,提供了一种科学领域语义地图(semantic map,Smap)构建方法。首先以Doc2Vec捕获文献间的高维语义特征,然后利用UMAP(uniform manifold approximation and projection)对文献语义临近性进行非线性降维,最后以核密度估计根据文献分布异质性刻画领域知识结构。在实证分析阶段,本研究对文献规模覆盖了从千级到百万级的4个学科领域,进行了领域可视化、知识层级结构识别以及动态演化分析。进而,本研究借助引用关系、关键词以及数据集的分类体系,通过量化Smap地图上文献分布的局部纯粹性以及全局地图距离和研究差异的相关性,验证了所提方法的有效性。本研究通过与随机实验对比,进一步地量化了有效性的显著程度。本研究为当前科学领域可视化方法提供了有益补充,可为大规模科技文献数据驱动的科技情报服务提供分析工具。

Next Generation Semantic and Spatial Joint Perception——Neural Metric-Semantic Understanding

Efficient perception of the real world is a long-standing effort of computer vision.Mod⁃ern visual computing techniques have succeeded in attaching semantic labels to thousands of daily objects and reconstructing dense depth maps of complex scenes.However,simultaneous se⁃mantic and spatial joint perception,so-called dense 3D semantic mapping,estimating the 3D ge⁃ometry of a scene and attaching semantic labels to the geometry,remains a challenging problem that,if solved,would make structured vision understanding and editing more widely accessible.Concurrently,progress in computer vision and machine learning has motivated us to pursue the capability of understanding and digitally reconstructing the surrounding world.Neural metric-se⁃mantic understanding is a new and rapidly emerging field that combines differentiable machine learning techniques with physical knowledge from computer vision,e.g.,the integration of visualinertial simultaneous localization and mapping(SLAM),mesh reconstruction,and semantic un⁃derstanding.In this paper,we attempt to summarize the recent trends and applications of neural metric-semantic understanding.Starting with an overview of the underlying computer vision and machine learning concepts,we discuss critical aspects of such perception approaches.Specifical⁃ly,our emphasis is on fully leveraging the joint semantic and 3D information.Later on,many im⁃portant applications of the perception capability such as novel view synthesis and semantic aug⁃mented reality(AR)contents manipulation are also presented.Finally,we conclude with a dis⁃cussion of the technical implications of the technology under a 5G edge computing scenario.

Multi-temporal urban semantic understanding based on GF-2 remote sensing imagery:from tri-temporal datasets to multi-task mapping

High resolution satellite images are becoming increasingly available for urban multi-temporal semantic understanding.However,few datasets can be used for land-use/land-cover(LULC)classification,binary change detection(BCD)and semantic change detection(SCD)simultaneously because classification datasets always have one time phase and BCD datasets focus only on the changed location,ignoring the changed classes.Public SCD datasets are rare but much needed.To solve the above problems,a tri-temporal SCD dataset made up of Gaofen-2(GF-2)remote sensing imagery(with 11 LULC classes and 60 change directions)was built in this study,namely,the Wuhan Urban Semantic Understanding(WUSU)dataset.Popular deep learning based methods for LULC classification,BCD and SCD are tested to verify the reliability of WUSU.A Siamese-based multi-task joint framework with a multi-task joint loss(MJ loss)named ChangeMJ is proposed to restore the object boundaries and obtains the best results in LULC classification,BCD and SCD,compared to the state-of-the-art(SOTA)methods.Finally,a large spatial-scale mapping for Wuhan central urban area is carried out to verify that the WUsU dataset and the ChangeMJ framework have good application values.

A labor-free index-guided semantic segmentation approach for urban vegetation mapping from high-resolution true color imagery

Accurate and timely information on urban vegetation(UV)can be used as an important indicator to estimate the health of cities.Due to the low cost of RGB cameras,true color imagery(TCI)has been widely used for high spatial resolution UV mapping.However,the current index-based and classifier-based UV mapping approaches face problems of the poor ability to accurately distinguish UV and the high reliance on massive annotated samples,respectively.To address this issue,an index-guided semantic segmentation(IGSS)framework is proposed in this paper.Firstly,a novel cross-scale vegetation index(CSVI)is calculated by the combination of TCI and Sentinel-2 images,and the index value can be used to provide an initial UV map.Secondly,reliable UV and non-UV samples are automatically generated for training the semantic segmentation model,and then the refined UV map can be produced.The experimental results show that the proposed CSVI outperformed the existingfive RGB vegetation indices in highlighting UV cover and suppressing complex backgrounds,and the proposed IGSS workflow achieved satisfactory results with an OA of 87.72%∼88.16%and an F1 score of 87.73%∼88.37%,which is comparable with the fully-supervised method.

基于路径规划特点的语义目标导航方法

为了解决语义目标导航任务中存在的探索效率低、深度不精准等问题,本文构建了一个解决语义目标导航任务的框架,在语义地图构建模块中引入了深度图边缘处理以及地图纠错机制;在探索模块中引入了覆盖范围最大化算法;在路径规划模块中引入了替代点机制。本文在一个3D仿真环境下进行了实验。实验结果表明,本文提出的解决方案明显提升了语义目标导航任务的性能。此外,本文所提方法成功应用到了四足机器人上,从而验证了其在现实场景下的泛化性。

基于可调场景语义标注范围的家庭室内语义地图构建

针对家庭室内环境语义地图建图速度较慢和在门口场景语义标注易出现错误等问题,提出一种基于可调场景语义标注范围的家庭室内语义地图构建方法。首先根据YOLOv5s识别的物体大小赋予相应的场景置信度,基于该场景置信度设置阈值使得语义标注范围限制在机器人当前所在区域,确保场景切换时语义标注范围不会立即改变。然后基于人工势场虚拟力“引力斥力”原理,实现语义标注范围的扩大或缩小。最后结合阈值和动态语义标注范围,避免在门口场景中出现语义标注错误。实验结果表明:与Places205-VGG16神经网络建立家庭室内语义地图相比,所提方法平均效率和平均精准率分别提升了11.0%和7.8%,在家庭室内环境中具有一定的优越性。

语义通信的数学理论

自从1948年经典信息论诞生以来,在其指导下,现代通信技术已经逼近了理论性能极限,例如信息熵H(U)、信道容量C=max_(p(x))I(X;Y)以及率失真函数R(D)=min_(p(x|x):Ed(x,x)≤D)I(X;X)。长期以来,由于经典信息论只研究语法信息,限制了通信科学的进一步发展。近年来,研究语义信息处理与传输的通信技术获得了学术界的普遍关注,语义通信开辟了未来通信技术发展的新方向,但还缺乏一般性的数学指导理论。为了解决这一难题,构建了语义信息论的理论框架,对语义信息的度量体系与语义通信的理论极限进行了系统性阐述。首先,通过深入分析各类信源的数据特征,以及各种下游任务的需求,总结归纳出语义信息的普遍属性——同义性。由此指出语义信息是语法信息的上级概念,是许多等效或相似语法信息的抽象特征,表征隐藏在数据或消息背后的含义或内容。将语义信息与语法信息之间的关系命名为同义映射,这是一种“一对多”映射,即一个语义符号可以由许多不同的语法符号表示。基于同义映射f这一核心概念,引入语义熵H_(s)(U)作为语义信息的基本度量指标,表示为信源概率分布与同义映射的泛函。在此基础上,引入上/下语义互信息I^(s)(X;Y)(I_(s)(X;Y)),语义信道容量C_(s)=max_(f_(xy))max_(p_((x)))I^(s)(X;Y)以及语义率失真函数R_(s)(D)=min_({f_(x),f_(x)})min_(p(x|x):Ed_(s)(x,x)≤D)I_(s)(X;X),从而构建了完整的语义信息度量体系。这些语义信息度量是经典信息度量的自然延伸,都由同义映射约束,如果采用“一对一”映射,则可以退化为传统的信息度量。由此可见,语义信息度量体系包含语法信息度量,前者与后者具有兼容性。其次,证明了3个重要的语义编码定理,以揭示语义通信的性能优势。基于同义映射,引入新的数学工具——语义渐近均分(AEP),详细探讨了同义典型序列的数学性质,并应用随机编码和同义典型序列译码/编码,证明了语义无失真信源编码定理、语义信道编码定理和语义限失真信源编码定理。类似于经典信息论,这些基本编码定理也都是存在性定理,但它们指出了语义通信系统的性能极限,在语义信息论中起着关键作用。由同义映射和这些基本编码定理可以推断,语义通信系统的性能优于经典通信系统,即语义熵小于信息熵H_(s)(U)≤H(U),语义信道容量大于经典信道容量C_(s)≥C,以及语义率失真函数小于经典率失真函数R_(s)(D)≤R(D)。最后,讨论了连续条件下的语义信息度量。此时,同义映射转换为连续随机变量分布区间的划分方式。相应地,划分后的子区间被命名为同义区间,其平均长度定义为同义长度S。特别是对于限带高斯信道,得到了一个新的信道容量公式C_(s)=B log[S^(4)(1+P/N_(0)B)],其中,平均同义长度S表征了信息的辨识能力。这一容量公式是经典信道容量的重要扩展,当S=1时,该公式退化为著名的香农信道容量公式。综上所述,语义信息论依据同义映射这一语义信息的本质特征,构建了语义信息的度量体系,引入新的数学工具,证明了语义编码的基本定理,论证了语义通信系统的性能极限,揭示了未来语义通信的巨大性能潜力。

复杂环境下基于自适应极线约束的AGV视觉SLAM算法

针对传统视觉同步定位与地图构建(SLAM)算法不能有效处理复杂环境中的动态及潜在动态目标而影响定位与建图性能的问题,提出一种基于Mask R-CNN神经网络以及ORB-SLAM3算法改进的视觉SLAM方法。针对动态目标,提出一种基于语义信息的运动一致性检验算法,使用自适应阈值的极线约束方法实现图像中动态特征点的精确剔除;针对潜在动态目标,提出一种改进的长期数据关联方法,通过增大关键帧选取密度及优化关键帧中的潜在动态目标信息,对算法的回环优化和地图融合过程进行改进,提高回环优化效果与地图复用性。在TUM数据集和真实场景中进行验证,实验结果表明与ORB-SLAM3算法相比,采用TUM数据集在低动态场景和高动态场景中的绝对轨迹均方根误差分别减小8.5%和65.6%;在真实场景下测试,所提算法的定位精度提高了62.5%。

面向地图匹配的室内位置-语义模型设计

传统的室内空间信息模型或者过于复杂导致操作性较差,或者没有充分表达地图匹配所需的空间要素信息,阻碍地图匹配方法的高效实施。该研究将以构建能够有效支撑地图匹配的室内空间信息模型作为研究目标,此模型应明确室内空间要素的表达内容,并应确定具体的表达方法。模型的构建应以地图匹配方法的机理为基础,以室内地图匹配的实际需要为导向,以表达支撑室内地图匹配的空间信息为目的,分别对室内空间要素的位置及属性,室内空间要素之间的拓扑关系进行表达。该研究构建面向地图匹配的室内位置-语义模型。实验表明此模型较为简洁且信息完备,可在保证地图匹配实际需要的同时提高运行效率。

动态场景下基于语义分割的视觉SLAM方法

针对在动态场景下视觉同步定位与建图(SLAM)鲁棒性差、定位与建图精度易受动态物体干扰的问题,设计一种基于改进DeepLabv3plus与多视图几何的语义视觉SLAM算法。以语义分割网络DeepLabv3plus为基础,采用轻量级卷积网络MobileNetV2进行特征提取,并使用深度可分离卷积代替空洞空间金字塔池化模块中的标准卷积,同时引入注意力机制,提出改进的语义分割网络DeepLabv3plus。将改进后的语义分割网络DeepLabv3plus与多视图几何结合,提出动态点检测方法,以提高视觉SLAM在动态场景下的鲁棒性。在此基础上,构建包含语义信息和几何信息的三维语义静态地图。在TUM数据集上的实验结果表明,与ORB-SLAM2相比,该算法在高动态序列下的绝对轨迹误差的均方根误差值和标准差(SD)值最高分别提升98%和97%。