Next Generation Semantic and Spatial Joint Perception——Neural Metric-Semantic Understanding

Efficient perception of the real world is a long-standing effort of computer vision.Mod⁃ern visual computing techniques have succeeded in attaching semantic labels to thousands of daily objects and reconstructing dense depth maps of complex scenes.However,simultaneous se⁃mantic and spatial joint perception,so-called dense 3D semantic mapping,estimating the 3D ge⁃ometry of a scene and attaching semantic labels to the geometry,remains a challenging problem that,if solved,would make structured vision understanding and editing more widely accessible.Concurrently,progress in computer vision and machine learning has motivated us to pursue the capability of understanding and digitally reconstructing the surrounding world.Neural metric-se⁃mantic understanding is a new and rapidly emerging field that combines differentiable machine learning techniques with physical knowledge from computer vision,e.g.,the integration of visualinertial simultaneous localization and mapping(SLAM),mesh reconstruction,and semantic un⁃derstanding.In this paper,we attempt to summarize the recent trends and applications of neural metric-semantic understanding.Starting with an overview of the underlying computer vision and machine learning concepts,we discuss critical aspects of such perception approaches.Specifical⁃ly,our emphasis is on fully leveraging the joint semantic and 3D information.Later on,many im⁃portant applications of the perception capability such as novel view synthesis and semantic aug⁃mented reality(AR)contents manipulation are also presented.Finally,we conclude with a dis⁃cussion of the technical implications of the technology under a 5G edge computing scenario.

Uncovering the relationship among spatial vitality,perception,and environment of urban underground space in the metro zone

With the rapid development of rail transit,effectively developing urban underground space(UUS)in the metro zone has become an important approach to expanding urban space.However,UUS is currently facing problems,such as an uneven distribution or even loss of vitality,which restricts the utilization efficiency of the space.Thus,we established a UUS environmental assessment system based on the‘‘comfort-aesthetic-function-traffic-structure”using space syntax,instrumental measurement,and questionnaire surveys.By constructing a partial-least-square structural equation model,the internal relationships between the UUS environment and corresponding space vitality and space perception under different study areas,namely the underground transportation-oriented space(transportation space for short)and the underground commercial-oriented space(commercial space for short),were studied in detail throughout the working day.Results indicate the following:(1)the UUS in the metro zone environment influences spatial vitality.The vitality distribution of transportation space is significantly affected by the spatial traffic and structure.The vitality distribution of commercial space is significantly affected by the spatial function and traffic.(2)The environment of UUSs in the metro zone influences users’s perception.The perception of transportation space is significantly affected by aesthetics and comfort.The perception of commercial space is significantly affected by aesthetic,comfort,and spatial functions.(3)The user’s perception affects vitality,and the effect is more significant in commercial space.This study provides an in-depth understanding of the relationship between the complex environment and its spatial vitality as well as the spatial perception of the UUS in metro zones.Our research results provide a novel approach and theoretical basis for the development and application of UUS vitality in various cities.

Test re-test reliability of virtual acoustic space identification (VASI) test in young adults with normal hearing

Background:Recent developments in virtual acoustic technology has levered promising applications in the field of auditory sciences,especially in spatial perception.While conventional auditory spatial assessment using loudspeakers,interaural differences and/or questionnaires are limited by the availability and cost of instruments,the use of virtual acoustic space identification(VASI)test has widespread applications in spatial test battery as it overcomes these constraints.Purpose:The lack of test-retest reliability data of VASI test narrows its direct application in auditory spatial assessment,which is explored in the present study.Methods:Data from 75 normal-hearing young adults(mean age:25.11 y±4.65 SD)was collected in three sessions:baseline,within 15 min of baseline(intra-session),and one week after baseline session(inter-session).Test-retest reliability was assessed using the intra-class correlation coefficient(ICC),coefficient of variation(CV),and cluster plots.Results:The results showed excellent reliability for both accuracy and reaction time measures of VASI,with ICC values of 0.93 and 0.87,respectively.The CV values for overall VASI accuracy and reaction time 9.66% and 11.88%,respectively.This was also complemented by the cluster plot analyses,which showed 93.33% and 96.00% of temporal stability in the accuracy and reaction time measures,indicative of high test-retest reliability of VASI test in auditory spatial assessment.Conclusions:The high temporal stability(test-retest reliability)of VASI test validates its application in spatial hearing test battery.

Earth Observation Brain(EOB):an intelligent earth observation system

Since the twenty-first century,with the rapid development of high-resolution earth observation satellites,the earth observation satellite system has developed from the initial single satellite observation model to the current satellite constellation formed by light and small satellites observation model.All-weather and all-directional fine earth observation can now be realized.In the future,the satellite constellation,communication satellites,navigation satellites,and aircrafts are linked through dynamic linking network to form an air-space information network to realize real-time services of intelligent air-space information.To further enhance the perception,cognition,and quick response ability of the network,we propose the concept and model of the Earth Observation Brain(EOB)−the intelligent earth system based on events perception in this paper.Then,some key technologies needed to be solved in the EOB are also described.An application example is illustrated to show the process of perception and cognition in the primary stage of the EOB.In the future,EOB can observe what change of what object,the when and where to push these right information to mobile terminal of right people at the right time and right place.Global users can obtain any data,information,and knowledge in real-time through the EOB.