Investigation of Inside-Out Tracking Methods for Six Degrees of Freedom Pose Estimation of a Smartphone in Augmented Reality

Six degrees of freedom(6DoF)input interfaces are essential formanipulating virtual objects through translation or rotation in three-dimensional(3D)space.A traditional outside-in tracking controller requires the installation of expensive hardware in advance.While inside-out tracking controllers have been proposed,they often suffer from limitations such as interaction limited to the tracking range of the sensor(e.g.,a sensor on the head-mounted display(HMD))or the need for pose value modification to function as an input interface(e.g.,a sensor on the controller).This study investigates 6DoF pose estimation methods without restricting the tracking range,using a smartphone as a controller in augmented reality(AR)environments.Our approach involves proposing methods for estimating the initial pose of the controller and correcting the pose using an inside-out tracking approach.In addition,seven pose estimation algorithms were presented as candidates depending on the tracking range of the device sensor,the tracking method(e.g.,marker recognition,visual-inertial odometry(VIO)),and whether modification of the initial pose is necessary.Through two experiments(discrete and continuous data),the performance of the algorithms was evaluated.The results demonstrate enhanced final pose accuracy achieved by correcting the initial pose.Furthermore,the importance of selecting the tracking algorithm based on the tracking range of the devices and the actual input value of the 3D interaction was emphasized.

Single-center experience with Knee+^(TM) augmented reality navigation system in primary total knee arthroplasty

BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.

Large-scale spatial data visualization method based on augmented reality

Background A task assigned to space exploration satellites involves detecting the physical environment within a certain space.However,space detection data are complex and abstract.These data are not conducive for researchers'visual perceptions of the evolution and interaction of events in the space environment.Methods A time-series dynamic data sampling method for large-scale space was proposed for sample detection data in space and time,and the corresponding relationships between data location features and other attribute features were established.A tone-mapping method based on statistical histogram equalization was proposed and applied to the final attribute feature data.The visualization process is optimized for rendering by merging materials,reducing the number of patches,and performing other operations.Results The results of sampling,feature extraction,and uniform visualization of the detection data of complex types,long duration spans,and uneven spatial distributions were obtained.The real-time visualization of large-scale spatial structures using augmented reality devices,particularly low-performance devices,was also investigated.Conclusions The proposed visualization system can reconstruct the three-dimensional structure of a large-scale space,express the structure and changes in the spatial environment using augmented reality,and assist in intuitively discovering spatial environmental events and evolutionary rules.

Augmented Reality Marketing:Breakthroughs and Opportunities

The impact of augmented reality(AR)technology on consumer behavior has increasingly attracted academic attention.While early research has provided valuable insights,many challenges remain.This article reviews recent studies,analyzing AR’s technical features,marketing concepts,and action mechanisms from a consumer perspective.By refining existing frameworks and introducing a new model based on situation awareness theory,the paper offers a deeper exploration of AR marketing.Finally,it proposes directions for future research in this emerging field.

航空发动机叶片装配执行过程智能检测及AR引导

为了提高航空发动机叶片装配执行过程的作业智能化程度,本文提出一种航空发动机叶片装配执行过程智能检测及AR引导方法,该方法包括叶片编码识别、物料AR出入库和齐套摆放过程状态检测3个环节。针对航空发动机叶片物料缺乏自动化识别和智能化纠错的问题,搭建基于编码识别的叶片物料管理架构,提出基于图像处理的叶片编码图像前处理增强操作,并利用贝叶斯纠错对识别结果进行正误判断和纠错校正的后处理操作,提高了叶片编码识别准确率;在物料人工出入库环节,利用AR增强可视化信息辅助用户快速执行作业任务,降低了叶片物料选取作业的时间;针对叶片物料齐套准备过程,构建了基于检测比对的防错纠错系统,避免发生人为错误。所提叶片装配执行过程智能检测及AR增强辅助引导方法可以有效减少人力物力和时间消耗,在推动航空发动机迈向智能化和自动化生产上起到技术支撑作用。

工业机器人教学实训平台的AR可视化应用设计

工业机器人教学实训平台运行机理较为复杂抽象,学生无法在有限的实训时间内完全理解平台整体运行机理,难以安全高效地完成实训任务。针对上述问题,提出以多层次架构设计实训平台AR可视化应用,在Unity3D中实现原型系统开发。通过KEPServerEX获取多源设备数据,以标志物注册配合空间锚点技术实现平台模型的跟踪注册。在Holo Lens2设备上将平台结构信息、运行状态以及操作指引等内容实现AR可视化,学生可通过手势、视点等多元手段获悉平台状态与实训指引。实践表明,工业机器人教学实训平台AR可视化应用能帮助学生把控平台运行状态,安全高效地完成实训任务。

基于AR技术的高档数控机床运维平台设计研究

为响应中国制造业智能转型需求,针对传统二维交互界面的数控机床运维系统在人机交互过程中容易造成信息认知偏差、维修错误等问题,研究探讨了AR技术的优势及其运用于高档数控机床运维领域的可能性,构建了一套基于AR技术的高档数控机床运维平台框架,并以Unity3D为载体,设计开发了基于AR技术的高档数控机床运维平台。实现了运维人员和数控机床之间面对面的AR人机交互,提升了数控机床运维的高效及智能管理效率。

VR/AR-AdaptFace:面向虚拟现实与增强现实的自适应多模态面部替换模型

随着VR/AR技术的迅猛发展,用户对于沉浸式体验的需求日益增长。同时,虚拟人脸技术亦趋成熟。基于此,本文探索将高度拟真的虚拟人脸融入VR/AR,以增强用户体验的自然度与沉浸感。然而,在虚拟数字人领域,图像生成及换脸技术在VR/AR环境下仍遇诸多挑战,尤其是唇形合成模型在动态场景及多语言环境下的性能需进一步优化。为解决上述问题,本文提出VR/AR-AdaptFace模型,一个面向虚拟现实与增强现实的自适应多模态面部替换方案。该模型由两大模块构成:“文颜绘真”模块,采用先进的文本至图像转换技术和特定类别先验保存策略,优化虚拟人脸生成,并通过注意力机制大幅提升图像质量;“语唇映生”模块,依托强大的生成器、唇形同步判别器及视觉质量判别器,实现语音与唇形的精准同步,为VR/AR场景中的动态交互带来更加逼真的体验。

基于移动增强现实关键技术(MAR)的工业遗址复原展现研究——以宁厂古镇三车间制盐遗址为例

数字化技术的快速发展为工业遗址复原展现提供了新的方法与途径。从工业遗址复原展现现状问题的研究入手,结合移动增强技术(MAR)的优点,以宁厂古镇三车间制盐遗址为例,提出数据采集、虚拟复原、构建MAR系统、发布与测试的复原展现思路,并予以实践。实践表明,移动增强技术可以为公众提供沉浸式、场景式的交互体验,有效促进工业遗址文化价值的传播与传承。验证了移动增强技术应用于工业遗址复原展现的可行性,以期为工业遗址复原展现工作提供实践参考。

基于AR技术的双重预防机制信息化平台

推行双重预防机制是解决安全生产领域“认不清”和“想不到”等突出问题的重要手段,但在企业实行风险分级管控与隐患排查治理过程中存在数据信息化滞后、数据信息缺失和数据真实性难以考证等问题,严重影响了双重预防机制的建设与运行。基于增强现实(augmented reality,AR)技术,结合Web应用开发、Android应用开发与WebRTC技术,以风险分级管控与隐患排查治理的双重预防机制为主要依据,搭建了包含网页端与AR眼镜端应用的实时信息化平台。经企业初步验证使用表明,该平台能够及时有效地激活双重预防机制的运行,提高企业对安全风险的分析决策效能以及落实安全主体责任。

Chinese expert consensus on laparoscopic hepatic segmentectomy and subsegmentectomy navigated by augmented-and mixed-reality technology combined with indocyanine green fluorescence imaging

Augmented-and mixed-reality technologies have pioneered the realization of real-time fusion and interactive projection for laparoscopic surgeries.Indocyanine green fluorescence imaging technology has enabled anatomical,functional,and radical hepatectomy through tumor identification and localization of target hepatic segments,driving a transformative shift in themanagement of hepatic surgical diseases,moving away from traditional,empirical diagnostic and treatment approaches toward digital,intelligent ones.The Hepatic Surgery Group of the Surgery Branch of the Chinese Medical Association,Digital Medicine Branch of the Chinese Medical Association,Digital Intelligent Surgery Committee of the Chinese Society of ResearchHospitals,and Liver Cancer Committee of the Chinese Medical Doctor Association organized the relevant experts in China to formulate this consensus.This consensus provides a comprehensive outline of the principles,advantages,processes,and key considerations associated with the application of augmented reality and mixed-reality technology combined with indocyanine green fluorescence imaging technology for hepatic segmental and subsegmental resection.The purpose is to streamline and standardize the application of these technologies.

Visualization of real-time displacement time history superimposed with dynamic experiments using wireless smart sensors and augmented reality

Wireless smart sensors(WSS)process field data and inform inspectors about the infrastructure health and safety.In bridge engineering,inspectors need reliable data about changes in displacements under loads to make correct decisions about repairs and replacements.Access to displacement information in the field and in real-time remains a challenge as inspectors do not see the data in real time.Displacement data from WSS in the field undergoes additional processing and is seen at a different location.If inspectors were able to see structural displacements in real-time at the locations of interest,they could conduct additional observations,creating a new,information-based,decision-making reality in the field.This paper develops a new,human-centered interface that provides inspectors with real-time access to actionable structural data during inspection and monitoring enhanced by augmented reality(AR).It summarizes and evaluates the development and validation of the new human-infrastructure interface in laboratory experiments.The experiments demonstrate that the interface that processes all calculations in the AR device accurately estimates dynamic displacements in comparison with the laser.Using this new AR interface tool,inspectors can observe and compare displacement data,share it across space and time,visualize displacements in time history,and understand structural deflection more accurately through a displacement time history visualization.

Defect inspection of indoor components in buildings using deep learning object detection and augmented reality

Visual inspection is commonly adopted for building operation,maintenance,and safety.The durability and defects of components or materials in buildings can be quickly assessed through visual inspection.However,implementations of visual inspection are substantially time-consuming,labor-intensive,and error-prone because useful auxiliary tools that can instantly highlight defects or damage locations from images are not available.Therefore,an advanced building inspection framework is developed and implemented with augmented reality(AR)and real-time damage detection in this study.In this framework,engineers should walk around and film every corner of the building interior to generate the three-dimensional(3D)environment through ARKit.Meanwhile,a trained YOLOv5 model real-time detects defects during this process,even in a large-scale field,and the defect locations indicating the detected defects are then marked in this 3D environment.The defects areas can be measured with centimeter-level accuracy with the light detection and ranging(LiDAR)on devices.All required damage information,including defect positions and sizes,is collected at a time and can be rendered in the 2D and 3D views.Finally,this visual inspection can be efficiently conducted,and the previously generated environment can also be loaded to re-localize existing defect marks for future maintenance and change observation.Moreover,the proposed framework is also implemented and verified by an underground parking lot in a building to detect and quantify surface defects on concrete components.As seen in the results,the conventional building inspection is significantly improved with the aid of the proposed framework in terms of damage localization,damage quantification,and inspection efficiency.

Three-dimensional automatic artificial intelligence driven augmented-reality selective biopsy during nerve-sparing robot-assisted radical prostatectomy:A feasibility and accuracy study

Objective:To evaluate the accuracy of our new three-dimensional(3D)automatic augmented reality(AAR)system guided by artificial intelligence in the identification of tumour’s location at the level of the preserved neurovascular bundle(NVB)at the end of the extirpative phase of nerve-sparing robot-assisted radical prostatectomy.Methods:In this prospective study,we enrolled patients with prostate cancer(clinical stages cT1ce3,cN0,and cM0)with a positive index lesion at target biopsy,suspicious for capsular contact or extracapsular extension at preoperative multiparametric magnetic resonance imaging.Patients underwent robot-assisted radical prostatectomy at San Luigi Gonzaga Hospital(Orbassano,Turin,Italy),from December 2020 to December 2021.At the end of extirpative phase,thanks to our new AAR artificial intelligence driven system,the virtual prostate 3D model allowed to identify the tumour’s location at the level of the preserved NVB and to perform a selective excisional biopsy,sparing the remaining portion of the bundle.Perioperative and postoperative data were evaluated,especially focusing on the positive surgical margin(PSM)rates,potency,continence recovery,and biochemical recurrence.Results:Thirty-four patients were enrolled.In 15(44.1%)cases,the target lesion was in contact with the prostatic capsule at multiparametric magnetic resonance imaging(Wheeler grade L2)while in 19(55.9%)cases extracapsular extension was detected(Wheeler grade L3).3D AAR guided biopsies were negative in all pathological tumour stage 2(pT2)patients while they revealed the presence of cancer in 14 cases in the pT3 cohort(14/16;87.5%).PSM rates were 0%and 7.1%in the pathological stages pT2 and pT3(<3 mm,Gleason score 3),respectively.Conclusion:With the proposed 3D AAR system,it is possible to correctly identify the lesion’s location on the NVB in 87.5%of pT3 patients and perform a 3D-guided tailored nerve-sparing even in locally advanced diseases,without compromising the oncological safety in terms of PSM rates.

互动仪式链下AR社交应用设计研究——以“物布空间”为例

探究AR社交应用的设计方法。以互动仪式链框架对AR社交应用进行剖析,通过典型案例“物布空间”进行研究论证,并结合分析归纳的设计特征,以互动仪式链视角提出了AR社交应用的设计策略。提出“虚拟与现实融合的互动场景,形成差异化社交情境”“虚拟信息为焦点的互动行为,链接持续性社交关系”“三维元素为特征的互动界面,形成沉浸式情感体验”3个维度的设计策略,为AR社交应用设计研究与实践提供指导。

Monocular Visual SLAM for Markerless Tracking Algorithm to Augmented Reality

Augmented Reality(AR)tries to seamlessly integrate virtual content into the real world of the user.Ideally,the virtual content would behave exactly like real objects.This necessitates a correct and precise estimation of the user’s viewpoint(or that of a camera)with regard to the virtual content’s coordinate sys-tem.Therefore,the real-time establishment of 3-dimension(3D)maps in real scenes is particularly important for augmented reality technology.So in this paper,we integrate Simultaneous Localization and Mapping(SLAM)technology into augmented reality.Our research is to implement an augmented reality system without markers using the ORB-SLAM2 framework algorithm.In this paper we propose an improved method for Oriented FAST and Rotated BRIEF(ORB)feature extraction and optimized key frame selection,as well as the use of the Progressive Sample Consensus(PROSAC)algorithm for planar estimation of augmented reality implementations,thus solving the problem of increased sys-tem runtime because of the loss of large amounts of texture information in images.In this paper,we get better results by comparing experiments and data analysis.However,there are some improved methods of PROSAC algorithm which are more suitable for the detection of plane feature points.

基于AR技术的植保机作业布局与优化分析

以植保机作业优化仿真过程为研究对象,基于AR增强现实技术对植保机的转向、行进及作业过程进行仿真,构建植保机优化作业布局。通过AR增强现实对植保机进行识别,利用3dsMax构建植保机三维虚拟仿真模型,在Unity3D模块中对植保机作业环境进行虚拟搭建,以提升植保机作业仿真的真实性。

基于感觉统合理论的AR技术辅助自闭症儿童出行系统设计

旨在基于感觉统合理论指导AR技术参与辅助自闭症儿童出行系统的设计实践。文章从相关概念界定出发,论证感觉统合理论与AR技术介入辅助自闭症儿童出行系统设计的可行性,通过对自闭症儿童及自闭症儿童家长的用户研究,总结设计痛点并整合出设计原则,进而建立设计策略。进行了感觉统合理论下的AR技术辅助自闭症儿童出行系统设计实践加以佐证。以期帮助自闭症儿童更好地适应出行场景、融入社会生活。

AR技术在铁路牵引供电故障检测中的应用

智能技术以解放双手、高效交互及辅助决策为理念,其中AR凭借其所拥有的独特优势,可以广泛应用到智慧医疗、设备检修、金融服务等行业,有效提高行业内的工作效率和企业的效能。该文介绍AR技术在铁路牵引供电故障检测中的应用,并具体提出一种基于AR技术的铁路牵引供电运维检测装置与系统设计框架,使用效果表明,利用此装置与系统可以实现设备巡检的智能化,能够有效提高设备巡检的准确率和效率。

阵列波导AR显示系统出瞳面光均匀性优化及漏光抑制

阵列波导增强现实显示系统部分反射镜反射率参数组合多,手动优化耗时且难以获得优化的参数组合,导致显示均匀性不佳。为此结合Python与LightTools对阵列波导各个部分反射镜反射率进行自动优化,获得优化的反射率参数组合;基于人眼角分辨率极限要求,利用蒙特卡罗方法对波导基板上下表面和部分反射镜阵列的平行公差要求进行仿真分析,在此基础上优化设计了阵列波导显示系统,提出了在阵列波导出光面镀增透膜的漏光抑制方法,进而开发了视场角为40°、出瞳均匀性达83%的阵列波导增强现实显示样机。样机在颜色均一性和还原性等指标方面有显著优势,实现了高出瞳面光均匀性,且明显降低了漏光。