Mobile Web Augmented Reality in 5G and Beyond:Challenges, Opportunities, and Future Directions

The popularity of wearable devices and smartphones has fueled the development of Mobile Augmented Reality(MAR),which provides immersive experiences over the real world using techniques,such as computer vision and deep learning.However,the hardware-specific MAR is costly and heavy,and the App-based MAR requires an additional download and installation and it also lacks cross-platform ability.These limitations hamper the pervasive promotion of MAR.This paper argues that mobile Web AR(MWAR)holds the potential to become a practical and pervasive solution that can effectively scale to millions of end-users because MWAR can be developed as a lightweight,cross-platform,and low-cost solution for end-to-end delivery of MAR.The main challenges for making MWAR a reality lie in the low efficiency for dense computing in Web browsers,a large delay for real-time interactions over mobile networks,and the lack of standardization.The good news is that the newly emerging 5G and Beyond 5G(B5G)cellular networks can mitigate these issues to some extent via techniques such as network slicing,device-to-device communication,and mobile edge computing.In this paper,we first give an overview of the challenges and opportunities of MWAR in the 5G era.Then we describe our design and development of a generic service-oriented framework(called MWAR5)to provide a scalable,flexible,and easy to deploy MWAR solution.We evaluate the performance of our MWAR5 system in an actually deployed 5G trial network under the collaborative configurations,which shows encouraging results.Moreover,we also share the experiences and insights from our development and deployment,including some exciting future directions of MWAR over 5G and B5G networks.

3D Mobile Augmented Reality Interface for Laboratory Experiments

In fields such as science and engineering, virtual environment is commonly used to provide replacements for practical hands-on laboratories. Sometimes, these environments take the form of a remote interface to the physical laboratory apparatus and at other times, in the form of a complete software implementation that simulates the laboratory apparatus. In this paper, we report on the use of a semi-immersive 3D mobile Augmented Reality (mAR) interface and limited simulations as a replacement for practical hands-on laboratories in science and engineering. The 3D-mAR based interfaces implementations for three different experiments (from micro-electronics, power and communications engineering) are presented;the discovered limitations are discussed along with the results of an evaluation by science and engineering students from two different institutions and plans for future work.

An Extended Application form of Mobile Computing ── Augmented Reality

With the rapid developments of wireless communication and microelectronic technology, the bandwidth of wireless communication is becoming wider than ever, up to 100Gbps and the computer can be designed as small as a match with powerful computing and controlling capability. These rapid developments have extended the mobile computing. There are many application forms of mobile computing, such as mobile databases, mobile data management, wearable computing etc. A great branch of mobile computing, Augmented Reality (AR), which is the combination of mobile computing and wearable computers was discussed.

Design and Implementation of Museum Educational Content Based on Mobile Augmented Reality

In the digital age,museums are becoming increasingly integrated with media technology.The interactive mode of museum education brought about by new digital communication technology can increase the audience’s participation and interest in museum education content.This paper attempts to use Mobile Augment Reality(MAR)technology to design museum education content under the guidance of the principle of abstraction hierarchy from the theory of education optimization.With the help of MAR technology,museum education content can move up and down at different levels of information abstraction.The purpose is to help the audience move between perceptual knowledge and rational knowledge and improve their ability to understand.The evaluation results show that this style of interactive teaching content can promote visitors’understanding,thinking and interest in the cultural content of museum exhibits.It is also can helps visitors use a more professional-level vocabulary and more detailed descriptions about the cultural content of museum exhibits.The design and implementation method detailed in this paper has a certain guiding value for museum education development as well as utilization of MAR technology.

Outdoor guide system based on the mobile augmented reality technology

This paper proposes an outdoor guide system using vision-based augmented reality(AR) on mobile devices.Augmented reality provides a virtual-real fusion display interface for outdoor guide.Vision-based methods are more accurate than GPS or other hardware-based methods.However,vision-based methods require more resources and relatively strong computing power of mobile devices.A C/S framework for vision based augmented reality system is introduced in this paper.In a server,a vocabulary tree is used for location recognition.In a mobile device,BRISK feature is combined with optical flow methods to track the offline keyframe.The system is tested on UKbench datasets and in real environment.Experimental results show that the proposed vision-based augmented reality system works well and yields relatively high recognition rate and that the mobile device achieves realtime recognition performance.

Realization of Mobile Augmented Reality System Based on Image Recognition

With the development of computation technology,the augmented reality(AR)is widely applied in many fields as well as the image recognition.However,the AR application on mobile platform is not developed enough in the past decades due to the capability of the mobile processors.In recent years,the performance of mobile processors has changed rapidly,which makes it comparable to the desktop processors.This paper proposed and realized an AR system to be used on the Android mobile platform based on the image recognition through EasyAR engine and Unity 3D development tools.In this system,the image recognition could be done locally and/or using cloud recognition.Test results show that the cloud-based recognition is more efficient and accuracy than the local recognition for the mobile AR when there are more images to be recognized at the same time.

Rise of Augmented Reality: Current and Future Application Areas

The massive technological advancements around the world have created significant challenging competition among companies where each of the companies tries to attract the customers using different techniques. One of the recent tech- niques is Augmented Reality (AR). The AR is a new technology which is capable of presenting possibilities that are difficult for other technologies to offer and meet. Nowadays, numerous augmented reality applications have been used in the industry of different kinds and disseminated all over the world. AR will really alter the way individuals view the world. The AR is yet in its initial phases of research and development at different colleges and high-tech institutes. Throughout the last years, AR apps became transportable and generally available on various devices. Besides, AR be- gins to occupy its place in our audio-visual media and to be used in various fields in our life in tangible and exciting ways such as news, sports and is used in many domains in our life such as electronic commerce, promotion, design, and business. In addition, AR is used to facilitate the learning whereas it enables students to access location-specific infor- mation provided through various sources. Such growth and spread of AR applications pushes organizations to compete one another, every one of them exerts its best to gain the customers. This paper provides a comprehensive study of AR including its history, architecture, applications, current challenges and future trends.

基于移动增强现实关键技术(MAR)的工业遗址复原展现研究——以宁厂古镇三车间制盐遗址为例

数字化技术的快速发展为工业遗址复原展现提供了新的方法与途径。从工业遗址复原展现现状问题的研究入手,结合移动增强技术(MAR)的优点,以宁厂古镇三车间制盐遗址为例,提出数据采集、虚拟复原、构建MAR系统、发布与测试的复原展现思路,并予以实践。实践表明,移动增强技术可以为公众提供沉浸式、场景式的交互体验,有效促进工业遗址文化价值的传播与传承。验证了移动增强技术应用于工业遗址复原展现的可行性,以期为工业遗址复原展现工作提供实践参考。

Three-dimensional CityGML building models in mobile augmented reality: a smartphone-based pose tracking system

Many augmented reality(AR)systems are developed for entertainment,but AR and particularly mobile AR potentially have more application possibilities in other fields.For example,in civil engineering or city planning,AR could be used in combination with CityGML building models to enhance some typical workflows in planning,execution and operation processes.A concrete example is the geo-referenced on-site visualization of planned buildings or building parts,to simplify planning processes and optimize the communication between the participating decision-makers.One of the main challenges for the visualization lies in the pose tracking,i.e.the real-time estimation of the translation and rotation of the mobile device to align the virtual objects with reality.In this paper,we introduce a proof-of-concept fine-grained mobile AR CityGML-based pose tracking system aimed at the mentioned applications.The system estimates poses by combining 3D CityGML data with information derived from 2D camera images and an inertial measurement unit and is fully self-sufficient and operates without external infrastructure.The results of our evaluation show that CityGML and low-cost off-the-shelf mobile devices,such as smartphones,already provide performant and accurate mobile pose tracking for AR in civil engineering and city planning.

Mixed reality head mounted displays for enhanced indoor point cloud segmentation with virtual seeds

Mixed Reality(MR)Head Mounted Displays(HMDs)offer a hitherto underutilized set of advantages compared to conventional 3D scanners.These benefits,inherent to MR-HMDs albeit not originally intended for such appli-cations,encompass the freedom of hand movement,hand tracking capabilities,and real-time mesh visualization.This study leverages these attributes to enhance indoor scanning process.The primary innovation lies in the con-ceptualization of manual-positioned MR virtual seeds for the purpose of indoor point cloud segmentation via a region-growing approach.The proposed methodology is effectively implemented using the HoloLens 2 platform.An application is designed to enable the remote placement of virtual tags based on the user’s visual focus on the MR-HMD display.This non-intrusive interface is further enriched with expedited tag saving and deletion functionalities,as well as augmented tag visualization through overlaying them on real-world objects.To assess the practicality of the proposed method,a comprehensive real-world case study spanning an area of 330 s^(2) is conducted.Remarkably,the survey demonstrates remarkable efficiency,with 20 virtual tags swiftly deployed,each requiring a mere 2 s for precise positioning.Subsequently,these virtual tags are employed as seeds in a region-growing algorithm for point cloud segmentation.The accuracy of virtual tag positioning is found to be exceptional,with an average error of 2.4±1.8 cm.Importantly,the user experience is significantly enhanced,leading to improved seed positioning and,consequently,more accurate final segmentation results.

A Geometry-Based Point Cloud Reduction Method for MobileAugmented Reality System

In this paper, a geometry-based point cloud reduction method is proposed, and a real-time mobile augmentedreality system is explored for applications in urban environments. We formulate a new objective function which combinesthe point reconstruction errors and constraints on spatial point distribution. Based on this formulation, a mixed integerprogramming scheme is utilized to solve the points reduction problem. The mobile augmented reality system explored inthis paper is composed of the ofttine and online stages. At the offline stage, we build up the localization database usingstructure from motion and compress the point cloud by the proposed point cloud reduction method. While at the onlinestage, we compute the camera pose in real time by combining an image-based localization algorithm and a continuous posetracking algorithm. Experimental results on benchmark and real data show that compared with the existing methods, thisgeometry-based point cloud reduction method selects a point cloud subset which helps the image-based localization methodto achieve higher success rate. Also, the experiments conducted on a mobile platform show that the reduced point cloud notonly reduces the time consuming for initialization and re-initialization, but also makes the memory footprint small, resultinga scalable and real-time mobile augmented reality system.

基于MAR的非物质文化遗产数字化保护系统研究

针对目前非物质文化遗产保护形式单一、交互性差的问题,基于移动增强现实技术设计并实现了一个非遗数字化在线展示系统。以泥咕咕非遗保护为例,设计系统云端和移动端的主要功能模块,提出基于三维扫描和网格迭代简化相结合的快速建模方法,采用ORB-FV和光流跟踪算法实现图像的在线识别和跟踪注册,基于Unity3D完成系统开发。测试结果表明,该系统不仅满足了非遗产品移动端增强展示的真实感和流畅度要求,而且实现了模型增强显示后的实时交互涂色展示功能,增强了非遗保护和传播的趣味性和互动性。

DRAM:A DRL-based resource allocation scheme for MAR in MEC

Mobile Edge Computing(MEC)and 5G technology allow clients to access computing resources at the network frontier,which paves the way for applying Mobile Augmented Reality(MAR)applications.Under the MEC paradigm,MAR clients can offload complex tasks to the MEC server and enhance the human perception of the world by merging the received virtual information with the real environment.However,the resource allocation problem arises as a critical challenge in circumstances where several MAR clients compete for limited resources at the network frontier.In this paper,we aim to design an online resource allocation scheme on the MEC server that takes both high quality of experience and good fairness performance for MAR clients into consideration.We first formulate this problem as a Markov decision process and tackle the challenge of applying the deep reinforcement learning paradigm.Then,we propose DRAM,a Deep reinforcement learning-based Resource allocation scheme for mobile Augmented reality service in MEC.We also propose a self-adaptive algorithm on the MAR client that is derived based on the analysis of the MAR service to tackle client adaptation problems.The simulation results demonstrated that DRAM can provide high quality of experience and simultaneously achieve good fairness performance by coordinating with clients’adaptation algorithms.

A Web Service for Managing Spatial Context Dedicated to Serious Games on and for Smartphones

The GeoEduc3D project aims to provide educational games for smartphones based on Geomatics and use augmented reality techniques in order to make these games more immersive. To improve the immersive and interactive aspects of those games, we focused on the exploitation of spatial context in this particular application framework (serious games, augmented reality, smart phones, and multi-users environment). Our work has thus led to the design of a solution dedicated to the management of spatial context in a multi-players environment on and for smartphones. Several contributions have been made: modeling spatial context, proposing a service-oriented architecture to manage this context, defining a Web Service Spatial Context (WSCS) and implementation of a WSCS prototype and a mobile client according to an environment exploiting FourSquare, a geo-social application.

面向元宇宙移动增强现实应用的异质内容 主动缓存与个性化交付机制

针对移动增强现实(MAR)应用异质数据内容传输和响应时延的问题,提出了面向元宇宙移动增强现实应用的异质内容主动缓存与个性化交付机制,首先综合考虑了用户特征和边缘节点服务能力,提出了用户行为和资源感知的边缘协作服务域构建方法;进而,基于协作服务域设计基于存储空间划分和用户偏好预测的异质内容预缓存机制;特别地,针对前景内容引入了个性化推荐机制,并设计了前景内容的差异化策略。仿真结果表明,所提机制在缓存命中率、前景内容和异质内容的平均响应时延方面均优于NCPCR策略、CCS-AGP策略和AIEC-RSC策略。

实景移动增强现实的实验室安全教育应用构建

实验安全是实验教学正常运行的根本,在实践性教学中发挥重要作用。然而,目前的实验室安全教育体系由于教学手段单一、教学内容缺乏适应性,以及考核较为抽象等问题,影响了实验室安全教育的效果。针对目前实验室安全教育的不足,提出了面向实验室安全教育内容的移动增强现实学习应用。在实验场所实景构建以移动增强现实为交互手段的自主化、探索化学习过程。借助移动增强现实的富交互学习行为,通过多维度数据反馈以及安全教育测试系统构建实验室教学的安全准入机制,保障高校实验室运行安全。

磁共振成像影像导航及靶区选择在经颅磁刺激治疗中的应用

背景:在经颅磁刺激临床实际应用中,是否能准确瞄准刺激脑区可影响经颅磁刺激的实际治疗效果。近年来,随着神经导航系统、移动式增强现实技术的发展,以及多种不同模态磁共振成像数据分析方法的出现,刺激靶区定位的准确性和靶区选择的优化有望得到进一步的提高。目的:综述基于磁共振成像的影像导航定位原理及在经颅磁刺激中的应用,归纳不同模态磁共振成像数据分析在指导经颅磁刺激靶区选择上的作用。方法:应用计算机在PubMed、中国知网和万方数据库检索相关文献,以“transcranial magnetic stimulation,coil positioning,neuronavigation,augmented reality,magnetic resonance imaging,经颅磁刺激,线圈定位,神经导航,增强现实,核磁共振成像,原理”为主要检索词,最终纳入63篇文献进行分析。结果与结论:①传统经颅磁刺激线圈放置方法中,“5 cm规则”、国际脑电10-20定位法最为常用,这些方法具有简便、经济的优点,但过于依赖操作者的经验,存在不同操作者的技术差异。②基于立体定向技术发展而来的神经导航系统是目前可视化程度及准确度最高的辅助经颅磁刺激线圈放置的引导方式,它通过MRI数据采集、脑三维重建、头模配准及立体几何定位等步骤来实现可视化定位,具有较高的临床治疗及科研应用价值,但因其设备较为昂贵,目前在医疗机构中尚未普及。③对于不同层次的医疗单位,移动式移动式增强现实技术不失为一种经济、高效的神经导航系统替代方案,它通过磁共振成像数据采集、构建二维/三维图像、虚拟图像与真实脑图像叠加来实现头皮下脑组织可视化定位,具有直观性强、成本低廉的优势,可在基层医疗单位推广应用。④尽管相对脑电10-20定位策略,可视化定位的临床疗效优越性目前尚未充分体现,但随着研究者对人脑多种不同模态磁共振成像扫描数据的挖掘,有望进一步优化经颅磁刺激治疗靶区选择的策略,提高经颅磁刺激治疗应答率和个性化程度,这在将来是极具潜力和挑战的研究方向。

手机室内场景要素实例化现实增强方法研究进展

室内手机导航与位置服务是当前的研究热点,场景要素实例化现实增强方法是其中重要的组成部分。实例分割是场景要素感知中一项具有挑战性的基本任务,增强现实是数字孪生建筑物地图应用的有效途径,两者在室内定位导航领域有着重要意义。当前,增强现实技术主要应用在对场景中的语义增强,智能手机室内导航AR增强也只是停留在视觉可视化效果方面,尚没有真正深入到对场景中要素实例的增强层面。针对该问题,提出手机场景要素实例化AR研究思路,通过识别室内场景中的对象并与建筑物地图进行匹配,将建筑物地图中对应存储的要素信息利用AR技术进行增强显示,进而辅助行人进行室内导航与位置服务等相关应用,提升用户室内定位导航等位置服务的信息化水平。对智能手机端视频的实例分割和增强现实方法进行了系统的梳理,并分析了相关方法的特点和适用场景,总结了移动端实例分割和增强现实的研究进展,最后探讨了室内场景要素实例化现实增强方法在导航与位置服务领域的应用前景。

基于可见光逆反射通信的移动增强现实定向交互

移动增强现实应用中,用户频繁地与环境中不同的智能物体交互完成任务,其完成效率和用户体验由所采用的定向交互技术决定.然而,从交互手段上来说,现阶段定向交互依赖于Wi-Fi、BLE等无线技术,其信号全向传播的特点使其不能利用用户的空间环境(即位置和方向)来缩短互动时间,带来了不必要的时间成本;从交互界面上来说,现阶段基于视觉的界面存在可靠性低、扩展性低等问题,进一步限制了系统的通用性.本文介绍了RetroAR——基于可见光逆反射通信的定向交互系统.RetroAR利用了光的定向传播特性来保留用户的空间环境,并依靠逆反射通信实现用户与目标设备之间无连接的快速定向交互.系统实验表明,RetroAR最远可在4 m距离支持100°视角的交互,同时实现厘米级的六自由度(6-Degrees Of Freedom,6-DoF)三维跟踪.用户研究表明,与基于Wi-Fi的解决方案相比,RetroAR将非接触式控制的交互时间减少了2倍,并且具有更好的用户体验.RetroAR借助可见光逆反射通信来利用用户空间环境,保持交互过程中的直观性.用户可以“所指即所控”的方式与多个目标进行互动,实现类似自然交互的快速定向交互.