Rectification of magnetic force tracker using neural network inaugmented reality system

Nonlinear errors always exist in data obtained from tracker in augmented reality （AR）, which badly influence the effect of AR. This paper proposes to rectify the errors using BP neural network. As BP neural network is prone to getting into local extrema and convergence is slow, genetic algorithm is employed to optimize the initial weights and threshold of neural network. This paper discusses how to set the crucial parameters in the algorithm. Experimental results show that the method ensures that the neural network achieves global convergence quickly and correctly. Tracking precision of AR system is improved after the tracker is rectified, and the third dimension of AR system is enhanced.

基于Arduino的AR智能头盔设计与开发

文章基于Arduino的增强现实(Augmented Reality,AR)技术设计并开发了智能头盔,主要工作包括硬件设计、软件设计、测试与验证。设计的AR智能头盔可以对现实场景实时AR增强,并支持用户个性化配置。文章旨在提供一种低成本、易制作、高可定制化的AR智能头盔解决方案。

新零售消费趋势下基于AR技术的中式卷烟营销研究

近年来移动互联网技术高速发展,随之而来的零售业态面临着强竞争性迭代升级。新零售背景下,零售业的线上线下的快速融合成为必然趋势,对于中式卷烟品牌培育而言,依托线上线下融合下的体验营销对于卷烟品牌培育有着至关重要的作用。在新零售背景下,应用增强现实技术(Augmented Reality,AR)等新型技术,创新体验营销场景与体验策略,创建提升卷烟消费者的产品感知度、体验度、友好度的新型方式,进而有效增强其对卷烟品牌的认同度并扩展为有效的社交裂变,探索新零售发展背景下中式卷烟体验营销的新模式。因此文章将体验营销作为关键出发点,探索以AR为场景开发方式,以中式卷烟品牌培育为核心,提出优化卷烟消费者用户体验的针对性建议。

基于AR技术的卷烟营销研究

近年来移动互联网技术高速发展,新消费环境下零售业态面临着竞争迭代升级,零售业线上线下的快速融合成为必然趋势,对于中式卷烟品牌培育而言,依托线上线下融合下的体验营销对于卷烟品牌培育有至关重要的影响。本文着重开展了体验营销场景与体验策略创新研究,通过应用增强现实技术(AR,Augmented Reality)等新型技术,创建提升卷烟消费者的产品感知度、体验度、友好度的新型方式,进而增强消费者对卷烟品牌的认同度并扩展为有效的社交裂变,探索新零售发展背景下中式卷烟体验营销的新模式。并在QD市场开展实证研究,以体验营销作为关键出发点,以中式卷烟品牌培育为核心,打造以AR为场景的特色市场开发方式,制定并实施不断优化卷烟消费者用户体验的针对性策略。

Study on Robot Grasping System of SSVEP-BCI Based on Augmented Reality Stimulus

Although notable progress has been made in the study of Steady-State Visual Evoked Potential(SSVEP)-based Brain-Computer Interface(BCI),several factors that limit the practical applications of BCIs still exist.One of these factors is the importability of the stimulator.In this study,Augmented Reality(AR)technology was introduced to present the visual stimuli of SSVEP-BCI,while the robot grasping experiment was designed to verify the applicability of the AR-BCI system.The offline experiment was designed to determine the best stimulus time,while the online experiment was used to complete the robot grasping task.The offline experiment revealed that better information transfer rate performance could be achieved when the stimulation time is 2 s.Results of the online experiment indicate that all 12 subjects could control the robot to complete the robot grasping task,which indicates the applicability of the AR-SSVEP-humanoid robot(NAO)system.This study verified the reliability of the AR-BCI system and indicated the applicability of the AR-SSVEP-NAO system in robot grasping tasks.

A comprehensive survey of augmented reality assembly research

In the past two decades, augmented reality （AR） has received a growing amount of attention by researchers in the manufacturing technology community, because AR can be applied to address a wide range of problems throughout the assembly phase in the lifecycle of a product, e.g., planning, design, ergonomics assessment, operation guidance and training. However, to the best of authors＇ knowledge, there has not been any comprehensive review of AR-based assembly systems. This paper aims to provide a concise overview of the technical features, characteristics and broad range of applications of AR- based assembly systems published between 1990 and 2015. Among these selected articles, two thirds of them were published between 2005 and 2015, and they are considered as recent pertinent works which will be discussed in detail. In addition, the current limitation factors and future trends in the development will also be discussed.

Novel Visualization Tool for Percutaneous Renal Puncture Training Using Augmented Reality Technology

We aim to develop a novel visualization tool for percutaneous renal puncture training based on augmented reality(AR)and compare the needle placement performance of this AR system with ultrasound-guidedfreehand navigation in phantoms.A head-mounted display-based AR navigation system was developed usingthe Unity3D software and Visual Studio to enable the overlay of the preoperative needle path and the complexanatomical structures onto a phantom in real time.The spatial location of the stationary phantom and the percutaneous instrument motion were traced by a Qualisys motion capture system.To evaluate the tracking accuracy,15 participants(7 males and 8 females)performed a single needle insertion using AR navigation(the number ofpunctures n=75)and ultrasound-guided freehand navigation(n=75).The needle placement error was measuredas the Euclidean distance between the actual needle tip and the virtual target by MicronTracker.All participantsdemonstrated a superior needle insertion efficiency when using the AR-assisted puncture method compared withthe ultrasound-guided freehand method.The needle insertion error of the ultrasound-guided method showed anincreased error compared with the AR method(5.54 mm±2.59 mm,4.34 mm±2.10 mm,respectively,p<0.05).The ultrasound-guided needle placements showed an increased time compared with the AR method(19.08 s±3.59 s,15.14 s±2.72 s,respectively,p<0.0001).Our AR training system facilitates the needle placement performance and solves hand-eye coordination problems.The system has the potential to increase efficiency andeffectiveness of percutaneous renal puncture training.

虚实结合的《移动通信》在线课程设计与实践

在5G技术、人工智能、增强现实等新技术迅猛发展的背景下,从改革高职在线课堂教学手段出发,构建了“虚实结合、云端协同”的《移动通信》课程教学改革实施方案。针对传统教学方式抽象、枯燥,理论教学与实践教学相脱节等问题,以《移动通信》课程为例,融入虚拟可视化教学手段,构建云(部署于云中心的精品在线开放课程及相关资源)-端(应用于端侧的课堂教学实践)协同架构,开发可视化体验式的虚实结合的课堂教学内容和教学资源,满足高职扩招背景下虚实结合的远程在线课程教学的需要。

“移动通信”课程虚拟可视化教学实践

本文基于高职教学模式改革发展的需求,围绕虚拟可视化辅助教学模式,以“移动通信”课程为例,分析该课程开展虚拟可视化教学的可行性,重构课程并开发可视化体验式课堂教学内容,构建虚拟可视化素材库,设计虚拟可视化互动系统,形成与课程结构相匹配的基于工作过程的虚实结合的教学方法,促进工学结合等实践教学模式的变革,赋予“互联网+职业教育”新的内涵,以期为其他专业课实施虚拟可视化教学提供启发与借鉴。

Camera localization for augmented reality and indoor positioning:a vision-based 3D feature database approach

The recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of Augmented Reality(AR),robotics,indoor GIS and self-driving.Camera localization is often a key and enabling technology among these applications.In this paper,we developed a novel camera localization workflow based on a highly accurate 3D prior map optimized by our RGBD SLAM method in conjunction with a deep learning routine trained using consecutive video frames labeled with high precision camera pose.Furthermore,an AR registration method tightly coupled with a game engine is proposed,which incorporates the proposed localization algorithm and aligns the real Kinetic camera with a virtual camera of the game engine to facilitate AR application development in an integrated manner.The experimental results show that the localization accuracy can achieve an average error of 35 cm based on a fine-tuned prior 3D feature database at 3 cm accuracy compared against the ground-truth 3D LiDAR map.The influence of the localization accuracy on the visual effect of AR overlay is also demonstrated and the alignment of the real and virtual camera streamlines the implementation of AR fire emergency response demo in a Virtual Geographic Environment.

Haptic and visual augmented reality interface for programming welding robots

It is a challenging task for operators to program a remote robot for welding manipulation depending only on the visual information from the remote site. This paper proposes an intuitive user interface for programming welding robots remotely using augmented reality （AR） with haptic feedback. The proposed system uses a depth camera to reconstruct the surfaces of workpieces. A haptic input device is used to allow users to define welding paths along these surfaces. An AR user interface is developed to allow users to visualize and adjust the orientation of the welding torch. Compared with the traditional robotic welding path programming methods which rely on prior CAD models or contact between the robot end-effector and the workpiece, this proposed approach allows for fast and intuitive remote robotic welding path programming with- out prior knowledge of CAD models of the workpieces. The experimental results show that the proposed approach is a user-friendly interface and can assist users in obtaining an accurate welding path.

Shared experiences across multiple devices:a new digital interactive experience method for cultural heritage based on mixed reality

In order to study the role of the new technological concept of shared experiences in the digital interactive experience of cultural heritage and apply it to the digital interactive experience of cultural heritage to solve the current problems in this field,starting from the mixed reality(MR) technology that the shared experiences rely on,proper software and hardware platforms were investigated and selected,a universal shared experiences solution was designed,and an experimental project based on the proposed solution was made to verify its feasibility.In the end,a proven and workable shared experiences solution was obtained.This solution included a proposed MR spatial alignment method,and it integrated the existing MR content production process and standard network synchronization functions.Furthermore,it is concluded that the introduction and reasonable use of new technologies can help the development of the digital interactive experience of cultural heritage.The shared experiences solution for the digital interactive experience of cultural heritage balances investment issues in the exhibition,display effect,and user experience.It can speed up the promotion of cultural heritage and bring the vitality of MR technology to relevant projects.

AR assistance for efficient dynamic target search

When searching for a dynamic target in an unknown real world scene,search efficiency is greatly reduced if users lack information about the spatial structure of the scene.Most target search studies,especially in robotics,focus on determining either the shortest path when the target’s position is known,or a strategy to find the target as quickly as possible when the target’s position is unknown.However,the target’s position is often known intermittently in the real world,e.g.,in the case of using surveillance cameras.Our goal is to help user find a dynamic target efficiently in the real world when the target’s position is intermittently known.In order to achieve this purpose,we have designed an AR guidance assistance system to provide optimal current directional guidance to users,based on searching a prediction graph.We assume that a certain number of depth cameras are fixed in a real scene to obtain dynamic target’s position.The system automatically analyzes all possible meetings between the user and the target,and generates optimal directional guidance to help the user catch up with the target.A user study was used to evaluate our method,and its results showed that compared to free search and a top-view method,our method significantly improves target search efficiency.

Progress in self-powered sensors—Moving toward artificial intelligent and neuromorphic system

Wearable and flexible electronics are shaping our life with their unique advantages of light weight,good compliance,and desirable comfortability.With marching into the era of Internet of Things(IoT),numerous sensor nodes are distributed throughout networks to capture,process,and transmit diverse sensory information,which gives rise to the demand on self-powered sensors to reduce the power consumption.Meanwhile,the rapid development of artificial intelligence(AI)and fifth-generation(5G)technologies provides an opportunity to enable smart-decision making and instantaneous data transmission in IoT systems.Due to continuously increased sensor and dataset number,conventional computing based on von Neumann architecture cannot meet the needs of brain-like high-efficient sensing and computing applications anymore.Neuromorphic electronics,drawing inspiration from the human brain,provide an alternative approach for efficient and low-power-consumption information processing.Hence,this review presents the general technology roadmap of self-powered sensors with detail discussion on their diversified applications in healthcare,human machine interactions,smart homes,etc.Via leveraging AI and virtual reality/augmented reality(VR/AR)techniques,the development of single sensors to intelligent integrated systems is reviewed in terms of step-by-step system integration and algorithm improvement.In order to realize efficient sensing and computing,brain-inspired neuromorphic electronics are next briefly discussed.Last,it concludes and highlights both challenges and opportunities from the aspects of materials,minimization,integration,multimodal information fusion,and artificial sensory system.

Novel AR-based interface for human-robot interaction and visualization

Intuitive and efficient interfaces for human- robot interaction （HRI） have been a challenging issue in robotics as it is essential for the prevalence of robots supporting humans in key areas of activities. This paper presents a novel augmented reality （AR） based interface to facilitate human-virtual robot interaction. A number of human-virtual robot interaction methods have been for- mulated and implemented with respect to the various types of operations needed in different robotic applications. A Euclidean distance-based method is developed to assist the users in the interaction with the virtual robot and the spatial entities in an AR environment. A monitor-based visualization mode is adopted as it enables the users to perceive the virtual contents associated with different interaction methods, and the virtual content augmented in the real environment is informative and useful to the users during their interaction with the virtual robot. Case researches are presented to demonstrate the successful implementation of the AR-based HRI interface in planning robot pick-and-place operations and path following operations.

Robust camera pose estimation by viewpoint classification using deep learning

Camera pose estimation with respect to target scenes is an important technology for superimposing virtual information in augmented reality(AR). However, it is difficult to estimate the camera pose for all possible view angles because feature descriptors such as SIFT are not completely invariant from every perspective. We propose a novel method of robust camera pose estimation using multiple feature descriptor databases generated for each partitioned viewpoint, in which the feature descriptor of each keypoint is almost invariant. Our method estimates the viewpoint class for each input image using deep learning based on a set of training images prepared for each viewpoint class. We give two ways to prepare these images for deep learning and generating databases. In the first method, images are generated using a projection matrix to ensure robust learning in a range of environments with changing backgrounds.The second method uses real images to learn a given environment around a planar pattern. Our evaluation results confirm that our approach increases the number of correct matches and the accuracy of camera pose estimation compared to the conventional method.

State-of-the-art survey on digital twin implementations

Digital twin(DT)has garnered attention in both industry and academia.With advances in big data and internet of things(IoTs)technologies,the infrastructure for DT implementation is becoming more readily available.As an emerging technology,there are both potential and challenges.DT is a promising methodology to leverage the modern data explosion to aid engineers,managers,healthcare experts and politicians in managing production lines,patient health and smart cities by providing a comprehensive and high fidelity monitoring,prognostics and diagnostics tools.New research and surveys into the topic are published regularly,as interest in this technology is high although there is a lack of standardization to the definition of a DT.Due to the large amount of information present in a DT system and the dual cyber and physical nature of a DT,augmented reality(AR)is a suitable technology for data visualization and interaction with DTs.This paper seeks to classify different types of DT implementations that have been reported,highlights some researches that have used AR as data visualization tool in DT,and examines the more recent approaches to solve outstanding challenges in DT and the integration of DT and AR.

AR-assisted robot welding programming

Robotic welding demands high accuracy and precision.However,robot programming is often a tedious and time-consuming process that requires expert knowledge.This paper presents an augmented reality assisted robot welding task programming(ARWP)system using a user-friendly augmented reality(AR)interface that simplifies and speeds up the programming of robotic welding tasks.The ARWP system makes the programming of robot welding tasks more user-friendly and reduces the need for trained programmers and expertise in specific robotic systems.The AR interface simplifies the definition of a welding path as well as the welding gun orientation,and the system;the system can locate the welding seam of a workpiece quickly and generate a viable welding path based on the user input.The developed system is integrated with the touch-sensing capability of welding robots in order to locate the welding path accurately based on the user input,for fillet welding.The system is applicable to other welding processes and methods of seam localization.The system implementation is described and evaluated with a case study.

3D bare-hand interactions enabling ubiquitous interactions with smart objects

Ubiquitous augmented reality(UAR)implementation can benefit smart shop floor operations significantly.UAR from a user's first-person view can support and provide the user with suitable and comprehensive information without him/her being distracted from ongoing tasks.A natural hand-based interaction interface,namely,a mobile bare-hand interface(MBHI),is proposed to assist a user in exploring and navigating a large amount of information for a task in the user's first-person view.The integration of a smart shop floor and UAR-based MBHI is particularly challenging.A real shop floor environment is composed of challenging conditions for the implementation of UAR,e.g.,messy backgrounds and significant changes in illumination conditions.Meanwhile,the MBHI is required to provide precise and quick responses to minimize the dificulty of a user's task.In this study,a wearable UAR system integrated with an MBH is proposed to augment the shop floor environment with smart information.A case study is implemented to demonstrate the practicality and effectiveness of the proposed UAR and MBHI system.

增强现实地图研究与应用

随着数字地图技术的不断革新,产生了将虚拟信息与真实世界融合的新型地图技术,即增强现实(augmented reality,AR)地图。从可视化、定位、测绘、分析几个方面阐述了AR地图的关键技术,介绍了AR地图的空间映射原理和可视化效果,提出了结合视觉定位和惯导定位的cm级室内定位方法,实现了基于AR地图的AR导航、AR测量、AR目标采集、AR地理围栏等关键技术并将其应用于AR智慧园区与AR室内场馆。