Prospects and challenges in augmented reality displays

Augmented reality(AR)displays are attracting significant attention and efforts.In this paper,we review the adopted device configurations of see-through displays,summarize the current development status and highlight future challenges in micro-displays.A brief introduction to optical gratings is presented to help understand the challenging design of grating-based waveguide for AR displays.Finally,we discuss the most recent progress in diffraction grating and its implications.

Metasurface-enabled augmented reality display: a review

Augmented reality(AR)display,which superimposes virtual images on ambient scene,can visually blend the physical world and the digital world and thus opens a new vista for human–machine interaction.AR display is considered as one of the next-generation display technologies and has been drawing huge attention from both academia and industry.Current AR display systems operate based on a combination of various refractive,reflective,and diffractive optical elements,such as lenses,prisms,mirrors,and gratings.Constrained by the underlying physical mechanisms,these conventional elements only provide limited light-field modulation capability and suffer from issues such as bulky volume and considerable dispersion,resulting in large size,severe chromatic aberration,and narrow field of view of the composed AR display system.Recent years have witnessed the emerging of a new type of optical elements—metasurfaces,which are planar arrays of subwavelength electromagnetic structures that feature an ultracompact footprint and flexible light-field modulation capability,and are widely believed to be an enabling tool for overcoming the limitations faced by current AR displays.Here,we aim to provide a comprehensive review on the recent development of metasurface-enabled AR display technology.We first familiarize readers with the fundamentals of AR display,covering its basic working principle,existing conventional-optics-based solutions,as well as the associated pros and cons.We then introduce the concept of optical metasurfaces,emphasizing typical operating mechanisms,and representative phase modulation methods.We elaborate on three kinds of metasurface devices,namely,metalenses,metacouplers,and metaholograms,which have empowered different forms of AR displays.Their physical principles,device designs,and the performance improvement of the associated AR displays are explained in details.In the end,we discuss the existing challenges of metasurface optics for AR display applications and provide our perspective on future research endeavors.

Metasurfaces for near-eye display applications

Virtual reality(VR)and augmented reality(AR)are revolutionizing our lives.Near-eye displays are crucial technologies for VR and AR.Despite the rapid advances in near-eye display technologies,there are still challenges such as large field of view,high resolution,high image quality,natural free 3D effect,and compact form factor.Great efforts have been devoted to striking a balance between visual performance and device compactness.While traditional optics are nearing their limitations in addressing these challenges,ultra-thin metasurface optics,with their high light-modulating capabilities,may present a promising solution.In this review,we first introduce VR and AR near-eye displays,and then briefly explain the working principles of light-modulating metasurfaces,review recent developments in metasurface devices geared toward near-eye display applications,delved into several advanced natural 3D near-eye display technologies based on metasurfaces,and finally discuss about the remaining challenges and future perspectives associated with metasurfaces for near-eye display applications.

Holographic techniques for augmented reality and virtual reality near-eye displays

Near-eye displays are the main platform devices for many augmented reality(AR)and virtual reality(VR)applications.As a wearable device,a near-eye display should have a compact form factor and be lightweight.Furthermore,a large field of view and sufficient eyebox are crucial for immersive viewing conditions.Natural three-dimensional(3D)image presentation with proper focus cues is another requirement that enables a comfortable viewing experience and natural user interaction.Finally,in the case of AR,the device should allow for an optical see-through view of the real world.Conventional bulk optics and two-dimensional display panels exhibit clear limitations when implementing these requirements.Holographic techniques have been applied to near-eye displays in various aspects to overcome the limitations of conventional optics.The wavefront reconstruction capability of holographic techniques has been extensively exploited to develop optical see-through 3D holographic near-eye displays of glass-like form factors.In this article,the application of holographic techniques to AR and VR near-eye displays is reviewed.Various applications are introduced,such as static holographic optical components and dynamic holographic display devices.Current issues and recent progress are also reviewed,providing a comprehensive overview of holographic techniques that are applied to AR and VR near-eye displays.

Design of Learning Media in Mixed Reality for Lao Education

To improve and develop education systems,the communication between instructors and learners in a class during the learning process is of utmost importance.Currently the presentations of 3D models using mixed reality(MR)technology can be used to avoid misinterpretations of oral and 2D model presentations.As an independent concept and MR applications,MR combines the excellent of each virtual reality(VR)and augmented reality(AR).This work aims to present the descriptions of MR systems,which include its devices,applications,and literature reviews and proposes computer vision tracking using the AR Toolkit Tracking Library.The focus of this work will be on creating 3D models and implementing in Unity 3D using the Vuforia SDK platform to develop VR and AR applications for architectural presentations.

Augmented reality assisted surgery： a urologic training tool

Augmented reality is widely used in aeronautics and is a developing concept within surgery. In this pilot study, we developed an application for use on Google Glass optical head-mounted display to train urology residents in how to place an inflatable penile prosthesis. We use the phrase Augmented Reality Assisted Surgery to describe this novel application of augmented reality in the setting of surgery. The application demonstrates the steps of the surgical procedure of inflatable penile prosthesis placement. It also contains software that allows for detection of interest points using a camera feed from the optical head-mounted display to enable faculty to interact with residents during placement of the penile prosthesis. Urology trainees and faculty who volunteered to take part in the study were given time to experience the technology in the operative or perioperative setting and asked to complete a feedback survey. From 30 total participants using a lO-point scale, educational usefulness was rated 8.6, ease of navigation was rated 7.6, likelihood to use was rated 7.4, and distraction in operating room was rated 4.9. When stratified between trainees and faculty, trainees found the technology more educationally useful, and less distracting. Overall, 81% of the participants want this technology in their residency program, and 93% see this technology in the operating room in the future. Further development of this technology is warranted before full release, and further studies are necessary to better characterize the effectiveness of Augmented Reality Assisted Surgery in urologic surgical training.

3D augmented reality teleoperated robot system based on dual vision

A 3D augmented reality navigation system using stereoscopic images is developed for teleoperated robot systems. The accurate matching between the simulated model and the video image of the actual robot can be realized, which helps the operator to accomplish the remote control task correctly and reliably. The system introduces the disparity map translation transformation method to take parallax images for stereoscopic displays, providing the operator an immersive 3D experience. Meanwhile, a fast and accurate registration method of dynamic stereo video is proposed, and effective integration of a virtual robot and the real stereo scene can be achieved. Preliminary experiments show that operation error of the system is maintained at less than 2.2 mm and the average error is 0.854 7, 0.909 3 and 0.697 2 mm at x, y, z direction respectively. Lots of experiments such as pressing the button, pulling the drawer and so on are also conducted to evaluate the performance of the system. The feasibility studies show that the depth information of structure can be rapidly and recognized in remote environment site. The augmented reality of the image overlay system could increase the operating accuracy and reduce the procedure time as a result of intuitive 3D viewing.

Waveguide-based augmented reality displays:perspectives and challenges

Augmented reality(AR)displays,as the next generation platform for spatial computing and digital twins,enable users to view digital images superimposed on real-world environment,fostering a deeper level of human-digital interactions.However,as a critical element in an AR system,optical combiners face unprecedented challenges to match the exceptional performance requirements of human vision system while keeping the headset ultracompact and lightweight.After decades of extensive device and material research efforts,and heavy investment in manufacturing technologies,several promising waveguide combiners have been developed.In this review paper,we focus on the perspectives and challenges of optical waveguide combiners for AR displays.We will begin by introducing the basic device structures and operation principles of different AR architectures,and then delve into different waveguide combiners,including geometric and diffractive waveguide combiners.Some commonly used in-couplers and out-couplers,such as prisms,mirrors,surface relief gratings,volume holographic gratings,polarization volume gratings,and metasurface-based couplers,will be discussed,and their properties analyzed in detail.Additionally,we will explore recent advances in waveguide combiner design and modeling,such as exit pupil expansion,wide field of view,geometric architectures of waveguide couplers,full-color propagation,and brightness and color uniformity optimization.Finally,we will discuss the bottlenecks and future development trends in waveguide combiner technologies.The objective of this review is to provide a comprehensive overview of the current state of waveguide combiner technologies,analyze their pros and cons,and then present the future challenges of AR displays.

Holographic optics in planar optical systems for next generation small form factor mixed reality headsets

Helmet Mounted Displays(HMDs),such as in Virtual Reality(VR),Augmented Reality(AR),Mixed reality(MR),and Smart Glasses have the potential to revolutionize the way we live our private and professional lives,as in communicating,working,teaching and learning,shopping and getting entertained.Such HMD devices have to satisfy draconian requirements in weight,size,form factor,power,compute,wireless communication and of course display,imaging and sensing performances.We review in this paper the various optical technologies and architectures that have been developed in the past 10 years to provide adequate solutions for the drastic requirements of consumer HMDs,a market that has yet to become mature in the next years,unlike the existing enterprise and defense markets that have already adopted VR and AR headsets as practical tools to improve greatly effectiveness and productivity.We focus specifically our attention on the optical combiner element,a crucial element in Optical See-Through(OST)HMDs that combines the see-through scene with a world locked digital image.As for the technological platform,we chose optical waveguide combiners,although there is also a considerable effort today dedicated to free-space combiners.Flat and thin optics as in micro-optics,holographics,diffractives,metasurfaces and other nanostructured optical elements are key building blocks to achieve the target form factor.

Recent research progress of master mold manufacturing by nanoimprint technique for the novel microoptics devices

The consumer demand for emerging technologies such as augmented reality(AR),autopilot,and three-dimensional(3D)internet has rapidly promoted the application of novel optical display devices in innovative industries.However,the micro/nanomanufacturing of high-resolution optical display devices is the primary issue restricting their development.The manufacturing technology of micro/nanostructures,methods of display mechanisms,display materials,and mass production of display devices are major technical obstacles.To comprehensively understand the latest state-of-the-art and trigger new technological breakthroughs,this study reviews the recent research progress of master molds produced using nanoimprint technology for new optical devices,particularly AR glasses,new-generation light-emitting diode car lighting,and naked-eye 3D display mechanisms,and their manufacturing techniques of master molds.The focus is on the relationships among the manufacturing process,microstructure,and display of a new optical device.Nanoimprint master molds are reviewed for the manufacturing and application of new optical devices,and the challenges and prospects of the new optical device diffraction grating nanoimprint technology are discussed.