Digital twin-and extended reality-based telepresence for collaborative robot programming in the 6G perspective

In the context of Industry 4.0,a paradigm shift from traditional industrial manipulators to Collaborative Robots(CRs)is ongoing,with the latter serving ever more closely humans as auxiliary tools in many production processes.In this scenario,continuous technological advancements offer new opportunities for further innovating robotics and other areas of next-generation industry.For example,6G could play a prominent role due to its human-centric view of the industrial domains.In particular,its expected dependability features will pave the way for new applications exploiting highly effective Digital Twin(DT)-and eXtended Reality(XR)-based telepresence.In this work,a novel application for the above technologies allowing two distant users to collaborate in the programming of a CR is proposed.The approach encompasses demanding data flows(e.g.,point cloud-based streaming of collaborating users and robotic environment),with network latency and bandwidth constraints.Results obtained by analyzing this approach from the viewpoint of network requirements in a setup designed to emulate 6G connectivity indicate that the expected performance of forthcoming mobile networks will make it fully feasible in principle.

Virtual reality tools for training in gastrointestinal endoscopy:A systematic review

BACKGROUND Virtual reality(VR)has emerged as an innovative technology in endoscopy training,providing a simulated environment that closely resembles real-life scenarios and offering trainees a valuable platform to acquire and enhance their endoscopic skills.This systematic review will critically evaluate the effectiveness and feasibility of VR-based training compared to traditional methods.AIM To evaluate the effectiveness and feasibility of VR-based training compared to traditional methods.By examining the current state of the field,this review seeks to identify gaps,challenges,and opportunities for further research and implementation of VR in endoscopic training.METHODS The study is a systematic review,following the guidelines for reporting systematic reviews set out by the PRISMA statement.A comprehensive search command was designed and implemented and run in September 2023 to identify relevant studies available,from electronic databases such as PubMed,Scopus,Cochrane,and Google Scholar.The results were systematically reviewed.RESULTS Sixteen articles were included in the final analysis.The total number of participants was 523.Five studies focused on both upper endoscopy and colonoscopy training,two on upper endoscopy training only,eight on colonoscopy training only,and one on sigmoidoscopy training only.Gastrointestinal Mentor virtual endoscopy simulator was commonly used.Fifteen reported positive results,indicating that VR-based training was feasible and acceptable for endoscopy learners.VR technology helped the trainees enhance their skills in manipulating the endoscope,reducing the procedure time or increasing the technical accuracy,in VR scenarios and real patients.Some studies show that the patient discomfort level decreased significantly.However,some studies show there were no significant differences in patient discomfort and pain scores between VR group and other groups.CONCLUSION VR training is effective for endoscopy training.There are several well-designed randomized controlled trials with large sample sizes,proving the potential of this innovative tool.Thus,VR should be more widely adopted in endoscopy training.Furthermore,combining VR training with conventional methods could be a promising approach that should be implemented in training.

Augmented Reality Marketing:Breakthroughs and Opportunities

The impact of augmented reality(AR)technology on consumer behavior has increasingly attracted academic attention.While early research has provided valuable insights,many challenges remain.This article reviews recent studies,analyzing AR’s technical features,marketing concepts,and action mechanisms from a consumer perspective.By refining existing frameworks and introducing a new model based on situation awareness theory,the paper offers a deeper exploration of AR marketing.Finally,it proposes directions for future research in this emerging field.

Leveraging Augmented Reality,Semantic-Segmentation,and VANETs for Enhanced Driver’s Safety Assistance

Overtaking is a crucial maneuver in road transportation that requires a clear view of the road ahead.However,limited visibility of ahead vehicles can often make it challenging for drivers to assess the safety of overtaking maneuvers,leading to accidents and fatalities.In this paper,we consider atrous convolution,a powerful tool for explicitly adjusting the field-of-view of a filter as well as controlling the resolution of feature responses generated by Deep Convolutional Neural Networks in the context of semantic image segmentation.This article explores the potential of seeing-through vehicles as a solution to enhance overtaking safety.See-through vehicles leverage advanced technologies such as cameras,sensors,and displays to provide drivers with a real-time view of the vehicle ahead,including the areas hidden from their direct line of sight.To address the problems of safe passing and occlusion by huge vehicles,we designed a see-through vehicle system in this study,we employed a windshield display in the back car together with cameras in both cars.The server within the back car was used to segment the car,and the segmented portion of the car displayed the video from the front car.Our see-through system improves the driver’s field of vision and helps him change lanes,cross a large car that is blocking their view,and safely overtake other vehicles.Our network was trained and tested on the Cityscape dataset using semantic segmentation.This transparent technique will instruct the driver on the concealed traffic situation that the front vehicle has obscured.For our findings,we have achieved 97.1% F1-score.The article also discusses the challenges and opportunities of implementing see-through vehicles in real-world scenarios,including technical,regulatory,and user acceptance factors.

Investigation of Inside-Out Tracking Methods for Six Degrees of Freedom Pose Estimation of a Smartphone in Augmented Reality

Six degrees of freedom(6DoF)input interfaces are essential formanipulating virtual objects through translation or rotation in three-dimensional(3D)space.A traditional outside-in tracking controller requires the installation of expensive hardware in advance.While inside-out tracking controllers have been proposed,they often suffer from limitations such as interaction limited to the tracking range of the sensor(e.g.,a sensor on the head-mounted display(HMD))or the need for pose value modification to function as an input interface(e.g.,a sensor on the controller).This study investigates 6DoF pose estimation methods without restricting the tracking range,using a smartphone as a controller in augmented reality(AR)environments.Our approach involves proposing methods for estimating the initial pose of the controller and correcting the pose using an inside-out tracking approach.In addition,seven pose estimation algorithms were presented as candidates depending on the tracking range of the device sensor,the tracking method(e.g.,marker recognition,visual-inertial odometry(VIO)),and whether modification of the initial pose is necessary.Through two experiments(discrete and continuous data),the performance of the algorithms was evaluated.The results demonstrate enhanced final pose accuracy achieved by correcting the initial pose.Furthermore,the importance of selecting the tracking algorithm based on the tracking range of the devices and the actual input value of the 3D interaction was emphasized.

Breaking the optical efficiency limit of virtual reality with a nonreciprocal polarization rotator

A catadioptric lens structure,also known as pancake lens,has been widely used in virtual reality(VR)displays to reduce the formfactor.However,the utilization of a half mirror(HM)to fold the optical path thrice leads to a significant optical loss.The theoretical maximum optical efficiency is merely 25%.To transcend this optical efficiency constraint while retaining the foldable characteristic inherent to traditional pancake optics,in this paper,we propose a theoretically lossless folded optical system to replace the HM with a nonreciprocal polarization rotator.In our feasibility demonstration experiment,we used a commercial Faraday rotator(FR)and reflective polarizers to replace the lossy HM.The theoretically predicted 100%efficiency can be achieved approximately by using two high-extinction-ratio reflective polarizers.In addition,we evaluated the ghost images using a micro-OLED panel in our imaging system.Indeed,the ghost images can be suppressed to undetectable level if the optics are with antireflection coating.Our novel pancake optical system holds great potential for revolutionizing next-generation VR displays with lightweight,compact formfactor,and low power consumption.

Virtual reality for preoperative patient education: Impact on satisfaction, usability, and burnout from the perspective of new nurses

BACKGROUND Traditional paper-based preoperative patient education is a struggle for new nurses and requires extensive training.In this situation,virtual reality technology can help the new nurses.Despite its potential benefits,there are studies on patient satisfaction but there is limited information on the usability of virtual reality(VR)technology for new nurses in giving preoperative education to patients.AIM To investigate the impact on satisfaction,usability,and burnout of a system using VR technology in preoperative patient education.METHODS The study involved 20 nurses from the plastic surgery ward and 80 patients admitted between April and May 2019.Each nurse taught four patients:Two using traditional verbal education and two using virtual reality.The System Usability Scale,After-Scenario Questionnaire,and Maslach Burnout Inventory(MBI)were employed to evaluate the impact of these education methods.RESULTS The VR education groups showed a statistically higher satisfaction than the traditional verbal education groups.Among the three subscales of the MBI,emotional exhaustion and personal accomplishment improved statistically significantly.VR was also better in terms of usability.CONCLUSION This study suggests VR enhances usability and reduces burnout in nurses,but further research is needed to assess its impact on depersonalization and objective measures like stress and heart rate.

Chemical simulation teaching system based on virtual reality and gesture interaction

Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based on virtual reality and gesture interaction.Methods The parameters of the models were obtained through actual investigation,whereby Blender and 3DS MAX were used to model and import these parameters into a physics engine.By establishing an interface for the physics engine,gesture interaction hardware,and virtual reality(VR)helmet,a highly realistic chemical experiment environment was created.Using code script logic,particle systems,as well as other systems,chemical phenomena were simulated.Furthermore,we created an online teaching platform using streaming media and databases to address the problems of distance teaching.Results The proposed system was evaluated against two mainstream products in the market.In the experiments,the proposed system outperformed the other products in terms of fidelity and practicality.Conclusions The proposed system which offers realistic simulations and practicability,can help improve the high school chemistry experimental education.

Towards engineering a portable platform for laparoscopic pre-training in virtual reality with haptic feedback

Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training through practical exercises on abstracted and reduced models.Methods The authors strive for a portable,easy to use and cost-effective Virtual Reality-based(VR)laparoscopic pre-training platform and therefore address the question of how such a system has to be designed to achieve the quality of today's gold standard using real tissue specimens.Current VR controllers are limited regarding haptic feedback.Since haptic feedback is necessary or at least beneficial for laparoscopic surgery training,the platform to be developed consists of a newly designed prototype laparoscopic VR controller with haptic feedback,a commercially available head-mounted display,a VR environment for simulating a laparoscopic surgery,and a training concept.Results To take full advantage of benefits such as repeatability and cost-effectiveness of VR-based training,the system shall not require a tissue sample for haptic feedback.It is currently calculated and visually displayed to the user in the VR environment.On the prototype controller,a first axis was provided with perceptible feedback for test purposes.Two of the prototype VR controllers can be combined to simulate a typical both-handed use case,e.g.,laparoscopic suturing.A Unity based VR prototype allows the execution of simple standard pre-trainings.Conclusions The first prototype enables full operation of a virtual laparoscopic instrument in VR.In addition,the simulation can compute simple interaction forces.Major challenges lie in a realistic real-time tissue simulation and calculation of forces for the haptic feedback.Mechanical weaknesses were identified in the first hardware prototype,which will be improved in subsequent versions.All degrees of freedom of the controller are to be provided with haptic feedback.To make forces tangible in the simulation,characteristic values need to be determined using real tissue samples.The system has yet to be validated by cross-comparing real and VR haptics with surgeons.

Enhancing preoperative patient education through virtual reality:A leap forward in nursing practice

Integration of virtual reality(VR)technology into preoperative patient education has shown potential to improve nursing practice.The study by Kim et al examines the impact of VR on nurse satisfaction,usability,and burnout.A prospective study involving 20 nurses and 80 patients was conducted,comparing traditional paper-based education with VR-based education in the plastic surgery ward at Chungnam National University Hospital.Findings demonstrated that VR significantly enhanced satisfaction and usability among nurses,while also reducing emotional exhaustion,a critical factor in nurse burnout.Consistency in education quality was also improved,ensuring uniform patient care.These results underscore the importance of VR in reducing the stress of repetitive tasks,improving job satisfaction,and potentially enhancing nurse retention.Future research should explore the broader applications of VR in healthcare and address the logistical challenges of integrating this technology into routine clinical practice.The study by Kim et al highlights VR’s transformative potential in preoperative education for both patients and healthcare providers.

Virtual reality:The bridge between medical education and clinical practice

After perusing the paper by Kim et al,I discovered that this is an interesting manuscript and a successful study.Virtual reality(VR)is an emerging and promising technology employed in the domain of medical practice and medical education over the past decade.In the era of big data,VR is constantly progressing in the fields of medical education and clinical diagnosis and treatment.As a novel scientific and technological tool,VR not only overcomes multiple limitations of the traditional medical teaching mode but also reduces the reliance on personnel and equipment.VR can simulate the real clinical situation,stimulate the enthusiasm of young doctors and nurses for clinical study,and simultaneously safeguard and promote medical safety and doctor-patient harmony.Favorable outcomes have been attained in clinical teaching and diagnosis and treatment activities.While enhancing the training conditions of medical cosmetics and elevating the level of clinical practice and teaching,the risks resulting from improper clinical diagnosis and treatment have been circumvented.All of this is evident and comprehensible.

Large-scale spatial data visualization method based on augmented reality

Background A task assigned to space exploration satellites involves detecting the physical environment within a certain space.However,space detection data are complex and abstract.These data are not conducive for researchers'visual perceptions of the evolution and interaction of events in the space environment.Methods A time-series dynamic data sampling method for large-scale space was proposed for sample detection data in space and time,and the corresponding relationships between data location features and other attribute features were established.A tone-mapping method based on statistical histogram equalization was proposed and applied to the final attribute feature data.The visualization process is optimized for rendering by merging materials,reducing the number of patches,and performing other operations.Results The results of sampling,feature extraction,and uniform visualization of the detection data of complex types,long duration spans,and uneven spatial distributions were obtained.The real-time visualization of large-scale spatial structures using augmented reality devices,particularly low-performance devices,was also investigated.Conclusions The proposed visualization system can reconstruct the three-dimensional structure of a large-scale space,express the structure and changes in the spatial environment using augmented reality,and assist in intuitively discovering spatial environmental events and evolutionary rules.

Single-center experience with Knee+^(TM) augmented reality navigation system in primary total knee arthroplasty

BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.

Tree: Reducing the use of restrictive practices on psychiatric wards through virtual reality immersive technology training

BACKGROUND Restrictive practices(RPs)are defined by measures linked to physical and chemical restraints to reduce the movement or control behaviours during any emergency.Seclusion is an equal part of RPs intended to isolate and reduce the sensory stimulation to safeguard the patient and those within the vicinity.Using interventions by way of virtual reality(VR)could assist with reducing the need for RPs as it could help reduce anxiety or agitation by way of placing users into realistic and immersive environments.This could also aid staff to and change current RPs.AIM To assess the feasibility and effectiveness of using a VR platform to provide reduction in RP training.METHODS A randomised controlled feasibility study,accompanied by evaluations at 1 month and 6 months,was conducted within inpatient psychiatric wards at Southern Health National Health Service Foundation Trust,United Kingdom.Virti VR scenarios were used on VR headsets to provide training on reducing RPs in 3 inpatient psychiatric wards.Outcome measures included general self-efficacy scale,generalised anxiety disorder assessment 7(GAD-7),Burnout Assessment Tool 12,the Everyday Discrimination Scale,and the Compassionate Engagement and Action Scale.RESULTS Findings revealed statistically significant differences between the VR and treatment as usual groups,in the Everyday Discrimination Scale items Q8 and Q9:P=0.023 and P=0.040 respectively,indicating higher levels of perceived discrimination in the VR group.There were no significant differences between groups in terms of general self-efficacy,generalised anxiety disorder assessment 9,and Burnout Assessment Tool 12 scores.A significant difference was observed within the VR group for compassionate engagement from others(P=0.005)over time.Most respondents recorded System Usability Scale scores above 70,with an average score of 71.79.There was a significant reduction in rates of RPs in the VR group vs treatment as usual group with a fluctuating variability observed in the VR group likely due to external factors not captured in the study.CONCLUSION Ongoing advancement of VR technology enables the possibility of creating scenarios and simulations tailored to healthcare environments that empower staff by providing more comprehensive and effective training for handling situations.

Personalized assessment and training of neurosurgical skills in virtual reality:An interpretable machine learning approach

Background Virtual reality technology has been widely used in surgical simulators,providing new opportunities for assessing and training surgical skills.Machine learning algorithms are commonly used to analyze and evaluate the performance of participants.However,their interpretability limits the personalization of the training for individual participants.Methods Seventy-nine participants were recruited and divided into three groups based on their skill level in intracranial tumor resection.Data on the use of surgical tools were collected using a surgical simulator.Feature selection was performed using the Minimum Redundancy Maximum Relevance and SVM-RFE algorithms to obtain the final metrics for training the machine learning model.Five machine learning algorithms were trained to predict the skill level,and the support vector machine performed the best,with an accuracy of 92.41%and Area Under Curve value of 0.98253.The machine learning model was interpreted using Shapley values to identify the important factors contributing to the skill level of each participant.Results This study demonstrates the effectiveness of machine learning in differentiating the evaluation and training of virtual reality neurosurgical performances.The use of Shapley values enables targeted training by identifying deficiencies in individual skills.Conclusions This study provides insights into the use of machine learning for personalized training in virtual reality neurosurgery.The interpretability of the machine learning models enables the development of individualized training programs.In addition,this study highlighted the potential of explanatory models in training external skills.

Virtual Reality-Based Mirror Therapy for Upper Extremity Function among Stroke Patients:A Meta-analysis and Systematic Review

[Objectives]To investigate the evidence-based effect of virtual reality-based mirror therapy system(VR-MT)on upper extremity function among stroke patients.[Methods]A systematic electronic searching of the Medline,PubMed,Web of Science and CNKI was initially performed up to June 10,2024.The risk of bias of the included studies was evaluated using RevMan 5.4 software based on the Cochrane Handbook for Systematic Reviews.The random-effects model or fixed-effects models was employed to estimate the standardized mean difference(SMD).The subgroup analyses were conducted exploring theVR-MT type(immersive or non-immersive)and comparing with MT or control group.[Results]In total 8 studies with a total of 273 stroke patients were included in this review.The pooled analysis of these trials showed a statistically significant enhancement inFMA-UE scores(6 studies,SMD=0.72,[95%CI 0.37 to 1.06];P<0.0001,I^(2)=31%)and Box and Block Test(BBT)(3 studies,SMD=0.49,[95%C/0.05 to 0.93];P=0.03,I^(2)=0%),rather than Manual Function Test(MFT)scores(3 studies,SMD=0.38,[95%CI-0.09 to 0.84];P=0.11,I^(2)=0%)following the application of reality-based mirror therapy.Additionally,the subgroup analysis results indicated that immersive VR-MT can significantly improve FMA-UE(5studies,SMD=0.73,[95%CI 0.24 to 1.23];P=0.004,I^(2)=43%).In contrast,the overall effect of non-immersive VR-MT was non-significant(2 studies,SMD=0.33,[95%CI-0.69 to 1.34];P=0.53,I^(2)=72%).[Conclusions]In this systematic review and meta-analysis,our findings indicate that immersiveVR-MT has the potential to improve upper extremity function among stroke patients.

Research on Visualization and Interactive Design of Plant Configuration Virtual Reality in the Context of Digitalization

This study aims to explore the application of digital technology in landscape design,focusing on the research of virtual reality visualization and interactive design in the process of plant configuration.Through an in-depth analysis of digital technology,the study outlines its important role in landscape design,especially in the application of plant configuration.The current application status of virtual reality technology in landscape design is discussed,as well as how interactive design can enhance user experience and participation.Furthermore,the achievements and challenges of digital technology in landscape design are summarized.Finally,it proposes future research directions and suggestions,aiming to provide new ideas and methods for practice and research in the field of landscape design and promote the further application and development of digital technology in landscape design.

Design of Medical Rehabilitation Experience System Based on Extended Reality

This project integrates Extended Reality technology with medical rehabilitation,integrating cloud rendering.interactive experience systems,adaptation platforms,and digital scene resources.It is a innovative application research in the era of metaverset medical rehabilitation.The project uses the Maya engine for the creation of digital assets,with Unity as the main functional development platform and PICO4 as the core device,to achieve interactive and fun experience effects for various rehabilitation training devices.It solves the problems of patients being dull,low participation,lack of confidence in traditional rehabilitation training.as well as the inability of medical staff to monitor patient progress in real time and adjust rehabilitation plans better.

Review of Digital Campus Construction Led by Virtual Reality Technology

In recent years,the government has issued a series of documents to promote the construction of digital campuses.This initiative serves to encourage the deep integration of information technology and intelligent technology education and digital reform,the combination of virtual reality and campus management is the need for innovative thinking and economic and social development,and then better change our learning style and living environment.The construction of the digital campus is based on virtual reality technology,BIM,GIS,and three-dimensional modeling technology to provide an immersive platform for students,promote the integration of virtual reality technology and education,help teachers,students,and parents to understand all kinds of education information and resources,to achieve their interoperability.From the off-campus environment to the school teaching equipment,teachers to teaching quality certification,and learning,to extracurricular entertainment,opening ceremonies to graduation parties,to bring more efficient,convenient,and safe campus life for teachers,students,and staff in school,and break the traditional information restrictions.

Discussion on the Construction and Application of Virtual Reality-Based Curriculum Resources for Kindergarten Education

The construction of early childhood curriculum resources serves as a vital vehicle and precondition for the implementation of collective teaching activities in kindergartens.However,traditional curriculum resources based on graphic language or the linguistic descriptions of early childhood educators often fail to provide children with an immersive experience during collective teaching activities,leading to a lack of initiative and interest in learning.This paper discusses the methods for building a kindergarten curriculum resource library and,based on this foundation,employs virtual reality technology for three-dimensional modeling of the library to enrich the curriculum resources in kindergartens.Furthermore,this paper proposes to train early childhood educators in virtual reality technology to enhance their abilities to operate and utilize virtual reality equipment,which can emphasize the children’s central role in the learning process,better achieve educational goals,and improve teaching outcomes.