Canada is a multicultural country which was mainly established by immigrants.Just because of that,Canadian government has carried out the policy of multiculturalism since1970s.However,it has encountered many problems ...Canada is a multicultural country which was mainly established by immigrants.Just because of that,Canadian government has carried out the policy of multiculturalism since1970s.However,it has encountered many problems such as policy conflicts,national identity,democracy-inquiry and racial discrimination,etc.Hence the Canadian multiculturalism has been in a dilemma.展开更多
Economic globalization has catalyzed the global flow of economic factors and set the stage for the rise of fast-developing emerging economies with worldwide influence.Research has found that the emerging economies of ...Economic globalization has catalyzed the global flow of economic factors and set the stage for the rise of fast-developing emerging economies with worldwide influence.Research has found that the emerging economies of China,Brazil,Russia,India,and South Africa have experienced significantly varying changes in their per capita GDP over time.Over the past decade,China has maintained a steady rate of economic growth with subdued volatility.In contrast,Brazil,Russia,India,and South Africa have all registered more volatility in their rates of growth.China has no significant correlation with other emerging economies in terms of imports and exports in goods and services and foreign direct investment(FDI),and their mutual influences are not evident.Going forward,for sustainable socio-economic development,China needs to optimize its industrial structure,promote the enhancement of ecological and institutional environments,and bring about all-round socio-economic progress,by initiating a new round of opening up at a higher level,broadening the space for international cooperation,building an open economy,promoting equal opportunity,and pursuing an inclusive path of growth.展开更多
It bus been 40 years since China embarked on "crossing the river by feel ing the stones," navigating through the untrodden path and uncharted waters to drive its iconic modemization effort through reform and opening...It bus been 40 years since China embarked on "crossing the river by feel ing the stones," navigating through the untrodden path and uncharted waters to drive its iconic modemization effort through reform and opening up.展开更多
Augmented reality(AR)and virtual reality(VR)are two novel display technologies that are under updates.The essential feature of AR/VR is the full-color display that requires high pixel densities.To generate three-color...Augmented reality(AR)and virtual reality(VR)are two novel display technologies that are under updates.The essential feature of AR/VR is the full-color display that requires high pixel densities.To generate three-color pixels,the fluorescent color conversion layer inevitably includes green and red pixels.To fabricate such sort of display kits,inkjet printing is a promising way to position the color conversion layers.In this review article,the progress of AR/VR technologies is first reviewed,and in succession,the state of the art of inkjet printing,as well as two key issues-the optimization of ink and the reduction of coffee-ring effects,are introduced.Finally,some potential problems associated with the color converting layer are highlighted.展开更多
In the context of Industry 4.0,a paradigm shift from traditional industrial manipulators to Collaborative Robots(CRs)is ongoing,with the latter serving ever more closely humans as auxiliary tools in many production pr...In the context of Industry 4.0,a paradigm shift from traditional industrial manipulators to Collaborative Robots(CRs)is ongoing,with the latter serving ever more closely humans as auxiliary tools in many production processes.In this scenario,continuous technological advancements offer new opportunities for further innovating robotics and other areas of next-generation industry.For example,6G could play a prominent role due to its human-centric view of the industrial domains.In particular,its expected dependability features will pave the way for new applications exploiting highly effective Digital Twin(DT)-and eXtended Reality(XR)-based telepresence.In this work,a novel application for the above technologies allowing two distant users to collaborate in the programming of a CR is proposed.The approach encompasses demanding data flows(e.g.,point cloud-based streaming of collaborating users and robotic environment),with network latency and bandwidth constraints.Results obtained by analyzing this approach from the viewpoint of network requirements in a setup designed to emulate 6G connectivity indicate that the expected performance of forthcoming mobile networks will make it fully feasible in principle.展开更多
In this work,we present a reconfigurable data glove design to capture different modes of human hand-object interactions,which are critical in training embodied artificial intelligence(AI)agents for fine manipulation t...In this work,we present a reconfigurable data glove design to capture different modes of human hand-object interactions,which are critical in training embodied artificial intelligence(AI)agents for fine manipulation tasks.To achieve various downstream tasks with distinct features,our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time.In the tactile-sensing mode,the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material;this design minimizes interference during complex hand movements.The virtual reality(VR)mode enables real-time interaction in a physically plausible fashion:A caging-based approach is devised to determine stable grasps by detecting collision events.Leveraging a state-of-the-art finite element method,the simulation mode collects data on fine-grained four-dimensionalmanipulation events comprising hand and object motions in three-dimensional space and how the object's physical properties(e.g.,stress and energy)change in accordance with manipulation over time.Notably,the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions.In a series of experiments,we characterize our data glove in terms of individual sensors and the overall system.More specifically,we evaluate the system's three modes by①recording hand gestures and associated forces,②improving manipulation fluency in VR,and③producing realistic simulation effects of various tool uses,respectively.Based on these three modes,our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments,thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.展开更多
Current rates of mental illness are worrisome.Mental illness mainly affects females and younger age groups.The use of the internet to deliver mental health care has been growing since 2020 and includes the implementat...Current rates of mental illness are worrisome.Mental illness mainly affects females and younger age groups.The use of the internet to deliver mental health care has been growing since 2020 and includes the implementation of novel mental health treatments using virtual reality,augmented reality,and artificial intelligence.A new three dimensional digital environment,known as the metaverse,has emerged as the next version of the Internet.Artificial intelligence,augmented reality,and virtual reality will create fully immersive,experiential,and interactive online environments in the metaverse.People will use a unique avatar to do anything they do in their“real”lives,including seeking and receiving mental health care.In this opinion review,we reflect on how the metaverse could reshape how we deliver mental health treatment,its opportunities,and its challenges.展开更多
Overtaking is a crucial maneuver in road transportation that requires a clear view of the road ahead.However,limited visibility of ahead vehicles can often make it challenging for drivers to assess the safety of overt...Overtaking is a crucial maneuver in road transportation that requires a clear view of the road ahead.However,limited visibility of ahead vehicles can often make it challenging for drivers to assess the safety of overtaking maneuvers,leading to accidents and fatalities.In this paper,we consider atrous convolution,a powerful tool for explicitly adjusting the field-of-view of a filter as well as controlling the resolution of feature responses generated by Deep Convolutional Neural Networks in the context of semantic image segmentation.This article explores the potential of seeing-through vehicles as a solution to enhance overtaking safety.See-through vehicles leverage advanced technologies such as cameras,sensors,and displays to provide drivers with a real-time view of the vehicle ahead,including the areas hidden from their direct line of sight.To address the problems of safe passing and occlusion by huge vehicles,we designed a see-through vehicle system in this study,we employed a windshield display in the back car together with cameras in both cars.The server within the back car was used to segment the car,and the segmented portion of the car displayed the video from the front car.Our see-through system improves the driver’s field of vision and helps him change lanes,cross a large car that is blocking their view,and safely overtake other vehicles.Our network was trained and tested on the Cityscape dataset using semantic segmentation.This transparent technique will instruct the driver on the concealed traffic situation that the front vehicle has obscured.For our findings,we have achieved 97.1% F1-score.The article also discusses the challenges and opportunities of implementing see-through vehicles in real-world scenarios,including technical,regulatory,and user acceptance factors.展开更多
Six degrees of freedom(6DoF)input interfaces are essential formanipulating virtual objects through translation or rotation in three-dimensional(3D)space.A traditional outside-in tracking controller requires the instal...Six degrees of freedom(6DoF)input interfaces are essential formanipulating virtual objects through translation or rotation in three-dimensional(3D)space.A traditional outside-in tracking controller requires the installation of expensive hardware in advance.While inside-out tracking controllers have been proposed,they often suffer from limitations such as interaction limited to the tracking range of the sensor(e.g.,a sensor on the head-mounted display(HMD))or the need for pose value modification to function as an input interface(e.g.,a sensor on the controller).This study investigates 6DoF pose estimation methods without restricting the tracking range,using a smartphone as a controller in augmented reality(AR)environments.Our approach involves proposing methods for estimating the initial pose of the controller and correcting the pose using an inside-out tracking approach.In addition,seven pose estimation algorithms were presented as candidates depending on the tracking range of the device sensor,the tracking method(e.g.,marker recognition,visual-inertial odometry(VIO)),and whether modification of the initial pose is necessary.Through two experiments(discrete and continuous data),the performance of the algorithms was evaluated.The results demonstrate enhanced final pose accuracy achieved by correcting the initial pose.Furthermore,the importance of selecting the tracking algorithm based on the tracking range of the devices and the actual input value of the 3D interaction was emphasized.展开更多
A catadioptric lens structure,also known as pancake lens,has been widely used in virtual reality(VR)displays to reduce the formfactor.However,the utilization of a half mirror(HM)to fold the optical path thrice leads t...A catadioptric lens structure,also known as pancake lens,has been widely used in virtual reality(VR)displays to reduce the formfactor.However,the utilization of a half mirror(HM)to fold the optical path thrice leads to a significant optical loss.The theoretical maximum optical efficiency is merely 25%.To transcend this optical efficiency constraint while retaining the foldable characteristic inherent to traditional pancake optics,in this paper,we propose a theoretically lossless folded optical system to replace the HM with a nonreciprocal polarization rotator.In our feasibility demonstration experiment,we used a commercial Faraday rotator(FR)and reflective polarizers to replace the lossy HM.The theoretically predicted 100%efficiency can be achieved approximately by using two high-extinction-ratio reflective polarizers.In addition,we evaluated the ghost images using a micro-OLED panel in our imaging system.Indeed,the ghost images can be suppressed to undetectable level if the optics are with antireflection coating.Our novel pancake optical system holds great potential for revolutionizing next-generation VR displays with lightweight,compact formfactor,and low power consumption.展开更多
The National Natural Science Foundation of China is one of the major funding agencies for neuro rehabilitation research in China.This study reviews the frontier directions and achievements in the field of neurorehabil...The National Natural Science Foundation of China is one of the major funding agencies for neuro rehabilitation research in China.This study reviews the frontier directions and achievements in the field of neurorehabilitation in China and wo rldwide.We used data from the Web of Science Core Collection(WoSCC) database to analyze the publications and data provided by the National Natural Science Foundation of China to analyze funding information.In addition,the prospects for neurorehabilitation research in China are discussed.From 2010 to 2022,a total of 74,220 publications in neurorehabilitation were identified,with there being an overall upward tendency.During this period,the National Natural Science Foundation of China has funded 476 research projects with a total funding of 192.38 million RMB to support neuro rehabilitation research in China.With the support of the National Natural Science Foundation of China,China has made some achievements in neurorehabilitation research.Research related to neurorehabilitation is believed to be making steady and significant progress in China.展开更多
Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training thro...Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training through practical exercises on abstracted and reduced models.Methods The authors strive for a portable,easy to use and cost-effective Virtual Reality-based(VR)laparoscopic pre-training platform and therefore address the question of how such a system has to be designed to achieve the quality of today's gold standard using real tissue specimens.Current VR controllers are limited regarding haptic feedback.Since haptic feedback is necessary or at least beneficial for laparoscopic surgery training,the platform to be developed consists of a newly designed prototype laparoscopic VR controller with haptic feedback,a commercially available head-mounted display,a VR environment for simulating a laparoscopic surgery,and a training concept.Results To take full advantage of benefits such as repeatability and cost-effectiveness of VR-based training,the system shall not require a tissue sample for haptic feedback.It is currently calculated and visually displayed to the user in the VR environment.On the prototype controller,a first axis was provided with perceptible feedback for test purposes.Two of the prototype VR controllers can be combined to simulate a typical both-handed use case,e.g.,laparoscopic suturing.A Unity based VR prototype allows the execution of simple standard pre-trainings.Conclusions The first prototype enables full operation of a virtual laparoscopic instrument in VR.In addition,the simulation can compute simple interaction forces.Major challenges lie in a realistic real-time tissue simulation and calculation of forces for the haptic feedback.Mechanical weaknesses were identified in the first hardware prototype,which will be improved in subsequent versions.All degrees of freedom of the controller are to be provided with haptic feedback.To make forces tangible in the simulation,characteristic values need to be determined using real tissue samples.The system has yet to be validated by cross-comparing real and VR haptics with surgeons.展开更多
Background The sense of touch plays a crucial role in interactive behavior within virtual spaces,particularly when visual attention is absent.Although haptic feedback has been widely used to compensate for the lack of...Background The sense of touch plays a crucial role in interactive behavior within virtual spaces,particularly when visual attention is absent.Although haptic feedback has been widely used to compensate for the lack of visual cues,the use of tactile information as a predictive feedforward cue to guide hand movements remains unexplored and lacks theoretical understanding.Methods This study introduces a fingertip aero-haptic rendering method to investigate its effectiveness in directing hand movements during eyes-free spatial interactions.The wearable device incorporates a multichannel micro-airflow chamber to deliver adjustable tactile effects on the fingertips.Results The first study verified that tactile directional feedforward cues significantly improve user capabilities in eyes-free target acquisition and that users rely heavily on haptic indications rather than spatial memory to control their hands.A subsequent study examined the impact of enriched tactile feedforward cues on assisting users in determining precise target positions during eyes-free interactions,and assessed the required learning efforts.Conclusions The haptic feedforward effect holds great practical promise in eyeless design for virtual reality.We aim to integrate cognitive models and tactile feedforward cues in the future,and apply richer tactile feedforward information to alleviate users'perceptual deficiencies.展开更多
López del Hoyo et al collections reported the meta verse based on the virtual reality,augmented reality and artificial intelligence could be used in the therapy of mental health,although there were still some cha...López del Hoyo et al collections reported the meta verse based on the virtual reality,augmented reality and artificial intelligence could be used in the therapy of mental health,although there were still some challenges.This manuscript reported that the meta verse is a prospective method to improve the prognosis of mental health problems.展开更多
Background Physical entity interactions in mixed reality(MR)environments aim to harness human capabilities in manipulating physical objects,thereby enhancing virtual environment(VEs)functionality.In MR,a common strate...Background Physical entity interactions in mixed reality(MR)environments aim to harness human capabilities in manipulating physical objects,thereby enhancing virtual environment(VEs)functionality.In MR,a common strategy is to use virtual agents as substitutes for physical entities,balancing interaction efficiency with environmental immersion.However,the impact of virtual agent size and form on interaction performance remains unclear.Methods Two experiments were conducted to explore how virtual agent size and form affect interaction performance,immersion,and preference in MR environments.The first experiment assessed five virtual agent sizes(25%,50%,75%,100%,and 125%of physical size).The second experiment tested four types of frames(no frame,consistent frame,half frame,and surrounding frame)across all agent sizes.Participants,utilizing a head mounted display,performed tasks involving moving cups,typing words,and using a mouse.They completed questionnaires assessing aspects such as the virtual environment effects,interaction effects,collision concerns,and preferences.Results Results from the first experiment revealed that agents matching physical object size produced the best overall performance.The second experiment demonstrated that consistent framing notably enhances interaction accuracy and speed but reduces immersion.To balance efficiency and immersion,frameless agents matching physical object sizes were deemed optimal.Conclusions Virtual agents matching physical entity sizes enhance user experience and interaction performance.Conversely,familiar frames from 2D interfaces detrimentally affect interaction and immersion in virtual spaces.This study provides valuable insights for the future development of MR systems.展开更多
Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based ...Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based on virtual reality and gesture interaction.Methods The parameters of the models were obtained through actual investigation,whereby Blender and 3DS MAX were used to model and import these parameters into a physics engine.By establishing an interface for the physics engine,gesture interaction hardware,and virtual reality(VR)helmet,a highly realistic chemical experiment environment was created.Using code script logic,particle systems,as well as other systems,chemical phenomena were simulated.Furthermore,we created an online teaching platform using streaming media and databases to address the problems of distance teaching.Results The proposed system was evaluated against two mainstream products in the market.In the experiments,the proposed system outperformed the other products in terms of fidelity and practicality.Conclusions The proposed system which offers realistic simulations and practicability,can help improve the high school chemistry experimental education.展开更多
Background Traditional methods for monitoring mining equipment rely primarily on visual inspections,which are time-consuming,inefficient,and hazardous.This article introduces a novel approach to monitoring mission-cri...Background Traditional methods for monitoring mining equipment rely primarily on visual inspections,which are time-consuming,inefficient,and hazardous.This article introduces a novel approach to monitoring mission-critical systems and services in the mining industry by integrating virtual reality(VR)and digital twin(DT)technologies.VR-based DTs enable remote equipment monitoring,advanced analysis of machine health,enhanced visualization,and improved decision making.Methods This article presents an architecture for VR-based DT development,including the developmental stages,activities,and stakeholders involved.A case study on the condition monitoring of a conveyor belt using real-time synthetic vibration sensor data was conducted using the proposed methodology.The study demonstrated the application of the methodology in remote monitoring and identified the need for further development for implementation in active mining operations.The article also discusses interdisciplinarity,choice of tools,computational resources,time and cost,human involvement,user acceptance,frequency of inspection,multiuser environment,potential risks,and applications beyond the mining industry.Results The findings of this study provide a foundation for future research in the domain of VR-based DTs for remote equipment monitoring and a novel application area for VR in mining.展开更多
BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolvi...BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.展开更多
Background A task assigned to space exploration satellites involves detecting the physical environment within a certain space.However,space detection data are complex and abstract.These data are not conducive for rese...Background A task assigned to space exploration satellites involves detecting the physical environment within a certain space.However,space detection data are complex and abstract.These data are not conducive for researchers'visual perceptions of the evolution and interaction of events in the space environment.Methods A time-series dynamic data sampling method for large-scale space was proposed for sample detection data in space and time,and the corresponding relationships between data location features and other attribute features were established.A tone-mapping method based on statistical histogram equalization was proposed and applied to the final attribute feature data.The visualization process is optimized for rendering by merging materials,reducing the number of patches,and performing other operations.Results The results of sampling,feature extraction,and uniform visualization of the detection data of complex types,long duration spans,and uneven spatial distributions were obtained.The real-time visualization of large-scale spatial structures using augmented reality devices,particularly low-performance devices,was also investigated.Conclusions The proposed visualization system can reconstruct the three-dimensional structure of a large-scale space,express the structure and changes in the spatial environment using augmented reality,and assist in intuitively discovering spatial environmental events and evolutionary rules.展开更多
文摘Canada is a multicultural country which was mainly established by immigrants.Just because of that,Canadian government has carried out the policy of multiculturalism since1970s.However,it has encountered many problems such as policy conflicts,national identity,democracy-inquiry and racial discrimination,etc.Hence the Canadian multiculturalism has been in a dilemma.
基金by the National Natural Science Foundation of China(NSFC)project“Study on China’s Pathway of Actions for Developing into a Manufacturing Powerhouse”(Grant No.:71673296).
文摘Economic globalization has catalyzed the global flow of economic factors and set the stage for the rise of fast-developing emerging economies with worldwide influence.Research has found that the emerging economies of China,Brazil,Russia,India,and South Africa have experienced significantly varying changes in their per capita GDP over time.Over the past decade,China has maintained a steady rate of economic growth with subdued volatility.In contrast,Brazil,Russia,India,and South Africa have all registered more volatility in their rates of growth.China has no significant correlation with other emerging economies in terms of imports and exports in goods and services and foreign direct investment(FDI),and their mutual influences are not evident.Going forward,for sustainable socio-economic development,China needs to optimize its industrial structure,promote the enhancement of ecological and institutional environments,and bring about all-round socio-economic progress,by initiating a new round of opening up at a higher level,broadening the space for international cooperation,building an open economy,promoting equal opportunity,and pursuing an inclusive path of growth.
文摘It bus been 40 years since China embarked on "crossing the river by feel ing the stones," navigating through the untrodden path and uncharted waters to drive its iconic modemization effort through reform and opening up.
基金supported by the National Natural Science Foundation of China (11904302)the Fundamental Research Funds for the Central Universities (Grant No.20720190005)+1 种基金the Major Science and Technology Project of Xiamen in China (3502Z20191015)Hong Kong University of Science and Technology-Foshan Joint Research Program (FSUST19-FYTRI11)
文摘Augmented reality(AR)and virtual reality(VR)are two novel display technologies that are under updates.The essential feature of AR/VR is the full-color display that requires high pixel densities.To generate three-color pixels,the fluorescent color conversion layer inevitably includes green and red pixels.To fabricate such sort of display kits,inkjet printing is a promising way to position the color conversion layers.In this review article,the progress of AR/VR technologies is first reviewed,and in succession,the state of the art of inkjet printing,as well as two key issues-the optimization of ink and the reduction of coffee-ring effects,are introduced.Finally,some potential problems associated with the color converting layer are highlighted.
基金funded by the European Commission through the H2020 project Hexa-X(Grant Agreement no.101015956).
文摘In the context of Industry 4.0,a paradigm shift from traditional industrial manipulators to Collaborative Robots(CRs)is ongoing,with the latter serving ever more closely humans as auxiliary tools in many production processes.In this scenario,continuous technological advancements offer new opportunities for further innovating robotics and other areas of next-generation industry.For example,6G could play a prominent role due to its human-centric view of the industrial domains.In particular,its expected dependability features will pave the way for new applications exploiting highly effective Digital Twin(DT)-and eXtended Reality(XR)-based telepresence.In this work,a novel application for the above technologies allowing two distant users to collaborate in the programming of a CR is proposed.The approach encompasses demanding data flows(e.g.,point cloud-based streaming of collaborating users and robotic environment),with network latency and bandwidth constraints.Results obtained by analyzing this approach from the viewpoint of network requirements in a setup designed to emulate 6G connectivity indicate that the expected performance of forthcoming mobile networks will make it fully feasible in principle.
基金the National Key Research and Development Program of China(2021ZD0150200)the Beijing Nova Program.
文摘In this work,we present a reconfigurable data glove design to capture different modes of human hand-object interactions,which are critical in training embodied artificial intelligence(AI)agents for fine manipulation tasks.To achieve various downstream tasks with distinct features,our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time.In the tactile-sensing mode,the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material;this design minimizes interference during complex hand movements.The virtual reality(VR)mode enables real-time interaction in a physically plausible fashion:A caging-based approach is devised to determine stable grasps by detecting collision events.Leveraging a state-of-the-art finite element method,the simulation mode collects data on fine-grained four-dimensionalmanipulation events comprising hand and object motions in three-dimensional space and how the object's physical properties(e.g.,stress and energy)change in accordance with manipulation over time.Notably,the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions.In a series of experiments,we characterize our data glove in terms of individual sensors and the overall system.More specifically,we evaluate the system's three modes by①recording hand gestures and associated forces,②improving manipulation fluency in VR,and③producing realistic simulation effects of various tool uses,respectively.Based on these three modes,our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments,thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.
基金Supported by Instituto de Salud CarlosⅢ(ISCⅢ),with group funds the Research Network on Chronicity,Primary Care and Health Promotion(RICAPPS,RD21/0016/0005)that is part of the Results-Oriented Cooperative Research Networks in Health(RICORS)(CarlosⅢHealth Institute),co-funded by the European Union“NextGeneration EU/PRTR”funds and with group funds Mental health research group in Primary Care(B17_23R),which is part of the Department of Innovation,Research,and University in the Government of Aragón(Spain).
文摘Current rates of mental illness are worrisome.Mental illness mainly affects females and younger age groups.The use of the internet to deliver mental health care has been growing since 2020 and includes the implementation of novel mental health treatments using virtual reality,augmented reality,and artificial intelligence.A new three dimensional digital environment,known as the metaverse,has emerged as the next version of the Internet.Artificial intelligence,augmented reality,and virtual reality will create fully immersive,experiential,and interactive online environments in the metaverse.People will use a unique avatar to do anything they do in their“real”lives,including seeking and receiving mental health care.In this opinion review,we reflect on how the metaverse could reshape how we deliver mental health treatment,its opportunities,and its challenges.
基金financially supported by the Ministry of Trade,Industry and Energy(MOTIE)and Korea Institute for Advancement of Technology(KIAT)through the International Cooperative R&D Program(Project No.P0016038)supported by the MSIT(Ministry of Sci-ence and ICT),Korea,under the ITRC(Information Technology Research Center)support program(IITP-2022-RS-2022-00156354)supervised by the IITP(Institute for Information&Communications Technology Planning&Evaluation).
文摘Overtaking is a crucial maneuver in road transportation that requires a clear view of the road ahead.However,limited visibility of ahead vehicles can often make it challenging for drivers to assess the safety of overtaking maneuvers,leading to accidents and fatalities.In this paper,we consider atrous convolution,a powerful tool for explicitly adjusting the field-of-view of a filter as well as controlling the resolution of feature responses generated by Deep Convolutional Neural Networks in the context of semantic image segmentation.This article explores the potential of seeing-through vehicles as a solution to enhance overtaking safety.See-through vehicles leverage advanced technologies such as cameras,sensors,and displays to provide drivers with a real-time view of the vehicle ahead,including the areas hidden from their direct line of sight.To address the problems of safe passing and occlusion by huge vehicles,we designed a see-through vehicle system in this study,we employed a windshield display in the back car together with cameras in both cars.The server within the back car was used to segment the car,and the segmented portion of the car displayed the video from the front car.Our see-through system improves the driver’s field of vision and helps him change lanes,cross a large car that is blocking their view,and safely overtake other vehicles.Our network was trained and tested on the Cityscape dataset using semantic segmentation.This transparent technique will instruct the driver on the concealed traffic situation that the front vehicle has obscured.For our findings,we have achieved 97.1% F1-score.The article also discusses the challenges and opportunities of implementing see-through vehicles in real-world scenarios,including technical,regulatory,and user acceptance factors.
文摘Six degrees of freedom(6DoF)input interfaces are essential formanipulating virtual objects through translation or rotation in three-dimensional(3D)space.A traditional outside-in tracking controller requires the installation of expensive hardware in advance.While inside-out tracking controllers have been proposed,they often suffer from limitations such as interaction limited to the tracking range of the sensor(e.g.,a sensor on the head-mounted display(HMD))or the need for pose value modification to function as an input interface(e.g.,a sensor on the controller).This study investigates 6DoF pose estimation methods without restricting the tracking range,using a smartphone as a controller in augmented reality(AR)environments.Our approach involves proposing methods for estimating the initial pose of the controller and correcting the pose using an inside-out tracking approach.In addition,seven pose estimation algorithms were presented as candidates depending on the tracking range of the device sensor,the tracking method(e.g.,marker recognition,visual-inertial odometry(VIO)),and whether modification of the initial pose is necessary.Through two experiments(discrete and continuous data),the performance of the algorithms was evaluated.The results demonstrate enhanced final pose accuracy achieved by correcting the initial pose.Furthermore,the importance of selecting the tracking algorithm based on the tracking range of the devices and the actual input value of the 3D interaction was emphasized.
文摘A catadioptric lens structure,also known as pancake lens,has been widely used in virtual reality(VR)displays to reduce the formfactor.However,the utilization of a half mirror(HM)to fold the optical path thrice leads to a significant optical loss.The theoretical maximum optical efficiency is merely 25%.To transcend this optical efficiency constraint while retaining the foldable characteristic inherent to traditional pancake optics,in this paper,we propose a theoretically lossless folded optical system to replace the HM with a nonreciprocal polarization rotator.In our feasibility demonstration experiment,we used a commercial Faraday rotator(FR)and reflective polarizers to replace the lossy HM.The theoretically predicted 100%efficiency can be achieved approximately by using two high-extinction-ratio reflective polarizers.In addition,we evaluated the ghost images using a micro-OLED panel in our imaging system.Indeed,the ghost images can be suppressed to undetectable level if the optics are with antireflection coating.Our novel pancake optical system holds great potential for revolutionizing next-generation VR displays with lightweight,compact formfactor,and low power consumption.
文摘The National Natural Science Foundation of China is one of the major funding agencies for neuro rehabilitation research in China.This study reviews the frontier directions and achievements in the field of neurorehabilitation in China and wo rldwide.We used data from the Web of Science Core Collection(WoSCC) database to analyze the publications and data provided by the National Natural Science Foundation of China to analyze funding information.In addition,the prospects for neurorehabilitation research in China are discussed.From 2010 to 2022,a total of 74,220 publications in neurorehabilitation were identified,with there being an overall upward tendency.During this period,the National Natural Science Foundation of China has funded 476 research projects with a total funding of 192.38 million RMB to support neuro rehabilitation research in China.With the support of the National Natural Science Foundation of China,China has made some achievements in neurorehabilitation research.Research related to neurorehabilitation is believed to be making steady and significant progress in China.
文摘Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training through practical exercises on abstracted and reduced models.Methods The authors strive for a portable,easy to use and cost-effective Virtual Reality-based(VR)laparoscopic pre-training platform and therefore address the question of how such a system has to be designed to achieve the quality of today's gold standard using real tissue specimens.Current VR controllers are limited regarding haptic feedback.Since haptic feedback is necessary or at least beneficial for laparoscopic surgery training,the platform to be developed consists of a newly designed prototype laparoscopic VR controller with haptic feedback,a commercially available head-mounted display,a VR environment for simulating a laparoscopic surgery,and a training concept.Results To take full advantage of benefits such as repeatability and cost-effectiveness of VR-based training,the system shall not require a tissue sample for haptic feedback.It is currently calculated and visually displayed to the user in the VR environment.On the prototype controller,a first axis was provided with perceptible feedback for test purposes.Two of the prototype VR controllers can be combined to simulate a typical both-handed use case,e.g.,laparoscopic suturing.A Unity based VR prototype allows the execution of simple standard pre-trainings.Conclusions The first prototype enables full operation of a virtual laparoscopic instrument in VR.In addition,the simulation can compute simple interaction forces.Major challenges lie in a realistic real-time tissue simulation and calculation of forces for the haptic feedback.Mechanical weaknesses were identified in the first hardware prototype,which will be improved in subsequent versions.All degrees of freedom of the controller are to be provided with haptic feedback.To make forces tangible in the simulation,characteristic values need to be determined using real tissue samples.The system has yet to be validated by cross-comparing real and VR haptics with surgeons.
文摘Background The sense of touch plays a crucial role in interactive behavior within virtual spaces,particularly when visual attention is absent.Although haptic feedback has been widely used to compensate for the lack of visual cues,the use of tactile information as a predictive feedforward cue to guide hand movements remains unexplored and lacks theoretical understanding.Methods This study introduces a fingertip aero-haptic rendering method to investigate its effectiveness in directing hand movements during eyes-free spatial interactions.The wearable device incorporates a multichannel micro-airflow chamber to deliver adjustable tactile effects on the fingertips.Results The first study verified that tactile directional feedforward cues significantly improve user capabilities in eyes-free target acquisition and that users rely heavily on haptic indications rather than spatial memory to control their hands.A subsequent study examined the impact of enriched tactile feedforward cues on assisting users in determining precise target positions during eyes-free interactions,and assessed the required learning efforts.Conclusions The haptic feedforward effect holds great practical promise in eyeless design for virtual reality.We aim to integrate cognitive models and tactile feedforward cues in the future,and apply richer tactile feedforward information to alleviate users'perceptual deficiencies.
文摘López del Hoyo et al collections reported the meta verse based on the virtual reality,augmented reality and artificial intelligence could be used in the therapy of mental health,although there were still some challenges.This manuscript reported that the meta verse is a prospective method to improve the prognosis of mental health problems.
基金the Strategic research and consulting project of Chinese Academy of Engineering(2023-HY-14).
文摘Background Physical entity interactions in mixed reality(MR)environments aim to harness human capabilities in manipulating physical objects,thereby enhancing virtual environment(VEs)functionality.In MR,a common strategy is to use virtual agents as substitutes for physical entities,balancing interaction efficiency with environmental immersion.However,the impact of virtual agent size and form on interaction performance remains unclear.Methods Two experiments were conducted to explore how virtual agent size and form affect interaction performance,immersion,and preference in MR environments.The first experiment assessed five virtual agent sizes(25%,50%,75%,100%,and 125%of physical size).The second experiment tested four types of frames(no frame,consistent frame,half frame,and surrounding frame)across all agent sizes.Participants,utilizing a head mounted display,performed tasks involving moving cups,typing words,and using a mouse.They completed questionnaires assessing aspects such as the virtual environment effects,interaction effects,collision concerns,and preferences.Results Results from the first experiment revealed that agents matching physical object size produced the best overall performance.The second experiment demonstrated that consistent framing notably enhances interaction accuracy and speed but reduces immersion.To balance efficiency and immersion,frameless agents matching physical object sizes were deemed optimal.Conclusions Virtual agents matching physical entity sizes enhance user experience and interaction performance.Conversely,familiar frames from 2D interfaces detrimentally affect interaction and immersion in virtual spaces.This study provides valuable insights for the future development of MR systems.
基金National Innovation and Entrepreneurship Program for College Students(202218213001)Science and Technology Innovation Strategy of Guangdong Province(Science and Technology Innovation Cultivation of University Students 2020329182130C000002).
文摘Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based on virtual reality and gesture interaction.Methods The parameters of the models were obtained through actual investigation,whereby Blender and 3DS MAX were used to model and import these parameters into a physics engine.By establishing an interface for the physics engine,gesture interaction hardware,and virtual reality(VR)helmet,a highly realistic chemical experiment environment was created.Using code script logic,particle systems,as well as other systems,chemical phenomena were simulated.Furthermore,we created an online teaching platform using streaming media and databases to address the problems of distance teaching.Results The proposed system was evaluated against two mainstream products in the market.In the experiments,the proposed system outperformed the other products in terms of fidelity and practicality.Conclusions The proposed system which offers realistic simulations and practicability,can help improve the high school chemistry experimental education.
基金the Natural Sciences and Engineering Research Council of Canada(NSERC)under GR012389.
文摘Background Traditional methods for monitoring mining equipment rely primarily on visual inspections,which are time-consuming,inefficient,and hazardous.This article introduces a novel approach to monitoring mission-critical systems and services in the mining industry by integrating virtual reality(VR)and digital twin(DT)technologies.VR-based DTs enable remote equipment monitoring,advanced analysis of machine health,enhanced visualization,and improved decision making.Methods This article presents an architecture for VR-based DT development,including the developmental stages,activities,and stakeholders involved.A case study on the condition monitoring of a conveyor belt using real-time synthetic vibration sensor data was conducted using the proposed methodology.The study demonstrated the application of the methodology in remote monitoring and identified the need for further development for implementation in active mining operations.The article also discusses interdisciplinarity,choice of tools,computational resources,time and cost,human involvement,user acceptance,frequency of inspection,multiuser environment,potential risks,and applications beyond the mining industry.Results The findings of this study provide a foundation for future research in the domain of VR-based DTs for remote equipment monitoring and a novel application area for VR in mining.
文摘BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.
文摘Background A task assigned to space exploration satellites involves detecting the physical environment within a certain space.However,space detection data are complex and abstract.These data are not conducive for researchers'visual perceptions of the evolution and interaction of events in the space environment.Methods A time-series dynamic data sampling method for large-scale space was proposed for sample detection data in space and time,and the corresponding relationships between data location features and other attribute features were established.A tone-mapping method based on statistical histogram equalization was proposed and applied to the final attribute feature data.The visualization process is optimized for rendering by merging materials,reducing the number of patches,and performing other operations.Results The results of sampling,feature extraction,and uniform visualization of the detection data of complex types,long duration spans,and uneven spatial distributions were obtained.The real-time visualization of large-scale spatial structures using augmented reality devices,particularly low-performance devices,was also investigated.Conclusions The proposed visualization system can reconstruct the three-dimensional structure of a large-scale space,express the structure and changes in the spatial environment using augmented reality,and assist in intuitively discovering spatial environmental events and evolutionary rules.