Chemical simulation teaching system based on virtual reality and gesture interaction

Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based on virtual reality and gesture interaction.Methods The parameters of the models were obtained through actual investigation,whereby Blender and 3DS MAX were used to model and import these parameters into a physics engine.By establishing an interface for the physics engine,gesture interaction hardware,and virtual reality(VR)helmet,a highly realistic chemical experiment environment was created.Using code script logic,particle systems,as well as other systems,chemical phenomena were simulated.Furthermore,we created an online teaching platform using streaming media and databases to address the problems of distance teaching.Results The proposed system was evaluated against two mainstream products in the market.In the experiments,the proposed system outperformed the other products in terms of fidelity and practicality.Conclusions The proposed system which offers realistic simulations and practicability,can help improve the high school chemistry experimental education.

Effects of virtual agents on interaction efficiency and environmental immersion in MR environments

Background Physical entity interactions in mixed reality(MR)environments aim to harness human capabilities in manipulating physical objects,thereby enhancing virtual environment(VEs)functionality.In MR,a common strategy is to use virtual agents as substitutes for physical entities,balancing interaction efficiency with environmental immersion.However,the impact of virtual agent size and form on interaction performance remains unclear.Methods Two experiments were conducted to explore how virtual agent size and form affect interaction performance,immersion,and preference in MR environments.The first experiment assessed five virtual agent sizes(25%,50%,75%,100%,and 125%of physical size).The second experiment tested four types of frames(no frame,consistent frame,half frame,and surrounding frame)across all agent sizes.Participants,utilizing a head mounted display,performed tasks involving moving cups,typing words,and using a mouse.They completed questionnaires assessing aspects such as the virtual environment effects,interaction effects,collision concerns,and preferences.Results Results from the first experiment revealed that agents matching physical object size produced the best overall performance.The second experiment demonstrated that consistent framing notably enhances interaction accuracy and speed but reduces immersion.To balance efficiency and immersion,frameless agents matching physical object sizes were deemed optimal.Conclusions Virtual agents matching physical entity sizes enhance user experience and interaction performance.Conversely,familiar frames from 2D interfaces detrimentally affect interaction and immersion in virtual spaces.This study provides valuable insights for the future development of MR systems.

Research on Visualization and Interactive Design of Plant Configuration Virtual Reality in the Context of Digitalization

This study aims to explore the application of digital technology in landscape design,focusing on the research of virtual reality visualization and interactive design in the process of plant configuration.Through an in-depth analysis of digital technology,the study outlines its important role in landscape design,especially in the application of plant configuration.The current application status of virtual reality technology in landscape design is discussed,as well as how interactive design can enhance user experience and participation.Furthermore,the achievements and challenges of digital technology in landscape design are summarized.Finally,it proposes future research directions and suggestions,aiming to provide new ideas and methods for practice and research in the field of landscape design and promote the further application and development of digital technology in landscape design.

Application of the Somatosensory Interaction Technology Combined with Virtual Reality Technology on Upper Limbs Function in Cerebrovascular Disease Patients

Objective: To explore the effects of the somatosensory interaction technology combined with virtual reality technology on upper limbs function and activities of daily living (ADL) in cerebrovascular disease patients. Methods: Form January, 2019 to December, 2019, 80 cerebrovascular disease patients were recruited, and had been divided into control group (n = 40) and observation group (n = 40), randomly. The control groups received conventional rehabilitation treatment, for 40 minutes per day, while observation group received conventional rehabilitation treatment, for 20 minutes per day, and virtual reality technology treatment, 20 minutes per day, 5 days a week for 4 weeks. Wolf Motor Function Test (WMFT), Fugl-Meyer Assessment-Upper Extremities (FMA-UE) and modified Barthel index (MBI) were used to assess the motor function of the upper limbs and ADL before and after treatment. Results: Before treatment, the scores of WMFT, FMA-UE and MBI were no significant difference between two groups (P > 0.05). The scores improved in both groups after treatment (P < 0.01), and were higher in the observation group than in the control group (P < 0.05). Conclusion: The somatosensory interaction technology combined with virtual reality technology could facilitate to improve the upper limbs function and ADL in cerebrovascular disease patients.

Gesture interaction in virtual reality

With the development of virtual reality(VR)and human-computer interaction technology,how to use natural and efficient interaction methods in the virtual environment has become a hot topic of research.Gesture is one of the most important communication methods of human beings,which can effectively express users'demands.In the past few decades,gesture-based interaction has made significant progress.This article focuses on the gesture interaction technology and discusses the definition and classification of gestures,input devices for gesture interaction,and gesture interaction recognition technology.The application of gesture interaction technology in virtual reality is studied,the existing problems in the current gesture interaction are summarized,and the future development is prospected.

Parallel Sensing in Metaverses: Virtual-Real Interactive Smart Systems for “6S” Sensing

In the construction of Metaverses,sensors that are referred to as the“bridge of information transmission”,play a key role.The functionality and efficiency of today’s sensors,which operate in a manner similar to physical sensing,are frequently constrained by their hardware and software.In this research,we proposed the Parallel Sensing framework,which includes background,concept,basic methods and typical application of parallel sensing.In our formulation,sensors are redefined as the integration of real physical sensors and virtual software-defined sensors based on parallel intelligence,in order to boost the performance of the sensors.Each sensor will have a parallel counterpart in the virtual world within the framework of parallel sensing.Digital sensors serve as the brain of sensors and maintain the same properties as physical sensors.Parallel sensing allows physical sensors to operate in discrete time periods to conserve energy,while cloud-based descriptive,predictive,and prescriptive sensors operate continuously to offer compensation data and serve as guardians.To better illustrate parallel sensing concept,we show some example applications of parallel sensing such as parallel vision,parallel point cloud and parallel light fields,both of which are designed by construct virtual sensors to extend small real data to virtual big data and then boost the performance of perception models.Experimental results demonstrate the effective of parallel sensing framework.The interaction between the real and virtual worlds enables sensors to operate actively,allowing them to intelligently adapt to various scenarios and ultimately attain the goal of“Cognitive,Parallel,Crypto,Federated,Social and Ecologic”6S sensing.

Personalized assessment and training of neurosurgical skills in virtual reality:An interpretable machine learning approach

Background Virtual reality technology has been widely used in surgical simulators,providing new opportunities for assessing and training surgical skills.Machine learning algorithms are commonly used to analyze and evaluate the performance of participants.However,their interpretability limits the personalization of the training for individual participants.Methods Seventy-nine participants were recruited and divided into three groups based on their skill level in intracranial tumor resection.Data on the use of surgical tools were collected using a surgical simulator.Feature selection was performed using the Minimum Redundancy Maximum Relevance and SVM-RFE algorithms to obtain the final metrics for training the machine learning model.Five machine learning algorithms were trained to predict the skill level,and the support vector machine performed the best,with an accuracy of 92.41%and Area Under Curve value of 0.98253.The machine learning model was interpreted using Shapley values to identify the important factors contributing to the skill level of each participant.Results This study demonstrates the effectiveness of machine learning in differentiating the evaluation and training of virtual reality neurosurgical performances.The use of Shapley values enables targeted training by identifying deficiencies in individual skills.Conclusions This study provides insights into the use of machine learning for personalized training in virtual reality neurosurgery.The interpretability of the machine learning models enables the development of individualized training programs.In addition,this study highlighted the potential of explanatory models in training external skills.

Virtual Keyboard:A Real-Time Hand Gesture Recognition-Based Character Input System Using LSTM and Mediapipe Holistic

In the digital age,non-touch communication technologies are reshaping human-device interactions and raising security concerns.A major challenge in current technology is the misinterpretation of gestures by sensors and cameras,often caused by environmental factors.This issue has spurred the need for advanced data processing methods to achieve more accurate gesture recognition and predictions.Our study presents a novel virtual keyboard allowing character input via distinct hand gestures,focusing on two key aspects:hand gesture recognition and character input mechanisms.We developed a novel model with LSTM and fully connected layers for enhanced sequential data processing and hand gesture recognition.We also integrated CNN,max-pooling,and dropout layers for improved spatial feature extraction.This model architecture processes both temporal and spatial aspects of hand gestures,using LSTM to extract complex patterns from frame sequences for a comprehensive understanding of input data.Our unique dataset,essential for training the model,includes 1,662 landmarks from dynamic hand gestures,33 postures,and 468 face landmarks,all captured in real-time using advanced pose estimation.The model demonstrated high accuracy,achieving 98.52%in hand gesture recognition and over 97%in character input across different scenarios.Its excellent performance in real-time testing underlines its practicality and effectiveness,marking a significant advancement in enhancing human-device interactions in the digital age.

Real Time Math Simulation of Contact Interaction during Spacecraft Docking and Berthing

Contact reactions of guide surfaces of assembly interfaces lead to the decreasing of theirs lateral and angular misalignments. The focus of this paper is the development of algorithms for computation of guide surfaces contact forces with acceptable engineering accuracy for real time simulation of assembly operations. Therefore, each complex guide surface is described as a set of contacting elements. Each contacting element for one＇s part can be represented by a finite set of geometric primitives which geometry is described by low order algebraic equations. So contact conditions and geometric parameters for all pairs of primitives are determined by analytical expressions. Math models are developed for two classes of contact interaction. The first class includes all cases when each contacting surface has several degrees of freedom of motion. Therefore, contact reactions introduced into differential equations of motion are calculated by using contacting elements penetrations, stiffness and damping parameters. The second class corresponds to all cases when one of contacting surfaces has insignificant inertia and only one degree of freedom of relative displacement counteracted by a spring. Here contact reactions are calculated from spring tension with any practical accuracy. This is very useful in some practical applications. Presented algorithms provide real time simulation together with some approaches for reduction of redundant comnutations.

Factors That Influence Healthcare Professionals’ Online Interaction in a Virtual Community of Practice

Online technologies have facilitated the development of Virtual Communities of Practice (virtual CoPs) to support health professionals collaborate online to share knowledge, improve performance and support the spread of innovation and best practices. Research, however,shows that many virtual CoPs do not achieve their expected potential because online interaction among healthcare professionals is generally low. Focusing on health visitors, who are UK qualified midwives or nurses who have undertaken additional qualifications as specialist public health workers in the community, the paper examines the factors that influence online interaction among health visitors collaborating to share knowledge and experience in a virtual CoP. The paper makes suggestions for how to improve online interaction among health professionals in virtual CoPs by increasing the size of membership in order to take advantage of both posting and viewing contributions, facilitating moderation to improve networking among geographically dispersed members groups and im-proving the topic relevance in order to stimulate contributions.

Effective Learner-Lecturer Interaction Working With a Virtual Learning Environment

Using the Internet to learn a language creates wide opportunities to enhance learning （Association of teachers of English in Catalonia （APAC）, 2010）. The Internet activities promote learners＇ self-monitoring ability, encourage the use of multimedia and network technology, and develop students＇ cooperation and participation. During the latest years, there have been many changes in education as these new technologies, including VLEs （Virtual Learning Environments）, which have become an important part in the teaching/learning process. According to Tech Terms Computer Dictionary （2012）, VLE is a virtual classroom where teachers and students communicate. VLEs have evolved as at an early stage, they were only ways of transmitting information： Teachers uploaded the multimedia resources and students read this information. At a higher stage, VLEs have become interactive. This means that students become active. We have designed a virtual environment where students, weekly, must contribute their opinions and comments in response to a required activity uploaded by the teacher. In this paper, we describe this weekly task and analyze students＇ opinion about this planned activity. The students become an active subject in this field. In this paper, we show how VLEs are no longer a means of transmitting information but a means of interaction as well as a way of motivating our students to be involved in their learning process

Mixed Reality-based Interactive Technology for Aircraft Cabin Assembly

Due to the narrowness of space and the complexity of structure, the assembly of aircraft cabin has become one of the major bottlenecks in the whole manufacturing process. To solve the problem, at the beginning of aircraft design, the different stages of the lifecycle of aircraft must be thought about, which include the trial manufacture, assembly, maintenance, recycling and destruction of the product. Recently, thanks to the development of the virtual reality and augmented reality, some low-cost and fast solutions are found for the product assembly. This paper presents a mixed reality-based interactive technology for the aircraft cabin assembly, which can enhance the efficiency of the assemblage in a virtual environment in terms of vision, information and operation. In the mixed reality-based assembly environment, the physical scene can be obtained by a camera and then generated by a computer. The virtual parts, the features of visual assembly, the navigation information, the physical parts and the physical assembly environment will be mixed and presented in the same assembly scene. The mixed or the augmented information will provide some assembling information as a detailed assembly instruction in the mixed reality-based assembly environment. Constraint proxy and its match rules help to reconstruct and visualize the restriction relationship among different parts, and to avoid the complex calculation of constraint＇s match. Finally, a desktop prototype system of virtual assembly has been built to assist the assembly verification and training with the virtual hand.

A SEMI-ANALYTICAL AND SEMI-NUMERICAL METHOD FOR SOLVING 2-D SOUND-STRUCTURE INTERACTION PROBLEMS

Based on the transfer matrix method and the virtual source simulation technique, this paper proposes a novel semi-analytical and semi-numerical method for solving 2-D sound- structure interaction problems under a harmonic excitation.Within any integration segment, as long as its length is small enough,along the circumferential curvilinear coordinate,the non- homogeneous matrix differential equation of an elastic ring of complex geometrical shape can be rewritten in terms of the homogeneous one by the method of extended homogeneous capacity proposed in this paper.For the exterior fluid domain,the multi-circular virtual source simulation technique is adopted.The source density distributed on each virtual circular curve may be ex- panded as the Fourier's series.Combining with the inverse fast Fourier transformation,a higher accuracy and efficiency method for solving 2-D exterior Helmholtz's problems is presented in this paper.In the aspect of solution to the coupling equations,the state vectors of elastic ring induced by the given harmonic excitation and generalized forces of coefficients of the Fourier series can be obtained respectively by using a high precision integration scheme combined with the method of extended homogeneous capacity put forward in this paper.According to the superposition princi- ple and compatibility conditions at the interface between the elastic ring and fluid,the algebraic equation of system can be directly constructed by using the least square approximation.Examples of acoustic radiation from two typical fluid-loaded elastic rings under a harmonic concentrated force are presented.Numerical results show that the method proposed is more efficient than the mixed FE-BE method in common use.

Nanomaterial-based flexible sensors for metaverse and virtual reality applications

Nanomaterial-based flexible sensors(NMFSs)can be tightly attached to the human skin or integrated with clothing to monitor human physiological information,provide medical data,or explore metaverse spaces.Nanomaterials have been widely incorporated into flexible sensors due to their facile processing,material compatibility,and unique properties.This review highlights the recent advancements in NMFSs involving various nanomaterial frameworks such as nanoparticles,nanowires,and nanofilms.Different triggering interaction interfaces between NMFSs and metaverse/virtual reality(VR)applications,e.g.skin-mechanics-triggered,temperature-triggered,magnetically triggered,and neural-triggered interfaces,are discussed.In the context of interfacing physical and virtual worlds,machine learning(ML)has emerged as a promising tool for processing sensor data for controlling avatars in metaverse/VR worlds,and many ML algorithms have been proposed for virtual interaction technologies.This paper discusses the advantages,disadvantages,and prospects of NMFSs in metaverse/VR applications.

Study on the mechanism of interaction for coal and methane gas

Although two moulds for methane gas in coal with the free state and adsorption state have been popularly considered, the derivation between the real methane gas state equation in coal and the perfect gas state equation has been fuzzily considered and the mechanism of interaction for coal aromatics and methane gas molecules has not been understood. Then these problems have been discussed in this paper applied the principle of statistical thermo mechanics and quantum chemistry as well as based on the numerical calculating of experiential data in quantum chemistry. Therefore, it is revealed by research results that the experience state equation for real methane gas in coal, which is put forward in this paper, is closer to actual situation and the interaction process for methane gas adsorption on the surface of coal aromatics can be formulated by Morse potential function. Furthermore it is most stable through this research that the structural mould for methane gas molecule adsorption on the surface of coal nuclear with one gas molecule on top of another aromatics in regular triangle cone has been understood, and it is a physical adsorption for methane gas adsorption with single layer molecule on the surface of coal nuclear.

Novel 3-dimensional virtual hepatectomy simulation combined with real-time deformation

AIM:To develop a novel 3-dimensional(3D) virtual hepatectomy simulation software,Liversim,to visualize the real-time deformation of the liver.METHODS:We developed a novel real-time virtual hepatectomy simulation software program called Liversim. The software provides 4 basic functions:viewing 3D models from arbitrary directions,changing the colors and opacities of the models,deforming the models based on user interaction,and incising the liver parenchyma and intrahepatic vessels based on user operations. From April 2010 through 2013,99 patients underwent virtual hepatectomies that used the conventional software program SYNAPSE VINCENT preoperatively. Between April 2012 and October 2013,11 patients received virtual hepatectomies using the novel software program Liversim; these hepatectomies were performed both preoperatively and at the same that the actual hepatectomy was performed in an operating room. The perioperative outcomes were analyzed between the patients for whom SYNAPSE VINCENT was used and those for whom Liversim wasused. Furthermore,medical students and surgical residents were asked to complete questionnaires regarding the new software.RESULTS:There were no obvious discrepancies(i.e.,the emergence of branches in the portal vein or hepatic vein or the depth and direction of the resection line) between our simulation and the actual surgery during the resection process. The median operating time was 304 min(range,110 to 846) in the VINCENT group and 397 min(range,232 to 497) in the Liversim group(P = 0.30). The median amount of intraoperative bleeding was 510 m L(range,18 to 5120) in the VINCENT group and 470 m L(range,130 to 1600) in the Liversim group(P = 0.44). The median postoperative stay was 12 d(range,6 to 100) in the VINCENT group and 13 d(range,9 to 21) in the Liversim group(P = 0.36). There were no significant differences in the preoperative outcomes between the two groups. Liversim was not found to be clinically inferior to SYNAPSE VINCENT. Both students and surgical residents reported that the Liversim image was almost the same as the actual hepatectomy.CONCLUSION:Virtual hepatectomy with real-time deformation of the liver using Liversim is useful for the safe performance of hepatectomies and for surgical education.

Immersive Robot Control in Virtual Reality to Command Robots in Space Missions

We present an approach to control a semi-autonomous robot team remotely under low bandwidth conditions with a single operator. Our approach utilises virtual reality and autonomous robots to create an immersive user interface for multi-robot control. This saves a big amount of bandwidth, just because there is no need to transfer a constant steam of camera images. The virtual environment for control only has to be transferred once to the control station and only has to be updated when the map is out of date. Also, the camera position can easily be changed in virtual reality for more overview on the robots situation. The parts of this approach can easily be transferred to applications on earth e.g. for semi-autonomous robots in hazardous areas or under water applications.

Virtual Reality Technology Applied on Maintenance of Painted Walls of Buildings

In a building, the paint coating applied to interior walls conveys their aesthetic character and also performs an important function of protection. It is a construction component which is exposed to agents of deterioration related to its use, needing the regular evaluation of its state of repair. The completed model supports the performance of such periodic inspections and the monitoring of interior wall maintenance, using Virtual Reality (VR) technology. Used during an inspection visit, the application allows users to consult a database of irregularities, normally associated with paint coating, classified by the most probable causes and by recommended repair methodologies. In addition, with this model, a chromatic scale related to the degree of deterioration of the coating, defined as a function of the time between the dates of the application of the paint and the scheduled repainting, can be attributed to each element of coating monitored. This use of VR technology allows inspections and the evaluation of the degree of wear and tear of materials to be carried out in a highly direct and intuitive manner. The computer application has a positive contribution to make in the field of construction, using as it does Information Technology (IT) tools which give access to innovative technology with it capacity for interaction and visualization.

VREG: A Virtual Reality Educational Game with Arabic Content Using Android Smart Phone

In the last few decades, technology found its way from being a wild sci-fi dream, only seen in movies, to being an important element of all aspects of life. The technological industry bloomed quite rapidly where the technological solutions have made any process you can think of easier, faster and more efficient. Educational methodologies started off simple, having a teacher using simply his/her words and some writing along with it. With technology entering the field of education, teaching methods were enhanced and the quality of its outcomes was improved. However, sadly in Arab countries that was not the story. In other words, the use of new education technologies is still limited to basic and simple tasks. In fact, to add to the agony, most of the programs used are poorly implemented and rarely applied. Interestingly, as many studies confirm and from the simple observation of kids playing, it is found that most children are very attracted to new technologies and their applications. Those interests not only changed what a normal childhood would look like, but also introduced a multi-billion industry of electronic games. Virtual Reality (VR) games were introduced a few years back starting a gaming revolution, allowing gamers to enter the world of their dreams. Similar to most technologies, VR started to become more affordable with time. Multiple companies have competed to give us amazing VR products such as Samsung Gear. In this manuscript, we present a Virtual Reality Educational Game (VREG), which is an interactive 3D game with Arabic content that targets the students between age 4 and 8 and can be played on a smart phone. VREG includes multiple games that cover the different concepts of Arabic and English languages alongside math, aiming to help the student to learn through playing. In order to make sure that our game reaches wanted education standards, we put it to the test in two private schools;the feedback was very positive from both teachers and students where the majority of users enjoyed their experiments in VR games.

Adaptive navigation assistance based on eye movement features in virtual reality

Background Navigation assistance is essential for users when roaming virtual reality scenes;however,the traditional navigation method requires users to manually request a map for viewing,which leads to low immersion and poor user experience.Methods To address this issue,we first collected data on who required navigation assistance in a virtual reality environment,including various eye movement features,such as gaze fixation,pupil size,and gaze angle.Subsequently,we used the boosting-based XGBoost algorithm to train a prediction model and finally used it to predict whether users require navigation assistance in a roaming task.Results After evaluating the performance of the model,the accuracy,precision,recall,and F1-score of our model reached approximately 95%.In addition,by applying the model to a virtual reality scene,an adaptive navigation assistance system based on the real-time eye movement data of the user was implemented.Conclusions Compared with traditional navigation assistance methods,our new adaptive navigation assistance method could enable the user to be more immersive and effective while roaming in a virtual reality(VR)environment.