Research on Visualization and Interactive Design of Plant Configuration Virtual Reality in the Context of Digitalization

This study aims to explore the application of digital technology in landscape design,focusing on the research of virtual reality visualization and interactive design in the process of plant configuration.Through an in-depth analysis of digital technology,the study outlines its important role in landscape design,especially in the application of plant configuration.The current application status of virtual reality technology in landscape design is discussed,as well as how interactive design can enhance user experience and participation.Furthermore,the achievements and challenges of digital technology in landscape design are summarized.Finally,it proposes future research directions and suggestions,aiming to provide new ideas and methods for practice and research in the field of landscape design and promote the further application and development of digital technology in landscape design.

Personalized assessment and training of neurosurgical skills in virtual reality:An interpretable machine learning approach

Background Virtual reality technology has been widely used in surgical simulators,providing new opportunities for assessing and training surgical skills.Machine learning algorithms are commonly used to analyze and evaluate the performance of participants.However,their interpretability limits the personalization of the training for individual participants.Methods Seventy-nine participants were recruited and divided into three groups based on their skill level in intracranial tumor resection.Data on the use of surgical tools were collected using a surgical simulator.Feature selection was performed using the Minimum Redundancy Maximum Relevance and SVM-RFE algorithms to obtain the final metrics for training the machine learning model.Five machine learning algorithms were trained to predict the skill level,and the support vector machine performed the best,with an accuracy of 92.41%and Area Under Curve value of 0.98253.The machine learning model was interpreted using Shapley values to identify the important factors contributing to the skill level of each participant.Results This study demonstrates the effectiveness of machine learning in differentiating the evaluation and training of virtual reality neurosurgical performances.The use of Shapley values enables targeted training by identifying deficiencies in individual skills.Conclusions This study provides insights into the use of machine learning for personalized training in virtual reality neurosurgery.The interpretability of the machine learning models enables the development of individualized training programs.In addition,this study highlighted the potential of explanatory models in training external skills.

Effects of virtual agents on interaction efficiency and environmental immersion in MR environments

Background Physical entity interactions in mixed reality(MR)environments aim to harness human capabilities in manipulating physical objects,thereby enhancing virtual environment(VEs)functionality.In MR,a common strategy is to use virtual agents as substitutes for physical entities,balancing interaction efficiency with environmental immersion.However,the impact of virtual agent size and form on interaction performance remains unclear.Methods Two experiments were conducted to explore how virtual agent size and form affect interaction performance,immersion,and preference in MR environments.The first experiment assessed five virtual agent sizes(25%,50%,75%,100%,and 125%of physical size).The second experiment tested four types of frames(no frame,consistent frame,half frame,and surrounding frame)across all agent sizes.Participants,utilizing a head mounted display,performed tasks involving moving cups,typing words,and using a mouse.They completed questionnaires assessing aspects such as the virtual environment effects,interaction effects,collision concerns,and preferences.Results Results from the first experiment revealed that agents matching physical object size produced the best overall performance.The second experiment demonstrated that consistent framing notably enhances interaction accuracy and speed but reduces immersion.To balance efficiency and immersion,frameless agents matching physical object sizes were deemed optimal.Conclusions Virtual agents matching physical entity sizes enhance user experience and interaction performance.Conversely,familiar frames from 2D interfaces detrimentally affect interaction and immersion in virtual spaces.This study provides valuable insights for the future development of MR systems.

Chemical simulation teaching system based on virtual reality and gesture interaction

Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based on virtual reality and gesture interaction.Methods The parameters of the models were obtained through actual investigation,whereby Blender and 3DS MAX were used to model and import these parameters into a physics engine.By establishing an interface for the physics engine,gesture interaction hardware,and virtual reality(VR)helmet,a highly realistic chemical experiment environment was created.Using code script logic,particle systems,as well as other systems,chemical phenomena were simulated.Furthermore,we created an online teaching platform using streaming media and databases to address the problems of distance teaching.Results The proposed system was evaluated against two mainstream products in the market.In the experiments,the proposed system outperformed the other products in terms of fidelity and practicality.Conclusions The proposed system which offers realistic simulations and practicability,can help improve the high school chemistry experimental education.

Nanomaterial-based flexible sensors for metaverse and virtual reality applications

Nanomaterial-based flexible sensors(NMFSs)can be tightly attached to the human skin or integrated with clothing to monitor human physiological information,provide medical data,or explore metaverse spaces.Nanomaterials have been widely incorporated into flexible sensors due to their facile processing,material compatibility,and unique properties.This review highlights the recent advancements in NMFSs involving various nanomaterial frameworks such as nanoparticles,nanowires,and nanofilms.Different triggering interaction interfaces between NMFSs and metaverse/virtual reality(VR)applications,e.g.skin-mechanics-triggered,temperature-triggered,magnetically triggered,and neural-triggered interfaces,are discussed.In the context of interfacing physical and virtual worlds,machine learning(ML)has emerged as a promising tool for processing sensor data for controlling avatars in metaverse/VR worlds,and many ML algorithms have been proposed for virtual interaction technologies.This paper discusses the advantages,disadvantages,and prospects of NMFSs in metaverse/VR applications.

Eye-shaped keyboard for dual-hand text entry in virtual reality

We propose an eye-shaped keyboard for high-speed text entry in virtual reality (VR), having the shape of dual eyes with characters arranged along the curved eyelids, which ensures low density and short spacing of the keys. The eye-shaped keyboard references the QWERTY key sequence, allowing the users to benefit from their experience using the QWERTY keyboard. The user interacts with an eye-shaped keyboard using rays controlled with both the hands. A character can be entered in one step by moving the rays from the inner eye regions to regions of the characters. A high-speed auto-complete system was designed for the eye-shaped keyboard. We conducted a pilot study to determine the optimal parameters, and a user study to compare our eye-shaped keyboard with the QWERTY and circular keyboards. For beginners, the eye-shaped keyboard performed significantly more efficiently and accurately with less task load and hand movement than the circular keyboard. Compared with the QWERTY keyboard, the eye-shaped keyboard is more accurate and significantly reduces hand translation while maintaining similar efficiency. Finally, to evaluate the potential of eye-shaped keyboards, we conducted another user study. In this study, the participants were asked to type continuously for three days using the proposed eye-shaped keyboard, with two sessions per day. In each session, participants were asked to type for 20min, and then their typing performance was tested. The eye-shaped keyboard was proven to be efficient and promising, with an average speed of 19.89 words per minute (WPM) and mean uncorrected error rate of 1.939%. The maximum speed reached 24.97 WPM after six sessions and continued to increase.

Building Maintenance Supported on Virtual Reality Environments： Roofs and Walls

The paper presents the description of a research work that has its main objective as the development of a technological tool for supporting building maintenance with resort to new information and visualization technologies. Three main components of the building were analyzed： roof, facades and interior walls. The ceramic tile roof covering constitutes a component of the building envelope and fulfils an important function in its performance, namely in its protection against the permeation of moisture and rain water. Facade coating plays a significant role in the durability of buildings, since it constitutes the exterior layer that ensures wall protection against aggressive actions of physical, chemical or biological nature. The paint coating, applied to interior walls while improving their aesthetic character, performs an important function of protection against deterioration agents related to building use. A survey was conducted of the main anomalies that occur in these components, their respective causes and the adequate interventions, in order to plan maintenance strategies. The collected information serves as a basis for the implementation of applications using interactive visualization technologies to support the planning of building maintenance. During this work, the basic knowledge related to the materials, the techniques of rehabilitation and conservation and the planning of maintenance is outlined and discussed. In addition, methods of interconnecting this knowledge with the virtual applications were explored. The implemented prototypes were tested in real cases. This research work provides an innovative contribution to the field of maintenance, supported by emergent virtual reality technology.

Mixed Reality-based Interactive Technology for Aircraft Cabin Assembly

Due to the narrowness of space and the complexity of structure, the assembly of aircraft cabin has become one of the major bottlenecks in the whole manufacturing process. To solve the problem, at the beginning of aircraft design, the different stages of the lifecycle of aircraft must be thought about, which include the trial manufacture, assembly, maintenance, recycling and destruction of the product. Recently, thanks to the development of the virtual reality and augmented reality, some low-cost and fast solutions are found for the product assembly. This paper presents a mixed reality-based interactive technology for the aircraft cabin assembly, which can enhance the efficiency of the assemblage in a virtual environment in terms of vision, information and operation. In the mixed reality-based assembly environment, the physical scene can be obtained by a camera and then generated by a computer. The virtual parts, the features of visual assembly, the navigation information, the physical parts and the physical assembly environment will be mixed and presented in the same assembly scene. The mixed or the augmented information will provide some assembling information as a detailed assembly instruction in the mixed reality-based assembly environment. Constraint proxy and its match rules help to reconstruct and visualize the restriction relationship among different parts, and to avoid the complex calculation of constraint＇s match. Finally, a desktop prototype system of virtual assembly has been built to assist the assembly verification and training with the virtual hand.

Digital Twin for Human-Robot Interactive Welding and Welder Behavior Analysis

This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to enhance human users'operational productivity and comfort;while data-driven welder behavior analysis benefits to further novice welder training.This HRI system includes three modules:1)a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles;2)a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite;3)a DT system that is developed based on virtual reality(VR)as a digital replica of the physical human-robot interactive welding environment.The DT system bridges a human user and robot through a bi-directional information flow:a)transmitting demonstrated welding operations in VR to the robot in the physical environment;b)displaying the physical welding scenes to human users in VR.Compared to existing DT systems reported in the literatures,the developed one provides better capability in engaging human users in interacting with welding scenes,through an augmented VR.To verify the effectiveness,six welders,skilled with certain manual welding training and unskilled without any training,tested the system by completing the same welding job;three skilled welders produce satisfied welded workpieces,while the other three unskilled do not.A data-driven approach as a combination of fast Fourier transform(FFT),principal component analysis(PCA),and support vector machine(SVM)is developed to analyze their behaviors.Given an operation sequence,i.e.,motion speed sequence of the welding torch,frequency features are firstly extracted by FFT and then reduced in dimension through PCA,which are finally routed into SVM for classification.The trained model demonstrates a 94.44%classification accuracy in the testing dataset.The successful pattern recognition in skilled welder operations should benefit to accelerate novice welder training.

Application of the Somatosensory Interaction Technology Combined with Virtual Reality Technology on Upper Limbs Function in Cerebrovascular Disease Patients

Objective: To explore the effects of the somatosensory interaction technology combined with virtual reality technology on upper limbs function and activities of daily living (ADL) in cerebrovascular disease patients. Methods: Form January, 2019 to December, 2019, 80 cerebrovascular disease patients were recruited, and had been divided into control group (n = 40) and observation group (n = 40), randomly. The control groups received conventional rehabilitation treatment, for 40 minutes per day, while observation group received conventional rehabilitation treatment, for 20 minutes per day, and virtual reality technology treatment, 20 minutes per day, 5 days a week for 4 weeks. Wolf Motor Function Test (WMFT), Fugl-Meyer Assessment-Upper Extremities (FMA-UE) and modified Barthel index (MBI) were used to assess the motor function of the upper limbs and ADL before and after treatment. Results: Before treatment, the scores of WMFT, FMA-UE and MBI were no significant difference between two groups (P > 0.05). The scores improved in both groups after treatment (P < 0.01), and were higher in the observation group than in the control group (P < 0.05). Conclusion: The somatosensory interaction technology combined with virtual reality technology could facilitate to improve the upper limbs function and ADL in cerebrovascular disease patients.

Immersive Robot Control in Virtual Reality to Command Robots in Space Missions

We present an approach to control a semi-autonomous robot team remotely under low bandwidth conditions with a single operator. Our approach utilises virtual reality and autonomous robots to create an immersive user interface for multi-robot control. This saves a big amount of bandwidth, just because there is no need to transfer a constant steam of camera images. The virtual environment for control only has to be transferred once to the control station and only has to be updated when the map is out of date. Also, the camera position can easily be changed in virtual reality for more overview on the robots situation. The parts of this approach can easily be transferred to applications on earth e.g. for semi-autonomous robots in hazardous areas or under water applications.

Virtual Reality Technology Applied on Maintenance of Painted Walls of Buildings

In a building, the paint coating applied to interior walls conveys their aesthetic character and also performs an important function of protection. It is a construction component which is exposed to agents of deterioration related to its use, needing the regular evaluation of its state of repair. The completed model supports the performance of such periodic inspections and the monitoring of interior wall maintenance, using Virtual Reality (VR) technology. Used during an inspection visit, the application allows users to consult a database of irregularities, normally associated with paint coating, classified by the most probable causes and by recommended repair methodologies. In addition, with this model, a chromatic scale related to the degree of deterioration of the coating, defined as a function of the time between the dates of the application of the paint and the scheduled repainting, can be attributed to each element of coating monitored. This use of VR technology allows inspections and the evaluation of the degree of wear and tear of materials to be carried out in a highly direct and intuitive manner. The computer application has a positive contribution to make in the field of construction, using as it does Information Technology (IT) tools which give access to innovative technology with it capacity for interaction and visualization.

VREG: A Virtual Reality Educational Game with Arabic Content Using Android Smart Phone

In the last few decades, technology found its way from being a wild sci-fi dream, only seen in movies, to being an important element of all aspects of life. The technological industry bloomed quite rapidly where the technological solutions have made any process you can think of easier, faster and more efficient. Educational methodologies started off simple, having a teacher using simply his/her words and some writing along with it. With technology entering the field of education, teaching methods were enhanced and the quality of its outcomes was improved. However, sadly in Arab countries that was not the story. In other words, the use of new education technologies is still limited to basic and simple tasks. In fact, to add to the agony, most of the programs used are poorly implemented and rarely applied. Interestingly, as many studies confirm and from the simple observation of kids playing, it is found that most children are very attracted to new technologies and their applications. Those interests not only changed what a normal childhood would look like, but also introduced a multi-billion industry of electronic games. Virtual Reality (VR) games were introduced a few years back starting a gaming revolution, allowing gamers to enter the world of their dreams. Similar to most technologies, VR started to become more affordable with time. Multiple companies have competed to give us amazing VR products such as Samsung Gear. In this manuscript, we present a Virtual Reality Educational Game (VREG), which is an interactive 3D game with Arabic content that targets the students between age 4 and 8 and can be played on a smart phone. VREG includes multiple games that cover the different concepts of Arabic and English languages alongside math, aiming to help the student to learn through playing. In order to make sure that our game reaches wanted education standards, we put it to the test in two private schools;the feedback was very positive from both teachers and students where the majority of users enjoyed their experiments in VR games.

Gesture-based target acquisition in virtual and augmented reality

Background Gesture is a basic interaction channel that is frequently used by humans to communicate in daily life. In this paper, we explore to use gesture-based approaches for target acquisition in virtual and augmented reality. A typical process of gesture-based target acquisition is: when a user intends to acquire a target, she performs a gesture with her hands, head or other parts of the body, the computer senses and recognizes the gesture and infers the most possible target. Methods We build mental model and behavior model of the user to study two key parts of the interaction process. Mental model describes how user thinks up a gesture for acquiring a target, and can be the intuitive mapping between gestures and targets. Behavior model describes how user moves the body parts to perform the gestures, and the relationship between the gesture that user intends to perform and signals that computer senses. Results In this paper, we present and discuss three pieces of research that focus on the mental model and behavior model of gesture-based target acquisition in VR and AR. Conclusions We show that leveraging these two models, interaction experience and performance can be improved in VR and AR environments.

Classification and Selection of Prototyping Activities for Interaction Design

Because of the evolution of markets and technologies, prototyping concerns should be kept updated almost day by day. Moreover, user centered design moves the focus towards interaction issues. Prototyping activities matching such characteristics are already available, but they are not so diffused in the industrial domain. This is due to many reasons;an important one is that a rigorous classification of them is missing, as well as an effective helping tool for the selection of the best activities, given the design context. The research described in this paper aims at defining a new classification of prototyping activities, as well as at developing a selection algorithm to choose the best ones in an automatic way. These goals are pursued by defining a set of characteristics that allow describing accurately the prototyping activities. The resulting classification is made by five classes, based on eighteen characteristics. This classification is exploited by the first release of an algorithm for the selection of the best activities, chosen in order to satisfy design situations described thanks to a different set of eleven indices. Five experiences in the field have been used up to now as a starting point for validating the research outcomes.

Adaptive navigation assistance based on eye movement features in virtual reality

Background Navigation assistance is essential for users when roaming virtual reality scenes;however,the traditional navigation method requires users to manually request a map for viewing,which leads to low immersion and poor user experience.Methods To address this issue,we first collected data on who required navigation assistance in a virtual reality environment,including various eye movement features,such as gaze fixation,pupil size,and gaze angle.Subsequently,we used the boosting-based XGBoost algorithm to train a prediction model and finally used it to predict whether users require navigation assistance in a roaming task.Results After evaluating the performance of the model,the accuracy,precision,recall,and F1-score of our model reached approximately 95%.In addition,by applying the model to a virtual reality scene,an adaptive navigation assistance system based on the real-time eye movement data of the user was implemented.Conclusions Compared with traditional navigation assistance methods,our new adaptive navigation assistance method could enable the user to be more immersive and effective while roaming in a virtual reality(VR)environment.

自然人机交互技术在雷达显控终端中的应用

针对当前雷达显控终端承担的操控任务多、操作步骤复杂等问题,提出了利用自然人机交互技术提升人机交互效率的方法。采用触控显示器、触控一体机、视频处理模块、语音处理模块、指纹识别模块、虚拟现实(VR)交互设备等,通过增加触控、语音、图像处理、VR、生物特征鉴权等多种自然人机交互技术拓展了人机交互的方式;根据雷达显控终端的交互情景,采用选择型多模态融合交互方式,大幅提高了雷达显控终端的人机交互效率,使得人机交互更为自然和舒适。VR交互设备在多维信息展示、场景仿真与推演、模拟训练等场合发挥了明显的优势,自然化和人机融合的智能交互方式将是未来的发展趋势。

“双一流”背景下的“电路”课程建设探索

为建设和落实一批世界一流大学和一流学科的目标,以“电路”课程为例,针对课程中存在的痛点问题,课程团队遵循“以学生为中心,学生发展成效驱动”的创新教学理念,以电路内容为主线筑牢学生工程基础,融入课程思政,采用课前、课中、课后全程智慧教学和多元互动的方式,全方位评价学生能力,形成了“素质教育、知识引领”的双主线和“虚实结合、多元互动、强基立本、成效为先”的教学特色。

基于MOOC和虚拟现实的工程制图教学创新

为了应对因为卓越工程师教育培养计划和新工科建设的实施,对工程制图教学提出的更高要求,在传统工程制图教学方法的基础上,构建了工程制图课程的大型开放式网络课程(MOOC)和虚拟现实项目。学生可以通过线上视频随时预习与复习,将虚拟现实引入装配图教学,使学生更容易理解装拆顺序,增强学生的参与感。创新的教学方法以课程理论为基础,启发学生进行创新活动,增强学生的工程能力和创新能力,从而培养出适应工业发展的创新型人才。

用于触摸屏交互的可穿戴指端力反馈系统设计

力触觉再现技术在增强用户与触摸屏交互的真实感和沉浸感方面发挥着越来越重要的作用,设计了一种面向触摸屏图像信息感知的可穿戴指端力反馈装置。设计基于线绳牵引驱动的指端力反馈机构,将装置分为驱动和可穿戴部分,使其具备小巧、轻便、易佩戴的特点。设计测控系统,实现了输出反馈力的检测和控制。研究图像信息提取和力触觉建模算法,获取图像中物体的三维特征信息,配合力反馈装置和测控系统实现图像纹理高度、外形轮廓信息再现反馈。实验结果表明,该装置最大可以提供5.3 N的反馈力,能够帮助用户感知触摸屏中图像的高度和形状信息。