Adaptive navigation assistance based on eye movement features in virtual reality

Background Navigation assistance is essential for users when roaming virtual reality scenes;however,the traditional navigation method requires users to manually request a map for viewing,which leads to low immersion and poor user experience.Methods To address this issue,we first collected data on who required navigation assistance in a virtual reality environment,including various eye movement features,such as gaze fixation,pupil size,and gaze angle.Subsequently,we used the boosting-based XGBoost algorithm to train a prediction model and finally used it to predict whether users require navigation assistance in a roaming task.Results After evaluating the performance of the model,the accuracy,precision,recall,and F1-score of our model reached approximately 95%.In addition,by applying the model to a virtual reality scene,an adaptive navigation assistance system based on the real-time eye movement data of the user was implemented.Conclusions Compared with traditional navigation assistance methods,our new adaptive navigation assistance method could enable the user to be more immersive and effective while roaming in a virtual reality(VR)environment.

Current trends in three-dimensional visualization and real-time navigation as well as robot-assisted technologies in hepatobiliary surgery

With the continuous development of digital medicine,minimally invasive precision and safety have become the primary development trends in hepatobiliary surgery.Due to the specificity and complexity of hepatobiliary surgery,traditional preoperative imaging techniques such as computed tomography and magnetic resonance imaging cannot meet the need for identification of fine anatomical regions.Imaging-based three-dimensional(3D)reconstruction,virtual simulation of surgery and 3D printing optimize the surgical plan through preoperative assessment,improving the controllability and safety of intraoperative operations,and in difficult-to-reach areas of the posterior and superior liver,assistive robots reproduce the surgeon’s natural movements with stable cameras,reducing natural vibrations.Electromagnetic navigation in abdominal surgery solves the problem of conventional surgery still relying on direct visual observation or preoperative image assessment.We summarize and compare these recent trends in digital medical solutions for the future development and refinement of digital medicine in hepatobiliary surgery.

Virtual local target method for avoiding local minimum in potential field based robot navigation

A novel robot navigation algorithm with global path generation capability is presented. Local minimum is a most intractable but is an encountered frequently problem in potential field based robot navigation.Through appointing appropriately some virtual local targets on the journey, it can be solved effectively. The key concept employed in this algorithm are the rules that govern when and how to appoint these virtual local targets. When the robot finds itself in danger of local minimum, a virtual local target is appointed to replace the global goal temporarily according to the rules. After the virtual target is reached, the robot continues on its journey by heading towards the global goal. The algorithm prevents the robot from running into local minima anymore. Simulation results showed that it is very effective in complex obstacle environments.

Real-time virtual sonography visualization and its clinical application in biliopancreatic disease

AIM:To evaluate the usefulness of real-time virtual sonography(RVS)in biliary and pancreatic diseases.METHODS:This study included 15 patients with biliary and pancreatic diseases.RVS can be used to observe an ultrasound image in real time by merging the ultrasound image with a multiplanar reconstruction computed tomography(CT)image,using pre-scanned CT volume data.The ultrasound used was EUB-8500with a convex probe EUP-C514.The RVS images were evaluated based on 3 levels,namely,excellent,good and poor,by the displacement in position.RESULTS:By combining the objectivity of CT with free scanning using RVS,it was possible to easily interpret the relationship between lesions and the surrounding organs as well as the position of vascular structures.The resulting evaluation levels of the RVS images were12 excellent(pancreatic cancer,bile duct cancer,cholecystolithiasis and cholangiocellular carcinoma)and 3 good(pancreatic cancer and gallbladder cancer).Compared with conventional B-mode ultrasonography and CT,RVS images achieved a rate of 80%superior visualization and 20%better visualization.CONCLUSION:RVS has potential usefulness in objective visualization and diagnosis in the field of biliary and pancreatic diseases.

Effective Virtual Reality Based Building Navigation Using Dynamic Loading and Path Optimization

Although computer capabilities have been improved significantly, a large-scale virtual reality （VR） system demands much more in terms of memory and computation than the current computer systems can offer. This paper discusses two important issues related to VR performance and applications in building navigation. These are dynamic loading of models based on cell segmentation for the optimal VR operation, and the route optimization based on path planning for easy navigation. The VR model of engineering and information technology complex （EITC） building at the University of Manitoba is built as an example to show the feasibility of the proposed methods. The reality, enhanced by three-dimensional （3D） real-time interactivity and visualization, leads navigators into a state of the virtual building immersion.

A survey of real-time rendering on Web3D application

Background In recent years, with the rapid development of mobile Internet and Web3D technologies, a large number of web-based online 3D visualization applications have emerged. Web3D applications, including Web3D online tourism, Web3D online architecture, Web3D online education environment, Web3D online medical care, and Web3D online shopping are examples of these applications that leverage 3D rendering on the web. These applications have pushed the boundaries of traditional web applications that use text, sound, image, video, and 2D animation as their main communication media, and resorted to 3D virtual scenes as the main interaction object, enabling a user experience that delivers a strong sense of immersion. This paper approached the emerging Web3D applications that generate stronger impacts on people's lives through “real-time rendering technology”, which is the core technology of Web3D. This paper discusses all the major 3D graphics APIs of Web3D and the well-known Web3D engines at home and abroad and classify the real-time rendering frameworks of Web3D applications into different categories. Results Finally, this study analyzed the specific demand posed by different fields to Web3D applications by referring to the representative Web3D applications in each particular field. Conclusions Our survey results show that Web3D applications based on real-time rendering have in-depth sectors of society and even family, which is a trend that has influence on every line of industry.

Value of virtual bronchoscopic navigation and transbronchial ultrasound-guided sheath-guided exploration in diagnosis of peripheral lung cancer

BACKGROUND Peripheral lung cancer poses a substantial harm to human health,and it is easy to become exacerbated,potentially threatening the life and safety of patients AIM To assess the value of virtual bronchoscopic navigation(VBN)combined with transbronchial ultrasound-guided sheath-guided(EBUS-GS)exploration in the diagnosis of peripheral lung cancer.METHODS A total of 236 patients with peripheral lung cancer(nodule diameter range,8-30 mm;diagnosed using high-resolution computed tomography)were selected from three centers between October 2018 and December 2019.Patients who underwent EBUS-GS exploration alone were included in a control group,and those who received VBN in combination with EBUS-GS exploration were included in an observation group.The diagnostic rate and total operating time of differentsubgroups of the two groups were compared,and the time needed to determine the lesion was recorded.RESULTS There were no significant differences in diagnosis rate or total operation time between the two groups(P>0.05),and the time needed to determine the lesion in the observation group was less than that of the control group(P<0.05).CONCLUSION The combined use of VBN and EBUS-GS exploration technology has little effect on the diagnosis rate and total operation time of peripheral lung cancer,but it significantly shortens the time needed to determine the lesion and is a valuable diagnostic method.

Virtual bronchoscopic navigation without fluoroscopy guidance for peripheral pulmonary lesions in inexperienced pulmonologist

Objective:Fluoroscopy guidance is generally required for endobronchial ultrasonography with guide sheath(EBUS-GS)in peripheral pulmonary lesions(PPLs).Virtual bronchoscopic navigation(VBN)can guide the bronchoscope by creating virtual images of the bronchial route to the lesion.The diagnostic yield and safety profiles of VBN without fluoroscopy for PPLs have not been evaluated in inexperienced pulmonologist performing EBUS-GS.Methods:Between January 2016 and June 2017,consecutive patients with PPLs referred for EBUS-GS at a single cancer center were enrolled.The diagnostic yield as well as safety profiles was retrospectively analyzed,and our preliminary experience was shared.Results:A total of 109 patients with 109 lesions were included,99(90.8%)lesions were visible on EBUS imaging.According to the procedure time needed to locate the lesion on EBUS,24.8%(27/109)were deemed technically difficult procedures;however,no significant relationships were identified between candidate parameters and technically difficult procedures.The overall diagnosis yield was 74.3%(81/109),and the diagnostic yield of malignancy was 83.7%(77/92).Lesions larger than 20 mm[odds ratio(OR),2.758;95%confidence interval(95%CI),1.077-7.062;P=0.034]and probe of within type(OR,3.174;95%CI,1.151-8.757,P=0.026)were independent factors leading to a better diagnostic yield in multivariate analysis.About 30 practice procedures were needed to achieve a stable diagnostic yield,and the proportion of technically difficult procedures decreased and stabilized after 70 practice procedures.Regarding complications,one patient(0.9%)had intraoperative hemorrhage(100 mL)which was managed under endoscopy.Conclusions:VBN without fluoroscopy guidance is still useful and safe for PPLs diagnosis,especially for malignant diseases when performed by pulmonologist without previous experience of EBUS-GS.VBN may simplify the process of lesion positioning and further multi-center randomized studies are warranted.

Optimizing Real-Time Performance of 3D Virtual Mining Environment with MultiGen Creator

System optimization plays a crucial role in developing VR system after 3D modeling, affecting the system's Immersion and Interaction performance enormously. In this article, several key techniques of optimizing a virtual mining system were discussed: optimizing 3D models to keep the polygon number in VR system within target hardware's processing ability; optimizing texture database to save texture memory with perfect visual effect; optimizing database hierarchy structure to accelerate model retrieval; and optimizing LOD hierarchy structure to speed up rendering.

A State-of-the-Art Survey on Real-Time Issues in Embedded Systems Virtualization

Virtualization has gained great acceptance in the server and cloud computing arena. In recent years, it has also been widely applied to real-time embedded systems with stringent timing constraints. We present a comprehensive survey on real-time issues in virtualization for embedded systems, covering popular virtualization systems including KVM, Xen, L4 and others.

Navigation in virtual and real environment using brain computer interface:a progress report

A brain-computer interface (BCI) facilitates bypassing the peripheral nervous system and directly communicating with surrounding devices. Navigation technology using BCI has developed-from exploring the prototype paradigm in the virtual environment (VE) to accurately completing the locomotion intention of the operator in the form of a powered wheelchair or mobile robot in a real environment. This paper summarizes BCI navigation applications that have been used in both real and VEs in the past 20 years. Horizontal comparisons were conducted between various paradigms applied to BCI and their unique signal-processing methods. Owing to the shift in the control mode from synchronous to asynchronous, the development trend of navigation applications in the VE was also reviewed. The contrast between high level commands and low-level commands is introduced as the main line to review the two major applications of BCI navigation in real environments: mobile robots and unmanned aerial vehicles (UAVs). Finally, applications of BCI navigation to scenarios outside the laboratory;research challenges, including human factors in navigation application interaction design;and the feasibility of hybrid BCI for BCI navigation are discussed in detail.

An introduction to the 3-dimensional virtual library sites-navigation system at Capital Normal University Library

Capital Normal University Library(CNU Library) initiated the first practical application of a 3D virtual library sites-navigation system(an electronic kiosk version)among Chinese academic and research libraries in 2010. It was primarily based on the technologies of 3DsMax and Virtools. This paper concentrates on the discussion of the methods in creating the 3D model and in realizing the interaction among the data usage of the system. As a result, several important service functions of the system have been developed successfully so far for convenient public access. They include the functions of virtual-book searching, path navigation online, real-time message exchanges, and multi-media sharing, etc.

A study of novel real-time power balance strategy with virtual asynchronous machine control for regional integrated electric-thermal energy systems

The development of regional integrated electric-thermal energy systems(RIETES) is considered a promising direction for modern energy supply systems. These systems provide a significant potential to enhance the comprehensive utilization and efficient management of energy resources. Therein, the real-time power balance between supply and demand has emerged as one pressing concern for system stability operation. However, current methods focus more on minute-level and hour-level power optimal scheduling methods applied in RIETES. To achieve real-time power balance, this paper proposes one virtual asynchronous machine(VAM) control using heat with large inertia and electricity with fast response speed. First, the coupling timescale model is developed that considers the dynamic response time scales of both electric and thermal energy systems. Second, a real-time power balance strategy based on VAM control can be adopted to the load power variation and enhance the dynamic frequency response. Then, an adaptive inertia control method based on temperature variation is proposed, and the unified expression is further established. In addition, the small-signal stability of the proposed control strategy is validated. Finally, the effectiveness of this control strategy is confirmed through MATLAB/Simulink and HIL(Hardware-in-the-Loop) experiments.

RESEARCH OF MOVEMENT NAVIGATION BASED ON ASSEMBLY CONSTRAINT RECOGNITION

The requirements and features of virtual assembly movement navigator areanalyzed to help operators flexibly manipulate virtual objects, precisely locate or assemble virtualparts in virtual environment. With the degree-of-freedom analysis, the assembly constrainthierarchical model is constructed and the system's constraints are built dynamically. Thus, allobjects in virtual environment can be located reasonally by the navigator. Moreover, the assemblyconstraint recognition in the process of assembly and movement correction is also discussed.

Precise Point Positioning Technique with IGS Real-Time Service (RTS) for Maritime Applications

The maritime navigation accuracy requirements for radionavigation systems such as GPS are specified by the International Maritime Organization (IMO). Maritime navigation usually consists of three major phases identified as Ocean/Coastal/Port approach/Inland waterway, in port navigation and automatic docking with an accuracy requirement that ranges from 10 m to 0.1 m. With the advancement in autonomous GPS positioning techniques such as Precise Point Positioning (PPP) and with the advent of the new IGS-Real-Time-Service (RTS), it is necessary to assess the possibility of a wider role of the PPP-based positioning technique in maritime applications. This paper investigates the performance of an autonomous real-time PPP-positioning solution by using the IGS- RTS service for maritime applications that require an accurate positioning system. To examine the performance of the real-time IGS-RTS PPP-based technique for maritime applications, kinematic data from a dual frequency GPS receiver is investigated. It is shown that the real-time IGS-RTS PPP-based GPS positioning technique fulfills IMO requirements for maritime applications with an accuracy requirement ranges from 10 m for Ocean/Coastal/Port approach/Inland waterways navigation to 1.0 m for in port navigation but cannot fulfill the automatic docking application with an accuracy requirement of 0.10 m. To further investigate the real-time PPP-based GPS positioning technique, a comparison is made between the real-time IGS-RTS PPP-based positioning technique and the real-time PPP-based positioning by using the predicted part of the IGS Ultra-Rapid products and the real-time GPS positioning technique with the Wide Area Differential GPS service (WADGPS). It is shown that the IGS-RTS PPP-based positioning technique is superior to the IGS-Ultra-Rapid PPP-based and WADGPS-based positioning techniques.

RRVPE:A Robust and Real-Time Visual-Inertial-GNSS Pose Estimator for Aerial Robot Navigation

Self-localization and orientation estimation are the essential capabilities for mobile robot navigation.In this article,a robust and real-time visual-inertial-GNSS(Global Navigation Satellite System)tightly coupled pose estimation(RRVPE)method for aerial robot navigation is presented.The aerial robot carries a front-facing stereo camera for self-localization and an RGB-D camera to generate 3D voxel map.Ulteriorly,a GNSS receiver is used to continuously provide pseudorange,Doppler frequency shift and universal time coordinated(UTC)pulse signals to the pose estimator.The proposed system leverages the Kanade Lucas algorithm to track Shi-Tomasi features in each video frame,and the local factor graph solution process is bounded in a circumscribed container,which can immensely abandon the computational complexity in nonlinear optimization procedure.The proposed robot pose estimator can achieve camera-rate(30 Hz)performance on the aerial robot companion computer.We thoroughly experimented the RRVPE system in both simulated and practical circumstances,and the results demonstrate dramatic advantages over the state-of-the-art robot pose estimators.

Influences of Network Delay on Quality of Experience for Soft Objects in Networked Real-Time Game with Haptic Sense

In this paper, we investigate the influences of network delay on QoE (Quality of Experience) such as the operability of haptic interface device and the fairness between players for soft objects in a networked real-time game subjectively and objectively. We handle a networked balloon bursting game in which two players burst balloons (i.e., soft objects) in a 3D virtual space by using haptic interface devices, and the players compete for the number of burst balloons. As a result, we find that the operability depends on the network delay from the local terminal to the other terminal, and the fairness is mainly dependent on the difference in network delay between the players’ terminals. We confirm that there exists a trade-off relationship between the operability and the fairness. We also see that the contribution of the fairness is larger than that of the operability to the comprehensive quality (i.e., the weighted sum of the operability and fairness). Assessment results further show that the output timing of terminals should be adjusted to the terminal which has the latest output timing to maintain the fairness when the difference in network delay between the terminals is large. In this way, the comprehensive quality at each terminal can be maintained as high as possible.

混合现实技术在口腔颌面部肿瘤诊疗中的应用

口腔颌面部肿瘤是口腔颌面外科的常见疾病,以虚拟手术设计、手术导航为代表的数字化外科技术目前已成为口腔颌面部肿瘤外科诊治的主要辅助手段。然而,现有的数字化技术在三维可视化成像、术中手眼配合等方面仍存在提升空间。目前,以混合现实技术为代表的三维可视化技术已经在医学领域迅速发展和应用,通过将三维虚拟图像叠加到真实的手术环境中,可辅助实现医学影像实时立体呈现,已逐步应用于口腔颌面部肿瘤的诊治中。术前可基于影像数据构建肿瘤及其周边重要结构三维模型,在混合现实环境中,进行个性化术前评估和手术方案设计;术中将混合现实技术与手术导航技术相结合,可以在术区真实环境中实时显示三维虚拟模型,克服单纯使用导航技术的手眼协调问题,进一步提高口腔颌面部肿瘤手术的精确性和安全性。混合现实技术与远程互联网医疗技术相结合,还可以提供高质量的教学平台,推动地区口腔颌面外科诊疗水平的发展。混合现实技术的局限性主要在于术区影像遮挡、单纯使用时精度不足、配准时间长等。笔者结合国内外文献报道以及本课题组的临床实践,对混合现实技术在口腔颌面部肿瘤诊疗中的应用进行归纳与总结。

基于ROS的服务机器人实训课程虚拟仿真教学研究

针对目前服务机器人实训课程教学存在的设备成本高、学生操作机会不足等问题,该文提出基于ROS的仿真实训课程教学方案。依托真实系统和仿真实训室,构建基于ROS的服务机器人建模与运动仿真、建图与导航仿真、平面视觉检测仿真、三维视觉检测仿真、物品自动抓取综合应用仿真5个实训项目,详述每个项目的实施要点,实现虚实结合的实训模式。经过实践证明,虚拟仿真实验教学不仅提高实训教学效率,同时极大激发学生的学习兴趣与自主创新意识。

虚拟支气管镜导航在周围性肺部病变检查中的应用

目的 探讨虚拟支气管镜导航(VBN)在周围性肺部病变检查中的应用价值。方法 选取2022年5月至2023年6月河北北方学院附属第一医院收治的周围性肺部病变患者80例为研究对象,基于病变活检方法不同,将其分为单纯超声支气管镜组(NVBN组,40例),VBN联合超声支气管镜组(VBN组,40例)。以病理结果为金标准,比较两组的诊断准确率、病变到达率、开始活检时间、操作总时间及并发症的发生情况。结果 VBN组诊断总准确率高于NVBN组,差异有统计学意义(P<0.05)。VBN组病变到达率高于NVBN组,开始活检时间、操作总时间短于NVBN组,差异有统计学意义(P<0.05)。两组并发症总发生率比较,差异无统计学意义(P>0.05)。结论 VBN可提高周围性肺部病变的诊断准确率,缩短了操作时间、安全性高,值得临床推广应用。