基于MR混合现实的足球机器人目标跟踪仿真

机器人跟踪足球时,由于目标具有较强的运动随机性,且较小的目标尺度增大了跟踪框的自适应跟踪难度,目标跟踪时会将很多背景信息包含在内,导致目标模型的更新错误。为此,提出基于MR混合现实的足球机器人目标跟踪方法。构建MR混合现实平台,令足球机器人在虚拟现实场景下完成人机交互,在保证目标跟踪安全性的同时提升目标跟踪性能;采用帧差法获取目标模型,利用投影法对二值图像投影,并对差分图像滤波,以此获取足球机器人在目标模板区域及候选目标区域中的直方图。基于此,更新足球机器人目标模型,实现基于MR混合现实的足球机器人目标跟踪。实验结果表明,当跟踪误差阈值为0.8mm时,研究方法的目标跟踪准确率和成功率均可达95%以上,平均耗时为55.17ms。当足球目标出现遮挡问题时,该方法仍能完成高精度跟踪。

Chinese expert consensus on laparoscopic hepatic segmentectomy and subsegmentectomy navigated by augmented-and mixed-reality technology combined with indocyanine green fluorescence imaging

Augmented-and mixed-reality technologies have pioneered the realization of real-time fusion and interactive projection for laparoscopic surgeries.Indocyanine green fluorescence imaging technology has enabled anatomical,functional,and radical hepatectomy through tumor identification and localization of target hepatic segments,driving a transformative shift in themanagement of hepatic surgical diseases,moving away from traditional,empirical diagnostic and treatment approaches toward digital,intelligent ones.The Hepatic Surgery Group of the Surgery Branch of the Chinese Medical Association,Digital Medicine Branch of the Chinese Medical Association,Digital Intelligent Surgery Committee of the Chinese Society of ResearchHospitals,and Liver Cancer Committee of the Chinese Medical Doctor Association organized the relevant experts in China to formulate this consensus.This consensus provides a comprehensive outline of the principles,advantages,processes,and key considerations associated with the application of augmented reality and mixed-reality technology combined with indocyanine green fluorescence imaging technology for hepatic segmental and subsegmental resection.The purpose is to streamline and standardize the application of these technologies.

Video Conference System in Mixed Reality Using a Hololens

The mixed reality conference system proposed in this paper is a robust,real-time video conference application software that makes up for the simple interaction and lack of immersion and realism of traditional video conference,which realizes the entire process of holographic video conference from client to cloud to the client.This paper mainly focuses on designing and implementing a video conference system based on AI segmentation technology and mixed reality.Several mixed reality conference system components are discussed,including data collection,data transmission,processing,and mixed reality presentation.The data layer is mainly used for data collection,integration,and video and audio codecs.The network layer uses Web-RTC to realize peer-to-peer data communication.The data processing layer is the core part of the system,mainly for human video matting and human-computer interaction,which is the key to realizing mixed reality conferences and improving the interactive experience.The presentation layer explicitly includes the login interface of the mixed reality conference system,the presentation of real-time matting of human subjects,and the presentation objects.With the mixed reality conference system,conference participants in different places can see each other in real-time in their mixed reality scene and share presentation content and 3D models based on mixed reality technology to have a more interactive and immersive experience.

Adaptive Consistent Management to Prevent System Collapse on Shared Object Manipulation in Mixed Reality

A concurrency control mechanism for collaborative work is akey element in a mixed reality environment. However, conventional lockingmechanisms restrict potential tasks or the support of non-owners, thusincreasing the working time because of waiting to avoid conflicts. Herein, wepropose an adaptive concurrency control approach that can reduce conflictsand work time. We classify shared object manipulation in mixed reality intodetailed goals and tasks. Then, we model the relationships among goal,task, and ownership. As the collaborative work progresses, the proposedsystem adapts the different concurrency control mechanisms of shared objectmanipulation according to the modeling of goal–task–ownership. With theproposed concurrency control scheme, users can hold shared objects andmove and rotate together in a mixed reality environment similar to realindustrial sites. Additionally, this system provides MS Hololens and Myosensors to recognize inputs from a user and provides results in a mixed realityenvironment. The proposed method is applied to install an air conditioneras a case study. Experimental results and user studies show that, comparedwith the conventional approach, the proposed method reduced the number ofconflicts, waiting time, and total working time.

Training on LSA lifeboat operation using Mixed Reality

Background This work aims to provide an overview of the Mixed Reality(MR)technology’s use in maritime industry for training purposes.Current training procedures cover a broad range of procedural operations for Life-Saving Appliances(LSA)lifeboats;however,several gaps and limitations have been identified related to the practical training that can be addressed through the use of MR.Augmented,Virtual and Mixed Reality applications are already used in various fields in maritime industry,but their full potential have not been yet exploited.SafePASS project aims to exploit MR advantages in the maritime training by introducing a relevant application focusing on use and maintenance of LSA lifeboats.Methods An MR Training application is proposed supporting the training of crew members in equipment usage and operation,as well as in maintenance activities and procedures.The application consists of the training tool that trains crew members on handling lifeboats,the training evaluation tool that allows trainers to assess the performance of trainees,and the maintenance tool that supports crew members to perform maintenance activities and procedures on lifeboats.For each tool,an indicative session and scenario workflow are implemented,along with the main supported interactions of the trainee with the equipment.Results The application has been tested and validated both in lab environment and using a real LSA lifeboat,resulting to improved experience for the users that provided feedback and recommendations for further development.The application has also been demonstrated onboard a cruise ship,showcasing the supported functionalities to relevant stakeholders that recognized the added value of the application and suggested potential future exploitation areas.Conclusions The MR Training application has been evaluated as very promising in providing a user-friendly training environment that can support crew members in LSA lifeboat operation and maintenance,while it is still subject to improvement and further expansion.

Web-based mixed reality video fusion with remote rendering

Background Mixed reality(MR)video fusion systems merge video imagery with 3D scenes to make the scene more realistic and help users understand the video content and temporal–spatial correlation between them,reducing the user′s cognitive load.MR video fusion are used in various applications;however,video fusion systems require powerful client machines because video streaming delivery,stitching,and rendering are computationally intensive.Moreover,huge bandwidth usage is another critical factor that affects the scalability of video-fusion systems.Methods Our framework proposes a fusion method for dynamically projecting video images into 3D models as textures.Results Several experiments on different metrics demonstrate the effectiveness of the proposed framework.Conclusions The framework proposed in this study can overcome client limitations by utilizing remote rendering.Furthermore,the framework we built is based on browsers.Therefore,the user can test the MR video fusion system with a laptop or tablet without installing any additional plug-ins or application programs.

Effects of virtual agents on interaction efficiency and environmental immersion in MR environments

Background Physical entity interactions in mixed reality(MR)environments aim to harness human capabilities in manipulating physical objects,thereby enhancing virtual environment(VEs)functionality.In MR,a common strategy is to use virtual agents as substitutes for physical entities,balancing interaction efficiency with environmental immersion.However,the impact of virtual agent size and form on interaction performance remains unclear.Methods Two experiments were conducted to explore how virtual agent size and form affect interaction performance,immersion,and preference in MR environments.The first experiment assessed five virtual agent sizes(25%,50%,75%,100%,and 125%of physical size).The second experiment tested four types of frames(no frame,consistent frame,half frame,and surrounding frame)across all agent sizes.Participants,utilizing a head mounted display,performed tasks involving moving cups,typing words,and using a mouse.They completed questionnaires assessing aspects such as the virtual environment effects,interaction effects,collision concerns,and preferences.Results Results from the first experiment revealed that agents matching physical object size produced the best overall performance.The second experiment demonstrated that consistent framing notably enhances interaction accuracy and speed but reduces immersion.To balance efficiency and immersion,frameless agents matching physical object sizes were deemed optimal.Conclusions Virtual agents matching physical entity sizes enhance user experience and interaction performance.Conversely,familiar frames from 2D interfaces detrimentally affect interaction and immersion in virtual spaces.This study provides valuable insights for the future development of MR systems.

基于MR的水电站专项应急预案演练平台

为解决水电站特种环境应急演练中存在的事故情景难以模拟、不安全、组织成本高等问题,构建一种基于混合现实(Mixed Reality,MR)技术进行险情可视化的应急演练仿真平台。针对电站地下厂房发电机组着火事故,采用Unity3D引擎结合情景构建与向导设计方法进行MR仿真平台的设计开发。测试结果表明,所设计的平台可实现在现实场景下进行虚拟操作的应急演练,平台模块设置得科学合理。该平台不仅拓宽了应急演练实训模式的边界,还可为水电站特种环境下应急演练提供新的参考依据。

Mixed neuroendocrine non-neuroendocrine neoplasms in gastroenteropancreatic tract

Mixed neuroendocrine non-neuroendocrine neoplasms(MiNENs)are a hetero-geneous group of malignant neoplasms that can settle in the gastroenteropan-creatic tract.They are composed of a neuroendocrine(NE)and a non-NE compo-nent in at least 30%of each tumour.The non-NE component can include different histological combinations of glandular,squamous,mucinous and sarcomatoid phenotypes,and one or both of the components can be low-or high grade malignant.Recent changes in the nomenclature of these neoplasms might lead to great deal of confusion,and the lack of specific clinical trials is the main reason why their management is difficult.The review aims to clarify the definition of MiNEN and analyze available evidence about their diagnosis and treatment options according to their location and extension through careful analysis of the available data.It would be important to reach a general consensus on their diagnosis in order to construct a classification that remains stable over time and facilitates the design of clinical trials that,due to their low incidence,will require long recruitment periods.

基于MR技术的计算机组装系统设计与实现

使用HoloLens2头盔和混合现实插件Mixed Reality Feature Tool开发出一款计算机组装混合现实交互应用。通过HoloLens2凝视、手部跟踪、UX构建基块、语音和听写系统、眼部跟踪等进行输入交互,并配合Unity3D中的Animation系统与UGUI系统,最终实现混合现实虚拟交互。Unity3D工程导出UWP,并生成VS工程,然后部署到HoloLens2中进行测试。测试结果表明,该系统实现了计算机组装的相关交互功能。研究为计算机组装的虚拟操作提供了一种混合现实解决方案。

Mixed neuroendocrine and adenocarcinoma of gastrointestinal tract:A complex diagnosis and therapeutic challenge

In this editorial we comment on the manuscript describing a case of adenocarcinoma mixed with a neuroendocrine carcinoma of the gastroesophageal junction.Mixed neuroendocrine and non-neuroendocrine neoplasms of the gastrointestinal system are rare heterogeneous group of tumors characterized by a high malignant potential,rapid growth,and poor prognosis.Due to the rarity of these cancers,the standard therapy is poorly defined.The diagnosis of these tumors is based on combination of morphological features,immunohistochemical and neuroendocrine and epithelial cell markers.Both endocrine and epithelial cell components can act independently of each other and thus,careful grading of each component separately is required.These cancers are aggressive in nature and the potential of each component has paramount importance in the choice of treatment and response.Regardless of the organ of origin,these tumors portend poor prognosis with increased proportion of neuroendocrine component.Multidisciplinary services and strategies are required for the management of these mixed malignancies to provide the best oncological outcomes.The etiopathogenesis of these mixed tumors remains obscure but poses interesting question.We briefly discuss a few salient points in this editorial.

Leveraging Augmented Reality,Semantic-Segmentation,and VANETs for Enhanced Driver’s Safety Assistance

Overtaking is a crucial maneuver in road transportation that requires a clear view of the road ahead.However,limited visibility of ahead vehicles can often make it challenging for drivers to assess the safety of overtaking maneuvers,leading to accidents and fatalities.In this paper,we consider atrous convolution,a powerful tool for explicitly adjusting the field-of-view of a filter as well as controlling the resolution of feature responses generated by Deep Convolutional Neural Networks in the context of semantic image segmentation.This article explores the potential of seeing-through vehicles as a solution to enhance overtaking safety.See-through vehicles leverage advanced technologies such as cameras,sensors,and displays to provide drivers with a real-time view of the vehicle ahead,including the areas hidden from their direct line of sight.To address the problems of safe passing and occlusion by huge vehicles,we designed a see-through vehicle system in this study,we employed a windshield display in the back car together with cameras in both cars.The server within the back car was used to segment the car,and the segmented portion of the car displayed the video from the front car.Our see-through system improves the driver’s field of vision and helps him change lanes,cross a large car that is blocking their view,and safely overtake other vehicles.Our network was trained and tested on the Cityscape dataset using semantic segmentation.This transparent technique will instruct the driver on the concealed traffic situation that the front vehicle has obscured.For our findings,we have achieved 97.1% F1-score.The article also discusses the challenges and opportunities of implementing see-through vehicles in real-world scenarios,including technical,regulatory,and user acceptance factors.

Dendroclimatological study of Sabina saltuaria and Abies faxoniana in the mixed forests of the Qionglai Mountains,eastern Tibetan Plateau

Tree-ring chronologies were developed for Sabina saltuaria and Abies faxoniana in mixed forests in the Qionglai Mountains of the eastern Tibetan Plateau.Climate-growth relationship analysis indicated that the two co-exist-ing species reponded similarly to climate factors,although S.saltuaria was more sensitive than A.faxoniana.The strong-est correlation was between S.saltuaria chronology and regional mean temperatures from June to November.Based on this relationship,a regional mean temperature from June to November for the period 1605-2016 was constructed.Reconstruction explained 37.3%of the temperature variance during th period 1961-2016.Six major warm periods and five major cold periods were identified.Spectral analysis detected significant interannual and multi-decadal cycles.Reconstruction also revealed the influence of the Atlantic Multi-decadal Oscillation,confirming its importance on climate change on the eastern Tibetan Plateau.

Soil properties and not plant factors affect both abundant and rare microbial taxa after thinning in a mixed stand of Cunninghamia lanceolata and Sassafras tzumu

Thinning is an effective management step for sustainable forest development,yet less attention is paid to the restoration of soil microbiota after thinning.In this study,both abundant and rare soil microbial communities(i.e.,bacterial,fungal),were evaluated under various thinning treatments in a mixed stand of Cunninghamia lanceolata and Sassafras tzumu using Mi Seq sequencing.Thinning did not significantly change either abundant or rare bacterial and fungal community composition,but affected their alpha diversity.The Shannon–Wiener indexes of rare fungal taxa under medium thinning were significantly lower than in the light thinning(P<0.05 level).Xanthobacteraceae dominated the abundant bacterial taxa,and Saitozyma and Mortierlla the abundant fungal taxa.The most common rare bacterial taxa varied;there was no prevalent rare fungal taxa under different thinnings.In addition,soil available nitrogen,total phosphorus,and p H had significant effects on rare bacterial taxa.Nutrients,especially available phosphorus,but not nitrogen,affected abundant and rare soil fungi.The results indicate that soil properties rather than plant factors affect abundant and rare microbial communities in soils of mixed stands.Thinning,through mediating soil properties,influences both abundant and rare bacterial and fungal communities in the mixed C.lanceolata and S.tzumu stand.

紫花苜蓿与全株玉米不同混合比例对TMR发酵的影响

试验旨在探究紫花苜蓿与全株玉米不同混合比例对全混合日粮(TMR)发酵的影响。将紫花苜蓿与全株玉米分别以10∶0(T1组)、7∶3(T2组)、5∶5(T3组)、3∶7(T4组)、0∶10(T5组)的比例混合作为粗饲料,再与精料以6∶4混合配制TMR,装入密封袋中厌氧发酵60 d,得到发酵全混合日粮(FTMR),取样测定其营养成分、发酵品质以及开封后第0、3、6、9、12天主要微生物的动态变化情况。结果表明:随着全株玉米比例的增加,FTMR中粗蛋白和粗脂肪含量显著降低(P<0.05)。T2组乳酸菌数量随着饲料在空气中暴露时间的延长而减少(P<0.05);随着全株玉米混合比例的增加,乳酸菌数量显著增加(P<0.05);随着全株玉米混合比例的增加,除T3组外,乙酸含量显著增加(P<0.05)。根据各组FTMR的营养成分、发酵品质、微生物的动态数量变化情况进行隶属函数分析,结果表明,最适紫花苜蓿与全株玉米比例为7∶3。综合各组FTMR的营养品质、发酵品质以及有氧暴露后微生物的变化情况,在中小型牧场乃至家庭式牧场中推荐紫花苜蓿和全株玉米混合发酵的适宜比例为7∶3。

Breaking the optical efficiency limit of virtual reality with a nonreciprocal polarization rotator

A catadioptric lens structure,also known as pancake lens,has been widely used in virtual reality(VR)displays to reduce the formfactor.However,the utilization of a half mirror(HM)to fold the optical path thrice leads to a significant optical loss.The theoretical maximum optical efficiency is merely 25%.To transcend this optical efficiency constraint while retaining the foldable characteristic inherent to traditional pancake optics,in this paper,we propose a theoretically lossless folded optical system to replace the HM with a nonreciprocal polarization rotator.In our feasibility demonstration experiment,we used a commercial Faraday rotator(FR)and reflective polarizers to replace the lossy HM.The theoretically predicted 100%efficiency can be achieved approximately by using two high-extinction-ratio reflective polarizers.In addition,we evaluated the ghost images using a micro-OLED panel in our imaging system.Indeed,the ghost images can be suppressed to undetectable level if the optics are with antireflection coating.Our novel pancake optical system holds great potential for revolutionizing next-generation VR displays with lightweight,compact formfactor,and low power consumption.

Towards engineering a portable platform for laparoscopic pre-training in virtual reality with haptic feedback

Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training through practical exercises on abstracted and reduced models.Methods The authors strive for a portable,easy to use and cost-effective Virtual Reality-based(VR)laparoscopic pre-training platform and therefore address the question of how such a system has to be designed to achieve the quality of today's gold standard using real tissue specimens.Current VR controllers are limited regarding haptic feedback.Since haptic feedback is necessary or at least beneficial for laparoscopic surgery training,the platform to be developed consists of a newly designed prototype laparoscopic VR controller with haptic feedback,a commercially available head-mounted display,a VR environment for simulating a laparoscopic surgery,and a training concept.Results To take full advantage of benefits such as repeatability and cost-effectiveness of VR-based training,the system shall not require a tissue sample for haptic feedback.It is currently calculated and visually displayed to the user in the VR environment.On the prototype controller,a first axis was provided with perceptible feedback for test purposes.Two of the prototype VR controllers can be combined to simulate a typical both-handed use case,e.g.,laparoscopic suturing.A Unity based VR prototype allows the execution of simple standard pre-trainings.Conclusions The first prototype enables full operation of a virtual laparoscopic instrument in VR.In addition,the simulation can compute simple interaction forces.Major challenges lie in a realistic real-time tissue simulation and calculation of forces for the haptic feedback.Mechanical weaknesses were identified in the first hardware prototype,which will be improved in subsequent versions.All degrees of freedom of the controller are to be provided with haptic feedback.To make forces tangible in the simulation,characteristic values need to be determined using real tissue samples.The system has yet to be validated by cross-comparing real and VR haptics with surgeons.

Stability Analysis and Performance Evaluation of Additive Mixed-Precision Runge-Kutta Methods

Additive Runge-Kutta methods designed for preserving highly accurate solutions in mixed-precision computation were previously proposed and analyzed.These specially designed methods use reduced precision for the implicit computations and full precision for the explicit computations.In this work,we analyze the stability properties of these methods and their sensitivity to the low-precision rounding errors,and demonstrate their performance in terms of accuracy and efficiency.We develop codes in FORTRAN and Julia to solve nonlinear systems of ODEs and PDEs using the mixed-precision additive Runge-Kutta(MP-ARK)methods.The convergence,accuracy,and runtime of these methods are explored.We show that for a given level of accuracy,suitably chosen MP-ARK methods may provide significant reductions in runtime.

On the Application of Mixed Models of Probability and Convex Set for Time-Variant Reliability Analysis

In time-variant reliability problems,there are a lot of uncertain variables from different sources.Therefore,it is important to consider these uncertainties in engineering.In addition,time-variant reliability problems typically involve a complexmultilevel nested optimization problem,which can result in an enormous amount of computation.To this end,this paper studies the time-variant reliability evaluation of structures with stochastic and bounded uncertainties using a mixed probability and convex set model.In this method,the stochastic process of a limit-state function with mixed uncertain parameters is first discretized and then converted into a timeindependent reliability problem.Further,to solve the double nested optimization problem in hybrid reliability calculation,an efficient iterative scheme is designed in standard uncertainty space to determine the most probable point(MPP).The limit state function is linearized at these points,and an innovative random variable is defined to solve the equivalent static reliability analysis model.The effectiveness of the proposed method is verified by two benchmark numerical examples and a practical engineering problem.

Personalized assessment and training of neurosurgical skills in virtual reality:An interpretable machine learning approach

Background Virtual reality technology has been widely used in surgical simulators,providing new opportunities for assessing and training surgical skills.Machine learning algorithms are commonly used to analyze and evaluate the performance of participants.However,their interpretability limits the personalization of the training for individual participants.Methods Seventy-nine participants were recruited and divided into three groups based on their skill level in intracranial tumor resection.Data on the use of surgical tools were collected using a surgical simulator.Feature selection was performed using the Minimum Redundancy Maximum Relevance and SVM-RFE algorithms to obtain the final metrics for training the machine learning model.Five machine learning algorithms were trained to predict the skill level,and the support vector machine performed the best,with an accuracy of 92.41%and Area Under Curve value of 0.98253.The machine learning model was interpreted using Shapley values to identify the important factors contributing to the skill level of each participant.Results This study demonstrates the effectiveness of machine learning in differentiating the evaluation and training of virtual reality neurosurgical performances.The use of Shapley values enables targeted training by identifying deficiencies in individual skills.Conclusions This study provides insights into the use of machine learning for personalized training in virtual reality neurosurgery.The interpretability of the machine learning models enables the development of individualized training programs.In addition,this study highlighted the potential of explanatory models in training external skills.