应对新型冠状病毒肺炎疫情的AR远程指导系统设计

Design of AR remote guidance system for response to COVID-19

目的:建立一套AR远程指导系统,以应对新型冠状病毒肺炎疫情下的医疗指导,避免交叉感染。方法:该系统包括服务器端和客户端(指导端和被指导端),由图像分割模块、图像融合模块和视频传输模块构成。其中,图像分割模块采用基于K-means色彩聚类的改进型算法进行前景提取;图像融合模块利用掩膜图像进行图像融合;视频传输模块采用H.264编码,使用HTTP-FLV协议进行数据拉流。同时,对系统进行功能测试和性能测试。结果:测试结果表明,该系统实现了指导端和被指导端远程就同一动态视野或场景进行高效、直观、精确的交互,系统延迟时间约1.61 s,平均每帧图像处理时间约0.064 s,具有较好的实时性和视频流畅度。结论:AR远程指导系统基本满足疫情期间远程指导的临床需求,对解决医疗资源短缺、避免交叉感染起到了十分重要的作用。

Objective To develop a set of AR remote guidance system to facilitate the medical consultation and avoid cross infection during the response to COVID-19.Methods The system was composed of a server end and client end,which involving three modules of image segmentation,image fusion and video transmission.The image segmentation module used an improved algorithm based on K-means color clustering for foreground extraction;the image fusion module took advantages of the mask images for image fusion;the video transmission module adopted H.264 coding and used HTTP-FLV protocol for data pushing.Tests were executed for the functions and performances of the system.Results The test results showed that the system enabled efficient,intuitive and accurate interaction between two parties remotely on the same dynamic field of view or scene,with a system latency of around 1.61 s,an average image processing time of about 0.064 s per frame as well as the advantages in timeliness and video smoothness.Conclusion The system developed realizes remote guidance during the response to COVID-19,saves medical resources and avoids cross infection.