基于卷积神经网络的半监督手术视频流程识别

Semi-supervised Surgical Video Workflow Recognition Based on Convolution Neural Network

实时鲁棒的开放性外科手术视频流程自动识别检测将是未来人工智能医疗手术室的核心组成部分,这一关键技术结合其他AI(Artificial Intelligence)技术就可以帮助医护人员自动化、智能化地完成多项术中的常规活动。利用人工智能和计算机视觉的方法进行手术流程识别检测需要对大量的数据进行学习,为了训练这种方法,需要大量地标记手术视频数据,然而在医学领域,对外科手术视频数据的标记需要专家知识,收集足够数量的标记外科手术视频数据是困难且耗时的。因此,文中以腹腔镜胆囊切除术视频数据为研究对象,通过半监督学习方法卷积自编码器对视频进行空间特征提取,结合从同视频上下文中的一对视频帧进行时序特征提取,将非结构化的手术视频数据结构化,从而构建低层手术视频特征到高层外科手术流程语义之间的桥梁,以低代价实现对手术视频流程的智能化识别检测,高效判定手术流程进展。在开源数据集上的实验的结果表明,使用该模型jacc系数达到71.3%,准确率为86.6%,取得了较好的实验效果。

The real-time and robust open surgery workflow automatic detection will be the core component of the future artificial intelligent medical operation room.The key technology combined with other artificial intelligence technologies can help medical staff to automatically and intelligently complete a number of routine activities in the operation.However,the use of artificial intelligence and computer vision for surgical workflow recognition requires a large amount of data to be learned.In order to train this method,a large amount of labeled surgical video data is required.However,in the medical field,the labeling of surgical video data requires expert knowledge,and collecting enough numbers of marked surgical video data is difficult and time-consuming.Therefore,in this paper,the video data of laparoscopic cholecystectomy data is taken as the research obJect,the video spatial feature extraction is carried out by convolution self-encoder with semi-supervised learning method,and combined with a pair of video frames in the context of the same video for sequential feature extraction.The unstructured surgical video data is structured to build a bridge between the video characteristics of low-level surgery and the semantics of high-level surgical procedures,trying to realize the intelligent recognition of the surgical workflow at a low cost,and effectively determining the progress of the surgical workflow.Finally,the Jaccard coefficient of the proposed algorithm in this paper on a public dataset is 71.3%and the accuracy is 86.6%,achieving good experimental results.