基于双目视觉的盆腔微创手术增强现实导航仿真系统的设计

Design of augmented reality navigation simulation system for pelvic minimally invasive surgery based on stereoscopic vision

基于内窥镜视觉导航的盆腔微创手术,往往因为病灶的复杂解剖位置以及内窥镜视野的局限性,对手术的顺利开展提出了重要的挑战。针对这个问题,设计并开发一套基于立体视觉的盆腔微创手术的增强现实(AR)导航仿真系统。首先,利用术前的CT影像重建骨盆3D模型以及盆腔的真实手术视频,生成带有纹理信息的骨盆3D模型,仿真一套具有真实轨迹的手术视频;然后,利用基于可视点颜色一致性的2D/3D配准技术,实现术前重建模型与手术视野的初始化配准。利用立体视觉跟踪算法,对手术过程中的内窥镜位置进行跟踪,根据内窥镜多自由度的变换矩阵来实现术前3D模型与手术视野的融合与增强现实导航。估计轨迹与真实轨迹的均方根误差为2.3933 mm,仿真实验表明,导航系统为视觉导航提供良好的增强现实显示效果。

Minimally invasive endoscopic surgery always remains a challenge due to the complexity of the anatomical location and the limitations of endoscopic vision.An Augmented Reality(AR)navigation system was designed for simulation of pelvic minimally invasive surgery.Firstly,a 3D model of pelvis which was segmented and reconstructed from the preoperative CT(Computed Tomography)was textured mapping with the real pelvic surgical video,and then a surgical video with the ground truth pose was simulated.The blank model was initially registered with the intraoperative video by a 2D/3D registration based on color consistency of visible surface points.After that,an accurate tracking of intraoperative endoscopy was performed using a stereoscopic tracking algorithm.According to the multi-DOFs(Degree Of Freedoms)transformation matrix of endoscopy,the preoperative 3D model could then be fused to the intraoperative vision to achieve an AR navigation.The experimental results show that the root mean square error of the estimated trajectory compared to the ground truth is 2.393 3 mm,which reveals that the system can achieve a good AR display for visual navigation.