基于深度学习的超声心动图动态图像切面识别研究

Deep learning-based two-dimensional echocardiographic dynamic image view recognition

目的提出一种基于深度学习的切面识别模型SlowFast-Echo,进行二维经胸超声心动图动态图像的切面类型自动识别。方法选取2022年8月至12月在南京大学医学院附属鼓楼医院超声医学科完成二维经胸超声心动图检查的722例受检者(含心尖二腔、心尖三腔与心尖四腔等9类临床检查常用切面,共2243个动态图像),各类切面图像按照5∶2∶3的比例划分为训练集、验证集和测试集。进行SlowFast-Echo模型的训练和验证后,以准确率、精度、召回率、F1分数对模型的切面识别性能进行定量评价,以类激活映射图对模型的可解释性进行定性评价,以模型实地部署到超声医学科后的表现进行实用性评价。结果SlowFast-Echo模型对测试集动态图像切面类型预测的整体准确率、精度、召回率与F1分数分别为0.9866、0.9847、0.9872与0.9859;显著性热力图表明模型关注区域与超声科医师基本一致,如模型准确地定位到了肋骨旁短轴大血管水平切面(PSAXGV)显著的主动脉及主动脉瓣、胸骨旁短轴二尖瓣水平切面(PSAXMV)的二尖瓣与胸骨旁短轴乳头肌水平切面(PSAXPM)的乳头肌。实地部署后模型切面识别的整体准确率、精度、召回率与F1分数分别为0.9903、0.9865、0.9868与0.9865;在RTX 3060 GPU上单个动态图像的平均推理时间平均值为(303.2±119.3)ms,基本满足采图后即时处理的临床需求。结论本研究提出的SlowFast-Echo模型有着良好的二维经胸超声心动图动态图像切面识别性能与推理实时性,实用性较强,具有较好的应用前景。

Objective To propose a deep learning-based view recognition model,SlowFast-Echo,for the automatic view recognition of two-dimensional(2D)transthoracic echocardiographic dynamic images.Methods From August to December 2022,722 patients who underwent 2D transthoracic echocardiography at the Department of Ultrasound Medicine,Affiliated Hospital of Medical School,Nanjing University(9 types of clinically commonly used views[including apical two-chamber,apical three-chamber,and apical four-chamber views],with a total of 2243 dynamic images)were selected,and the images of each view were divided into training set,validation set,and test set in a ratio of 5:2:3.After training and validation of the SlowFast-Echo model,the performance of the model was evaluated quantitatively in terms of accuracy,precision,recall,and F1 score,qualitatively in terms of the interpretability of the model with regard to class activation mapping,and practically in terms of the performance of the model after field deployment to the ultrasound medicine department.Results The overall accuracy,precision,recall,and F1 score of the SlowFast-Echo model for dynamic image view recognition in the test set were 0.9866,0.9847,0.9872,and 0.9859,respectively,and the significance heatmap indicated that the model's regions of interest were generally consistent with those drawn by the physicians;e.g.,the model accurately pinpointed the significant aorta and aortic valve in parasternal short axis view of great vessel(PSAXGV)view,mitral valve in parasternal short axis view of left ventricle at mitral value level(PSAXMV)view,and papillary muscles in parasternal short axis view of left ventricle at papillary muscle level(PSAXPM)view.The overall accuracy,precision,recall,and F1 score of the model for view recognition after deployment were 0.9903,0.9865,0.9868,and 0.9865,respectively,and the average inference time on RTX 3060 GPU for a single dynamic image was(303.2±119.3)ms,which basically meets the clinical demand for immediate processing after image acquisition.Conclusion The SlowFast-Echo model proposed in this study has good performance in view recognition of 2D transthoracic echocardiographic dynamic images and inference in real time,which is practically useful.