甲状腺结节人工智能自动分割和分类系统的建立和验证

Establishment and verification of an artificial intelligence system for automatic segmentation and classification of thyroid nodules

目的开发一种能自动分割和诊断甲状腺结节良恶性的人工智能(AI)系统。方法收集872例2017年10月至2018年10月于上海市第十人民医院行穿刺活检确认的甲状腺结节患者的超声图像,利用AI方法对这些图片进行处理、检测等并最终反馈结果,建立AI系统,并对AI系统进行验证及内部测试。按照6∶2∶2的比例将所有收集的超声图像分为训练集、验证集和内部测试集进行初步验证测试。纳入外院209例甲状腺结节患者(共209个结节)超声图像再次进行验证,以穿刺或外科手术病理结果为诊断标准,计算低年资医师组、高年资医师组和AI系统诊断甲状腺结节良恶性的敏感度、特异度、准确性、阳性预测值、阴性预测值,并绘制三者诊断甲状腺结节良恶性的受试者操作特征曲线,计算曲线下面积(AUC),采用Delong检验比较AI系统与低年资医师组、高年资医师组的诊断效能。结果AI系统结节自动分割率在验证集、内部测试集和外部测试集上分别为98.8%、98.9%、98.1%。在外部测试集中,AI系统的诊断敏感度、特异度及准确性与低年资医师组、高年资医师组比较,差异均无统计学意义(P均>0.017)。而AI系统诊断甲状腺结节良恶性的AUC优于低年资医师组[0.885(95%CI:0.842~0.929)vs 0.823(95%CI:0.771~0.875),P=0.022],而与高年资医师组[0.932(95%CI:0.897~0.966)]类似(P=0.096)。结论本研究开发了一种能自动分割及诊断甲状腺结节良恶性的AI系统,其在外部测试集中具有较高的诊断效能,有望辅助低年资医师更准确鉴别甲状腺结节良恶性。

Objective To develop an artificial intelligence(AI)system that can automatically segment and diagnose benign and malignant thyroid nodules.Methods The ultrasound images of 872 patients with thyroid nodules confirmed by puncture biopsy at Shanghai Tenth People's Hospital from October 2017 to October 2018 were collected,and the results were processed,monitored,and finally fed back by AI methods.Then,an AI system was established,and the system was verified and tested internally.According to a ratio of 6:2:2,all the collected ultrasound images were divided into training set,validation set,and internal test set for preliminary verification test.The ultrasound images of 209 patients with thyroid nodules(a total of 209 nodules)in other hospitals were re-verified,and the sensitivity,specificity,accuracy,positive predictive value,and negative predictive value of a junior physician group,a senior physician group,and the AI system in the diagnosis of benign and malignant thyroid nodules were calculated using the pathological results of puncture biopsy or surgery as the diagnostic criteria.The receiver operation characteristic curves of the three in the diagnosis of benign and malignant thyroid nodules were plotted,and the area under the curve(AUC)was calculated.The Delong test was used to compare the diagnostic performance of the AI system with junior physicians and senior physicians.Results The automatic nodule segmentation rates of the AI system were 98.8%,98.9%,and 98.1%in the validation set,internal test set,and external test set,respectively.In the external test set,there were no significant differences in the diagnostic sensitivity,specificity,or accuracy between the AI system and the junior or senior physician group(P>0.017 for all).The AUC of the AI system in the diagnosis of benign and malignant thyroid nodules was better than that of junior physicians[0.885(95%CI:0.842-0.929)vs 0.823(95%CI:0.771-0.875),P=0.022],but similar to that of senior physicians[0.932(95%CI:0.897-0.966)](P=0.096).Conclusion We have developed an AI system that can automatically segment and diagnose benign and malignant thyroid nodules,which has high diagnostic efficacy in the external test set,and it is expected to assist junior physicians to more accurately identify benign and malignant thyroid nodules.