人工智能辅助阅片与单纯人工阅片在女性阴道微生态系统形态学诊断中的比对研究

Comparative Study of Artificial Intelligence-assisted Analysis and Manual Visual Analysis in Gynecological Microbiome Diagnosis

目的评估不同级别检验员对革兰染色涂片阴道微生态形态学评价的基线水平,分析检验员在电子阅片和镜下阅片的差异,探究使用人工智能分析系统独立进行微生态评价以及辅助检验员进行微生态评价的能力表现,评价人工智能分析系统在临床中的应用价值。方法该研究样本来源于北京协和医院《中国女性人群下生殖道微生态菌群基线研究项目》,收集2021年5月~2021年7月女性阴道分泌物涂片共385例,经革兰染色和图像采集后,分别进行检验员等级考核以及人工显微镜镜下阅片、人工电子阅片、人工智能(artificial intelligence,AI)独立阅片和AI辅助检验员阅片。在确定镜下阅片金标准和电子阅片金标准之后,分析两种不同阅片方式在AV评分和Nugent评分的差异,比较不同级别检验员、AI,以及经AI辅助后,在AV评分和Nugent评分上的能力表现。结果镜下阅片和电子阅片在需氧菌性阴道炎(aerobic vaginitis,AV)和细菌性阴道病(bacterial vaginosis,BV)(含BV中间型)诊断的Kappa一致性分析分别为0.91和0.93(P<0.01)。AI独立阅片在AV和BV(含BV中间型)诊断的准确度分别为0.85和0.92,灵敏度分别为0.86和0.88,Kappa值分别为0.62和0.79。初级检验员在电子阅片下的AV和BV(含BV中间型)诊断的准确度分别为0.85±0.02和0.89±0.01,灵敏度分别为0.64±0.06和0.84±0.07,Kappa值分别为0.55±0.07和0.72±0.04。高级检验员在电子阅片下的AV和BV(含BV中间型)诊断的准确度分别为0.92±0.03和0.91±0.03,灵敏度分别为0.87±0.02和0.92±0.04,Kappa值分别为0.78±0.07和0.79±0.06。经AI辅助诊断后,初级检验员AV和BV(含BV中间型)诊断的Kappa值提升至0.77±0.04和0.78±0.02,高级检验员AV和BV(含BV中间型)诊断的Kappa值提升至0.82±0.05和0.85±0.01。结论镜下阅片和电子阅片的一致性非常高,电子阅片或可替代镜下阅片成为一种新的阅片方式。AI独立阅片诊断AV和BV(含BV中间型)的能力优于普通检验员,比高级检验员略差。不同级别检验员经AI辅助诊断后,AV和BV(含BV中间型)的诊断能力均有提升,其中初级检验员提升明显,能力接近高级检验员的水平,且各检验员之间的偏差缩小明显。整体结果表明,使用人工智能Descartes-Image妇科微生态辅助分析软件不仅能提升检验员诊断能力,还能减小检验员之间的偏差,使诊断结果不容易因为人为因素而出现较大波动,保证了结果的稳定性和可靠性。

Objective To evaluate the accuracy of morphological evaluation of vaginal microecology on Gram-stained vaginal smears by operators of differing levels of experience,determine differences between analyses using previously captured images versus live microscope images,explore the use of an artificial intelligence(AI)analysis system to independently conduct vaginal microecological evaluation,assist operators in vaginal microecological evaluation,and determine the application value of the AI analysis system in a clinical setting.Methods A total of 385 cases of female vaginal secretion smears from May 2021 to July 2021 were collected.After gram dyeing and image acquisition,the inspector’s grade assessment,manual microscope film reading,manual electronic film reading,AI independent film reading and AI auxiliary inspector film reading were conducted respectively.After determining the gold standard of microscopic viewing and the gold standard of image viewing,the differences in AV score and Nugent score of two different viewing methods were analyzed,and the performance of different operators,AI,and AI-assisted performance on AV and Nugent scores were compared.Results The Kappa concordance analysis of microscopic viewing and image viewing in the diagnosis of AV and BV(including BV intermediate)were 0.91 and 0.93,respectively(P<0.01).The accuracy of AI independent analysis in the diagnosis of AV and BV(including intermediate BV)were 0.85 and 0.92,sensitivity was 0.86 and 0.88 respectively,and the Kappa value was 0.62 and 0.79 respectively.The diagnostic accuracy of AV and BV(including intermediate BV)by junior operators using image viewing were 0.85±0.02 and 0.89±0.01,the sensitivity were 0.64±0.06 and 0.84±0.07,and the Kappa value were 0.55±0.07 and 0.72±0.04 respectively.The diagnostic accuracy of AV and BV(including intermediate BV)by senior operators using image viewing was 0.92±0.03 and 0.91±0.03,the sensitivity was 0.87±0.02 and 0.92±0.04 respectively,and the Kappa values was 0.78±0.07 and 0.79±0.06 respectively.After AIassisted diagnosis,the Kappa values of AV and BV(including intermediate BV)diagnosed by junior operators were increased to 0.77±0.04 and 0.78±0.02,and the Kappa values of senior operators AV and BV(including intermediate BV)diagnosis were increased to 0.82±0.05 and 0.85±0.01.Conclusion The consistency between microscopic viewing and image viewing was very high,suggesting that image viewing could replace microscopic viewing as a new viewing method.The ability of AI to independently analyze and diagnose AV and BV(including intermediate BV)was better than that of junior operators,and slightly inferior to senior operators.With AI-assisted analysis,the diagnostic capabilities for AV and BV(including intermediate BV)among both junior and senior operators improved.The performance of junior operators improved significantly and nearly approached the performance of senior operators,significantly reducing the performance gap between junior and senior operators.The overall results indicated that the use of the Turing Microbial Gynecology Microbiome Auxiliary Analysis System not only improved the diagnostic ability of the operators,but also reduce the deviation between different operators thereby reducing fluctuations due to human factors and improving reliability of diagnosis.