人工智能医疗器械质量评价测试集样本量分析

Analysis of Sample Size of Artificial Intelligence Medical Device Test Set for Quality Evaluation

目的本文探讨了人工智能医疗器械质量评价测试集样本量的测算方法,并通过试验进行验证。方法选取两种辅助诊断人工智能医疗器械,以不同的样本量计算召回率(灵敏度)、特异度、精确度指标,分析样本量与结果偏差的关系。结果分别通过理论计算和试验分析的方式计算了糖尿病视网膜病变眼底图像及肺部CT影像辅助诊断软件满足检测的样本量要求,分别为700例和500例。结论本文对于有流行病学统计病种产品的样本量估算方法进行了验证,对于无流行病学统计病种的样本量进行试验分析,得出了两类已上市产品的推荐测试样本量大小。在满足测试要求的前提下,最大限度地降低测试集样本数量,节约社会资源。

Objective To explore the method of measuring the sample quantity of artificial intelligence medical device test set for quality evaluation,and verify it through experiments.Methods Two kinds of artificial intelligence medical devices for aided diagnosis were selected to calculate the recall rate(sensitivity),specificity and accuracy with different sample quantity,and the relationship between sample size and result deviation was analyzed.Results According to the theoretical calculation and experimental analysis,the sample sizes of the fundus images of diabetic retinopathy and the auxiliary diagnostic software of lung CT images that met the test requirements were calculated,which were 700 cases and 500 cases respectively.Conclusion In this paper,the sample quantity estimation method for products with statistical epidemiological diseases is verified,and the sample quantity of products without statistical epidemiological diseases is analyzed experimentally,and the recommended test sample quantity for two types of products already on the market are obtained.On the premise of meeting the test requirements,the number of test set samples should be minimized to save social resources.