ACC和AUC在ATR算法评估中的应用

Application of ACC and AUC in ATR algorithm evaluation

ATR算法评估领域常用的评估指标为正确识别率ACC,但ACC自身存在诸多缺陷,仅用ACC指标评估结论具有一定的盲目性和误导性。基于ROC曲线的性能评估指标AUC反映了识别算法在多门限下的整体性能,克服了ACC指标的缺陷。在概述ACC及AUC各自含义和性质的基础上,全面揭示了二者之间内在的联系,AUC值固定时,加权和非加权ACC均值与AUC间存在线性关系;另一方面在ACC固定的情况下,若PTP已知,ACC、AUCmax和AUCmin之间也存在线性关系,最后阐释了AUC的优越性。

The most commonly used evaluation measure in the domain of Automatic Target Recognition （ATR） algorithm evaluation is Accuracy（ACC）, however, the ACC measure has many defects. The evaluation conclusions drawn under ACC alone may be blind and misleading.The Area Under the Curve （AUC） based on Receiver Operating Characteristic （ROC） curve can depict the whole performance of the algorithm under multi- threshold and overcome the inherent defects of ACC. The concept and characteristics of ACC and AUC are discussed respectively, and the relationship between the two measures is analyzed in detail. When the AUC value is fixed, the weighted and un-weighted mean value of ACC has linear relationship with AUC; meanwhile, when the ACC value is fixed and the probability of true positive PTP is given, the ACC also has linear relationship with the maximum and minimum value of AUC. In the end, the superiority of the AUC evaluation measure is analyzed.