用AUC评估分类器的预测性能

Using AUC to Evaluate Predictive Performance of Classifiers

准确率一直被作为分类器预测性能的主要评估标准，但是它存在着诸多的缺点和不足。本文将准确率与AUC（the area under the Receiver Operating Characteristic curve）进行了理论上的对比分析，并分别使用AUC和准确率对3种分类学习算法在15个两类数据集上进行了评估。综合理论和实验两个方面的结果，显示了AUC不但优于而且应该替代准确率，成为更好的分类器性能的评估度量。同时，用AUC对3种分类学习算法的重新评估，进一步证实了基于贝叶斯定理的Naive Bayes和TAN-CMI分类算法优于决策树分类算法C4．5。

Accuracy has been used as a main evaluation criterion for predictive performance of classifiers. However, it has many shortcomings and disadvantages. In this paper, we compared accuracy to AUC（ the area under the Receiver Operating Characteristic curve） measure in theory and used respectively AUC and accuracy to evaluate three classification learning algorithms on fifteen binary datasets. Theoretical and experimental results show that AUC is not only a better measure than accuracy but also should replace it in comparing classifiers. Furthermore, using AUC to re-evaluate three classification algorithms shows classification algorithm NaiveBayes and TAN-CMI based on Bayes theorem are better than decision tree classification algorithm CA.5 in performance.