不平衡数据下模型评价指标的有效性探讨

Study on the Validity of Model Evaluation Index Under Imbalanced Data

模型评价指标对于衡量模型的表现尤为关键,只有正确合理的评价指标才能更好地反映模型的性能优劣。由于AUC和KS指标在被用于评价信用评级模型时,均存在忽视了数据的不平衡性和类别误判代价不等价性的不足,故文章从代价敏感矩阵出发计算总损失,进而提出新的评价指标——AKS指标。进一步地,模拟分析的结果表明,由AKS指标确定的阈值较KS指标更合理,且不同情形下AUC、KS和AKS指标的有效性分析结果表明AKS指标可以较好地衡量模型的分类性能。

The evaluation index of model is particularly critical to measuring the performance of the model.The correct and reasonable evaluation indexes can better reflect the performance of the model.In view of the fact that when AUC and KS indicators are used to evaluate the credit rating model,they both ignore the data imbalance and the category misjudgment cost inequality,this paper calculates the total loss from the cost-sensitive matrix,and then proposes a new evaluation index—AKS index.Furthermore,the results of simulation analysis show that the threshold determined by AKS index is more reasonable than KS index,and the validity analysis results of AUC,KS and AKS index under different circumstances show that AKS index can better measure the classification performance of the model.