Evaluation measures play an important role in the design of new approaches, and often quality is measured by assessing the relevance of the obtained result set. While many evaluation measures based on precision/recall...Evaluation measures play an important role in the design of new approaches, and often quality is measured by assessing the relevance of the obtained result set. While many evaluation measures based on precision/recall are based on a binary relevance model, ranking correlation coefficients are better suited for multi-class problems. State-of-the-art rank- ing correlation coefficients like Kendall＇s T and Spearman＇s p do not allow the user to specify similarities between differ- ing object classes and thus treat the transposition of objects from similar classes the same way as that of objects from dissimilar classes. We propose ClasSi, a new ranking corre- lation coefficient which deals with class label rankings and employs a class distance function to model the similarities between the classes. We also introduce a graphical representation of ClasSi which describes how the correlation evolves throughout the ranking.展开更多
文摘Evaluation measures play an important role in the design of new approaches, and often quality is measured by assessing the relevance of the obtained result set. While many evaluation measures based on precision/recall are based on a binary relevance model, ranking correlation coefficients are better suited for multi-class problems. State-of-the-art rank- ing correlation coefficients like Kendall＇s T and Spearman＇s p do not allow the user to specify similarities between differ- ing object classes and thus treat the transposition of objects from similar classes the same way as that of objects from dissimilar classes. We propose ClasSi, a new ranking corre- lation coefficient which deals with class label rankings and employs a class distance function to model the similarities between the classes. We also introduce a graphical representation of ClasSi which describes how the correlation evolves throughout the ranking.
