On the Noise Resilience of Ranking Measures

Conference paper

DOI: 10.1007/978-3-319-46672-9_6

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9948)
Cite this paper as:
Berrar D. (2016) On the Noise Resilience of Ranking Measures. In: Hirose A., Ozawa S., Doya K., Ikeda K., Lee M., Liu D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science, vol 9948. Springer, Cham


Performance measures play a pivotal role in the evaluation and selection of machine learning models for a wide range of applications. Using both synthetic and real-world data sets, we investigated the resilience to noise of various ranking measures. Our experiments revealed that the area under the ROC curve (AUC) and a related measure, the truncated average Kolmogorov-Smirnov statistic (taKS), can reliably discriminate between models with truly different performance under various types and levels of noise. With increasing class skew, however, the H-measure and estimators of the area under the precision-recall curve become preferable measures. Because of its simple graphical interpretation and robustness, the lower trapezoid estimator of the area under the precision-recall curve is recommended for highly imbalanced data sets.


Ranking Classification Noise Robustness ROC curve AUC H-measure taKS Precision-recall curve 

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.School of Arts and Sciences, College of EngineeringShibaura Institute of TechnologyMinuma-kuJapan

Personalised recommendations