An Empirical Comparison of Ideal and Empirical ROC-Based Reject Rules

  • Claudio Marrocco
  • Mario Molinara
  • Francesco Tortorella
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4571)


Two class classifiers are used in many complex problems in which the classification results could have serious consequences. In such situations the cost for a wrong classification can be so high that can be convenient to avoid a decision and reject the sample. This paper presents a comparison between two different reject rules (the Chow’s and the ROC rule). In particular, the experiments show that the Chow’s rule is inappropriate when the estimates of the a posteriori probabilities are not reliable.


ROC curve reject option two-class classification cost-sensitive classification decision theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Webb, A.R.: Statistical Pattern Recognition. John Wiley and Sons Ltd, West Sussex (2002)zbMATHGoogle Scholar
  2. 2.
    Chow, C.K.: On optimum recognition error and reject tradeoff. IEEE Trans. Information Theory IT 10, 41–46 (1970)CrossRefGoogle Scholar
  3. 3.
    Tortorella, F.: A ROC-based reject rule for dichotomizers. Pattern Recognition Letters 26, 167–180 (2005)CrossRefGoogle Scholar
  4. 4.
    Santos-Pereira, C.M., Pires, A.M.: On optimal reject rules and ROC curves, Pattern Recognition Letters. Pattern Recognition Letters 26, 943–952 (2005)CrossRefGoogle Scholar
  5. 5.
    Xie, J., Qiu, Z., Wu, J.: Bootstrap methods for reject rules of Fisher LDA. In: Proc. 18th Int. Conf. on Pattern Recognition, 3rd edn., pp. 425–428. IEEE Press, NJ (2006)Google Scholar
  6. 6.
    van Trees, H.L.: Detection, Estimation, and Modulation Theory. Wiley, New York (1968)zbMATHGoogle Scholar
  7. 7.
    Provost, F., Fawcett, T.: Robust classification for imprecise environments. Machine Learning 42, 203–231 (2001)zbMATHCrossRefGoogle Scholar
  8. 8.
    Mukhopadhyay, N.: Probability and Statistical Inference. Marcel Dekker Inc., New York (2000)zbMATHGoogle Scholar
  9. 9.
    Garthwaite, P.H., Jolliffe, I.T., Jones, B.: Statistical Inference, 2nd edn. Oxford University Press, Oxford (2002)zbMATHGoogle Scholar
  10. 10.
    Tortorella, F.: Reducing the classification cost of support vector classifiers through an ROC-based reject rule. Pattern Analysis and Applications 7, 128–143 (2004)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Blake, C., Keogh, E., Merz, C.J.: UCI Repository of Machine Learning Databases, Irvine, University of California, Department of Information and Computer Science (1998),
  12. 12.
    Huang, J., Ling, C.X.: Using AUC and accuracy in evaluating learning algorithms, IEEE Trans. Knowledgde and Data Engineering 17, 299–310 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Claudio Marrocco
    • 1
  • Mario Molinara
    • 1
  • Francesco Tortorella
    • 1
  1. 1.DAEIMI, Università degli Studi di Cassino, Via G. Di Biasio 43, 03043 Cassino (FR)Italia

Personalised recommendations