An Optimal Reject Rule for Binary Classifiers

  • Francesco Tortorella
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)


Binary classifiers are used in many complex classification problems in which the classification result could have serious consequences. Thus, they should ensure a very high reliability to avoid erroneous decisions. Unfortunately, this is rarely the case in real situations where the cost for a wrong classification could be so high that it should be convenient to reject the sample which gives raise to an unreliable result. However, as far as we know, a reject option specifically devised for binary classifiers has not been yet proposed. This paper presents an optimal reject rule for binary classifiers, based on the Receiver Operating Characteristic curve. The rule is optimal since it maximizes a classification utility function, defined on the basis of classification and error costs peculiar for the application at hand. Experiments performed with a data set publicly available confirmed the effectiveness of the proposed reject rule.


Optimal Threshold Binary Classifier Decision Threshold Level Curve Cost Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Chow, C.K.: An Optimum Character Recognition System Using Decision Functions. IRE Trans. Electronic Computers EC-6 (1957) 247–254CrossRefGoogle Scholar
  2. 2.
    Chow, C.K.: On Optimum Recognition Error and Reject Tradeoff. IEEE Trans. Inf. Th. IT-10 (1970) 41–46CrossRefGoogle Scholar
  3. 3.
    Dubuisson, B., Masson, M.: A Statistical Decision Rule with Incomplete Knowledge about Classes. Pattern Recognition 26 (1993) 155–165CrossRefGoogle Scholar
  4. 4.
    Muzzolini, R., Yang, Y.-H., Pierson, R.: Classifier Design with Incomplete Knowledge. Pattern Recognition 31 (1998) 345–369CrossRefGoogle Scholar
  5. 5.
    Cordella, L.P., De Stefano, C., Tortorella, F., Vento, M.: A Method for Improving Classification Reliability of Multilayer Perceptrons. IEEE Trans. Neur. Net. 6 (1995) 1140–1147CrossRefGoogle Scholar
  6. 6.
    Bradley, A.P.: The use of the Area under the ROC Curve in the Evaluation of Machine Learning Algorithms. Pattern Recognition 30 (1997) 1145–1159CrossRefGoogle Scholar
  7. 7.
    Provost, F., Fawcett, T.: Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions. Proc. 3rd Int. Conf. on Knowledge Discovery and Data Mining (KDD-97)Google Scholar
  8. 8.
    Blake, C., Keogh, E., Merz, C.J.: UCI Repository of machine learning databases, [] Irvine, CA: University of California, Department of Information and Computer Science, 1998Google Scholar
  9. 9.
    Hoekstra, A., Kraaijved, M.A., de Ridder, D., Schmidt, W.F., Ypma, A.: The complete SPRLIB & ANNLIB. Statistical Pattern recognition and Artificial Neural Network Library. 2nd edn. Version 3.1. User’s Guide and Reference Manual, Pattern Recognition Group, Faculty of Applied Physics, Delft University of Technology, Delft, The Netherlands (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Francesco Tortorella
    • 1
  1. 1.Dipartimento di Automazione, Elettromagnetismo, Ingegneria dell’Informazione e Matematica IndustrialeUniversità degli Studi di CassinoCassino (FR)Italy

Personalised recommendations