An Optimal Reject Rule for Binary Classifiers
Binary classifiers are used in many complex classification problems in which the classification result could have serious consequences. Thus, they should ensure a very high reliability to avoid erroneous decisions. Unfortunately, this is rarely the case in real situations where the cost for a wrong classification could be so high that it should be convenient to reject the sample which gives raise to an unreliable result. However, as far as we know, a reject option specifically devised for binary classifiers has not been yet proposed. This paper presents an optimal reject rule for binary classifiers, based on the Receiver Operating Characteristic curve. The rule is optimal since it maximizes a classification utility function, defined on the basis of classification and error costs peculiar for the application at hand. Experiments performed with a data set publicly available confirmed the effectiveness of the proposed reject rule.
KeywordsOptimal Threshold Binary Classifier Decision Threshold Level Curve Cost Matrix
- 7.Provost, F., Fawcett, T.: Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions. Proc. 3rd Int. Conf. on Knowledge Discovery and Data Mining (KDD-97)Google Scholar
- 9.Hoekstra, A., Kraaijved, M.A., de Ridder, D., Schmidt, W.F., Ypma, A.: The complete SPRLIB & ANNLIB. Statistical Pattern recognition and Artificial Neural Network Library. 2nd edn. Version 3.1. User’s Guide and Reference Manual, Pattern Recognition Group, Faculty of Applied Physics, Delft University of Technology, Delft, The Netherlands (1998)Google Scholar