Classification with Rejection: Concepts and Evaluations

  • Wladyslaw HomendaEmail author
  • Marcin Luckner
  • Witold Pedrycz
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 364)


Standard classification process allocates all processed elements to given classes. Such type of classification assumes that there are only native and no foreign elements, i.e., all processed elements are included in given classes. The quality of standard classification can be measured by two factors: numbers of correctly and incorrectly classified elements, called True Positives and False Positives. Admitting foreign elements in standard classification process increases False Positives and, in this way, deteriorates quality of classification. In this context, it is desired to reject foreign elements, i.e., not to assign them to any of given classes. Rejecting foreign elements will reduce the number of false positives, but can also reject native elements reducing True Positives as side effect. Therefore, it is important to build well-designed rejection, which will reject significant part of foreigners and only few natives. In this paper, evaluations of classification with rejection concepts are presented. Three main models: a classification without rejection, a classification with rejection, and a classification with reclassification are presented. The concepts are illustrated by flexible ensembles of binary classifiers with evaluations of each model. The proposed models can be used, in particular, as classifiers working with noised data, where recognized input is not limited to elements of known classes.


Rejection rule Binary classifiers ensemble Reclassification 



The research is supported by the National Science Center, grant No 2012/07/B/ST6/01501, decision no UMO-2012/07/B/ST6/01501.


  1. 1.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–27 (2011)CrossRefGoogle Scholar
  2. 2.
    Chow, C.: On optimum recognition error and reject tradeoff. IEEE Trans. Inf. Theory 16(1), 41–46 (1970)CrossRefzbMATHGoogle Scholar
  3. 3.
    Fumera, G., Roli, F.: Analysis of error-reject trade-off in linearly combined multiple classifiers. Pattern Recogn. 37, 1245–1265 (2004)CrossRefzbMATHGoogle Scholar
  4. 4.
    Graves, D., Pedrycz, W.: Kernel-based fuzzy clustering and fuzzy clustering: a comparative experimental study (2010)Google Scholar
  5. 5.
    Ha, T.M.: Optimum tradeoff between class-selective rejection error and average number of classes. Eng. Appl. Artif. Intell. 10(6), 525–529 (1997)CrossRefGoogle Scholar
  6. 6.
    Ishibuchi, H., Nakashima, T., Morisawa, T.: Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets Syst. 103, 223–238 (1999)Google Scholar
  7. 7.
    Li, M., Sethi, I.K.: Confidence-based classifier design. Pattern Recogn. 39(7), 1230–1240 (2006)CrossRefzbMATHGoogle Scholar
  8. 8.
    Lou, Z., Liu, K., Yang, J.Y., Suen, C.Y.: Rejection criteria and pairwise discrimination of handwritten numerals based on structural features. Pattern Anal. Appl. 2(3), 228–238 (1999)CrossRefGoogle Scholar
  9. 9.
    Luckner, M.: Comparison of hierarchical svm structures in letters recognition task. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J. (eds.) Computational Intelligence: Methods and Applications, pp. 291–302. Challenging Problems of Science, Academic Publishing House EXIT, Warsaw (2008)Google Scholar
  10. 10.
    Mascarilla, L., Frélicot, C.: Reject strategies driven combination of pattern classifiers. Pattern Anal. Appl. 5(2), 234–243 (2002)CrossRefMathSciNetzbMATHGoogle Scholar
  11. 11.
    Pierce, S.G., Worden, K., Manson, G.: Evaluation of neural network performance and generalisation using thresholding functions. Neural Comput. Appl. 16, 109–124 (2007)CrossRefGoogle Scholar
  12. 12.
    Pillai, I., Fumera, G., Roli, F.: A classification approach with a reject option for multi-label problems. In: Maino, G., Foresti, G.L. (eds.) ICIAP (1). Lecture Notes in Computer Science, vol. 6978, pp. 98–107. Springer (2011)Google Scholar
  13. 13.
    Schölkopf, B., et al.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)Google Scholar
  14. 14.
    Simeone, P., Marrocco, C., Tortorella, F.: Design of reject rules for ECOC classification systems. Pattern Recogn. 45(2), 863–875 (2012)Google Scholar
  15. 15.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  16. 16.
    Xie, J., Qiu, Z., Wu, J.: Bootstrap methods for reject rules of fisher LDA. Pattern Recogn. 425–428 (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Wladyslaw Homenda
    • 1
    Email author
  • Marcin Luckner
    • 1
  • Witold Pedrycz
    • 2
    • 3
  1. 1.Faculty of Mathematics and Information ScienceWarsaw University of TechnologyWarsawPoland
  2. 2.System Research Institute, Polish Academy of SciencesWarsawPoland
  3. 3.Department of Electrical and Computer EngineeringUniversity of Alberta EdmontonAlbertaCanada

Personalised recommendations