Machine Learning

, Volume 6, Issue 1, pp 67–80

Information-Based Evaluation Criterion for Classifier's Performance

  • Igor Kononenko
  • Ivan Bratko

DOI: 10.1023/A:1022642017308

Cite this article as:
Kononenko, I. & Bratko, I. Machine Learning (1991) 6: 67. doi:10.1023/A:1022642017308


In the past few years many systems for learning decision rules from examples were developed. As different systems allow different types of answers when classifying new instances, it is difficult to appropriately evaluate the systems' classification power in comparison with other classification systems or in comparison with human experts. Classification accuracy is usually used as a measure of classification performance. This measure is, however, known to have several defects. A fair evaluation criterion should exclude the influence of the class probabilities which may enable a completely uninformed classifier to trivially achieve high classification accuracy. In this paper a method for evaluating the information score of a classifier's answers is proposed. It excludes the influence of prior probabilities, deals with various types of imperfect or probabilistic answers and can be used also for comparing the performance in different domains.

Classifier evaluation criteria machine learning information theory 

Copyright information

© Kluwer Academic Publishers 1991

Authors and Affiliations

  • Igor Kononenko
    • 1
  • Ivan Bratko
    • 2
  1. 1.Faculty of Electrical and Computer EngineeringLjubljanaYugoslavia
  2. 2.Faculty of Electrical and Computer EngineeringJozef Stefan InstituteLjubljanaYugoslavia

Personalised recommendations