Discrimination-Based Criteria for the Evaluation of Classifiers

  • Thanh Ha Dang
  • Christophe Marsala
  • Bernadette Bouchon-Meunier
  • Alain Boucher
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4027)


Evaluating the performance of classifiers is a difficult task in machine learning. Many criteria have been proposed and used in such a process. Each criterion measures some facets of classifiers. However, none is good enough for all cases. In this communication, we justify the use of discrimination measures for evaluating classifiers. The justification is mainly based on a hierarchical model for discrimination measures, which was introduced and used in the induction of decision trees.


Hierarchical Model Information Gain Shannon Entropy Discrimination Power Inductive Learning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alvarez, I.: Explaining the result of a decision tree to the end-user. In: Proceedings of the 16th European Conference on Artificial Intelligence, ECAI 2004, pp. 411–415 (2004)Google Scholar
  2. 2.
    Bradley, A.P.: The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7), 1145–1159 (1997)CrossRefGoogle Scholar
  3. 3.
    Caruana, R., Joachims, T., Backstrom, L.: KDD-Cup 2004: Results and analysis. SIGKDD Explorations 6(2), 95–108 (2004)CrossRefGoogle Scholar
  4. 4.
    Dang, T.H.: Entropies et leurs applications en apprentissage inductif (rapport de pré-soutenance). Technical report, Université Paris 6, France (2005)Google Scholar
  5. 5.
    Dang, T.H., Bouchon-Meunier, B., Marsala, C.: Measures of information for inductive learning. In: Proc. of Information Processing and Management of Uncertainty in Knownledge-Based Systems, IPMU 2004, Perugia - Italy, July 2004, pp. 1495–1502 (2004)Google Scholar
  6. 6.
    Fawcett, T.: Roc graphs: Notes and practical considerations for researchers. Machine Learning (1-2) (2004)Google Scholar
  7. 7.
    Kononenko, I., Bratko, I.: Information-based evaluation criterion for classifier’s performance. Machine Learning 6, 67–80 (1991)Google Scholar
  8. 8.
    Korb, K.B., Hope, L.R., Hughes, M.J.: The evaluation of predictive learners: Some theoretical and empirical results. In: EMCL 2001: Proceedings of the 12th European Conference on Machine Learning, London, UK, pp. 276–287. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  9. 9.
    Marsala, C.: Apprentissage inductif en présence de données imprécises: construction et utilisation d’arbres de décision flous. Ph.D thesis, Université Paris 6, France (1998)Google Scholar
  10. 10.
    Marsala, C., Bouchon-Meunier, B., Ramer, A.: Hierarchical model for discrimination measures. In: Proc. of the eight IFSA 1999 World Congres, Taipei - Taiwan, August 1999, pp. 339–343 (1999)Google Scholar
  11. 11.
    Provost, F.J., Fawcett, T.: Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In: Knowledge Discovery and Data Mining, pp. 43–48 (1997)Google Scholar
  12. 12.
    Provost, F.J., Fawcett, T., Kohavi, R.: The case against accuracy estimation for comparing induction algorithms. In: ICML 1998: Proceedings of the Fifteenth International Conference on Machine Learning, pp. 445–453. Morgan Kaufmann Publishers Inc., San Francisco (1998)Google Scholar
  13. 13.
    Quinlan, J.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)Google Scholar
  14. 14.
    Ramer, A., Bouchon-Meunier, B., Marsala, C.: An alytical structure of hierarchical discrimination. In: Proc. of the IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE, Seoul - Korea, August 1999, pp. 1050–1053 (1999)Google Scholar
  15. 15.
    Hettich, C.B.S., Merz, C.: UCI repository of machine learning databases (1998)Google Scholar
  16. 16.
    Shannon, C.: A mathematical theory of communication. Bell System Technical Journal 27, 379–423, 623–656 (1948)MATHMathSciNetGoogle Scholar
  17. 17.
    Stermann, F., Longuet, N.: Document technique de DTGen, rapport de stage de fin d’étude, DESS IA. Technical report, Laboratoire d’Informatique de Paris 6, Avril (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Thanh Ha Dang
    • 1
    • 2
  • Christophe Marsala
    • 1
  • Bernadette Bouchon-Meunier
    • 1
  • Alain Boucher
    • 2
  1. 1.DAPA, LIP6Université Pierre et Marie Curie – Paris6, CNRS UMR 7606ParisFrance
  2. 2.Institut de la Francophonie pour l’Informatique, Equipe MSIHanoiVietnam

Personalised recommendations