Information Theory and Classification Error in Probabilistic Classifiers

  • Aritz Páerez
  • Pedro Larrañaga
  • Iñaki Inza
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4265)


This work shows, using bivariate continuous artificial domains, the relation that seems to exist between some measures based on the information theory and the expected classification error.

The relations that seem to be found in this work could be applied to the improvement of the classifiers which assign a posteriori probabilities to each class value. They also could be used in other tasks related to the supervised classification such as feature subset selection or discretization.


Mutual Information Posteriori Probability Feature Subset Selection Kernel Component Intelligent Information System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. John Wiley and Sons, Chichester (1991)zbMATHCrossRefGoogle Scholar
  2. 2.
    Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the 13th International Conference on Artificial Intelligence, pp. 1022–1027 (1993)Google Scholar
  3. 3.
    Hall, M.A., Smith, L.A.: Feature subset selection: A correlation based filter approach. In: Proceeding of the Fourth International Conference on Neural Information Processing and Intelligent Information Systems, pp. 855–858 (1997)Google Scholar
  4. 4.
    Páerez, A., Larrañaga, P., Inza, I.: Supervised classification with conditional Gaussian networks: Increasing the structure complexity from naive Bayes. International Journal of Approximate Reasoning (in press, 2006)Google Scholar
  5. 5.
    Silverman, B.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, London (1986)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Aritz Páerez
    • 1
  • Pedro Larrañaga
    • 1
  • Iñaki Inza
    • 1
  1. 1.Intelligent Systems Group, Computer Science and A.I Dept.University of The Basque CountrySpain

Personalised recommendations