More About Entropy

  • Max Bramer
Chapter
Part of the Undergraduate Topics in Computer Science book series (UTICS)

Abstract

This chapter returns to the subject of the entropy of a training set. It explains the concept of entropy in detail using the idea of coding information using bits. The important result that when using the TDIDT algorithm information gain must be positive or zero is discussed, followed by the use of information gain as a method of feature reduction for classification tasks.

Keywords

Information Gain Feature Reduction Frequency Table Exact Power Unknown Classification 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. [1]
    McSherry, D., & Stretch, C. (2003). Information gain (University of Ulster Technical Note). Google Scholar
  2. [2]
    Noordewier, M. O., Towell, G. G., & Shavlik, J. W. (1991). Training knowledge-based neural networks to recognize genes in DNA sequences. In Advances in neural information processing systems (Vol. 3). San Mateo: Morgan Kaufmann. Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Max Bramer
    • 1
  1. 1.School of ComputingUniversity of PortsmouthPortsmouthUK

Personalised recommendations