More About Entropy

  • Max Bramer
Part of the Undergraduate Topics in Computer Science book series (UTICS)


This chapter returns to the subject of the entropy of a training set. It explains the concept of entropy in detail using the idea of coding information using bits. The important result that when using the TDIDT algorithm information gain must be positive or zero is discussed, followed by the use of information gain as a method of feature reduction for classification tasks.


Information Gain Feature Reduction Frequency Table Exact Power Unknown Classification 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [1]
    McSherry, D., & Stretch, C. (2003). Information gain (University of Ulster Technical Note). Google Scholar
  2. [2]
    Noordewier, M. O., Towell, G. G., & Shavlik, J. W. (1991). Training knowledge-based neural networks to recognize genes in DNA sequences. In Advances in neural information processing systems (Vol. 3). San Mateo: Morgan Kaufmann. Google Scholar

Copyright information

© Springer-Verlag London Ltd. 2016

Authors and Affiliations

  • Max Bramer
    • 1
  1. 1.School of ComputingUniversity of PortsmouthPortsmouthUK

Personalised recommendations