More About Entropy

  • Max Bramer
Part of the Undergraduate Topics in Computer Science book series (UTICS)


This chapter returns to the subject of the entropy of a training set. It explains the concept of entropy in detail using the idea of coding information using bits. The important result that when using the TDIDT algorithm information gain must be positive or zero is discussed, followed by the use of information gain as a method of feature reduction for classification tasks.




  1. [1]
    McSherry, D., & Stretch, C. (2003). Information gain (University of Ulster Technical Note). Google Scholar
  2. [2]
    Noordewier, M. O., Towell, G. G., & Shavlik, J. W. (1991). Training knowledge-based neural networks to recognize genes in DNA sequences. In Advances in neural information processing systems (Vol. 3). San Mateo: Morgan Kaufmann. Google Scholar

Copyright information

© Springer-Verlag London Ltd. 2016

Authors and Affiliations

  • Max Bramer
    • 1
  1. 1.School of ComputingUniversity of PortsmouthPortsmouthUK

Personalised recommendations