More About Entropy
This chapter returns to the subject of the entropy of a training set. It explains the concept of entropy in detail using the idea of coding information using bits. The important result that when using the TDIDT algorithm information gain must be positive or zero is discussed, followed by the use of information gain as a method of feature reduction for classification tasks.
KeywordsInformation Gain Feature Reduction Frequency Table Exact Power Unknown Classification
- McSherry, D., & Stretch, C. (2003). Information gain (University of Ulster Technical Note). Google Scholar
- Noordewier, M. O., Towell, G. G., & Shavlik, J. W. (1991). Training knowledge-based neural networks to recognize genes in DNA sequences. In Advances in neural information processing systems (Vol. 3). San Mateo: Morgan Kaufmann. Google Scholar