The Entropy of Relations and a New Approach for Decision Tree Learning
The formula for scaling how much information in relations on the finite universe is proposed, which is called the entropy of relation R and denoted by H (R). Based on the concept of H (R), the entropy of predicates and the information of propositions are measured. We can use these measures to evaluate predicates and choose the most appropriate predicate for some given cartesian set. At last, H (R) is used to induce decision tree. The experiment show that the new induction algorithm denoted by IDIR do better than ID3 on the aspects of nodes and test time.
KeywordsBinary Relation Information Complexity Bijective Function Fuzzy Measure Fuzzy Partition
Unable to display preview. Download preview PDF.
- 1.Xuelong, Z.: Fundamentals of applied information theory. Tsinghua University press, Beijing (2001)Google Scholar
- 2.Uang, N.C.: Generalized information theory. University of scince and technology of China press, Hefei (1993)Google Scholar
- 5.Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
- 6.De MSantaras, R.L.: A distance-based attribute selection measure for decision tree induction. Machine Learning 6, 81–92 (1991)Google Scholar
- 7.Simovici, D.A., Jaroszewicz, S.: Generalized conditional entropy and decision trees. In: Proceedings of EGC France, pp. 369–380 (2003)Google Scholar
- 10.Kolman, B., Busby, R.C., Ross, S.: Discrete mathematical structures, 3rd edn. Prentice Hall, USA (1996)Google Scholar