Comparison of Information Theoretical Measures for Reduct Finding
The paper discusses the properties of an attribute selection criterion for building rough set reducts based on discernibility matrix and compares it with Shannon entropy and Gini index used for building decision trees. It has been shown theoretically and experimentally that entropy and Gini index tend to work better if the reduct is later used for prediction of previously unseen cases, and the criterion based on the discernibility matrix tends to work better for learning functional relationships where generalization is not an issue.
KeywordsShannon Entropy Gini Index Independent Attribute Validation Error Discernibility Matrix
Unable to display preview. Download preview PDF.
- 1.Bazan, J.: Methods of approximating infrence for synthesis of decision algorithms. PhD thesis, Warsaw University (in Polish) (1999)Google Scholar
- 2.Bazan, J., Nguyen, H.S., Nguyen, S.H., Synak, P., Wróblewski, J.: Rough set algorithms in classification problems. In: Rough Set Methods and Applications: New Developments in Knowledge Discovery in Information Systems. Studies in Fuzziness and Soft Computing, vol. 56, pp. 49–88. Physica-Verlag (2000)Google Scholar
- 4.Grzymala-Busse, J.: LERS—a system for learning from examples based on rough sets. In: Intelligent Decision Support. Handbook of Applications and Advances of the Rough Sets Theory, pp. 3–18. Kluwer Academic Publishers, Dordrecht (1992)Google Scholar
- 6.Korzeń, M., Jaroszewicz, S.: Finding reducts without building the discernibility matrix. In: Proc. of the 5th Int. Conf. on Intelligent Systems Design and Applications (ISDA 2005), pp. 450–455 (2005)Google Scholar
- 7.Nguyen, S.H., Nguyen, H.S.: Some efficient algorithms for rough set methods. In: Proceedings of the Conference of Information Processing and Management of Uncertainty in Knowledge-Based Systems IPMU 1996, Granada, Spain, July 1996, pp. 1451–1456 (1996)Google Scholar
- 9.Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
- 10.Simovici, D., Jaroszewicz, S.: A new metric splitting criterion for decision trees. Journal of Parallel, Emerging and Distributed Computing (to appear)Google Scholar
- 11.Stefanowski, J.: On rough set based approaches to induction of decision rules. In: Skowron, A., Polkowski, L. (eds.) Rough Sets in Knowledge Discovery, vol. 1, pp. 500–529. Physica Verlag, Heidelberg (1998)Google Scholar