Abstract
Prism has been developed as a modular classification rule generator following the separate and conquer approach since 1987 due to the replicated sub-tree problem occurring in Top-Down Induction of Decision Trees (TDIDT). A series of experiments have been done to compare the performance between Prism and TDIDT which proved that Prism may generally provide a similar level of accuracy as TDIDT but with fewer rules and fewer terms per rule. In addition, Prism is generally more tolerant to noise with consistently better accuracy than TDIDT. However, the authors have identified through some experiments that Prism may also give rule sets which tend to underfit training sets in some cases. This paper introduces a new modular classification rule generator, which follows the separate and conquer approach, in order to avoid the problems which arise with Prism. In this paper, the authors review the Prism method and its advantages compared with TDIDT as well as its disadvantages that are overcome by a new method using Information Entropy Based Rule Generation (IEBRG). The authors also set up an experimental study on the performance of the new method in classification accuracy and computational efficiency. The method is also evaluated comparatively with Prism.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufman (1993)
Hunt, E.B., Stone, P.J., Marin, J.: Experiments in Induction. Academic Press, New York (1966)
Michalski, R.S.: On the quasi-minimal solution of the general covering problem. In: Proceedings of the Fifth International Symposium on Information Processing, Bled, Yugoslavia, pp. 125–128 (1969)
Bramer, M.A.: Principles of Data Mining. Springer, London (2007)
Cendrowska, J.: PRISM: an algorithm for inducing modular rules. Int. J. Man Mach. Stud. 27, 349–370 (1987)
Shannon, C.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)
Stahl, F., Bramer, M.: Induction of modular classification rules: using Jmax-pruning. In: 30th SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge (2011)
Stahl, F., Bramer, M.: Jmax-pruning: a facility for the information theoretic pruning of modular classification rules. Knowl.-Based Syst. 29(2012), 12–19 (2012)
Deng, X.: A Covering-Based Algorithm for Classification: PRISM. CS831: Knowledge Discover in Databases (2012)
Kerber, R.: Chimerge: discretization of numeric attributes. In: AAAI’92 Proceedings of the tenth national conference on Artificial intelligence, pp. 123–128 (1992)
Bramer, M.A.: Inducer: a public domain workbench for data mining. Int. J. Syst. Sci. 36, 909–919 (2005)
Stahl, F., Bramer, M.: Computationally efficient induction of classification rules with the PMCRI and J-PMCRI frameworks. Knowl. Based Syst. 35(2012), 49–63 (2012)
Bramer, M.A.: Automatic induction of classification rules from examples using N-Prism. In: Research and Development in Intelligent Systems XVI, pp. 99–121. Springer, Berlin (2000)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases. Technical Report, University of California, Irvine, Department of Information and Computer Sciences (1998)
Bramer, M.A.: Using J-pruning to reduce overfitting of classification rules in noisy domains. In: Proceedings of 13th International Conference on Database and Expert Systems Applications— DEXA 2002, Aix-en-Provence, France, 2–6 Sept 2002
Bramer, M.A.: Using J-pruning to reduce overfitting in classification trees. In: Research and Development in Intelligent Systems XVIII, pp. 25–38. Springer, Berlin (2002)
Smyth, P., Goodman, R.M.: Rule induction using information theory. In: Piatetsky-Shapiro, G., Frawley, W.J. (eds.) Knowledge Discovery in Databases, pp. 159–176. AAAI Press (1991)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Liu, H., Gegov, A. (2016). Induction of Modular Classification Rules by Information Entropy Based Rule Generation. In: Sgurev, V., Yager, R., Kacprzyk, J., Jotsov, V. (eds) Innovative Issues in Intelligent Systems. Studies in Computational Intelligence, vol 623. Springer, Cham. https://doi.org/10.1007/978-3-319-27267-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-27267-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27266-5
Online ISBN: 978-3-319-27267-2
eBook Packages: EngineeringEngineering (R0)