Abstract
A system of rule extraction out of a complex hierarchical classifier is proposed in this paper. There are several methods for rule extraction out of trained artificial neural networks (ANN’s), but these methods do not scale well, i.e. results are satisfactory for small problems. For complicated problems hundreds of rules are produced, which are hard to govern.
In this paper a hierarchical classifier with a tree-like structure and simple ANN’s at nodes, is presented, which splits the original problem into several sub-problems that overlap. Node classifiers are all weak (i.e. with accuracy only better than random), and errors are corrected at lower levels. Single sub-problems constitute of examples that were hard to separate. Such architecture is able to classify better than single network models.
At the same time if–then rules are extracted, which only answer which sub-problem a given example belongs to. Such rules, by introducing hierarchy, are simpler and easier to modify by hand, giving also a better insight into the original classifier behaviour.
Research was funded by Jagiellonian University’s grant “Hierarchical classifiers”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hand, D., Mannila, H., Smyth, P.: Principles of Data Mining. MIT Press, Cambridge (2001)
Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning. Springer, Heidelberg (2001)
Setiono, R.: Feedforward neural network construction using cross validation. Neural Computation 13, 2865–2877 (2001)
Podolak, I.T., Biel, S., Bobrowski, M.: Hierarchical classifier. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Waśniewski, J. (eds.) PPAM 2005. LNCS, vol. 3911, pp. 591–598. Springer, Heidelberg (2006)
Schapire, R.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)
Haykin, S.: Neural networks, a comprehensive foundation. Prentice-Hall, Englewood Cliffs (1999)
Newman, D.J., Hettich, S., Blake, C.L.: Uci repository of machine learning databases (1998)
Kittler, J., Hojjatoleslami, A., Windeatt, T.: Strategies for combining classifiers emplying shared and distinct pattern representations. Pattern Recognition Letters 18, 1373–1377 (1997)
Bax, E.: Validation of voting committees. Neural Computation 10(4), 975–986 (1998)
Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of extracting rules from trained artificial neural networks. Knowledge–Based Systems 8(6), 373–389 (1995)
Duch, W., Setiono, R., Zurada, J.M.: Computational intelligence methods for rule-based data understanding. Proceedings of the IEEE 92(5), 771–805 (2004)
Setiono, R., Leow, W.K.: FERNN: An algorithm for fast extraction of rules from neural networks. Applied Intelligence 12(1-2), 15–25 (2000)
Zwitter, M., Soklic, M.: Primary tumor data set (1988)
Friedman-Hill, E.: Jess, the rule engine for java platform. Sandia National Laboratory (2001)
Podolak, I.T.: Hierarchical classifier with overlapping class groups. Expert Systems with Applications 34(3) (2007)
Witten, I.H., Frank, E.: Data mining: practical machine learning tools with Java implementations. Morgan Kaufmann, San Francisco (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Podolak, I.T. (2007). Hierarchical Rules for a Hierarchical Classifier. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2007. Lecture Notes in Computer Science, vol 4431. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71618-1_83
Download citation
DOI: https://doi.org/10.1007/978-3-540-71618-1_83
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71589-4
Online ISBN: 978-3-540-71618-1
eBook Packages: Computer ScienceComputer Science (R0)