Abstract
A novel architecture for a hierarchical classifier (HC) is defined. The objective is to combine several weak classifiers to form a strong one, but a different approach from those known, e.g. AdaBoost, is taken: the training set is split on the basis of previous classifier misclassification between output classes. The problem is split into overlapping subproblems, each classifying into a different set of output classes. This allows for a task size reduction as each sub-problem is smaller in the sense of lower number of output classes, and for higher accuracy. The HC proposes a different approach to the boosting approach.
The groups of output classes overlap, thus examples from a single class may end up in several subproblems. It is shown, that this approach ensures that such hierarchical classifier achieves better accuracy. A notion of generalized accuracy is introduced.
The sub-problems generation is simple as it is performed with a clustering algorithm operating on classifier outputs. We propose to use the Growing Neural Gas [1] algorithm, because of its good adaptiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fritzke, B.: A growing neural gas network leatns topologies. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems, pp. 625–632 (1995)
Hand, D., Mannila, H., Smyth, P.: Principles of data mining. MIT Press, Cambridge (2001)
Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning. Springer, Heidelberg (2001)
Schapire, R.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)
Eibl G Pfeiffer, K.P.: Multiclass boosting for weak classifiers. Machine Learning 5, 189–210 (2005)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)
Podolak, I.T.: Hierarchical classifier with overlapping class groups. Expert Systems with Applications 34(1), 673–682 (2008)
Podolak, I.T., Biel, S.: Hierarchical classifier. In: Wyrzykowski, R. (ed.) PPAM 2005. LNCS, vol. 3911, pp. 591–598. Springer, Heidelberg (2006)
Podolak, I.T.: Hierarchical rules for a hierarchical classifier. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds.) ICANNGA 2007. LNCS, vol. 4431, pp. 749–757. Springer, Heidelberg (2007)
Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of extracting rules from trained artificial neural networks. Knowledge–Based Systems 8, 373–389 (1995)
Duch, W., Setiono, R., Żurada, J.M.: Computational Intelligence Methods for Rule-Based Data Understanding. Proceedings of the IEEE 92(5), 771–805 (2004)
Setiono, R., Leow, W.K.: FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks. Applied Intelligence 12, 15–25 (2000)
Newman, D.J., Hettich, S., Blake, C.L.: UCI repository of machine learning databases (1998)
Zwitter, M., Sokolic, M.: Primary tumor data set (1998)
Setiono, R.: Feedforward neural network construction using cross validation. NeuralComputation 13, 2865–2877 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Podolak, I.T., Bartocha, K. (2009). A Hierarchical Classifier with Growing Neural Gas Clustering. In: Kolehmainen, M., Toivanen, P., Beliczynski, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2009. Lecture Notes in Computer Science, vol 5495. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04921-7_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-04921-7_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04920-0
Online ISBN: 978-3-642-04921-7
eBook Packages: Computer ScienceComputer Science (R0)