Abstract
The extreme learning machines (ELMs) have been proposed for generalized single-hidden-layer feedforward networks (SLFNs) which need not be neuron alike and perform well in both regression and classification applications. An active topic in ELMs is how to automatically determine network architectures for given applications. In this paper, we propose an extreme learning machine with adaptive growth of hidden nodes and incremental updating of output weights by an error-minimization-based method (AIE-ELM). AIE-ELM grows the randomly generated hidden nodes in an adaptive way in the sense that the existing hidden nodes may be replaced by some newly generated hidden nodes with better performance rather than always keeping those existing ones in other incremental ELMs. The output weights are updated incrementally in the same way of error minimized ELM (EM-ELM). Simulation results demonstrate and verify that our new approach can achieve a more compact network architecture than EM-ELM with better generalization performance.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal functions. IEEE Transactions on Information Theory 39, 930–945 (1993)
Bishop, C.M.: Neural networks for pattern recognition. Oxford University Press, New York (1995)
Black, C.L., Merz, C.J.: UCI repository of machine learning databases, department of information and computer sciences, University of California, Irvine, USA (1998), http://www.ics.uci.edu/~mlearn/mlrepository.html
Feng, G., Huang, G.-B., Lin, Q., Gay, R.: Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Transactions on Neural Networks 20(8), 1352–1357 (2009)
Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural network design. PWS Publshing Company, Boston (1996)
Haykin, S.: Neural networks: a comprehensive foundation. Macmillan College Publishing Company, New York (1994)
Huang, G.-B., Chen, L.: Convex incremental extreme learning machine. Neural Computing 70, 3056–3062 (2007)
Huang, G.-B., Chen, L., Siew, C.-K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks 17(4) (2006)
Huang, G.-B., Chen, L.: Enhanced random search based incremental extreme learning machine. Neural Computing 71, 3460–3468 (2008)
Huang, G.-B., Liang, N.-Y., Rong, H.-J., Saratchandran, P., Sundararajan, N.: On-line sequential extreme learning machine. In: The IASTED International Conference on Computational Intelligence (2005)
Huang, G.-B., Siew, C.-K.: Extreme learning machine: RBF network case. In: Proceedings of 8th International Conference Control, Automation, Robotics Vision, vol. 2, pp. 1029–1036 (2004)
Huang, G.-B., Siew, C.-K.: Extreme learning machine with randomly assigned RBF kernels. International Journal of Technology 11(1) (2005)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: A new learning scheme of feedforward neural networks. In: Proceedings of International Joint Conference Neural Networks, vol. 2, pp. 985–990 (2004)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neural Computing 70, 489–501 (2006)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Real-time learning capability of neural networks. IEEE Transactions on Neural Networks 17(4), 863–878 (2006)
Huang, G.-B., Zhu, Q.-Y., Zhi, M.-K., Siew, C.-K., Saratchandran, P., Sundararajan, N.: Can threshold networks be trained directly? IEEE Transactions on Circuits and Systems II 53(3), 187–191 (2006)
Lan, Y., Soh, Y.C., Huang, G.-B.: Ensemble of online sequential extreme learning machine. Neural Computing 72, 3391–3395 (2009)
Li, M.-B., Huang, G.-B., Saratchandran, P., Sundararajan, N.: Fully complex extreme learning machine. Neural Computing 68, 306–314 (2005)
Liang, N.-Y., Huang, G.-B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Transactions on Neural Networks 17(6), 1411–1423 (2006)
Looney, C.G.: Pattern recognition using neural networks: theory and algorithms for engineers and scientists. Oxford University Press, New York (1997)
Rong, H.-J., Huang, G.-B., Sundararajan, N., Saratchandran, P.: Online sequential fuzzy learning machine for function approximation and classfication problem. IEEE Transactions on System, Man and Cybernetics B 39(4), 1067–1072 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, R., Lan, Y., Huang, GB., Soh, Y.C. (2011). Extreme Learning Machine with Adaptive Growth of Hidden Nodes and Incremental Updating of Output Weights. In: Kamel, M., Karray, F., Gueaieb, W., Khamis, A. (eds) Autonomous and Intelligent Systems. AIS 2011. Lecture Notes in Computer Science(), vol 6752. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21538-4_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-21538-4_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21537-7
Online ISBN: 978-3-642-21538-4
eBook Packages: Computer ScienceComputer Science (R0)