Abstract
Extreme learning machine (ELM) is a fast learning algorithm for single hidden layer feed-forward neural network (SLFN) based on random input weights which usually requires large number of hidden nodes. Recently, novel constructive and destructive parsimonious (CP and DP)-ELM which provide the effectiveness generalization and compact hidden nodes have been proposed. However, the performance might be unstable due to the randomization either in ordinary ELM or CP and DP-ELM. In this study, analytical incremental learning (AIL) algorithm is proposed in which all weights of neural network are calculated analytically without any randomization. The hidden nodes of AIL are incrementally generated based on residual error using least square (LS) method. The results show the effectiveness of AIL which has not only smallest number of hidden nodes and more stable but also good generalization than those of ELM, CP and DP-ELM based on seven benchmark data sets evaluation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available in http://www.dcc.fc.up.pt/~ltorgo/Regression/DataSets.html.
References
Bobrow, J.E., Murray, W.: An algorithm for rls identification parameters that vary quickly with time. IEEE Trans. Autom. Control 38(2), 351–354 (1993)
Castaño, A., Fernández-Navarro, F., Hervás-Martínez, C.: PCA-ELM: a robust and pruned extreme learning machine approach based on principal component analysis. Neural Process. Lett. 37, 1–16 (2013)
Huang, G.B., Chen, L.: Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16), 3460–3468 (2008)
Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)
Petković, M.D., Stanimirović, P.S.: Generalized matrix inversion is not harder than matrix multiplication. J. Comput. Appl. Math. 230(1), 270–282 (2009)
Rong, H.J., Ong, Y.S., Tan, A.H., Zhu, Z.: A fast pruned-extreme learning machine for classification problem. Neurocomputing 72(1), 359–366 (2008)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Readings in Cognitive Science: A Perspective from Psychology and Artificial Intelligence, pp. 399–421. Kaufmann, San Mateo (1988)
Strassen, V.: Gaussian elimination is not optimal. Numer. Math. 13(4), 354–356 (1969)
Wang, N., Er, M.J., Han, M.: Parsimonious extreme learning machine using recursive orthogonal least squares. IEEE Trans. Neural Netw. Learn. Syst. 25(10), 1828–1841 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Alfarozi, S.A.I., Setiawan, N.A., Adji, T.B., Woraratpanya, K., Pasupa, K., Sugimoto, M. (2016). Analytical Incremental Learning: Fast Constructive Learning Method for Neural Network. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9948. Springer, Cham. https://doi.org/10.1007/978-3-319-46672-9_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-46672-9_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46671-2
Online ISBN: 978-3-319-46672-9
eBook Packages: Computer ScienceComputer Science (R0)