Abstract
Functional data learning is an extension of traditional data learning, that is, learning the data chosen from the Euclidean space \({\mathbb{R}^{n}}\) to a metric space. This paper focuses on the functional data learning with generalized single-hidden layer feedforward neural networks (GSLFNs) acting on some metric spaces. In addition, three learning algorithms, named Hilbert parallel overrelaxation backpropagation (H-PORBP) algorithm, ν-generalized support vector regression (ν-GSVR) and generalized extreme learning machine (G-ELM) are proposed to train the GSLFNs acting on some metric spaces. The experimental results on some metric spaces indicate that GELM with additive/RBF hidden-nodes has a faster learning speed, a better accuracy, and a better stability than HPORBP algorithm and ν-GSVR for training the functional data. The idea of GELM can be used to extend those improved extreme learning machines (ELMs) that act on the Euclidean space \({\mathbb{R}^{n}, }\) such as online sequential ELM, incremental ELM, pruning ELM and so on, to some metric spaces.
Similar content being viewed by others
References
Cao FL, Zhang R (2009) The errors of approximation for feedforward neural networks in the L p metric.. Math Comput Model 49(7–8):1563–1572
Cao FL, Zhang YQ, He ZR (2009) Interpolation and rate of convergence for a class of neural networks.. Appl Math Model 33(3):1441–1456
Cao FL, Lin SB, Xu ZB (2010) Constructive approximate interpolation by neural networks in the metric space. Math Comput Model 52:1674–1681
Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2011) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0049-5
Chen TP, Chen H, Liu RW (1995) Approximation capability in by multiplayer feedforward networks and related problems. IEEE Trans Neural Netw 6:25–30
Chen TP, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw 6:911–917
Conway JB (1985) A course in functional analysis. GTM96, Springer-Verlag, New York, USA
Corrieu P (2005) Function approximation on non-Euclidean spaces. Neural Netw 18:91–102
Cortes C, Vapnik V (1995) Support vector networks. Mach Learn 20(3):273–297
Cybenko G (1989) Approximation by superposition of sigmoidal functions. Math Control Signal Syst 2(4):303–314
Deng WY, Zheng QH, Lian SG, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74:447–456
Feng GR, Huang GB, Lin QP, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357
Fukuoka Y, Matsuki H, Minamitani H (1998) A modified backpropagation method to avoid false local minima. Neural Netw 11:1059–1072
Funahashi KI (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183–192
Guo GD, Chen S, Chen LF (2011) Soft subspace clustering with an improved feature weight self-adjustment mechanism. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0039-7
Heeswijk M, Miche Y, Oja E, Lendasse A (2011) GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16):2430–2437
Hornik K (1993) Some new results on neural network approximation. Neural Netw 6:1069–1072
Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks (IJCNN), vol 2, Budapest, Hungary, pp 985–990
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501
Huang GB, Zhu QY, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892
Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062
Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468
Huang GB, Ding X, Zhou X (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163
Huang GB, Wang DH, Lan Y (2011) Extreme learning machine: a survey. Int J Mach Learn Cybern 2(2):107–122
Kadirkamanathan V, Niranjan M (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5:954–975
Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395
Lan Y, Soh YC, Huang GB (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73:3193–3199
Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73:3028–3038
Li J, Han G, Wen J, Gao XB (2011) Robust tensor subspace learning for anomaly detection. Int J Mach Learn Cybern 2(2):89–98
Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Mangasarian OL, Solodov MV (1994) Serial and parallel backpropagation convergence via nonmonotone perturbed minimization. Optim method Softw 4(2):103–116
Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):1411–1423
Miche Y, Heeswijk M, Bas P, Simula O, Lendasse A (2011) TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 74(16):2413–2421
Muńoz A, Gonźlez J (2010) Representing functional data using support vector machines. Pattern Recog Lett 31:511–516
Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246–257
Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New York
Rossi F, Delannay N, Guez BC, Verleysen M (2005) Representation of functional data in neural networks. Neurocomputing 64:83–210
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by backpropagation errors. Nature 323:533–536
Schölkopf B, Mika S, Burges CJC, Knirsch P, Müller KR, Rätsch G (1999) Input space vs. feature space in kernel-basedmethods. IEEE Trans Neural Netw 10(5):1000–1017
Schölkopf B, Smola AJ, Williamson R, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245
Sharma A, Imoto S, Miyano S, Sharma V (2011) Null space based feature selection method for gene expression data. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0061-9
Vapnik V (1998) Statistical learning theory. Wiley, New York
Wang XZ, Dong CR (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567
Wang XZ, Chen AX, Feng HM (2011a) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525
Wang YG, Cao FL, Yuan YB (2011b) A study on effectiveness of extreme learning machine. Neurocomputing 74(16):2483–2490
Wu J, Wang ST, Chung FL (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271
Wu W, Wang J, Cheng MS, Li ZX (2011) Convergence analysis of online gradient method for BP neural networks. Neural Netw 24:91–98
Yamamuro S (1970) Differential calculus in topological linear spaces. Lect Notes Math, vol 374, Springer, Berlin
Acknowledgments
This research was financially supported by the Grant of the Second Stage of Brain Korea 21, the Ministry of Education, Science Technology (MEST) and Korea Industrial Technology Foundation (KOTEF) through the Human Resource Training Project for Regional Innovation. This work was supported by National Research Foundation of Korean Grant funded by the Korean Government (2009-0077772). In addition, the research was supported by the National Natural Science Foundation of China (Nos. 61101240, 90818020) and the Zhejiang Provincial Natural Science Foundation of China (No. Y6110117).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhao, J., Park, D.S., Lee, J. et al. Generalized extreme learning machine acting on a metric space. Soft Comput 16, 1503–1514 (2012). https://doi.org/10.1007/s00500-012-0825-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-012-0825-5