Skip to main content
Log in

Generalized extreme learning machine acting on a metric space

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Functional data learning is an extension of traditional data learning, that is, learning the data chosen from the Euclidean space \({\mathbb{R}^{n}}\) to a metric space. This paper focuses on the functional data learning with generalized single-hidden layer feedforward neural networks (GSLFNs) acting on some metric spaces. In addition, three learning algorithms, named Hilbert parallel overrelaxation backpropagation (H-PORBP) algorithm, ν-generalized support vector regression (ν-GSVR) and generalized extreme learning machine (G-ELM) are proposed to train the GSLFNs acting on some metric spaces. The experimental results on some metric spaces indicate that GELM with additive/RBF hidden-nodes has a faster learning speed, a better accuracy, and a better stability than HPORBP algorithm and ν-GSVR for training the functional data. The idea of GELM can be used to extend those improved extreme learning machines (ELMs) that act on the Euclidean space \({\mathbb{R}^{n}, }\) such as online sequential ELM, incremental ELM, pruning ELM and so on, to some metric spaces.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Cao FL, Zhang R (2009) The errors of approximation for feedforward neural networks in the L p metric.. Math Comput Model 49(7–8):1563–1572

    Article  MathSciNet  MATH  Google Scholar 

  • Cao FL, Zhang YQ, He ZR (2009) Interpolation and rate of convergence for a class of neural networks.. Appl Math Model 33(3):1441–1456

    Article  MathSciNet  MATH  Google Scholar 

  • Cao FL, Lin SB, Xu ZB (2010) Constructive approximate interpolation by neural networks in the metric space. Math Comput Model 52:1674–1681

    Article  MathSciNet  MATH  Google Scholar 

  • Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2011) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0049-5

  • Chen TP, Chen H, Liu RW (1995) Approximation capability in by multiplayer feedforward networks and related problems. IEEE Trans Neural Netw 6:25–30

    Article  MATH  Google Scholar 

  • Chen TP, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw 6:911–917

    Article  Google Scholar 

  • Conway JB (1985) A course in functional analysis. GTM96, Springer-Verlag, New York, USA

  • Corrieu P (2005) Function approximation on non-Euclidean spaces. Neural Netw 18:91–102

    Article  Google Scholar 

  • Cortes C, Vapnik V (1995) Support vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  • Cybenko G (1989) Approximation by superposition of sigmoidal functions. Math Control Signal Syst 2(4):303–314

    Article  MathSciNet  MATH  Google Scholar 

  • Deng WY, Zheng QH, Lian SG, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74:447–456

    Article  Google Scholar 

  • Feng GR, Huang GB, Lin QP, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357

    Article  Google Scholar 

  • Fukuoka Y, Matsuki H, Minamitani H (1998) A modified backpropagation method to avoid false local minima. Neural Netw 11:1059–1072

    Article  Google Scholar 

  • Funahashi KI (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183–192

    Article  Google Scholar 

  • Guo GD, Chen S, Chen LF (2011) Soft subspace clustering with an improved feature weight self-adjustment mechanism. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0039-7

  • Heeswijk M, Miche Y, Oja E, Lendasse A (2011) GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16):2430–2437

    Article  Google Scholar 

  • Hornik K (1993) Some new results on neural network approximation. Neural Netw 6:1069–1072

    Article  Google Scholar 

  • Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks (IJCNN), vol 2, Budapest, Hungary, pp 985–990

  • Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501

    Article  Google Scholar 

  • Huang GB, Zhu QY, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  • Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062

    Article  Google Scholar 

  • Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468

    Article  Google Scholar 

  • Huang GB, Ding X, Zhou X (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163

    Article  Google Scholar 

  • Huang GB, Wang DH, Lan Y (2011) Extreme learning machine: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  • Kadirkamanathan V, Niranjan M (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5:954–975

    Article  Google Scholar 

  • Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395

    Article  Google Scholar 

  • Lan Y, Soh YC, Huang GB (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73:3193–3199

    Google Scholar 

  • Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73:3028–3038

    Article  Google Scholar 

  • Li J, Han G, Wen J, Gao XB (2011) Robust tensor subspace learning for anomaly detection. Int J Mach Learn Cybern 2(2):89–98

    Article  Google Scholar 

  • Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  • Mangasarian OL, Solodov MV (1994) Serial and parallel backpropagation convergence via nonmonotone perturbed minimization. Optim method Softw 4(2):103–116

    Article  Google Scholar 

  • Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):1411–1423

    Google Scholar 

  • Miche Y, Heeswijk M, Bas P, Simula O, Lendasse A (2011) TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 74(16):2413–2421

    Article  Google Scholar 

  • Muńoz A, Gonźlez J (2010) Representing functional data using support vector machines. Pattern Recog Lett 31:511–516

    Article  Google Scholar 

  • Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246–257

    Article  Google Scholar 

  • Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New York

    MATH  Google Scholar 

  • Rossi F, Delannay N, Guez BC, Verleysen M (2005) Representation of functional data in neural networks. Neurocomputing 64:83–210

    Article  Google Scholar 

  • Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by backpropagation errors. Nature 323:533–536

    Article  Google Scholar 

  • Schölkopf B, Mika S, Burges CJC, Knirsch P, Müller KR, Rätsch G (1999) Input space vs. feature space in kernel-basedmethods. IEEE Trans Neural Netw 10(5):1000–1017

    Google Scholar 

  • Schölkopf B, Smola AJ, Williamson R, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245

    Article  Google Scholar 

  • Sharma A, Imoto S, Miyano S, Sharma V (2011) Null space based feature selection method for gene expression data. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0061-9

  • Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  • Wang XZ, Dong CR (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567

    Article  Google Scholar 

  • Wang XZ, Chen AX, Feng HM (2011a) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525

    Article  Google Scholar 

  • Wang YG, Cao FL, Yuan YB (2011b) A study on effectiveness of extreme learning machine. Neurocomputing 74(16):2483–2490

    Article  Google Scholar 

  • Wu J, Wang ST, Chung FL (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271

    Article  Google Scholar 

  • Wu W, Wang J, Cheng MS, Li ZX (2011) Convergence analysis of online gradient method for BP neural networks. Neural Netw 24:91–98

    Article  MATH  Google Scholar 

  • Yamamuro S (1970) Differential calculus in topological linear spaces. Lect Notes Math, vol 374, Springer, Berlin

Download references

Acknowledgments

This research was financially supported by the Grant of the Second Stage of Brain Korea 21, the Ministry of Education, Science Technology (MEST) and Korea Industrial Technology Foundation (KOTEF) through the Human Resource Training Project for Regional Innovation. This work was supported by National Research Foundation of Korean Grant funded by the Korean Government (2009-0077772). In addition, the research was supported by the National Natural Science Foundation of China (Nos. 61101240, 90818020) and the Zhejiang Provincial Natural Science Foundation of China (No. Y6110117).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Feilong Cao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhao, J., Park, D.S., Lee, J. et al. Generalized extreme learning machine acting on a metric space. Soft Comput 16, 1503–1514 (2012). https://doi.org/10.1007/s00500-012-0825-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0825-5

Keywords

Navigation