Skip to main content

Advertisement

SpringerLink
Log in

Extreme learning machine and its applications

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Recently, a novel learning algorithm for single-hidden-layer feedforward neural networks (SLFNs) named extreme learning machine (ELM) was proposed by Huang et al. The essence of ELM is that the learning parameters of hidden nodes, including input weights and biases, are randomly assigned and need not be tuned while the output weights can be analytically determined by the simple generalized inverse operation. The only parameter needed to be defined is the number of hidden nodes. Compared with other traditional learning algorithms for SLFNs, ELM provides extremely faster learning speed, better generalization performance and with least human intervention. This paper firstly introduces a brief review of ELM, describing the principle and algorithm of ELM. Then, we put emphasis on the improved methods or the typical variants of ELM, especially on incremental ELM, pruning ELM, error-minimized ELM, two-stage ELM, online sequential ELM, evolutionary ELM, voting-based ELM, ordinal ELM, fully complex ELM, and symmetric ELM. Next, the paper summarized the applications of ELM on classification, regression, function approximation, pattern recognition, forecasting and diagnosis, and so on. In the last, the paper discussed several open issues of ELM, which may be worthy of exploring in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Xu XZ, Ding SF, Shi ZZ, Zhu H (2012) Optimizing radial basis function neural network based on rough set and AP clustering algorithm. J Zhejiang Univ Sci A 13(2):131–138

    Google Scholar 

  2. Chen Y, Zheng WX (2012) Stochastic state estimation for neural networks with distributed delays and Markovian jump. Neural Netw 25:14–20

    Article  MATH  MathSciNet  Google Scholar 

  3. Ding SF, Su CY, Yu JZ (2011) An optimizing BP neural network algorithm based on genetic algorithm. Artif Intell Rev 36(2):153–162

    Article  Google Scholar 

  4. Francisco FN, César HM, Gutiérrez PA, Carbonero-Ruz M (2011) Evolutionary q-Gaussian radial basis function neural networks for multiclassification. Neural Netw 24(7):779–784

    Article  MATH  Google Scholar 

  5. Ding SF, Jia WK, Su CY, Zhang LW (2011) Research of neural network algorithm based on factor analysis and cluster analysis. Neural Comput Appl 20(2):297–302

    Article  Google Scholar 

  6. Razavi S, Tolson BA (2011) A new formulation for feedforward neural networks. IEEE Trans Neural Netw 22(10):1588–1598

    Google Scholar 

  7. Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks (IJCNN2004), vol 2, no 25–29, pp 985–990

  8. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  9. Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Google Scholar 

  10. Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3060–3068

    Google Scholar 

  11. Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062

    Article  Google Scholar 

  12. Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72:359–366

    Article  Google Scholar 

  13. Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163

    Article  Google Scholar 

  14. Lim JS, Lee S, Pang HS (2013) Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimations. Neural Comput Appl 22(3–4):569–576

    Article  Google Scholar 

  15. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529

    Article  Google Scholar 

  16. Wang L, Huang YP, Luo XY, Wang Z, Luo SW (2011) Image deblurring with filters learned by extreme learning machine. Neurocomputing 74:2464–2474

    Article  Google Scholar 

  17. Cao JW, Lin ZP, Huang GB, Liu N (2012) Voting based extreme learning machine. Inf Sci 185(1, 15):66–77

    Article  MathSciNet  Google Scholar 

  18. Zhu QY, Qin AK, Suganthan PN, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38:1759–1763

    Article  MATH  Google Scholar 

  19. Feng GR, Huang GB, Lin QP, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357

    Google Scholar 

  20. Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73(16–18):3028–3038

    Google Scholar 

  21. Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate on-line sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Google Scholar 

  22. Deng WY, Zheng QH, Lian SG, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74(1–3):447–456

    Google Scholar 

  23. Li MB, Huang GB, Saratchandran P, Sundararajan N (2005) Fully complex extreme learning machine. Neurocomputing 68:306–314

    Google Scholar 

  24. Liu XY, Li P, Gao CH (2013) Symmetric extreme learning machine. Neural Comput Appl 22(3–4):551–558

    Google Scholar 

  25. Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468

    Google Scholar 

  26. Huang GB, Li MB, Chen L, Siew CK (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71:576–583

    Google Scholar 

  27. Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395

    Google Scholar 

  28. Zhao JW, Wang ZH, Park DS (2012) Online sequential extreme learning machine with forgetting mechanism. Neurocomputing 87(15):79–89

    Google Scholar 

  29. Castano A, Fernandez-Navarro F, Hervas-Martinez C (2013) PCA-ELM: a robust and pruned extreme learning machine approach based on principal component analysis. Neural Process Lett 37(3):377–392

    Google Scholar 

  30. Zhang WB, Ji HB (2013) Fuzzy extreme learning machine for classification. Electron Lett 49(7):448–449

    MathSciNet  Google Scholar 

  31. Horata P, Chiewchanwattana S, Sunat K (2013) Robust extreme learning machine. Neurocomputing 102(SI):31–44

    Google Scholar 

  32. He Q, Shang TF, Zhuang FZ (2013) Parallel extreme learning machine for regression based on MapReduce. Neurocomputing 102(SI):52–58

    Google Scholar 

  33. Yu Qi, Miche Yoan, Eirola Emil (2013) Regularized extreme learning machine for regression with missing data. Neurocomputing 102(SI):45–51

    Google Scholar 

  34. Zong WW, Huang GB, Chen YQ (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242

    Google Scholar 

  35. Wang BT, Wang GR, Li JJ, Wang B (2012) Update strategy based on region classification using ELM for mobile object index. Soft Comput 16(9):1607–1615

    Google Scholar 

  36. Zheng WB, Qian YT, Lu HJ (2013) Text categorization based on regularization extreme learning machine. Neural Comput Appl 22(3–4):447–456

    Google Scholar 

  37. Karpagachelvi S, Arthanari M, Sivakumar M (2012) Classification of electrocardiogram signals with support vector machines and extreme learning machine. Neural Comput Appl 21(6):1331–1339

    Google Scholar 

  38. Kim J, Shin HS, Shin K, Lee M (2009) Robust algorithm for arrhythmia classification in ECG using extreme learning machine. Biomed Eng. doi:10.1186/1475-925X-8-31

  39. Lee Y, Lee H, Kim J, Shin HC, Lee M (2009) Classification of BMI control commands from rat’s neural signals using extreme learning machine. Biomed Eng. doi:10.1186/1475-925X-8-29

  40. Li GQ, Niu PF (2013) An enhanced extreme learning machine based on ridge regression for regression. Neural Comput Appl 22(3–4):803–810

    Google Scholar 

  41. Balasundaram S (2013) On extreme learning machine for e-insensitive regression in the primal by Newton method. Neural Comput Appl. doi:10.1007/s00521-011-0798-9

  42. Feng GR, Qian ZX, Zhang XP (2012) Evolutionary selection extreme learning machine optimization for regression. Soft Comput 16(9):1485–1491

    Google Scholar 

  43. Zong WW, Huang GB (2011) Face recognition based on extreme learning machine. W. Zong, G.-B. Huang Neurocomput 74:2541–2551

    Google Scholar 

  44. Mohammed AA, Minhas R, Jonathan WuQM, Sid-Ahmed MA (2011) Human face recognition based on multidimensional PCA and extreme learning machine. Pattern Recognit 44:2588–2597

    MATH  Google Scholar 

  45. Minhas R, Baradarani A, Seifzadeh S, Jonathan WuQM (2010) Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73:1906–1917

    Google Scholar 

  46. Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2012) Handwritten character recognition using wavelet energy and extreme learning machine. J Mach Learn Cyber 3:149–161

    Google Scholar 

  47. Lan Y, Hu ZJ, Soh YC, Huang GB (2013) An extreme learning machine approach for speaker recognition. Neural Comput Appl 22(3–4):417–425

    Google Scholar 

  48. Nian R, He B, Lendasse A (2013) 3D object recognition based on a geometrical topology model and extreme learning machine. Neural Comput Appl 22(3–4):427–433

    Google Scholar 

  49. Zhou ZH, Zhao JW, Cao FL (2013) Surface reconstruction based on extreme learning machine. Neural Comput Appl 23(2):283–292

    Google Scholar 

  50. Yang JC, Jiao YB, Xiong NX (2013) Fast face gender recognition by using local ternary pattern and extreme learning machine. KSII Trans Intern Inf Syst 7(7):1705–1720

    Google Scholar 

  51. Yang JC, Xie SJ, Yoon S (2013) Fingerprint matching based on extreme learning machine. Neural Comput Appl 22(3–4):435–445

    Google Scholar 

  52. Chen FL, Ou TY (2011) Sales forecasting system based on Gray extreme learning machine with Taguchi method in retail industry. Expert Syst Appl 38:1336–1345

    Google Scholar 

  53. Sun ZL et al (2008) Sales forecasting using extreme learning machine with applications in fashion retailing. Decis Support Syst 46:411–419

    Google Scholar 

  54. Hu XF, Zhao Z, Wang S, Wang FL, He DK, Wu SK (2008) Multi-stage extreme learning machine for fault diagnosis on hydraulic tube tester. Neural Comput Appl 17:399–403

    Google Scholar 

  55. Daliri MR (2012) A hybrid automatic system for the diagnosis of lung cancer based on genetic algorithm and fuzzy extreme learning machines. J Med Syst 36:1001–1005

    Google Scholar 

  56. Xu Y, Dai YY, Dong ZY, Zhang R, Meng K (2013) Extreme learning machine-based predictor for real-time frequency stability assessment of electric power systems. IET Gener Transm Distrib 7(4):391–397

    Google Scholar 

  57. Pan C, Park DS, Yang Y, Yoo HM (2012) Leukocyte image segmentation by visual attention and extreme learning machine. Neural Comput Appl 21(6):1217–1227

    Google Scholar 

  58. Pan C, Park DS, Lu HJ, Wu XP (2012) Color image segmentation by fixation-based active learning with ELM. Soft Comput 16(9):1569–1584

    Google Scholar 

  59. Malathi V, Marimuthu NS, Baskar S, Ramar K (2011) Application of extreme learning machine for series compensated transmission line protection. Eng Appl Artif Intell 24:880–887

    Google Scholar 

  60. Zhao LJ, Wang DH, Chai TY (2013) Estimation of effluent quality using PLS-based extreme learning machines. Neural Comput Appl 22(3–4):509–519

    Google Scholar 

  61. Li YJ, Li Y, Zhai JH, Shiu S (2012) RTS game strategy evaluation using extreme learning machine. Soft Comput 16(9):1627–1637

    Google Scholar 

  62. Li LN, Ouyang JH, Chen HL, Liu DY (2012) A computer aided diagnosis system for thyroid disease using extreme learning machine. J Med Syst 36(5):3327–3337

    Google Scholar 

  63. Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cyber 2:107–122

    Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation (No. 61379101), the 973 Program (No. 2013CB329502), the Basic Research Program (Natural Science Foundation) of Jiangsu Province of China (No. BK20130209), the Opening Foundation of the Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (No. IIP2010-1), and the Opening Foundation of Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, Beijing University of Posts and Telecommunications.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Shifei Ding, Xinzheng Xu & Ru Nie

  2. Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Science, Beijing, 100190, China

    Shifei Ding

Authors
  1. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinzheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ru Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shifei Ding.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ding, S., Xu, X. & Nie, R. Extreme learning machine and its applications. Neural Comput & Applic 25, 549–556 (2014). https://doi.org/10.1007/s00521-013-1522-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-013-1522-8

Keywords

Search

Navigation