Advertisement

Neural Computing and Applications

, Volume 25, Issue 3–4, pp 549–556 | Cite as

Extreme learning machine and its applications

  • Shifei Ding
  • Xinzheng Xu
  • Ru Nie
Review

Abstract

Recently, a novel learning algorithm for single-hidden-layer feedforward neural networks (SLFNs) named extreme learning machine (ELM) was proposed by Huang et al. The essence of ELM is that the learning parameters of hidden nodes, including input weights and biases, are randomly assigned and need not be tuned while the output weights can be analytically determined by the simple generalized inverse operation. The only parameter needed to be defined is the number of hidden nodes. Compared with other traditional learning algorithms for SLFNs, ELM provides extremely faster learning speed, better generalization performance and with least human intervention. This paper firstly introduces a brief review of ELM, describing the principle and algorithm of ELM. Then, we put emphasis on the improved methods or the typical variants of ELM, especially on incremental ELM, pruning ELM, error-minimized ELM, two-stage ELM, online sequential ELM, evolutionary ELM, voting-based ELM, ordinal ELM, fully complex ELM, and symmetric ELM. Next, the paper summarized the applications of ELM on classification, regression, function approximation, pattern recognition, forecasting and diagnosis, and so on. In the last, the paper discussed several open issues of ELM, which may be worthy of exploring in the future.

Keywords

Single-hidden-layer feedforward networks Neural Networks Extreme learning machine Classification Regression 

Notes

Acknowledgments

This work is supported by the National Natural Science Foundation (No. 61379101), the 973 Program (No. 2013CB329502), the Basic Research Program (Natural Science Foundation) of Jiangsu Province of China (No. BK20130209), the Opening Foundation of the Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (No. IIP2010-1), and the Opening Foundation of Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, Beijing University of Posts and Telecommunications.

References

  1. 1.
    Xu XZ, Ding SF, Shi ZZ, Zhu H (2012) Optimizing radial basis function neural network based on rough set and AP clustering algorithm. J Zhejiang Univ Sci A 13(2):131–138Google Scholar
  2. 2.
    Chen Y, Zheng WX (2012) Stochastic state estimation for neural networks with distributed delays and Markovian jump. Neural Netw 25:14–20CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Ding SF, Su CY, Yu JZ (2011) An optimizing BP neural network algorithm based on genetic algorithm. Artif Intell Rev 36(2):153–162CrossRefGoogle Scholar
  4. 4.
    Francisco FN, César HM, Gutiérrez PA, Carbonero-Ruz M (2011) Evolutionary q-Gaussian radial basis function neural networks for multiclassification. Neural Netw 24(7):779–784CrossRefzbMATHGoogle Scholar
  5. 5.
    Ding SF, Jia WK, Su CY, Zhang LW (2011) Research of neural network algorithm based on factor analysis and cluster analysis. Neural Comput Appl 20(2):297–302CrossRefGoogle Scholar
  6. 6.
    Razavi S, Tolson BA (2011) A new formulation for feedforward neural networks. IEEE Trans Neural Netw 22(10):1588–1598Google Scholar
  7. 7.
    Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks (IJCNN2004), vol 2, no 25–29, pp 985–990 Google Scholar
  8. 8.
    Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRefGoogle Scholar
  9. 9.
    Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892 Google Scholar
  10. 10.
    Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3060–3068Google Scholar
  11. 11.
    Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062CrossRefGoogle Scholar
  12. 12.
    Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72:359–366CrossRefGoogle Scholar
  13. 13.
    Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163CrossRefGoogle Scholar
  14. 14.
    Lim JS, Lee S, Pang HS (2013) Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimations. Neural Comput Appl 22(3–4):569–576CrossRefGoogle Scholar
  15. 15.
    Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529CrossRefGoogle Scholar
  16. 16.
    Wang L, Huang YP, Luo XY, Wang Z, Luo SW (2011) Image deblurring with filters learned by extreme learning machine. Neurocomputing 74:2464–2474CrossRefGoogle Scholar
  17. 17.
    Cao JW, Lin ZP, Huang GB, Liu N (2012) Voting based extreme learning machine. Inf Sci 185(1, 15):66–77CrossRefMathSciNetGoogle Scholar
  18. 18.
    Zhu QY, Qin AK, Suganthan PN, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38:1759–1763CrossRefzbMATHGoogle Scholar
  19. 19.
    Feng GR, Huang GB, Lin QP, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357Google Scholar
  20. 20.
    Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73(16–18):3028–3038Google Scholar
  21. 21.
    Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate on-line sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423Google Scholar
  22. 22.
    Deng WY, Zheng QH, Lian SG, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74(1–3):447–456Google Scholar
  23. 23.
    Li MB, Huang GB, Saratchandran P, Sundararajan N (2005) Fully complex extreme learning machine. Neurocomputing 68:306–314Google Scholar
  24. 24.
    Liu XY, Li P, Gao CH (2013) Symmetric extreme learning machine. Neural Comput Appl 22(3–4):551–558 Google Scholar
  25. 25.
    Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468Google Scholar
  26. 26.
    Huang GB, Li MB, Chen L, Siew CK (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71:576–583Google Scholar
  27. 27.
    Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395Google Scholar
  28. 28.
    Zhao JW, Wang ZH, Park DS (2012) Online sequential extreme learning machine with forgetting mechanism. Neurocomputing 87(15):79–89Google Scholar
  29. 29.
    Castano A, Fernandez-Navarro F, Hervas-Martinez C (2013) PCA-ELM: a robust and pruned extreme learning machine approach based on principal component analysis. Neural Process Lett 37(3):377–392Google Scholar
  30. 30.
    Zhang WB, Ji HB (2013) Fuzzy extreme learning machine for classification. Electron Lett 49(7):448–449MathSciNetGoogle Scholar
  31. 31.
    Horata P, Chiewchanwattana S, Sunat K (2013) Robust extreme learning machine. Neurocomputing 102(SI):31–44Google Scholar
  32. 32.
    He Q, Shang TF, Zhuang FZ (2013) Parallel extreme learning machine for regression based on MapReduce. Neurocomputing 102(SI):52–58Google Scholar
  33. 33.
    Yu Qi, Miche Yoan, Eirola Emil (2013) Regularized extreme learning machine for regression with missing data. Neurocomputing 102(SI):45–51Google Scholar
  34. 34.
    Zong WW, Huang GB, Chen YQ (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242Google Scholar
  35. 35.
    Wang BT, Wang GR, Li JJ, Wang B (2012) Update strategy based on region classification using ELM for mobile object index. Soft Comput 16(9):1607–1615Google Scholar
  36. 36.
    Zheng WB, Qian YT, Lu HJ (2013) Text categorization based on regularization extreme learning machine. Neural Comput Appl 22(3–4):447–456Google Scholar
  37. 37.
    Karpagachelvi S, Arthanari M, Sivakumar M (2012) Classification of electrocardiogram signals with support vector machines and extreme learning machine. Neural Comput Appl 21(6):1331–1339Google Scholar
  38. 38.
    Kim J, Shin HS, Shin K, Lee M (2009) Robust algorithm for arrhythmia classification in ECG using extreme learning machine. Biomed Eng. doi: 10.1186/1475-925X-8-31
  39. 39.
    Lee Y, Lee H, Kim J, Shin HC, Lee M (2009) Classification of BMI control commands from rat’s neural signals using extreme learning machine. Biomed Eng. doi: 10.1186/1475-925X-8-29
  40. 40.
    Li GQ, Niu PF (2013) An enhanced extreme learning machine based on ridge regression for regression. Neural Comput Appl 22(3–4):803–810Google Scholar
  41. 41.
    Balasundaram S (2013) On extreme learning machine for e-insensitive regression in the primal by Newton method. Neural Comput Appl. doi: 10.1007/s00521-011-0798-9
  42. 42.
    Feng GR, Qian ZX, Zhang XP (2012) Evolutionary selection extreme learning machine optimization for regression. Soft Comput 16(9):1485–1491Google Scholar
  43. 43.
    Zong WW, Huang GB (2011) Face recognition based on extreme learning machine. W. Zong, G.-B. Huang Neurocomput 74:2541–2551Google Scholar
  44. 44.
    Mohammed AA, Minhas R, Jonathan WuQM, Sid-Ahmed MA (2011) Human face recognition based on multidimensional PCA and extreme learning machine. Pattern Recognit 44:2588–2597zbMATHGoogle Scholar
  45. 45.
    Minhas R, Baradarani A, Seifzadeh S, Jonathan WuQM (2010) Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73:1906–1917Google Scholar
  46. 46.
    Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2012) Handwritten character recognition using wavelet energy and extreme learning machine. J Mach Learn Cyber 3:149–161Google Scholar
  47. 47.
    Lan Y, Hu ZJ, Soh YC, Huang GB (2013) An extreme learning machine approach for speaker recognition. Neural Comput Appl 22(3–4):417–425Google Scholar
  48. 48.
    Nian R, He B, Lendasse A (2013) 3D object recognition based on a geometrical topology model and extreme learning machine. Neural Comput Appl 22(3–4):427–433Google Scholar
  49. 49.
    Zhou ZH, Zhao JW, Cao FL (2013) Surface reconstruction based on extreme learning machine. Neural Comput Appl 23(2):283–292Google Scholar
  50. 50.
    Yang JC, Jiao YB, Xiong NX (2013) Fast face gender recognition by using local ternary pattern and extreme learning machine. KSII Trans Intern Inf Syst 7(7):1705–1720Google Scholar
  51. 51.
    Yang JC, Xie SJ, Yoon S (2013) Fingerprint matching based on extreme learning machine. Neural Comput Appl 22(3–4):435–445Google Scholar
  52. 52.
    Chen FL, Ou TY (2011) Sales forecasting system based on Gray extreme learning machine with Taguchi method in retail industry. Expert Syst Appl 38:1336–1345Google Scholar
  53. 53.
    Sun ZL et al (2008) Sales forecasting using extreme learning machine with applications in fashion retailing. Decis Support Syst 46:411–419Google Scholar
  54. 54.
    Hu XF, Zhao Z, Wang S, Wang FL, He DK, Wu SK (2008) Multi-stage extreme learning machine for fault diagnosis on hydraulic tube tester. Neural Comput Appl 17:399–403Google Scholar
  55. 55.
    Daliri MR (2012) A hybrid automatic system for the diagnosis of lung cancer based on genetic algorithm and fuzzy extreme learning machines. J Med Syst 36:1001–1005Google Scholar
  56. 56.
    Xu Y, Dai YY, Dong ZY, Zhang R, Meng K (2013) Extreme learning machine-based predictor for real-time frequency stability assessment of electric power systems. IET Gener Transm Distrib 7(4):391–397Google Scholar
  57. 57.
    Pan C, Park DS, Yang Y, Yoo HM (2012) Leukocyte image segmentation by visual attention and extreme learning machine. Neural Comput Appl 21(6):1217–1227Google Scholar
  58. 58.
    Pan C, Park DS, Lu HJ, Wu XP (2012) Color image segmentation by fixation-based active learning with ELM. Soft Comput 16(9):1569–1584Google Scholar
  59. 59.
    Malathi V, Marimuthu NS, Baskar S, Ramar K (2011) Application of extreme learning machine for series compensated transmission line protection. Eng Appl Artif Intell 24:880–887Google Scholar
  60. 60.
    Zhao LJ, Wang DH, Chai TY (2013) Estimation of effluent quality using PLS-based extreme learning machines. Neural Comput Appl 22(3–4):509–519Google Scholar
  61. 61.
    Li YJ, Li Y, Zhai JH, Shiu S (2012) RTS game strategy evaluation using extreme learning machine. Soft Comput 16(9):1627–1637Google Scholar
  62. 62.
    Li LN, Ouyang JH, Chen HL, Liu DY (2012) A computer aided diagnosis system for thyroid disease using extreme learning machine. J Med Syst 36(5):3327–3337Google Scholar
  63. 63.
    Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cyber 2:107–122Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyChina University of Mining and TechnologyXuzhouChina
  2. 2.Key Laboratory of Intelligent Information Processing, Institute of Computing TechnologyChinese Academy of ScienceBeijingChina

Personalised recommendations