Advertisement

Neural Computing and Applications

, Volume 27, Issue 2, pp 291–303 | Cite as

Self-adaptive extreme learning machine

  • Gai-Ge WangEmail author
  • Mei Lu
  • Yong-Quan Dong
  • Xiang-Jun Zhao
Extreme Learning Machine and Applications

Abstract

In order to overcome the disadvantage of the traditional algorithm for SLFN (single-hidden layer feedforward neural network), an improved algorithm for SLFN, called extreme learning machine (ELM), is proposed by Huang et al. However, ELM is sensitive to the neuron number in hidden layer and its selection is a difficult-to-solve problem. In this paper, a self-adaptive mechanism is introduced into the ELM. Herein, a new variant of ELM, called self-adaptive extreme learning machine (SaELM), is proposed. SaELM is a self-adaptive learning algorithm that can always select the best neuron number in hidden layer to form the neural networks. There is no need to adjust any parameters in the training process. In order to prove the performance of the SaELM, it is used to solve the Italian wine and iris classification problems. Through the comparisons between SaELM and the traditional back propagation, basic ELM and general regression neural network, the results have proven that SaELM has a faster learning speed and better generalization performance when solving the classification problem.

Keywords

Classification Self-adaptive Extreme learning machine Back propagation General regression neural network 

Notes

Acknowledgments

This work was supported by Research Fund for the Doctoral Program of Jiangsu Normal University (No. 13XLR041) and the National Natural Science Foundation of China (Nos. 61402207, 61100167, and 61272297), the Natural Science Foundation of Jiangsu Province, China, under Grant No. BK2011204, Qing Lan Project.

References

  1. 1.
    Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501. doi: 10.1016/j.neucom.2005.12.126 CrossRefGoogle Scholar
  2. 2.
    Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062. doi: 10.1016/j.neucom.2007.02.009 CrossRefGoogle Scholar
  3. 3.
    Huang G-B, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cyber 2(2):107–122. doi: 10.1007/s13042-011-0019-y CrossRefGoogle Scholar
  4. 4.
    Horata P, Chiewchanwattana S, Sunat K (2013) Robust extreme learning machine. Neurocomputing 102:31–44. doi: 10.1016/j.neucom.2011.12.045 CrossRefGoogle Scholar
  5. 5.
    Yang Y, Wang Y, Yuan X (2012) Bidirectional extreme learning machine for regression problem and its learning effectiveness. IEEE Trans Neural Netw Learn Syst 23(9):1498–1505. doi: 10.1109/TNNLS.2012.2202289 CrossRefGoogle Scholar
  6. 6.
    Jun W, Shitong W, Chung F (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cyber 2(4):261–271. doi: 10.1007/s13042-011-0024-1 CrossRefGoogle Scholar
  7. 7.
    Pouzols FM, Lendasse A (2010) Evolving fuzzy optimally pruned extreme learning machine for regression problems. Evol Syst 1(1):43–58. doi: 10.1007/s12530-010-9005-y CrossRefGoogle Scholar
  8. 8.
    Li G, Niu P (2011) An enhanced extreme learning machine based on ridge regression for regression. Neural Comput Appl 22(3–4):803–810. doi: 10.1007/s00521-011-0771-7 Google Scholar
  9. 9.
    Zhai J-H, Xu H-Y, Wang X-Z (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput 16(9):1493–1502. doi: 10.1007/s00500-012-0824-6 CrossRefGoogle Scholar
  10. 10.
    Zong W, Huang G-B, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242. doi: 10.1016/j.neucom.2012.08.010 CrossRefGoogle Scholar
  11. 11.
    He Q, Shang T, Zhuang F, Shi Z (2013) Parallel extreme learning machine for regression based on MapReduce. Neurocomputing 102:52–58. doi: 10.1016/j.neucom.2012.01.040 CrossRefGoogle Scholar
  12. 12.
    Wang D, Alhamdoosh M (2013) Evolutionary extreme learning machine ensembles with size control. Neurocomputing 102:98–110. doi: 10.1016/j.neucom.2011.12.046 CrossRefGoogle Scholar
  13. 13.
    Feng G, Huang G-B, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357. doi: 10.1109/TNN.2009.2024147 CrossRefGoogle Scholar
  14. 14.
    Wang L, Huang Y, Luo X, Wang Z, Luo S (2011) Image deblurring with filters learned by extreme learning machine. Neurocomputing 74(16):2464–2474. doi: 10.1016/j.neucom.2010.12.035 CrossRefGoogle Scholar
  15. 15.
    Iosifidis A, Tefas A, Pitas I (2013) Dynamic action recognition based on dynemes and extreme learning machine. Pattern Recogn Lett 34(15):1890–1898. doi: 10.1016/j.patrec.2012.10.019 CrossRefGoogle Scholar
  16. 16.
    Song Y, Crowcroft J, Zhang J (2012) Automatic epileptic seizure detection in EEGs based on optimized sample entropy and extreme learning machine. J Neurosci Methods 210(2):132–146. doi: 10.1016/j.jneumeth.2012.07.003 CrossRefGoogle Scholar
  17. 17.
    Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2011) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cyber 3(2):149–161. doi: 10.1007/s13042-011-0049-5 CrossRefGoogle Scholar
  18. 18.
    Zheng W, Qian Y, Lu H (2012) Text categorization based on regularization extreme learning machine. Neural Comput Appl 22(3–4):447–456. doi: 10.1007/s00521-011-0808-y Google Scholar
  19. 19.
    Hu X-F, Zhao Z, Wang S, Wang F-L, He D-K, Wu S-K (2007) Multi-stage extreme learning machine for fault diagnosis on hydraulic tube tester. Neural Comput Appl 17(4):399–403. doi: 10.1007/s00521-007-0139-1 CrossRefGoogle Scholar
  20. 20.
    Suresh S, Venkatesh Babu R, Kim HJ (2009) No-reference image quality assessment using modified extreme learning machine classifier. Appl Soft Compt 9(2):541–552. doi: 10.1016/j.asoc.2008.07.005 CrossRefGoogle Scholar
  21. 21.
    Chen FL, Ou TY (2011) Sales forecasting system based on Gray extreme learning machine with Taguchi method in retail industry. Expert Syst Appl 38(3):1336–1345. doi: 10.1016/j.eswa.2010.07.014 CrossRefGoogle Scholar
  22. 22.
    Malathi V, Marimuthu NS, Baskar S, Ramar K (2011) Application of extreme learning machine for series compensated transmission line protection. Eng Appl Artif Intel 24(5):880–887. doi: 10.1016/j.engappai.2011.03.003 CrossRefGoogle Scholar
  23. 23.
    Zhao X (2010) A perturbed particle swarm algorithm for numerical optimization. Appl Soft Compt 10(1):119–124. doi: 10.1016/j.asoc.2009.06.010 CrossRefGoogle Scholar
  24. 24.
    Zhao X, Liu Z, Yang X (2014) A multi-swarm cooperative multistage perturbation guiding particle swarm optimizer. Appl Soft Compt 22:77–93. doi: 10.1016/j.asoc.2014.04.042 CrossRefGoogle Scholar
  25. 25.
    Li X, Yin M (2012) Application of differential evolution algorithm on self-potential data. PLoS One 7(12):e51199. doi: 10.1371/journal.pone.0051199 CrossRefGoogle Scholar
  26. 26.
    Zou D, Liu H, Gao L, Li S (2011) An improved differential evolution algorithm for the task assignment problem. Eng Appl Artif Intel 24(4):616–624. doi: 10.1016/j.engappai.2010.12.002 CrossRefGoogle Scholar
  27. 27.
    Li X, Yin M (2014) Parameter estimation for chaotic systems by hybrid differential evolution algorithm and artificial bee colony algorithm. Nonlinear Dynam 77(1–2):61–71. doi: 10.1007/s11071-014-1273-9 MathSciNetCrossRefGoogle Scholar
  28. 28.
    Zhu QY, Qin AK, Suganthan PN, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recogn 38(10):1759–1763. doi: 10.1016/j.patcog.2005.03.028 CrossRefzbMATHGoogle Scholar
  29. 29.
    Cao J, Lin Z, Huang G-B (2012) Self-adaptive evolutionary extreme learning machine. Neural Process Lett 36(3):285–305. doi: 10.1007/s11063-012-9236-y CrossRefGoogle Scholar
  30. 30.
    Gandomi AH, Yang X-S, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29(1):17–35. doi: 10.1007/s00366-011-0241-y MathSciNetCrossRefGoogle Scholar
  31. 31.
    Yang XS, Deb S (2009) Cuckoo search via Lévy flights. In: Abraham A, Carvalho A, Herrera F, Pai V (eds) Proceeding of world congress on nature & biologically inspired computing (NaBIC 2009), Coimbatore, India, December 2009. IEEE Publications, USA, pp 210–214Google Scholar
  32. 32.
    Wang G-G, Guo L, Duan H, Wang H (2014) A new improved firefly algorithm for global numerical optimization. J Comput Theor Nanosci 11(2):477–485. doi: 10.1166/jctn.2014.3383 CrossRefGoogle Scholar
  33. 33.
    Yang XS (2010) Firefly algorithm, stochastic test functions and design optimisation. Int J Bio-Inspired Comput 2(2):78–84. doi: 10.1504/IJBIC.2010.032124 CrossRefGoogle Scholar
  34. 34.
    Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61. doi: 10.1016/j.advengsoft.2013.12.007 CrossRefGoogle Scholar
  35. 35.
    Geem ZW, Kim JH, Loganathan GV (2001) A new heuristic optimization algorithm: harmony search. Simulation 76(2):60–68. doi: 10.1177/003754970107600201 CrossRefGoogle Scholar
  36. 36.
    Wang G, Guo L, Wang H, Duan H, Liu L, Li J (2014) Incorporating mutation scheme into krill herd algorithm for global numerical optimization. Neural Comput Appl 24(3–4):853–871. doi: 10.1007/s00521-012-1304-8 CrossRefGoogle Scholar
  37. 37.
    Zou D, Gao L, Li S, Wu J (2011) Solving 0–1 knapsack problem by a novel global harmony search algorithm. Appl Soft Compt 11(2):1556–1564. doi: 10.1016/j.asoc.2010.07.019 CrossRefGoogle Scholar
  38. 38.
    Wang G, Guo L, Duan H, Wang H, Liu L, Shao M (2013) Hybridizing harmony search with biogeography based optimization for global numerical optimization. J Comput Theor Nanosci 10(10):2318–2328. doi: 10.1166/jctn.2013.3207 Google Scholar
  39. 39.
    Simon D (2008) Biogeography-based optimization. IEEE Trans Evolut Comput 12(6):702–713. doi: 10.1109/TEVC.2008.919004 CrossRefGoogle Scholar
  40. 40.
    Wang G-G, Gandomi AH, Alavi AH (2014) An effective krill herd algorithm with migration operator in biogeography-based optimization. Appl Math Model 38(9–10):2454–2462. doi: 10.1016/j.apm.2013.10.052 MathSciNetCrossRefGoogle Scholar
  41. 41.
    Saremi S, Mirjalili S, Lewis A (2014) Biogeography-based optimisation with chaos. Neural Comput Appl 25(5):1077–1097. doi: 10.1007/s00521-014-1597-x CrossRefGoogle Scholar
  42. 42.
    Li X, Yin M (2013) Multiobjective binary biogeography based optimization for feature selection using gene expression data. IEEE Trans Nanobiosci 12(4):343–353. doi: 10.1109/TNB.2013.2294716 CrossRefGoogle Scholar
  43. 43.
    Li X, Zhang J, Yin M (2014) Animal migration optimization: an optimization algorithm inspired by animal migration behavior. Neural Comput Appl 24(7–8):1867–1877. doi: 10.1007/s00521-013-1433-8 CrossRefGoogle Scholar
  44. 44.
    Mirjalili S, Wang G-G, Coelho LdS (2014) Binary optimization using hybrid particle swarm optimization and gravitational search algorithm. Neural Comput Appl 25(6):1423–1435. doi: 10.1007/s00521-014-1629-6 CrossRefGoogle Scholar
  45. 45.
    Mirjalili S, Lewis A (2014) Adaptive gbest-guided gravitational search algorithm. Neural Comput Appl 25(7–8):1569–1584. doi: 10.1007/s00521-014-1640-y CrossRefGoogle Scholar
  46. 46.
    Mirjalili S, Mohd Hashim SZ, Moradian Sardroudi H (2012) Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput 218(22):11125–11137. doi: 10.1016/j.amc.2012.04.069 MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Zhang Z, Zhang N, Feng Z (2014) Multi-satellite control resource scheduling based on ant colony optimization. Expert Syst Appl 41(6):2816–2823. doi: 10.1016/j.eswa.2013.10.014 CrossRefGoogle Scholar
  48. 48.
    Zhang Z, Feng Z (2012) Two-stage updating pheromone for invariant ant colony optimization algorithm. Expert Syst Appl 39(1):706–712. doi: 10.1016/j.eswa.2011.07.062 CrossRefGoogle Scholar
  49. 49.
    Gandomi AH (2014) Interior search algorithm (ISA): a novel approach for global optimization. ISA Trans 53(4):1168–1183. doi: 10.1016/j.isatra.2014.03.018 CrossRefGoogle Scholar
  50. 50.
    Gandomi AH, Yang X-S, Alavi AH, Talatahari S (2013) Bat algorithm for constrained optimization tasks. Neural Comput Appl 22(6):1239–1255. doi: 10.1007/s00521-012-1028-9 CrossRefGoogle Scholar
  51. 51.
    Yang XS, Gandomi AH (2012) Bat algorithm: a novel approach for global engineering optimization. Eng Comput 29(5):464–483. doi: 10.1108/02644401211235834 CrossRefGoogle Scholar
  52. 52.
    Gandomi AH, Alavi AH (2012) Krill herd: a new bio-inspired optimization algorithm. Commun Nonlinear Sci Numer Simul 17(12):4831–4845. doi: 10.1016/j.cnsns.2012.05.010 MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Wang G-G, Gandomi AH, Alavi AH (2014) Stud krill herd algorithm. Neurocomputing 128:363–370. doi: 10.1016/j.neucom.2013.08.031 CrossRefGoogle Scholar
  54. 54.
    Guo L, Wang G-G, Gandomi AH, Alavi AH, Duan H (2014) A new improved krill herd algorithm for global numerical optimization. Neurocomputing 138:392–402. doi: 10.1016/j.neucom.2014.01.023 CrossRefGoogle Scholar
  55. 55.
    Wang G-G, Gandomi AH, Alavi AH (2013) A chaotic particle-swarm krill herd algorithm for global numerical optimization. Kybernetes 42(6):962–978. doi: 10.1108/K-11-2012-0108 MathSciNetCrossRefGoogle Scholar
  56. 56.
    Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18. doi: 10.1016/j.swevo.2011.02.002 CrossRefGoogle Scholar
  57. 57.
    Wang G-G, Guo L, Gandomi AH, Hao G-S, Wang H (2014) Chaotic krill herd algorithm. Inf Sci 274:17–34. doi: 10.1016/j.ins.2014.02.123 MathSciNetCrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2015

Authors and Affiliations

  • Gai-Ge Wang
    • 1
    • 2
    • 3
    Email author
  • Mei Lu
    • 1
  • Yong-Quan Dong
    • 1
  • Xiang-Jun Zhao
    • 1
  1. 1.School of Computer Science and TechnologyJiangsu Normal UniversityXuzhouChina
  2. 2.Institute of Algorithm and Big Data AnalysisNortheast Normal UniversityChangchunChina
  3. 3.School of Computer Science and Information TechnologyNortheast Normal UniversityChangchunChina

Personalised recommendations