Neural Computing and Applications

, Volume 31, Issue 3, pp 637–652 | Cite as

An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction

  • Wei Zhang
  • Aiqiang XuEmail author
  • Dianfa Ping
  • Mingzhe Gao
Original Article


In order to curb the model expansion of the kernel learning methods and adapt the nonlinear dynamics in the process of the nonstationary time series online prediction, a new online sequential learning algorithm with sparse update and adaptive regularization scheme is proposed based on kernel-based incremental extreme learning machine (KB-IELM). For online sparsification, a new method is presented to select sparse dictionary based on the instantaneous information measure. This method utilizes a pruning strategy, which can prune the least “significant” centers, and preserves the important ones by online minimizing the redundancy of dictionary. For adaptive regularization scheme, a new objective function is constructed based on basic ELM model. New model has different structural risks in different nonlinear regions. At each training step, new added sample could be assigned optimal regularization factor by optimization procedure. Performance comparisons of the proposed method with other existing online sequential learning methods are presented using artificial and real-word nonstationary time series data. The results indicate that the proposed method can achieve higher prediction accuracy, better generalization performance and stability.


Time series prediction Extreme learning machine Online modeling Fixed budget Sparsity measures Adaptive regularization 



The research was supported by National Science Foundation of China under Grant Nos. 61571454.

Compliance with ethical standards

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.


  1. 1.
    Mourad E, Amiya N (2012) Comparison-based system-level fault diagnosis: a neural network approach. IEEE Trans Parallel Distrib Syst 23(6):1047–1059CrossRefGoogle Scholar
  2. 2.
    Tian Z, Qian C, Gu B, Yang L, Liu F (2015) Electric vehicle air conditioning system performance prediction based on artificial neural network. Appl Therm Eng 89:101–104CrossRefGoogle Scholar
  3. 3.
    Cambria E, Huang GB (2013) Extreme learning machine [trends and controversies]. IEEE Intell Syst 28(6):30–59CrossRefGoogle Scholar
  4. 4.
    Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and application. Neurocomputing 70(1–3):489–501CrossRefGoogle Scholar
  5. 5.
    Yin G, Zhang YT, Li ZN, Ren GQ, Fan HB (2014) Online fault diagnosis method based on incremental support vector data description and extreme learning machine with incremental output structure. Neurocomputing 128:224–231CrossRefGoogle Scholar
  6. 6.
    Rong HJ, Huang GB, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Transactions on systems, man and cybernetics—part B: cybernetics 39(4):1067–1072CrossRefGoogle Scholar
  7. 7.
    Mirza B, Lin ZP, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329CrossRefGoogle Scholar
  8. 8.
    Xia SX, Meng FR, Liu B, Zhou Y (2015) A kernel clustering-based possibilistic fuzzy extreme learning machine for class imbalance learning. Cogn Comput 7:74–85CrossRefGoogle Scholar
  9. 9.
    Li XD, Mao WJ, Wei Jiang (2016) Multiple-kernel-learning- based extreme learning machine for classification design. Neural Comput Appl 27:175–184CrossRefGoogle Scholar
  10. 10.
    Zhao SL, Chen BD, Zhu PP, Principe JC (2013) Fixed budget quantized kernel least-mean-square algorithm. Sig Process 93:2759–2770CrossRefGoogle Scholar
  11. 11.
    Huang GB, Zhou H, Ding X, Zhang R (2011) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern—Part B: Cybern 42(2):513–529CrossRefGoogle Scholar
  12. 12.
    Wang XY, Han M (2014) Online sequential extreme learning machine with kernels for nonstationary time series prediction. Neurocomputing 145:90–97CrossRefGoogle Scholar
  13. 13.
    Deng WY, Ong YS, Tan PS, Zheng QH (2016) Online sequential reduced kernel extreme learning machine. Neurocomputing 174:72–84CrossRefGoogle Scholar
  14. 14.
    Wong SY, Yap KS, Yap HJ, Tan SC (2015) A truly online learning algorithm using hybird fuzzy ARTMAP and online extreme learning machine for pattern classification. Neural Process Lett 42:585–602CrossRefGoogle Scholar
  15. 15.
    Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423CrossRefGoogle Scholar
  16. 16.
    Huynh HT, Won Y (2011) Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks. Pattern Recogn Lett 32:1930–1935CrossRefGoogle Scholar
  17. 17.
    Guo L, Hao JH, Liu M (2014) An incremental extreme learning machine for online sequential learning problems. Neurocomputing 128:50–58CrossRefGoogle Scholar
  18. 18.
    Fan HJ, Song Q, Yang XL, Xu Z (2015) Kernel online learning algorithm with state feedbacks. Knowl Based Syst 89:173–180CrossRefGoogle Scholar
  19. 19.
    Fan HJ, Song Q (2013) A sparse kernel algorithm for online time series data prediction. Expert Syst Appl 40:2174–2181CrossRefGoogle Scholar
  20. 20.
    Zhou XR, Liu ZJ, Zhu CX (2014) Online regularized and kernelized extreme learning machines with forgetting mechanism. Math Probl Eng. doi: 10.1155/2014/938548 Google Scholar
  21. 21.
    Zhou XR, Wang CS (2016) Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism. Neurocomputing 174:1147–1155CrossRefGoogle Scholar
  22. 22.
    Gu Y, Liu JF, Chen YQ, Jiang XL, Yu HC (2014) TOSELM: timeliness online sequential extreme learning machine. Neurocomputing 128:119–127CrossRefGoogle Scholar
  23. 23.
    Lim J, Lee S, Pang HS (2013) Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimation. Neural Comput Appl 22:569–576CrossRefGoogle Scholar
  24. 24.
    He X, Wang HL, Lu JH, Jiang W (2015) Online fault diagnosis of analog circuit based on limited-samples sequence extreme learning machine. Control Decis 30(3):455–460zbMATHGoogle Scholar
  25. 25.
    Scardapance S, Comminiello D, Scarpiniti M, Uncini A (2015) Online sequential extreme learning machine with kernel. IEEE Trans Neural Netw Learn. Syst 26(9):2214–2220MathSciNetCrossRefGoogle Scholar
  26. 26.
    Zhang YT, Ma C, Li ZN, Fan HB (2014) Online modeling of kernel extreme learning machine based on fast leave-one-out cross-validation. J Shanghai Jiaotong Univ 48(5):641–646Google Scholar
  27. 27.
    Shao ZF, Meng JE (2016) An online sequential learning algorithm for regularized extreme learning machine. Neurocomputing 173:778–788CrossRefGoogle Scholar
  28. 28.
    Lu XJ, Zhou C, Huang MH, Lv WB (2016) Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system. Neurocomputing 174:617–626CrossRefGoogle Scholar
  29. 29.
    Lin M, Zhang LJ, Jin R, Weng SF, Zhang CS (2016) Online kernel learning with nearly constant support vectors. Neurocomputing 179:26–36CrossRefGoogle Scholar
  30. 30.
    Honeine P (2015) Analyzing sparse dictionaries for online learning with kernels. IEEE Trans Signal Process 63(23):6343–6353MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Platt J (1991) A resource-allocating network for function interpolation. Neural Comput 3(2):213–225MathSciNetCrossRefGoogle Scholar
  32. 32.
    Engel Y, Mannor S, Meir R (2004) The kernel recursive least-squares algorithm. IEEE Trans Signal Process 52(8):2275–2285MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Richard C, Bermudez JCM, Honeine P (2009) Online prediction of time series data with kernels. IEEE Trans Signal Process 57(3):1058–1067MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Fan HJ, Song Q, Xu Z (2014) Online learning with kernel regularized least mean square algorithms. Expert Syst Appl 41:4349–4359CrossRefGoogle Scholar
  35. 35.
    Liu WF, Park I, Principe JC (2009) An information theoretic approach of designing sparse kernel adaptive filters. IEEE Trans Neural Netw 20(12):1950–1961CrossRefGoogle Scholar
  36. 36.
    Fan HJ, Song Q, Shrestha SB (2016) Kernel online learning with adaptive kernel width. Neurocomputing 175:233–242CrossRefGoogle Scholar
  37. 37.
    Zhao YP, Wang KK (2014) Fast cross validation for regularized extreme learning machine. J Syst Eng Electron 25(5):895–900CrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  • Wei Zhang
    • 1
  • Aiqiang Xu
    • 1
    Email author
  • Dianfa Ping
    • 2
  • Mingzhe Gao
    • 1
  1. 1.Office of Research and DevelopmentNaval Aeronautical and Astronautical UniversityYantaiPeople’s Republic of China
  2. 2.Department of Electronic and Information EngineeringNaval Aeronautical and Astronautical UniversityYantaiPeople’s Republic of China

Personalised recommendations