Skip to main content

Advertisement

SpringerLink
Log in

SVM hyperparameters tuning for recursive multi-step-ahead prediction

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Prediction of time series data is of relevance for many industrial applications. The prediction can be made in one-step and multi-step ahead. For predictive maintenance, multi-step-ahead prediction is of interest for projecting the evolution of the future conditions of the equipment of interest, computing the remaining useful life and taking corresponding maintenance decisions. Recursive prediction is one of the popular strategies for multi-step-ahead prediction. SVM is a popular data-driven approach that has been used for recursive multi-step-ahead prediction. Tuning the hyperparameters in SVM during the training process is challenging, and normally the hyperparameters are tuned by solving an optimization problem. This paper analyses the possible objectives of the optimization for tuning hyperparameters. Through experiments on one synthetic dataset and two real time series data, related to the prediction of wind speed in a region and leakage from the reactor coolant pump in a nuclear power plant, a bi-objective optimization combining mean absolute derivatives and accuracy on all prediction steps is shown to be the best choice for tuning SVM hyperparameters for recursive multi-step-ahead prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Aich U, Banerjee S (2013) Modeling of EDM responses by support vector machine regression with parameters selected by particle swarm optimization. Appl Math Model 38:2800–2818. doi:10.1016/j.apm.2013.10.073

    Article  Google Scholar 

  2. Ak R, Li Y, Vitelli V, Zio E, López Droguett E, Magno Couto Jacinto C (2013) NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment. Expert Syst Appl 40:1205–1212. doi:10.1016/j.eswa.2012.08.018

    Article  Google Scholar 

  3. Asefa T, Kemblowski M, McKee M, Khalil A (2006) Multi-time scale stream flow predictions: the support vector machines approach. J Hydrol 318:7–16. doi:10.1016/j.jhydrol.2005.06.001

    Article  Google Scholar 

  4. Aydin I, Karakose M, Akin E (2011) A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Appl Soft Comput 11:120–129. doi:10.1016/j.asoc.2009.11.003

    Article  Google Scholar 

  5. Bao Y, Xiong T, Hu Z (2014) Multi-step-ahead time series prediction using multiple-output support vector regression. Neurocomputing 129:482–493. doi:10.1016/j.neucom.2013.09.010

    Article  Google Scholar 

  6. Belegundu AD, Chandrupatla TR (2011) Optimization concepts and applications in engineering. Cambridge University Press

  7. Benkedjouh T, Medjaher K, Zerhouni N, Rechak S (2013) Remaining useful life estimation based on nonlinear feature reduction and support vector regression. Eng Appl Artif Intell 26:1751–1760. doi:10.1016/j.engappai.2013.02.006

    Article  Google Scholar 

  8. Benkedjouh T, Medjaher K, Zerhouni N, Rechak S (2015) Health assessment and life prediction of cutting tools based on support vector regression. J Intell Manuf 26:213–223. doi:10.1007/s10845-013-0774-6

    Article  Google Scholar 

  9. Blasco X, Herrero JM, Sanchis J, Martínez M (2008) A new graphical visualization of n-dimensional Pareto front for decision-making in multiobjective optimization. Inf Sci 178:3908–3924. doi:10.1016/j.ins.2008.06.010

    Article  MATH  Google Scholar 

  10. Canadian Weather Office (2012) The wind speed data. http://climate.weather.gc.ca/

  11. Chang C-C, Chou S-H (2015) Tuning of the hyperparameters for L2-loss SVMs with the RBF kernel by the maximum-margin principle and the jackknife technique. Pattern Recognit 48:3983–3992. doi:10.1016/j.patcog.2015.06.017

    Article  Google Scholar 

  12. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46:131–159. doi:10.1023/A:1012450327387

    Article  MATH  Google Scholar 

  13. Chen R, Liang C-Y, Hong W-C, Gu D-X (2015) Forecasting holiday daily tourist flow based on seasonal support vector regression with adaptive genetic algorithm. Appl Soft Comput 26:435–443. doi:10.1016/j.asoc.2014.10.022

    Article  Google Scholar 

  14. Chou J-SS, Cheng M-YY, Wu Y-WW, Pham A-DD (2014) Optimizing parameters of support vector machine using fast messy genetic algorithm for dispute classification. Expert Syst Appl 41:3955–3964. doi:10.1016/j.eswa.2013.12.035

    Article  Google Scholar 

  15. Ding S, Han Y, Yu J, Gu Y (2013) A fast fuzzy support vector machine based on information granulation. Neural Comput Appl 23(1):139–144

    Article  Google Scholar 

  16. Drucker H, Burges CJC, Kaufman L et al (1997) Support vector regression machines. In: Mozer MC, Jordan MI, Petsche T (eds) Advances in neural information processing systems. vol 9. MIT Press, Cambridge, pp 155–161

  17. Hu Z, Bao Y, Xiong T (2014) Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression. Appl Soft Comput 25:15–25. doi:10.1016/j.asoc.2014.09.007

    Article  Google Scholar 

  18. Igel C (2005) Multi-objective model selection for support vector machines. In: Coello CAC, Aguirre AH, Zitzler E (eds) Evolutionary multi-criterion optimization. Springer, Berlin, pp 534–546

  19. Jardine AKS, Lin D, Banjevic D (2006) A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mech Syst Signal Process 20:1483–1510. doi:10.1016/j.ymssp.2005.09.012

    Article  Google Scholar 

  20. Javed K, Gouriveau R, Zerhouni N, Nectoux P (2015) Enabling health monitoring approach based on vibration data for accurate prognostics. IEEE Trans Ind Electron 62:647–656. doi:10.1109/TIE.2014.2327917

    Article  Google Scholar 

  21. Karush W (1939) Minima of functions of several variables with inequalities as side constraints (doctoral dissertation, Master’s thesis, Dept. of Mathematics, Univ. of Chicago)

  22. Kim IY, De Weck OL (2005) Adaptive weighted-sum method for bi-objective optimization: Pareto front generation. Struct Multidiscip Optim 29:149–158. doi:10.1007/s00158-004-0465-1

    Article  Google Scholar 

  23. Kuhn HW, Tucker A (1951) Nonlinear programming. In: Proceedings of the second symposium on mathematical statistics and probability. doi:10.1007/BF01582292

  24. Lei Y, He Z, Zi Y, Hu Q (2007) Fault diagnosis of rotating machinery based on multiple ANFIS combination with GAs. Mech Syst Signal Process 21:2280–2294. doi:10.1016/j.ymssp.2006.11.003

    Article  Google Scholar 

  25. Lei Y, Liu Z, Wu X et al (2015) Health condition identification of multi-stage planetary gearboxes using a mRVM-based method. Mech Syst Signal Process 60–61:289–300. doi:10.1016/j.ymssp.2015.01.014

    Article  Google Scholar 

  26. Lin J-Y, Cheng C-T, Chau K-W (2006) Using support vector machines for long-term discharge prediction. Hydrol Sci J 51:599–612. doi:10.1623/hysj.51.4.599

    Article  Google Scholar 

  27. Liu R, Liu E, Yang J, Li M, Wang F (2006) Optimizing the hyper-parameters for SVM by combining evolution strategies with a grid search. Lect Notes Control Inf Sci 344:712–721. doi:10.1007/11816492_87

    Article  MATH  Google Scholar 

  28. Liu S, Xu L, Jiang Y, Li D, Chen Y, Li Z (2014) A hybrid WA–CPSO-LSSVR model for dissolved oxygen content prediction in crab culture. Eng Appl Artif Intell 29:114–124. doi:10.1016/j.engappai.2013.09.019

    Article  Google Scholar 

  29. Min JH, Lee Y-C (2005) Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters. Expert Syst Appl 28:603–614. doi:10.1016/j.eswa.2004.12.008

    Article  Google Scholar 

  30. Müller K, Smola A, Rätsch G, Schölkopf B, Kohlmorgen J, Vapnik V (1997) Predicting time series with support vector machines. In: Artificial neural networks ICANN97, pp 999–1004

  31. Namdari M, Jazayeri-Rad H (2014) Incipient fault diagnosis using support vector machines based on monitoring continuous decision functions. Eng Appl Artif Intell 28:22–35. doi:10.1016/j.engappai.2013.11.013

    Article  Google Scholar 

  32. Nuhic A, Terzimehic T, Soczka-Guth T, Buchholz M, Dietmayer K (2013) Health diagnosis and remaining useful life prognostics of lithium-ion batteries using data-driven methods. J Power Sources 239:680–688. doi:10.1016/j.jpowsour.2012.11.146

    Article  Google Scholar 

  33. Palm R (2007) Multiple-step-ahead prediction in control systems with Gaussian process models and TS-fuzzy models. Eng Appl Artif Intell 20:1023–1035. doi:10.1016/j.engappai.2007.02.003

    Article  Google Scholar 

  34. Rodgers JL, Nicewander WA (1988) Thirteen ways to look at the correlation coefficient. Am Stat 42:59–66. doi:10.2307/2685263

    Article  Google Scholar 

  35. Saha B, Goebel K, Poll S, Christophersen J (2009) Prognostics methods for battery health monitoring using a Bayesian framework. IEEE Trans Instrum Meas 58:291–297

    Article  Google Scholar 

  36. Sapankevych N, Sankar R (2009) Time series prediction using support vector machines: a survey. IEEE Comput Intell Mag 4:24–38. doi:10.1109/MCI.2009.932254

    Article  Google Scholar 

  37. Shen L, Chen H, Yu Z et al (2016) Evolving support vector machines using fruit fly optimization for medical data classification. Knowl Based Syst. doi:10.1016/j.knosys.2016.01.002

    Google Scholar 

  38. Sorjamaa A, Hao J, Reyhani N, Ji Y, Lendasse A (2007) Methodology for long-term prediction of time series. Neurocomputing 70:2861–2869. doi:10.1016/j.neucom.2006.06.015

    Article  Google Scholar 

  39. Taieb SB, Sorjamaa A, Bontempi G (2010) Multiple-output modeling for multi-step-ahead time series forecasting. Neurocomputing 73:1950–1957. doi:10.1016/j.neucom.2009.11.030

    Article  Google Scholar 

  40. Vichare NM, Pecht MG (2006) Prognostics and health management of electronics. IEEE Trans Compon Packag Technol 29:222–229. doi:10.1109/TCAPT.2006.870387

    Article  Google Scholar 

  41. Wang X, Han M (2015) Improved extreme learning machine for multivariate time series online sequential prediction. Eng Appl Artif Intell 40:28–36. doi:10.1016/j.engappai.2014.12.013

    Article  Google Scholar 

  42. Wang Z, He X, Gao D, Xue X (2013) An efficient Kernel-based matrixized least squares support vector machine. Neural Comput Appl 22(1):143–150

    Article  Google Scholar 

  43. Wang S, Han Z, Liu F, Tang Y (2015) Nonlinear system identification using least squares support vector machine tuned by an adaptive particle swarm optimization. Int J Mach Learn Cybern 6:981–992. doi:10.1007/s13042-015-0403-0

    Article  Google Scholar 

  44. Widodo A, Shim MC, Caesarendra W, Yang BS (2011) Intelligent prognostics for battery health monitoring based on sample entropy. Expert Syst Appl 38:11763–11769. doi:10.1016/j.eswa.2011.03.063

    Article  Google Scholar 

  45. Wu CL, Chau KW (2013) Prediction of rainfall time series using modular soft computingmethods. Eng Appl Artif Intell 26:997–1007. doi:10.1016/j.engappai.2012.05.023

    Article  Google Scholar 

  46. Wu C-H, Tzeng G-H, Goo Y-J, Fang W-C (2007) A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy. Expert Syst Appl 32:397–408. doi:10.1016/j.eswa.2005.12.008

    Article  Google Scholar 

  47. Zhang X, Qiu D, Chen F (2015) Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis. Neurocomputing 149:641–651. doi:10.1016/j.neucom.2014.08.010

    Article  Google Scholar 

  48. Zhao Z, Quan Q, Cai K-Y (2014) A profust reliability based approach to prognostics and health management. IEEE Trans Reliab 63:26–41. doi:10.1109/TR.2014.2299111

    Article  Google Scholar 

  49. Zio E (2012) Prognostics and health management of industrial equipment. Diagnostics and prognostics of engineering systems: methods and techniques, pp 333–356

  50. Zio E, Di Maio F (2012) Fatigue crack growth estimation by relevance vector machine. Expert Syst Appl 39:10681–10692. doi:10.1016/j.eswa.2012.02.199

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chair on System Science and the Energetic Challenge, EDF Foundation, Centralesupélec, Chatenay-Malabry, France

    Jie Liu & Enrico Zio

  2. Energy Department, Politecnico di Milano, Milan, Italy

    Enrico Zio

Authors
  1. Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enrico Zio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Enrico Zio.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Zio, E. SVM hyperparameters tuning for recursive multi-step-ahead prediction. Neural Comput & Applic 28, 3749–3763 (2017). https://doi.org/10.1007/s00521-016-2272-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2272-1

Keywords

Search

Navigation