Skip to main content
SpringerLink
Log in

An Ensemble Model of Arima and Ann with Restricted Boltzmann Machine Based on Decomposition of Discrete Wavelet Transform for Time Series Forecasting

  • Published:
Journal of Systems Science and Systems Engineering Aims and scope Submit manuscript

Abstract

Time series forecasting research area mainly focuses on developing effective forecasting models to improve prediction accuracy. An ensemble model composed of autoregressive integrated moving average (ARIMA), artificial neural network (ANN), restricted Boltzmann machines (RBM), and discrete wavelet transform (DWT) is presented in this paper. In the proposed model, DWT first decomposes time series into approximation and detail. Then Khashei and Bijari’s model, which is an ensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their both linear and nonlinear components and fit the relationship between the components as a function instead of additive relationship. Furthermore, RBM is used to perform pre-training for generating initial weights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detail are combined to obtain final forecasting. The forecasting capability of the proposed model is tested with three well-known time series: sunspot, Canadian lynx, exchange rate time series. The prediction performance is compared to the other six forecasting models. The results indicate that the proposed model gives the best performance in all three data sets and all three measures (i.e. MSE, MAE and MAPE).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ackley, D., E. Hinton, & Sejnowski T. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9: 147–169.

    Article  Google Scholar 

  2. Adamowski, J. & Chan, H. F. (2011). A wavelet neural network conjunction model for groundwater level forecasting. Journal of Hydrology, 407 (1): 28–40.

    Article  Google Scholar 

  3. Adamowski, J. & Sun, K. (2010). Development of a coupled wavelet transform and neural network method for flow forecasting of non-perennial rivers in semi-arid watersheds. Journal of Hydrology, 390 (1): 85–91.

    Article  Google Scholar 

  4. Campbell, M. & Walker, A. (1977). A survey of statistical work on the Mackenzie River series of annual Canadian lynx trappings for the years 1821–1934 and a new analysis. Journal of the Royal Statistical Society. Series A (general), 140 (4): 411–431.

    Article  Google Scholar 

  5. Chen, J. & Tang, X. (2017). Ensemble of multiple kNN classifiers for societal risk classification. Journal of Systems Science and Systems Engineering, 26 (4): 433–447.

    Article  Google Scholar 

  6. Cogranne, R., & Fridrich, J. (2015). Modeling and extending the ensemble classifier for steganalysis of digital images using hypothesis testing theory. IEEE Transactions on Information Forensics and Security, 10 (12): 2627–2642.

    Article  Google Scholar 

  7. Conejo, A.J., Plazas, M.A., Espinola, R. & Molina, A.B. (2005). Day-ahead electricity price forecasting using the wavelet transform and ARIMA models. IEEE Transactions on Power Systems, 20 (2): 1035–1042.

    Article  Google Scholar 

  8. Dayhoff, J.A. (1995). Neural Network Architectures: An Introduction. MIT press, Cambridge.

    Google Scholar 

  9. De Gooijer, J.G. & Hyndman, R.J. (2006). 25 Years of time series forecasting. International Journal of Forecasting, 22 (3): 443–473.

    Article  Google Scholar 

  10. Fard, A.K. & Akbari-Zadeh, M.R. (2014). A hybrid method based on wavelet, ANN and ARIMA model for short-term load forecasting. Journal of Experimental & Theoretical Articial Intelligence, 26 (2): 167–182.

    Article  Google Scholar 

  11. Fliege, N. J. (1994). Multirate Digital Signal Processing: Multirate Systems, Filter Banks, Wavelets. John Willy & Sons, Inc., Chichester.

    MATH  Google Scholar 

  12. Gao, W. & Tian, Z. (2009). Learning Granger causality graphs for multivariate nonlinear time series. Journal of Systems Science and Systems Engineering, 18 (1): 38–52.

    Article  Google Scholar 

  13. Hecht-Nielsen, R. (1989). Theory of the backpropagation neural network. In: International Joint Conference on Neural Networks, 593–605, 1989, IEEE.

    Chapter  Google Scholar 

  14. Hinton, G.E. & Sejnowski, T.J. (1986) Learning and relearning in Boltzmann machines. in: D.E. Rumelhart, J.L. McClelland (eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1, pp. 282–317: Foundations, MIT Press, Cambridge, MA.

    Google Scholar 

  15. Huang, N.E., Shen, Z., Long, S.R., Wu, M.C., Shih, H.H., Zheng, Q., Yen, N.-C., Tung, C.C. & Liu, H.H. (1998). The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. In: Proceedings of the Royal Society of London A: Mathematical, Physical And Engineering Sciences, 454(1971): 903–995, The Royal Society.

    Article  MathSciNet  MATH  Google Scholar 

  16. Kennedy, J. & Eberhart, R.C. (1995). Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, 1942–1948, 1995, IEEE.

    Chapter  Google Scholar 

  17. Khandelwal, I., Adhikari, R. & Verma, G. (2015). Time series forecasting using hybrid ARIMA and ANN models based on DWT decomposition. Procedia Computer Science, 48:173–179.

    Article  Google Scholar 

  18. Khashei, M. & Bijari, M. (2011). A novel hybridization of articial neural networks and ARIMA models for time series forecasting. Applied Soft Computing, 11 (2): 2664–2675.

    Article  Google Scholar 

  19. Lin, T. & Pourahmadi, M. (1998). Nonparametric and non-linear models and data mining in time series: a case-study on the Canadian lynx data. Journal of the Royal Statistical Society: Series C (Applied Statistics), 47 (2): 187–201.

    Article  MATH  Google Scholar 

  20. MacKay, D.J. (1992). A practical bayesian framework for backpropagation networks. Neural Computation, 4 (3): 448–472.

    Article  Google Scholar 

  21. Meese, R. & Rogoff, K. (1983). Empirical exchange rate models of the seventies: Do they fit out of sample? Journal of International Economics, 14 (1-2): 3–24.

    Article  Google Scholar 

  22. Nourani, V., Komasi, M. & Mano, A. (2009). A multivariate ANN-Wavelet approach for rainfall-runoff modeling. Water Resources Management, 23 (14): 2877–2894.

    Article  Google Scholar 

  23. Pannakkong, W. & Huynh, V. N. (2017). A new hybrid linear-nonlinear model based on decomposition of discrete wavelet transform for time series forecasting. In: International Symposium on Knowledge and Systems Sciences, 186–196, 2017, Springer.

    Chapter  Google Scholar 

  24. Partal, T. & Kisi, Ö. (2007). Wavelet and neuro-fuzzy conjunction model for precipitation forecasting. Journal of Hydrology, 342 (1): 199–212.

    Article  Google Scholar 

  25. Rai, B. & Singh, N. (2005). Forecasting automobile warranty performance in presence of ‘maturing data’ phenomena using multilayer perceptron neural network. Journal of Systems Science and Systems Engineering, 14 (2): 159–176.

    Article  Google Scholar 

  26. Takashi K., Shinsuke, K., Kunikazu, K. & Masanao, O. (2014). Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing, 137: 47–56.

    Article  Google Scholar 

  27. Tan, Z., Zhang, J., Wang, J. & Xu, J. (2010). Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models. Applied Energy, 87 (11): 3606–3610.

    Article  Google Scholar 

  28. Thury, G. (2003). Industrial production in Germany and Austria: A case study in structural time series modeling. Journal of Systems Science and Systems Engineering, 12 (2): 159–170.

    Article  Google Scholar 

  29. Thury, G. & Zhou, Mi. (2005) Calendar effects in monthly time series models. Journal of Systems Science and Systems Engineering, 14 (2): 218–230.

    Article  Google Scholar 

  30. Tiwari, M. K. & Chatterjee, C. (2010). Development of an accurate and reliable hourly flood forecasting model using WaveletBootstrapANN (WBANN) hybrid approach. Journal of Hydrology, 394 (3): 458–470.

    Article  Google Scholar 

  31. Twala, B. (2009). Combining classifiers for credit risk prediction. Journal of Systems Science and Systems Engineering, 18 (3): 292–311.

    Article  Google Scholar 

  32. Wei, S., Zuo, D. & Song, J. (2012). Improving prediction accuracy of river discharge time series using a wavelet-NAR articial neural network. Journal of Hydroinformatics, 14 (4): 974–991.

    Article  Google Scholar 

  33. Wong, C. & Li, W. (2000). On a mixture autoregressive model. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 62 (1): 95–115.

    MathSciNet  MATH  Google Scholar 

  34. Wu, W., Liu, M., Liu, Q. & Shen, W. (2016). A quantum multi-agent based neural network model for failure prediction. Journal of Systems Science and Systems Engineering, 25 (2): 210–228.

    Article  Google Scholar 

  35. Xiao, J., Xie, L., He, C. & Jiang, X. (2012). Dynamic classifier ensemble model for customer classification with imbalanced class distribution. Expert Systems with Applications, 39 (3): 3668–3675.

    Article  Google Scholar 

  36. Qiu, X., Ren, Y., Suganthan, P.N. & Amaratunga, G.A. (2017). Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting. Applied Soft Computing, 54: 246–255.

    Article  Google Scholar 

  37. Zhang, G.P. (2003). Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, 50: 159–175.

    Article  MATH  Google Scholar 

  38. Zhang, G., Patuwo, B.E. & Hu, M.Y. (1998). Forecasting with artificial neural networks: the state of the art. International Journal of Forecasting, 14 (1): 35–62.

    Article  Google Scholar 

  39. Zhou, H.C., Peng, Y. & Liang, G.H. (2008). The research of monthly discharge predictor-corrector model based on wavelet decomposition. Water Resources Management, 22 (2): 217–227.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for providing insightful comments and providing directions for additional work which has resulted in this paper. This work was supported by the SIIT Young Researcher Grant, under contract no. SIIT 2017-YRG-WP05.

Author information

Authors and Affiliations

  1. School of Manufacturing Systems and Mechanical Engineering, Sirindhorn International Institute of Technology, Thammasat University, Pathumthani, 12121, Thailand

    Warut Pannakkong

  2. Faculty of Economics, Chiang Mai University, Chiang Mai, 52000, Thailand

    Songsak Sriboonchitta

  3. School of Knowledge Science, Japan Advanced Institute of Science and Technology, Nomi, 923 1292, Japan

    Van-Nam Huynh

Authors
  1. Warut Pannakkong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Songsak Sriboonchitta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Van-Nam Huynh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Warut Pannakkong.

Additional information

This paper is a significantly extended and revised version of the conference paper presented at KSS-2017 (Pannakkong & Huynh, 2017).

Warut Pannakkong received his B.Engr. in Industrial Engineering and M.Engr. in Logistics and Supply Chain Systems Engineering both from Sirindhorn International Institute of Technology (SIIT), Thammasat University, Thailand in 2010 and 2014, respectively. He earned a PhD in Knowledge Science from Japan Advanced Institute of Science and Technology (JAIST) in 2017. Currently, he is a lecturer at School of Manufacturing Systems and Mechanical Engineering, SIIT. His research interests include time series forecasting, data mining, discrete event system simulation, production planning and control, and logistics and supply chain management.

Songsak Sriboonchitta received Bachelor’s and Master Degrees in Economics from Thammasat University, Thailand in 1972 and 1975 respectively and received PhD in Agricultural Economics from The University of Illinois at Urbana-Champaign in 1983. He is a professor of economics and has been working at the Faculty of Economics, Chiang Mai University since 1976. He was the Dean of the Faculty of Economics, Chiang Mai University during 2004–2008. Presently he is the President of Thailand Econometric Society and also Director of Centre of Excellence in Econometrics, Chiang Mai University. He has published one book and five edited books and more than 100 papers. His current research interests include econometrics, data mining and decision analysis.

Van-Nam Huynh is an associate professor at Japan Advanced Institute of Science and Technology, Japan. He received a PhD in Mathematics (1999) from the Institute of Information Technology, Vietnam Academy of Science and Technology, and a “Habilitation à Diriger des Recherches” (2012) at Université de Technologie de Compiègne, France. He was a post-doctoral fellow (2001—2002) awarded by Inoue Foundation for Science at Japan Advanced Institute of Science and Technology (JAIST). His research interests include decision theories, data mining and machine learning, computing and reasoning with words, information fusion, kansei information processing and application. Currently he is a member of the Editorial Board of the International Journal of Approximate Reasoning and an associate editor of the International Journal of Knowledge and Systems Science.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pannakkong, W., Sriboonchitta, S. & Huynh, VN. An Ensemble Model of Arima and Ann with Restricted Boltzmann Machine Based on Decomposition of Discrete Wavelet Transform for Time Series Forecasting. J. Syst. Sci. Syst. Eng. 27, 690–708 (2018). https://doi.org/10.1007/s11518-018-5390-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11518-018-5390-8

Keywords

Search

Navigation