Deep Belief Network Using Reinforcement Learning and Its Applications to Time Series Forecasting

  • Takaomi Hirata
  • Takashi KuremotoEmail author
  • Masanao Obayashi
  • Shingo Mabu
  • Kunikazu Kobayashi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9949)


Artificial neural networks (ANNs) typified by deep learning (DL) is one of the artificial intelligence technology which is attracting the most attention of researchers recently. However, the learning algorithm used in DL is usually with the famous error-backpropagation (BP) method. In this paper, we adopt a reinforcement learning (RL) algorithm “Stochastic Gradient Ascent (SGA)” proposed by Kimura and Kobayashi into a Deep Belief Net (DBN) with multiple restricted Boltzmann machines (RBMs) instead of BP learning method. A long-term prediction experiment, which used a benchmark of time series forecasting competition, was performed to verify the effectiveness of the proposed method.


Deep learning Restricted boltzmann machine Stochastic gradient ascent Reinforcement learning Error-backpropagation 



This work was supported by JSPS KAKENHI Grant No. 26330254 and No. 25330287.


  1. 1.
    Box, G.E.P., Pierce, D.A.: Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. J. Am. Stat. Ass. 65(332), 1509–1526 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representation by back-propagating errors. Nature 232(9), 533–536 (1986)CrossRefGoogle Scholar
  3. 3.
    Casdagli, M.: Nonlinear prediction of chaotic time series. Phys. D 35, 335–356 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Lendasse, A., Oja, E., Simula, O., Verleysen, M.: Time series prediction competition: the CATS benchmark. In: Proceedings of International Joint Conference on Neural Networks (IJCNN 2004), pp. 1615–1620 (2004)Google Scholar
  5. 5.
    Lendasse, A., Oja, E., Simula, O., Verleysen, M.: Time series prediction competition: the CATS benchmark. Neurocomputing 70, 2325–2329 (2007)CrossRefGoogle Scholar
  6. 6.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Kuremoto, T., Obayashi, M., Kobayashi, M.: Neural forecasting systems. In: Weber, C., Elshaw, M., Mayer, N.M. (eds.) Reinforcement Learning, Theory and Applications, Chap. 1, pp. 1–20. INTECH (2008)Google Scholar
  8. 8.
    Kuremoto, T., Kimura, S., Kobayashi, K., Obayashi, M.: Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing 137(5), 47–56 (2014)CrossRefGoogle Scholar
  9. 9.
    Kuremoto, T., Hirata, T., Obayashi, M., Mabu, S., Kobayashi, K.: Forecast chaotic time series data by DBNs. In: Proceedings of the 7th International Congress on Image and Signal Processing (CISP 2014), pp. 1304–1309, October 2014Google Scholar
  10. 10.
    Hirata, T., Kuremoto, T., Obayashi, M., Mabu, S.: Time series prediction using DBN and ARIMA. In: International Conference on Computer Application Technologies (CCATS 2015). Matsue, Japan, pp. 24–29, September 2015Google Scholar
  11. 11.
    Zhang, G.P.: Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50, 159–175 (2003)CrossRefzbMATHGoogle Scholar
  12. 12.
    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281–305 (2012)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Kimura, H., Kobayashi, S.: Reinforcement learning for continuous action using stochastic gradient ascent. In: Proceedings of 5th Intelligent Autonomous Systems, pp. 288–295 (1998)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Takaomi Hirata
    • 1
  • Takashi Kuremoto
    • 1
    Email author
  • Masanao Obayashi
    • 1
  • Shingo Mabu
    • 1
  • Kunikazu Kobayashi
    • 2
  1. 1.Graduate School of Science and EngineeringYamaguchi UniversityUbeJapan
  2. 2.School of Information Science and TechnologyAichi Prefectural UniversityNagakuteJapan

Personalised recommendations