Abstract
We propose a novel recurrent neural network model based on a combination of the echo state network (ESN) and the dynamic Bayesian network (DBN). Our contribution includes the following: (1) A new graph-based echo state network (GESN) model is presented for nonlinear system modeling. The GESN consists of four layers: an input layer, reservoir layer, graph model optimization layer and output layer. Unlike the original ESN model, the GESN model uses probability graph inference to optimize reservoir states, and the optimized GESN is more robust. (2) A novel DBN graph is proposed for describing random probability signals in the GESN. Simultaneously, an inference algorithm of the DBN is deduced according to Bayesian rules and probability graph theory. Finally, GESN performance is tested using the Mackey–Glass time series and laser time series data forecasting The simulations and comparison results show that the proposed model is promising.
Similar content being viewed by others
References
Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, London
Hopfield JJ, Tank DW (1985) Neural computation of decisions in optimization problems. Biol Cybern 52:141–152
Jaeger H, Haas H (2004) Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304:78–80
Gross JL, Yellen J (2011) Graph theory and its applications, 2nd edn. Chapman and Hall/CRC, Boca Raton
Li D, Han M, Wang J (2012) Chaotic time series prediction based on a novel robust echo state network. IEEE Trans Neural Netw Learn Syst 23(5):787–799
Luko M, Jaeger H (2009) Reservoir computing approaches to recurrent neural network training. Comput Sci Rev 3(3):127–149
Yildiz IB, Jaeger H, Kiebel SJ (2012) Re-visiting the echo state property. Neural Netw 35(21):1–9
Ozturk MC, Xu D, Prncipe JC (2007) Analysis and design of echo state networks. Neural Comput 19 (1):111–138
Deng Z, Zhang Y (2007) Collective behavior of a small-world recurrent neural system with scale-free distribution. IEEE Trans Neural Netw 18(5):1364–1375
Luko M (2012) Reservoir computing and self-organized neural hierarchies, Ph.D. dissertation, School Eng. Sci., Jacobs Univ., Bremen, Bremen Germany
Luko M (2010) On self-organizing reservoirs and their hierarchies, School Eng. Sci., Jacobs Univ. Bremen, Germany, Tech. Rep. 25
Rodan A, Tino P (2011) Minimum complexity echo state network. IEEE Trans Neural Netw 22(1):131–144
Strauss T, Wustlich W, Labahn R (2012) Design strategies for weight matrices of echo state networks. Neural Comput 24(12):3246–3276
Xue Y, Yang L, Haykin S (2007) Decoupled echo state networks with lateral inhibition. Neural Netw 20(3):365–376
Bas E, Egrioglu E, Hakan Aladag C, Yolcu U (2015) Fuzzy-time-series network used to forecast linear and nonlinear time series. Appl Intell 43:343–355
Bas E, Rezan Uslu V, Yolcu U, Egrioglu E (2014) A modified genetic algorithm for forecasting fuzzy time series. Appl Intell 41:453–463
Najibi E, Rostami H (2015) SCESN, SPESN, SWESN: Three recurrent neural echo state networks with clustered reservoirs for prediction of nonlinear and chaotic time series. Appl Intell 43:460–472
Lorenz EN (1963) Deterministic nonperiodic flow. J Atmos Sci 20:130–141
Mackey MC (1977) Glass L. Oscillation and chaos in physiological control systems. Sci, New Series 197 (4300):287– 289
Weigend AA, Gershenfeld NA (1994) Time series Prediction: forecasting the future and understanding the past. Harlow UK: Addison Wesley
Han M, Xi JH, Xu SG, Yin FL (2004) Prediction of chaotic time series based on the recurrent predictor neural network. IEEE Trans Signal Process 52(12):3409–3416
Acknowledgments
This research was supported by projects 60972095, 61271362, and 61671362 of the National Natural Science Foundation of China.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interests
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Xiao, Q. Recurrent neural network system using probability graph model optimization. Appl Intell 46, 889–897 (2017). https://doi.org/10.1007/s10489-016-0879-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-016-0879-7