Computational Performance of Echo State Networks with Dynamic Synapses

  • Ryota Mori
  • Gouhei Tanaka
  • Ryosho Nakane
  • Akira Hirose
  • Kazuyuki Aihara
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9947)

Abstract

The echo state network is a framework for temporal data processing, such as recognition, identification, classification and prediction. The echo state network generates spatiotemporal dynamics reflecting the history of an input sequence in the dynamical reservoir and constructs mapping from the input sequence to the output one in the readout. In the conventional dynamical reservoir consisting of sparsely connected neuron units, more neurons are required to create more time delay. In this study, we introduce the dynamic synapses into the dynamical reservoir for controlling the nonlinearity and the time constant. We apply the echo state network with dynamic synapses to several benchmark tasks. The results show that the dynamic synapses are effective for improving the performance in time series prediction tasks.

Keywords

Echo state networks Reservoir computing Dynamic synapses Short-term synaptic plasticity Time series prediction Recurrent neural networks 

References

  1. 1.
    Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks. Technical Report 148, GMD - German National Research Institute for Computer Science (2001)Google Scholar
  2. 2.
    Jaeger, H.: Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the “echo state network” approach. GMD-Forschungszentrum Informationstechnik (2002)Google Scholar
  3. 3.
    Markram, H., Tsodyks, M.: Redistribution of synaptic efficacy between neocortical pyramidal neurons. Nature 382, 807–810 (1996)CrossRefGoogle Scholar
  4. 4.
    Mongillo, G., Barak, O., Tsodyks, M.: Synaptic theory of working memory. Science 319, 1543 (2008)CrossRefGoogle Scholar
  5. 5.
    Tsodyks, M., Markram, H.: The neural code between neocortial pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. USA 94, 719–723 (1997)CrossRefGoogle Scholar
  6. 6.
    Tsodyks, M., Markram, H.: Differential signaling via the same axon of neocortical pyramidal neurons. Proc. Natl. Acad. Sci. USA 95, 5323–5328 (1998)CrossRefGoogle Scholar
  7. 7.
    Jaeger, H.: Short term memory in echo state networks. GMD-Report 152, German National Research Institute for Computer Science (2002)Google Scholar
  8. 8.
    Goudarzi, A., Banda, P., Lakin, M.R., Teuscher, C., Stefanovic, D.: A Comparative Study of Reservoir Computing for Tenporal Signal Processing, arXiv:1401.2224v1 [cs.NE] (2014)
  9. 9.
    Schrauwen, B., Verstraeten, D., Van Campenhout, J.: An overview of reservoir computing: theory, applications and implementations. In: Proceedings of the 15th European Symposium on Articial Neural Networks, pp. 471–482 (2007)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Ryota Mori
    • 1
  • Gouhei Tanaka
    • 1
    • 2
    • 3
  • Ryosho Nakane
    • 2
  • Akira Hirose
    • 2
  • Kazuyuki Aihara
    • 1
    • 2
    • 3
  1. 1.Graduate School of Information Science and TechnologyThe University of TokyoTokyoJapan
  2. 2.Graduate School of EngineeringThe University of TokyoTokyoJapan
  3. 3.Institue of Industrial ScienceThe University of TokyoTokyoJapan

Personalised recommendations