Skip to main content

An Introduction to Delay-Coupled Reservoir Computing

  • Conference paper

Part of the Springer Series in Bio-/Neuroinformatics book series (SSBN,volume 4)

Abstract

Reservoir computing has been successfully applied in difficult time series prediction tasks by injecting an input signal into a spatially extended reservoir of nonlinear subunits to perform history-dependent nonlinear computation. Recently, the network was replaced by a single nonlinear node, delay-coupled to itself. Instead of a spatial topology, subunits are arrayed in time along one delay span of the system. As a result, the reservoir exists only implicitly in a single delay differential equation, the numerical solving of which is costly.We give here a brief introduction to the general topic of delay-coupled reservoir computing and derive approximate analytical equations for the reservoir by solving the underlying system explicitly. The analytical approximation represents the system accurately and yields comparable performance in reservoir benchmark tasks, while reducing computational costs practically by several orders of magnitude. This has important implications with respect to electronic realizations of the reservoir and opens up new possibilities for optimization and theoretical investigation.

Keywords

  • Delay Differential Equation
  • Predictive Distribution
  • Virtual Node
  • Gaussian Process Regression
  • Bayesian Model Selection

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-09903-3_4
  • Chapter length: 28 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-09903-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.99
Price excludes VAT (USA)
Hardcover Book
USD   299.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Albert, A.: The Penrose-Moore Pseudo Inverse with Diverse Statistical Applications. Part I. The General Theory and Computational Methods. Defense Technical Information Center (1971)

    Google Scholar 

  2. Appeltant, L., Soriano, M.C., Van der Sande, G., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C.R., Fischer, I.: Information processing using a single dynamical node as complex system. Nature Communications 2, 468 (2011)

    CrossRef  Google Scholar 

  3. Berger, J.O.: Statistical Decision Theory and Bayesian Analysis. Springer Series in Statistics. Springer (1985)

    Google Scholar 

  4. Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus (2006)

    Google Scholar 

  5. Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. Chapman & Hall, New York (1993)

    CrossRef  MATH  Google Scholar 

  6. Ganguli, S., Huh, D., Sompolinsky, H.: Memory traces in dynamical systems. Proceedings of the National Academy of Sciences 105(48), 18970–18975 (2008)

    CrossRef  Google Scholar 

  7. Guo, S., Wu, J.: Bifurcation Theory of Functional Differential Equations. Applied Mathematical Sciences. Springer London, Limited (2013)

    Google Scholar 

  8. Hastie, T.J., Tibshirani, R.J.: Generalized Additive Models. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Taylor & Francis (1990)

    Google Scholar 

  9. Heuser, H.: Lehrbuch der Analysis. Number pt. 1 in Mathematische Leitfäden. Teubner Verlag (2009)

    Google Scholar 

  10. Hida, T., Hitsuda, M.: Gaussian Processes. Translations of Mathematical Monographs. American Mathematical Society (2007)

    Google Scholar 

  11. Hoerl, A.E., Kennard, R.W.: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)

    CrossRef  MATH  MathSciNet  Google Scholar 

  12. Huebner, U., Abraham, N.B., Weiss, C.O.: Dimensions and entropies of chaotic intensity pulsations in a single-mode far-infrared nh3 laser. Phys. Rev. A 40(11), 6354–6365 (1989)

    CrossRef  Google Scholar 

  13. Jäger, H.: The echo state approach to analysing and training recurrent neural networks. Technical report (2001)

    Google Scholar 

  14. Jaynes, E.T., Bretthorst, G.L.: Probability Theory: The Logic of Science. Cambridge University Press (2003)

    Google Scholar 

  15. Konishi, S., Kitagawa, G.: Information Criteria and Statistical Modeling. Springer Series in Statistics. Springer (2008)

    Google Scholar 

  16. Larger, L., Soriano, M.C., Brunner, D., Appeltant, L., Gutierrez, J.M., Pesquera, L., Mirasso, C.R., Fischer, I.: Photonic information processing beyond turing: an optoelectronic implementation of reservoir computing. Opt. Express 20(3), 3241–3249 (2012)

    CrossRef  Google Scholar 

  17. Lazar, A., Pipa, G., Triesch, J.: SORN: a self-organizing recurrent neural network. Frontiers in Computational Neuroscience 3 (2009)

    Google Scholar 

  18. Lindley, D.V.: The 1988 wald memorial lectures: The present position in bayesian statistics. Statistical Science 5(1), 44–65 (1990)

    MATH  MathSciNet  Google Scholar 

  19. Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Computation 14(11), 2531–2560 (2002)

    CrossRef  MATH  Google Scholar 

  20. Quarteroni, A., Sacco, R., Saleri, F.: Numerical Mathematics, 2nd edn. Texts in Applied Mathematics, vol. 37. Springer, Berlin (2006)

    Google Scholar 

  21. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. Adaptative computation and machine learning series. University Press Group Limited (2006)

    Google Scholar 

  22. Rugh, W.J.: Nonlinear system theory: the Volterra/Wiener approach. Johns Hopkins series in information sciences and systems. Johns Hopkins University Press (1981)

    Google Scholar 

  23. Schrauwen, B., Buesing, L., Legenstein, R.A.: On computational power and the order-chaos phase transition in reservoir computing. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) NIPS, pp. 1425–1432. Curran Associates, Inc. (2008)

    Google Scholar 

  24. Shampine, L.F., Thompson, S.: Solving ddes in matlab. Applied Numerical Mathematics 37, 441–458 (2001)

    CrossRef  MATH  MathSciNet  Google Scholar 

  25. Smith, H.: An Introduction to Delay Differential Equations with Applications to the Life Sciences. Texts in Applied Mathematics. Springer (2010)

    Google Scholar 

  26. Soriano, M.C., Ortín, S., Brunner, D., Larger, L., Mirasso, C.R., Fischer, I., Pesquera, L.: Optoelectronic reservoir computing: tackling noise-induced performance degradation. Optics Express 21(1), 12–20 (2013)

    CrossRef  Google Scholar 

  27. Sundararajan, S., Sathiya Keerthi, S.: Predictive approaches for choosing hyperparameters in gaussian processes. Neural Computation 13(5), 1103–1118 (2001)

    CrossRef  MATH  Google Scholar 

  28. Toutounji, H., Schumacher, J., Pipa, G.: Optimized Temporal Multiplexing for Reservoir Computing with a Single Delay-Coupled Node. In: The 2012 International Symposium on Nonlinear Theory and its Applications (NOLTA 2012) (2012)

    Google Scholar 

  29. Weigend, A., Gershenfeld, N. (eds.): Time series prediction: forecasting the future and understanding the past. SFI studies in the sciences of complexity. Addison-Wesley (1993)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Johannes Schumacher .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Schumacher, J., Toutounji, H., Pipa, G. (2015). An Introduction to Delay-Coupled Reservoir Computing. In: Koprinkova-Hristova, P., Mladenov, V., Kasabov, N. (eds) Artificial Neural Networks. Springer Series in Bio-/Neuroinformatics, vol 4. Springer, Cham. https://doi.org/10.1007/978-3-319-09903-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-09903-3_4

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-09902-6

  • Online ISBN: 978-3-319-09903-3

  • eBook Packages: EngineeringEngineering (R0)