Skip to main content
Log in

Reservoir Computing with an Ensemble of Time-Delay Reservoirs

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Reservoir computing (RC) has attracted a lot of attention in the field of machine learning because of its promising performance in a broad range of applications. However, it is difficult to implement standard RC in hardware. Reservoir computers with a single nonlinear neuron subject to delayed feedback (delay-based RC) allow efficient hardware implementation with similar performance to standard RC. We propose and study two different ways to build ensembles of delay-based RC with several delayed neurons (time-delay reservoirs): one using decoupled neurons and the other using coupled neurons through the feedback lines. In both cases, the outputs of the different neurons are linearly combined to solve some benchmark tasks. Simulation results show that these schemes achieve better performance than the single-neuron case. Moreover, the proposed architectures boost the RC processing speed with respect to the single-neuron case. Both schemes are found to be robust against small mismatches between delayed neuron parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Jaeger H, Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 2004;304:78–80.

    Article  CAS  PubMed  Google Scholar 

  2. Verstraeten D, Schrauwen B, D′Haene M, Stroobandt D. An experimental unification of reservoir computing methods. Neural Netw. 2007;20:391–403.

    Article  CAS  PubMed  Google Scholar 

  3. Rodan A, Tiño P. Minimum complexity echo state network. IEEE Trans Neural Netw. 2011;22:131–144.

    Article  PubMed  Google Scholar 

  4. Buteneers P, Verstraeten D, Van Nieuwenhuyse B, Stroobandt D, Raedt R, Vonck K, et al. Real-time detection of epileptic seizures in animal models using reservoir computing. Epilepsy Res. 2013;103(2-3): 124–134.

    Article  PubMed  Google Scholar 

  5. Meftah B, Lézoray O, Benyettou A. Novel approach using echo state networks for microscopic cellular image segmentation. Cogn Comput. 2016;8(2):237–245.

    Article  Google Scholar 

  6. Scardapane S, Uncini A. Semi-supervised echo state networks for audio classification. Cognitive Computation, pp 1–11. 2016.

  7. Lukos̃evic̃ius M, Jaeger H. Survey: reservoir computing approaches to recurrent neural network training. Comput Sci Rev. 2009;3(3):127–149.

    Article  Google Scholar 

  8. Appeltant L, Soriano MC, Van der Sande G, Danckaert J, Dambre J, Schrauwen B, et al. Information processing using a single dynamical node as complex system. Nat Commun. 2011;2:468.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Soriano MC, Ortín S, Keuninckx L, Appeltant L, Danckaert J, Pesquera L, et al. Delay-based reservoir computing: noise effects in a combined analog and digital implementation. IEEE Transactions on Neural Networks and Learning Systems. 2015 Feb;26(2):388–393.

    Article  PubMed  Google Scholar 

  10. Larger L, Soriano MC, Brunner D, Appeltant L, Gutiérrez JM, Pesquera L, et al. Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. Opt Express. 2012;20:3241–3249.

    Article  CAS  PubMed  Google Scholar 

  11. Duport F, Schneider B, Smerieri A, Haelterman M, Massar S. All-optical reservoir computing. Opt Express. 2012;20(20):22783–22795.

    Article  PubMed  Google Scholar 

  12. Brunner D, Soriano MC, Mirasso C, Fischer I. Parallel photonic information processing at gigabyte per second data rates using transient states. Nat Commun. 2013;4:1364.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Duport F, Smerieri A, Akrout A, Haelterman M, Massar S. Fully analogue photonic reservoir computer. Sci Report. 2016;6:22381.

    Article  CAS  Google Scholar 

  14. Ortín S, Pesquera L, Gutiérrez JM. In: Gilbert T, Kirkilionis M, and Nicolis G, editors. Memory and nonlinear mapping in reservoir computing with two uncoupled nonlinear delay nodes. Springer International Publishing; 2013. p. 895–899.

  15. Grigoryeva L, Henriques J, Larger L, Ortega JP. Stochastic nonlinear time series forecasting using time-delay reservoir computers: performance and universality. Neural Netw. 2014;55:59–71.

    Article  PubMed  Google Scholar 

  16. Cui H, Feng C, Chai Y, Liu RP, Liu Y. Effect of hybrid circle reservoir injected with wavelet-neurons on performance of echo state network. Neural Netw. 2014;57(0):141–151.

    Article  PubMed  Google Scholar 

  17. Wang S, Yang XJ, Wei CJ. Harnessing non-linearity by Sigmoid-wavelet Hybrid Echo State Networks (SWHESN). 2006 6th World Congress on Intelligent Control and Automation; 2006. p. 3014–3018.

  18. Jaeger H. Short term memory in echo state networks. Technical Report GMD Report 152, German National Research Center for Information Technology. 2001.

  19. Paquot Y, Duport F, Smerieri A, Dambre J, Schrauwen B, Haelterman M, et al. Optoelectronic reservoir computing. Scientific Reports. 2. 2012.

  20. Soriano MC, Ortín S, Brunner D, Larger L, Mirasso CR, Fischer I, et al. Optoelectronic reservoir computing: tackling noise-induced performance degradation. Opt Express. 2013;21(1):12–20.

    Article  CAS  PubMed  Google Scholar 

  21. Weigend AS, Gershenfeld NA. Time series prediction: forecasting the future and understanding the past. vol. 80 Addison-Wesley. 1993.

  22. Ortín S, Soriano M, Pesquera L, Brunner D, San-Martín D, Fischer I, et al. A unified framework for reservoir computing and extreme learning machines based on a single time-delayed neuron. Sci Report. 2015;5:14945.

    Article  Google Scholar 

  23. Xue Y, Yang L, Haykin S. Decoupled echo state networks with lateral inhibition. Neural Netw. 2007;20(3):365–376.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank J.M. Gutierrez for the helpful comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Silvia Ortín.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Funding

This work has been funding by the Ministerio de Economía y Competitividad (MINECO/FEDER, UE), Spain under project TEC2015-65212-C3-1-P. Silvia Ortín was supported by the Conselleria d’Innovació, Recerca i Turisme del Govern de les Illes Balears and the European Social Fund.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ortín, S., Pesquera, L. Reservoir Computing with an Ensemble of Time-Delay Reservoirs. Cogn Comput 9, 327–336 (2017). https://doi.org/10.1007/s12559-017-9463-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-017-9463-7

Keywords

Navigation