Abstract
Reservoir computing (RC) has attracted a lot of attention in the field of machine learning because of its promising performance in a broad range of applications. However, it is difficult to implement standard RC in hardware. Reservoir computers with a single nonlinear neuron subject to delayed feedback (delay-based RC) allow efficient hardware implementation with similar performance to standard RC. We propose and study two different ways to build ensembles of delay-based RC with several delayed neurons (time-delay reservoirs): one using decoupled neurons and the other using coupled neurons through the feedback lines. In both cases, the outputs of the different neurons are linearly combined to solve some benchmark tasks. Simulation results show that these schemes achieve better performance than the single-neuron case. Moreover, the proposed architectures boost the RC processing speed with respect to the single-neuron case. Both schemes are found to be robust against small mismatches between delayed neuron parameters.
Similar content being viewed by others
References
Jaeger H, Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 2004;304:78–80.
Verstraeten D, Schrauwen B, D′Haene M, Stroobandt D. An experimental unification of reservoir computing methods. Neural Netw. 2007;20:391–403.
Rodan A, Tiño P. Minimum complexity echo state network. IEEE Trans Neural Netw. 2011;22:131–144.
Buteneers P, Verstraeten D, Van Nieuwenhuyse B, Stroobandt D, Raedt R, Vonck K, et al. Real-time detection of epileptic seizures in animal models using reservoir computing. Epilepsy Res. 2013;103(2-3): 124–134.
Meftah B, Lézoray O, Benyettou A. Novel approach using echo state networks for microscopic cellular image segmentation. Cogn Comput. 2016;8(2):237–245.
Scardapane S, Uncini A. Semi-supervised echo state networks for audio classification. Cognitive Computation, pp 1–11. 2016.
Lukos̃evic̃ius M, Jaeger H. Survey: reservoir computing approaches to recurrent neural network training. Comput Sci Rev. 2009;3(3):127–149.
Appeltant L, Soriano MC, Van der Sande G, Danckaert J, Dambre J, Schrauwen B, et al. Information processing using a single dynamical node as complex system. Nat Commun. 2011;2:468.
Soriano MC, Ortín S, Keuninckx L, Appeltant L, Danckaert J, Pesquera L, et al. Delay-based reservoir computing: noise effects in a combined analog and digital implementation. IEEE Transactions on Neural Networks and Learning Systems. 2015 Feb;26(2):388–393.
Larger L, Soriano MC, Brunner D, Appeltant L, Gutiérrez JM, Pesquera L, et al. Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. Opt Express. 2012;20:3241–3249.
Duport F, Schneider B, Smerieri A, Haelterman M, Massar S. All-optical reservoir computing. Opt Express. 2012;20(20):22783–22795.
Brunner D, Soriano MC, Mirasso C, Fischer I. Parallel photonic information processing at gigabyte per second data rates using transient states. Nat Commun. 2013;4:1364.
Duport F, Smerieri A, Akrout A, Haelterman M, Massar S. Fully analogue photonic reservoir computer. Sci Report. 2016;6:22381.
Ortín S, Pesquera L, Gutiérrez JM. In: Gilbert T, Kirkilionis M, and Nicolis G, editors. Memory and nonlinear mapping in reservoir computing with two uncoupled nonlinear delay nodes. Springer International Publishing; 2013. p. 895–899.
Grigoryeva L, Henriques J, Larger L, Ortega JP. Stochastic nonlinear time series forecasting using time-delay reservoir computers: performance and universality. Neural Netw. 2014;55:59–71.
Cui H, Feng C, Chai Y, Liu RP, Liu Y. Effect of hybrid circle reservoir injected with wavelet-neurons on performance of echo state network. Neural Netw. 2014;57(0):141–151.
Wang S, Yang XJ, Wei CJ. Harnessing non-linearity by Sigmoid-wavelet Hybrid Echo State Networks (SWHESN). 2006 6th World Congress on Intelligent Control and Automation; 2006. p. 3014–3018.
Jaeger H. Short term memory in echo state networks. Technical Report GMD Report 152, German National Research Center for Information Technology. 2001.
Paquot Y, Duport F, Smerieri A, Dambre J, Schrauwen B, Haelterman M, et al. Optoelectronic reservoir computing. Scientific Reports. 2. 2012.
Soriano MC, Ortín S, Brunner D, Larger L, Mirasso CR, Fischer I, et al. Optoelectronic reservoir computing: tackling noise-induced performance degradation. Opt Express. 2013;21(1):12–20.
Weigend AS, Gershenfeld NA. Time series prediction: forecasting the future and understanding the past. vol. 80 Addison-Wesley. 1993.
Ortín S, Soriano M, Pesquera L, Brunner D, San-Martín D, Fischer I, et al. A unified framework for reservoir computing and extreme learning machines based on a single time-delayed neuron. Sci Report. 2015;5:14945.
Xue Y, Yang L, Haykin S. Decoupled echo state networks with lateral inhibition. Neural Netw. 2007;20(3):365–376.
Acknowledgements
We would like to thank J.M. Gutierrez for the helpful comments.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Funding
This work has been funding by the Ministerio de Economía y Competitividad (MINECO/FEDER, UE), Spain under project TEC2015-65212-C3-1-P. Silvia Ortín was supported by the Conselleria d’Innovació, Recerca i Turisme del Govern de les Illes Balears and the European Social Fund.
Ethical Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Rights and permissions
About this article
Cite this article
Ortín, S., Pesquera, L. Reservoir Computing with an Ensemble of Time-Delay Reservoirs. Cogn Comput 9, 327–336 (2017). https://doi.org/10.1007/s12559-017-9463-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-017-9463-7