Stability and Topology in Reservoir Computing
Recently Jaeger and others have put forth the paradigm of "reservoir computing" as a way of computing with highly recurrent neural networks. This reservoir is a collection of neurons randomly connected with each other of fixed weights. Amongst other things, it has been shown to be effective in temporal pattern recognition; and has been held as a model appropriate to explain how certain aspects of the brain work. (Particularly in its guise as “liquid state machine”, due to Maass et al.) In this work we show that although it is known that this model does have generalizability properties and thus is robust to errors in input, it is NOT resistant to errors in the model itself. Thus small malfunctions or distortions make previous training ineffective. Thus this model as currently presented cannot be thought of as appropriate as a biological model; and it also suggests limitations on the applicability in the pattern recognition sphere. However, we show that, with the enforcement of topological constraints on the reservoir, in particular that of small world topology, the model is indeed fault tolerant. Thus this implies that "natural" computational systems must have specific topologies and the uniform random connectivity is not appropriate.
KeywordsReservoir Computing Small world topology robustness Machine Learning
Unable to display preview. Download preview PDF.
- Albert, R., Barabási, A.-L.: Topology of evolving networks: local events and universality, vol. 85, pp. 5234–5237 (2000)Google Scholar
- Chklovskii, L.R.: Structural Properties of the Caenorhabditis elegans Neuronal Network (July 2009)Google Scholar
- Jaeger, H.: "The ëcho state" approach to analysing and training recurrent neural networks. German National Research Institute for Computer Science (2001)Google Scholar
- Jaeger, H.: The "echo state" approach to analysing and training recurrent neural networks. German National Research Institute for Computer Science (2001)Google Scholar
- Lukosevicius, M., Jaeger, H.: Reservoir Computing Approaches to Recurrent Neural Network Training. Computer Scinece Review, 127–149 (2009)Google Scholar
- Maass, W.: Paradigms for computing with Spiking Neurons. Springer, Heidelberg (1999)Google Scholar
- Maass, W., Natschlger, T., Markram, H.: Computational models for generic cortical microcircuits. In: Feng, J. (ed.) Computational Neuroscience: A Comprehensive Approach, pp. 575–605. Chapman & Hall/CRC, Boca Raton (2004)Google Scholar
- Widrow, B., Hoff, M.: Adaptive Switching Circuits. IRE WESCON Convention Record 4, 96–104 (1960)Google Scholar