Weak Sensitivity to Initial Conditions for Generating Temporal Patterns in Recurrent Neural Networks: A Reservoir Computing Approach
A function for generating temporal patterns such as melody of the music and motor commands for body movements is one of major roles in the brain. In this paper, we study how such temporal patterns can be generated from nonlinear dynamics of recurrent neural networks (RNNs) and clarify the hidden mechanism that supports the functional ability of RNNs from reservoir computing (RC) approach. We show that when the reservoir (random recurrent neural network) shows weak instability to initial conditions, the error of the output from the reservoir and the target pattern is sufficiently small and robust to noise. It is also shown that the output from the spontaneous activity of the trained system intermittently exhibits response-like activity to the trigger input, which may be related to recent experimental findings in the neuroscience.
KeywordsSpontaneous Activity Recurrent Neural Network Recursive Little Square Target Pattern Trigger Pulse
Unable to display preview. Download preview PDF.
- 1.Tsuda, I.: Behavioral and Brain Sciences 24, 793–810 (2001)Google Scholar
- 2.Sompolinsky, H., Crisanti, A., Sommers, H.J.: Phys. Rev. Lett. 61, 259 (1988)Google Scholar
- 3.Maas, W., Natschläger, T., Markam, H.: Neural Comp. 14, 2351 (2002)Google Scholar
- 4.Jaeger, H., Haas, H.: Science 304, 78 (2004)Google Scholar
- 5.Sussillo, D., Abbott, L.F.: Neuron 63, 544 (2009)Google Scholar
- 6.Costa, U.M.S., Lyra, M.L., Plastino, A.R., Tsallis, C.: Phys. Rev. E 56, 245 (1997)Google Scholar
- 7.Arieli, A., Sterkin, A., Grinvald, A., Aertsen, A.: Science 273, 1868 (1996)Google Scholar
- 8.Kenet, T., Bibitchko, D., Tsodyks, M., Grinvald, A., Arieli, A.: Nature 425, 954 (2003)Google Scholar