Abstract
We explore the role of local plasticity rules in learning statistical priors in a self-organizing recurrent neural network (SORN). The network receives input sequences composed of different symbols and learns the structure embedded in these sequences via a simple spike-timing-dependent plasticity rule, while synaptic normalization and intrinsic plasticity maintain a low level of activity. After learning, the network exhibits spontaneous activity that matches the stimulus-evoked activity during training and thus can be interpreted as samples from the network’s prior probability distribution over evoked activity states. Further, we show how learning the frequency and spatio-temporal characteristics of the input sequences influences network performance in several classification tasks. These results suggest a novel connection between low level learning mechanisms and high level concepts of statistical inference.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Fiser, J., Berkes, P., Orban, G., Lengyel, M.: Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14(3), 119–130 (2010)
Kver, H., Bao, S.: Cortical plasticity as a mechanism for storing bayesian priors in sensory perception. PLoS One 5(5), e10497 (2010)
Ringach, D.L.: Spontaneous and driven cortical activity: implications for computation. Curr. Opin. Neurobiol. 19(4), 439–444 (2009)
Han, F., Caporale, N., Dan, Y.: Reverberation of recent visual experience in spontaneous cortical waves. Neuron 60(2), 321–327 (2008)
Berkes, P., Orban, G.: Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331(6013), 83–87 (2011)
Lazar, A., Pipa, G., Triesch, J.: SORN: a self-organizing recurrent neural network. Front Comput. Neurosci. 3, 23 (2009)
Jaeger, H.: The ”echo state” approach to analysing and training recurrent neural networks. GMD Report 148, GMD - German National Research Institute for Computer Science (2001)
Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14, 2531–2560 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lazar, A., Pipa, G., Triesch, J. (2011). Emerging Bayesian Priors in a Self-Organizing Recurrent Network. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2011. ICANN 2011. Lecture Notes in Computer Science, vol 6792. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21738-8_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-21738-8_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21737-1
Online ISBN: 978-3-642-21738-8
eBook Packages: Computer ScienceComputer Science (R0)