Skip to main content

Emerging Bayesian Priors in a Self-Organizing Recurrent Network

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 6792)

Abstract

We explore the role of local plasticity rules in learning statistical priors in a self-organizing recurrent neural network (SORN). The network receives input sequences composed of different symbols and learns the structure embedded in these sequences via a simple spike-timing-dependent plasticity rule, while synaptic normalization and intrinsic plasticity maintain a low level of activity. After learning, the network exhibits spontaneous activity that matches the stimulus-evoked activity during training and thus can be interpreted as samples from the network’s prior probability distribution over evoked activity states. Further, we show how learning the frequency and spatio-temporal characteristics of the input sequences influences network performance in several classification tasks. These results suggest a novel connection between low level learning mechanisms and high level concepts of statistical inference.

Keywords

  • Spontaneous activity
  • statistical priors
  • Bayesian inference
  • STDP
  • intrinsic plasticity
  • recurrent networks

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-21738-8_17
  • Chapter length: 8 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-21738-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Fiser, J., Berkes, P., Orban, G., Lengyel, M.: Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14(3), 119–130 (2010)

    CrossRef  Google Scholar 

  2. Kver, H., Bao, S.: Cortical plasticity as a mechanism for storing bayesian priors in sensory perception. PLoS One 5(5), e10497 (2010)

    Google Scholar 

  3. Ringach, D.L.: Spontaneous and driven cortical activity: implications for computation. Curr. Opin. Neurobiol. 19(4), 439–444 (2009)

    CrossRef  Google Scholar 

  4. Han, F., Caporale, N., Dan, Y.: Reverberation of recent visual experience in spontaneous cortical waves. Neuron 60(2), 321–327 (2008)

    CrossRef  Google Scholar 

  5. Berkes, P., Orban, G.: Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331(6013), 83–87 (2011)

    CrossRef  Google Scholar 

  6. Lazar, A., Pipa, G., Triesch, J.: SORN: a self-organizing recurrent neural network. Front Comput. Neurosci. 3, 23 (2009)

    CrossRef  Google Scholar 

  7. Jaeger, H.: The ”echo state” approach to analysing and training recurrent neural networks. GMD Report 148, GMD - German National Research Institute for Computer Science (2001)

    Google Scholar 

  8. Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14, 2531–2560 (2002)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Lazar, A., Pipa, G., Triesch, J. (2011). Emerging Bayesian Priors in a Self-Organizing Recurrent Network. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2011. ICANN 2011. Lecture Notes in Computer Science, vol 6792. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21738-8_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-21738-8_17

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-21737-1

  • Online ISBN: 978-3-642-21738-8

  • eBook Packages: Computer ScienceComputer Science (R0)