Journal of Computational Neuroscience

, Volume 40, Issue 3, pp 317–329 | Cite as

Slow feature analysis with spiking neurons and its application to audio stimuli

  • Guillaume Bellec
  • Mathieu Galtier
  • Romain Brette
  • Pierre Yger
Article

Abstract

Extracting invariant features in an unsupervised manner is crucial to perform complex computation such as object recognition, analyzing music or understanding speech. While various algorithms have been proposed to perform such a task, Slow Feature Analysis (SFA) uses time as a means of detecting those invariants, extracting the slowly time-varying components in the input signals. In this work, we address the question of how such an algorithm can be implemented by neurons, and apply it in the context of audio stimuli. We propose a projected gradient implementation of SFA that can be adapted to a Hebbian like learning rule dealing with biologically plausible neuron models. Furthermore, we show that a Spike-Timing Dependent Plasticity learning rule, shaped as a smoothed second derivative, implements SFA for spiking neurons. The theory is supported by numerical simulations, and to illustrate a simple use of SFA, we have applied it to auditory signals. We show that a single SFA neuron can learn to extract the tempo in sound recordings.

Keywords

unsupervised learning plasticity slow feature analysis 

References

  1. Becker, S., & Hinton, G.E. (1992). Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature, 355, 161–163.CrossRefPubMedGoogle Scholar
  2. Berkes, P., & Wiskott, L. (2005). Slow Feature Analysis yields a rich repertoire of complex cell properties. Journal of Vision, 5(6), 579–602.CrossRefPubMedGoogle Scholar
  3. Bi, G.Q., & Poo, M.M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18, 10464–10472.PubMedGoogle Scholar
  4. Bliss, T.V.P., & Lomo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the preforant path. Journal of physiology, 232, 331–356.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Clopath, C., Büsing, L., Vasilaki, E., & Gerstner, W. (2010). Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nature neuroscience, 13(3), 344–52.CrossRefPubMedGoogle Scholar
  6. Dähne, S., Wilbert, N., & Wiskott, L. (2014). Slow Feature Analysis on retinal waves leads to V1 complex cells PLos computational biology 10(5):e1003564.Google Scholar
  7. Dan, Y., Atick, J.J., & Reid, R.C. (1996). Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. The Journal of neuroscience, 16, 3351–62.PubMedGoogle Scholar
  8. Dan, Y., & Poo, M. (2004). Spike Timing-Dependent plasticity of neural circuits. Neuron, 44(1), 23–30.CrossRefPubMedGoogle Scholar
  9. DiCarlo, J.J., Zoccolan, D., & Rust, N.C. (2012). How Does the Brain Solve Visual Object Recognition? Neuron, 73(3), 415–434.CrossRefPubMedPubMedCentralGoogle Scholar
  10. Einhäuser, W., Hipp, J., Eggert, J., Körner, E., & König, P. (2005). Learning viewpoint invariant object representations using a temporal coherence principle. Biological Cybernetics, 93, 79–90.CrossRefPubMedGoogle Scholar
  11. Falconbridge, M.S., Stamps, R.L., & Badcock, D.R. (2006). A simple Hebbian/anti-Hebbian network learns the sparse, independent components of natural images. Neural Computation, 18(2), 415–429.CrossRefPubMedGoogle Scholar
  12. Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 200, 194–200.CrossRefGoogle Scholar
  13. Franzius, M., Sprekeler, H., & Wiskott, L. (2007). Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Computational Biology, 3(8), e166.CrossRefPubMedPubMedCentralGoogle Scholar
  14. Galtier, M., & Wainrib, G. (2012). Multiscale analysis of slow-fast neuronal learning models with noise. Journal of Mathematical Neuroscience, 2, 13.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Mathieu, N. (2013). Galtier and Gilles Wainrib. A biological gradient descent for prediction through a combination of stdp and homeostatic plasticity. Neural computation, 25(11), 2815–2832.CrossRefGoogle Scholar
  16. Gerstner, W., Kempter, R., Leo van Hemmen, J., Wagner, H., & Van Hemmen, J.L. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383(2), 76–78.CrossRefPubMedGoogle Scholar
  17. Gibson, J.J. (1986). The Ecological Approach to Visual Perception.Google Scholar
  18. Goodman, D, & Brian, R.B. (2008). A simulator for spiking neural networks in python. Frontiers in Neuroinformatics, 2.Google Scholar
  19. Hebb, D.O.O. (1949). The organization of behavior: a neuropsychological theory. Science Education, 44, 335.Google Scholar
  20. Izhikevich, E.M., & Desai, N.S. (2003). Relating STDP to BCM. Neural Computation, 15(7), 1511–1523.CrossRefPubMedGoogle Scholar
  21. Kayser, C., Einhäuser, W., Dümmer, O., König, P., & Körding, K.P. (2001). Extracting slow subspaces from natural videos leads to complex cells. In Artificial Neural Networks - ICANN 2001, 2130, 1075–1080.CrossRefGoogle Scholar
  22. Kempter, R., Gerstner, W., & Leo Van Hemmen, J. (1999). Hebbian learning and spiking neurons. Physical Review E, 59(4), 4498.CrossRefGoogle Scholar
  23. Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275(5297), 213–5.CrossRefPubMedGoogle Scholar
  24. Marr, D. (1970). A theory for cerebral neocortex. Proceedings of the Royal Society of London. Series B, 176, 161–234.Google Scholar
  25. Oja, E. (1982). Simplified neuron model as a principal component analyzer. Journal of mathematical biology, 15(3), 267– 273.CrossRefPubMedGoogle Scholar
  26. Pozzorini, C., Naud, R., Mensi, S., & Gerstner, W. (2013). Temporal whitening by power-law adaptation in neocortical neurons. Nature neuroscience, 16(7), 942–8.CrossRefPubMedGoogle Scholar
  27. Quian Quiroga, R., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005). Invariant visual representation by single neurons in the human brain. Nature, 435(7045), 1102–7.CrossRefPubMedGoogle Scholar
  28. Rao, R.P., & Sejnowski, T.J. (2001). Spike-timing-dependent Hebbian plasticity as temporal difference learning. Neural Computation, 13(10), 2221–2237.CrossRefPubMedGoogle Scholar
  29. Sprekeler, H., Michaelis, C., & Slowness, L.W. (2007). An objective for Spike-Timing-Dependent Plasticity?. PLoS Computational Biology, 3(6), e112.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Stevenson, I.H., Cronin, B., Sur, M., & Kording, K.P. (2010). Sensory adaptation and short term plasticity as bayesian correction for a changing brain, Vol. 5.Google Scholar
  31. Toyoizumi, T., Pfister, J., Aihara, K., & gerstne, W (2007). optiMality model of unsupervised Spike-Timing-Dependent plasticity synaptic memory and weight distribution. Neural Computation, 19(3), 639–671.CrossRefPubMedGoogle Scholar
  32. Turner, R., & Sahani, M. (2007). A maximum-likelihood interpretation for Slow Feature Analysis. Neural computation, 19(4), 1022–38.CrossRefPubMedGoogle Scholar
  33. Wallis, G., & Rolls, E.T. (1997). Invariant face and object recognition in the visual system. Progress in neurobiology, 51, 167–194.CrossRefPubMedGoogle Scholar
  34. Wiskott, L., & Berkes, P. (2003). Is slowness a learning principle of the visual cortex, (Vol. 106.Google Scholar
  35. Wiskott, L., & Sejnowski, T.J. (2002). Slow Feature Analysis: unsupervised learning of invariances. Neural Computation, 14(4), 715–770.CrossRefPubMedGoogle Scholar
  36. Yger, Pierre, Boustani, S., Frégnac, Y., Destexhe, A., & El Boustani, S. (2012). Stable learning in stochastic network states. Journal of neuroscience, 32(1), 194–214.CrossRefPubMedGoogle Scholar
  37. Yger, P., & Harris, K.D. (2013). The Convallis rule for unsupervised learning in cortical networks. PLoS Computational Biology, 9(10), 1–32.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Guillaume Bellec
    • 1
    • 2
    • 3
  • Mathieu Galtier
    • 5
  • Romain Brette
    • 1
    • 2
    • 3
  • Pierre Yger
    • 1
    • 2
    • 3
    • 4
  1. 1.Institut de la Vision, Sorbonne Université, UPMC Univ Paris06 UMRS968ParisFrance
  2. 2.INSERMParisFrance
  3. 3.CNRSParisFrance
  4. 4.Institut d’Etudes de la CognitionENSParisFrance
  5. 5.European Institute for Theoretical Neuroscience CNRS UNIC UPR-3293ParisFrance

Personalised recommendations