Natural Computing

, 3:159 | Cite as

Finding independent components using spikes: A natural result of hebbian learning in a sparse spike coding scheme

  • Laurent Perrinet


As an alternative to classical representations in machine learning algorithms, we explore coding strategies using events as is observed for spiking neurons in the central nervous system. Focusing on visual processing, we have previously shown that we can define with an over-complete dictionary a sparse spike coding scheme by implementing lateral interactions that account for redundant information. Since this class of algorithms is both compatible with biological constraints and with neuro-physiological observations, it can provide a possible algorithm to explain the speed of visual processing despite the relatively slow time of response of single neurons. Here, I explore learning mechanisms to derive in an unsupervised manner an over-complete set of filters which provides a progressively sparser representation of the input. This work is based on a previous model of sparse coding from Olshausen et al. (1998) and the results leads to similar results, suggesting that this strategy provides a simple neural implementation of this algorithm and thus of Blind Source Separation. Moreover, this neuro-mimetic algorithm may be easily extended to realistic architectures of cortical columns in the primary visual cortex and we show results for different strategies of representation, leading to neuro-mimetic adaptive sparse spike coding schemes.

emergence hebbian learning natural images statistics over-complete representation parallel asynchronous processing sparse spike coding vision wavelet transform 


  1. Adrian E (1928) The Basis of Sensation: The Action of Sense Organs. Christophers, LondonGoogle Scholar
  2. Bell AJ and Sejnowski TJ (1997) The ‘independent components’ of natural scenes are edge filters. Vision Research 37(23): 3327–3338CrossRefGoogle Scholar
  3. Celebrini S, Thorpe SJ, Trotter Y and Imbert M (1993) Dynamics of orientation coding in area V1 of the awake primate. Vis Neurosci 5(10): 811–825CrossRefGoogle Scholar
  4. Chance F, Nelson SB and Abbott LF (1998) Synaptic depression and the temporal response characeristics of V1 simple cells. The Journal of Neuroscience 18: 4785–4799Google Scholar
  5. Comon P (1994) Independent Component analysis, a new concept? Signal Processing 36(3): 287–314zbMATHCrossRefGoogle Scholar
  6. Daugman J and Downing C (1995) Gabor Wavelets for Statistical Pattern Recognition, pp. 414–419. The MIT Press, Cambridge, MAGoogle Scholar
  7. Field DJ (1994) What is the goal of sensory coding? Neural Computation 6(4): 559–601MathSciNetGoogle Scholar
  8. Linsker R (1986) From basic network principles to neural architecture: emergence of spacial-opponent cells/orientation-selective cells/orientation columns. Proceedings of the National Academy of Sciences 83: 7508–7512, 8390–8394, 8779–8783CrossRefGoogle Scholar
  9. Mallat S (1998) A Wavelet Tour of Signal Processing. Academic PressGoogle Scholar
  10. Mallat S and Zhang Z (1993) Matching pursuit with time-frequency dictionaries. IEEE Transactions on Signal Processing 41(12): 3397–3414zbMATHCrossRefGoogle Scholar
  11. Mallat S and Zhong S (1992) Wavelet transform maxima and multiscale edgeGoogle Scholar
  12. Marr D (1982) Vision. W.H. Freeman and Company, NYGoogle Scholar
  13. Oja E (1982) A simplified neuron model as a principal component analyzer. J. Math. Biol. (15): 267–273zbMATHMathSciNetCrossRefGoogle Scholar
  14. Olshausen B and Field DJ (1996) Natural image statistics and efficient coding. Network 7: 333–339CrossRefGoogle Scholar
  15. Olshausen B and Field DJ (1998) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Research 37: 3311–3325CrossRefGoogle Scholar
  16. Olshausen B, Sallee P and Lewicki MS (1998) Learning sparse wavelet codes for natural images. In: Jordan MI, Kearns MJ and Solla SA (eds) Advances in Neural Information Processing Systems, Vol. 12. The MIT Press, Cambridge, MAGoogle Scholar
  17. Perrinet L and Samuelides M (2002a) Coherence detection in a spiking neuron via Hebbian learning. Neurocomputing 44–46(C): 133–139CrossRefGoogle Scholar
  18. Perrinet L and Samuelides M (2002b) Sparse image coding using an asynchronous spiking neural network. In: Proceedings of ESANN, pp. 313–318Google Scholar
  19. Perrinet L, Samuelides M and Thorpe S (2002) Sparse spike coding in an asynchronous feed-forward multi-layer neural network using Matching Pursuit. Neurocomputing, in pressGoogle Scholar
  20. Perrinet L, Samuelides M and Thorpe S (2003) Coding static natural images using spiking event times: do neurons cooperate?. SubmittedGoogle Scholar
  21. Rodieck RW (1965) Quantitative analysis of cat retinal ganglion cell response to visual stimuli. Vision Research 5: 583–601CrossRefGoogle Scholar
  22. Salinas E and Abbott L (2000) Do simple cells in primary visual cortex form a tight frame. Neural Computation 12(2): 313–335CrossRefGoogle Scholar
  23. Schwartz O and Simoncelli E (2001) Natural signal statistics and sensory gain control. Nature Neuroscience 4(8): 819–825CrossRefGoogle Scholar
  24. Thorpe SJ, Delorme A, Van Rullen R and Paquier W (2000) Reverse engineering of the visual system using networks of spiking neurons. In: IEEE International Symposium on Circuits and Systems, Vol. 4, pp. 405–408Google Scholar
  25. Thorpe SJ and Fabre-Thorpe M (2001) Neuroscience. Seeking categories in the brain. Science 291(5502): 260–263 Comment on: Science, 2001 Jan 12; 291(5502): 312–316CrossRefGoogle Scholar
  26. Thorpe SJ, Fize D and Marlot C (1996) Speed of processing in the human visual system. Nature 381: 520–522CrossRefGoogle Scholar
  27. Thorpe SJ and Gautrais J (1998) Rank order coding. In: Bower J (ed) Computational Neuroscience: Trends in Research 1998, pp. 113–118. Plenum Press, New YorkGoogle Scholar
  28. van Hateren J and van der Schaaf A (1998) Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B 265: 359–366CrossRefGoogle Scholar
  29. Van Rullen R and Thorpe SJ (2001) Rate coding versus temporal order coding: what the retina ganglion cells tell the visual cortex. Neural Computation 13(6): 1255–1283zbMATHCrossRefGoogle Scholar
  30. Vapnik V (1995) The Nature of Statistical Learning Theory. Springer, New YorkzbMATHGoogle Scholar
  31. Vinje WE and Gallant JL (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287: 1273–1276CrossRefGoogle Scholar
  32. Zibulevsky M and Pearlmutter BA (2001) Blind source separation by sparse decomposition in a signal dictionary. Neural Computation 13(4): 863–882zbMATHCrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Laurent Perrinet
    • 1
  1. 1.INCM-CNRS, 31, chemin Joseph AiguierMarseilleFrance (

Personalised recommendations