Abstract
As an alternative to classical representations in machine learning algorithms, we explore coding strategies using events as is observed for spiking neurons in the central nervous system. Focusing on visual processing, we have previously shown that we can define with an over-complete dictionary a sparse spike coding scheme by implementing lateral interactions that account for redundant information. Since this class of algorithms is both compatible with biological constraints and with neuro-physiological observations, it can provide a possible algorithm to explain the speed of visual processing despite the relatively slow time of response of single neurons. Here, I explore learning mechanisms to derive in an unsupervised manner an over-complete set of filters which provides a progressively sparser representation of the input. This work is based on a previous model of sparse coding from Olshausen et al. (1998) and the results leads to similar results, suggesting that this strategy provides a simple neural implementation of this algorithm and thus of Blind Source Separation. Moreover, this neuro-mimetic algorithm may be easily extended to realistic architectures of cortical columns in the primary visual cortex and we show results for different strategies of representation, leading to neuro-mimetic adaptive sparse spike coding schemes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adrian E (1928) The Basis of Sensation: The Action of Sense Organs. Christophers, London
Bell AJ and Sejnowski TJ (1997) The ‘independent components’ of natural scenes are edge filters. Vision Research 37(23): 3327–3338
Celebrini S, Thorpe SJ, Trotter Y and Imbert M (1993) Dynamics of orientation coding in area V1 of the awake primate. Vis Neurosci 5(10): 811–825
Chance F, Nelson SB and Abbott LF (1998) Synaptic depression and the temporal response characeristics of V1 simple cells. The Journal of Neuroscience 18: 4785–4799
Comon P (1994) Independent Component analysis, a new concept? Signal Processing 36(3): 287–314
Daugman J and Downing C (1995) Gabor Wavelets for Statistical Pattern Recognition, pp. 414–419. The MIT Press, Cambridge, MA
Field DJ (1994) What is the goal of sensory coding? Neural Computation 6(4): 559–601
Linsker R (1986) From basic network principles to neural architecture: emergence of spacial-opponent cells/orientation-selective cells/orientation columns. Proceedings of the National Academy of Sciences 83: 7508–7512, 8390–8394, 8779–8783
Mallat S (1998) A Wavelet Tour of Signal Processing. Academic Press
Mallat S and Zhang Z (1993) Matching pursuit with time-frequency dictionaries. IEEE Transactions on Signal Processing 41(12): 3397–3414
Mallat S and Zhong S (1992) Wavelet transform maxima and multiscale edge
Marr D (1982) Vision. W.H. Freeman and Company, NY
Oja E (1982) A simplified neuron model as a principal component analyzer. J. Math. Biol. (15): 267–273
Olshausen B and Field DJ (1996) Natural image statistics and efficient coding. Network 7: 333–339
Olshausen B and Field DJ (1998) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Research 37: 3311–3325
Olshausen B, Sallee P and Lewicki MS (1998) Learning sparse wavelet codes for natural images. In: Jordan MI, Kearns MJ and Solla SA (eds) Advances in Neural Information Processing Systems, Vol. 12. The MIT Press, Cambridge, MA
Perrinet L and Samuelides M (2002a) Coherence detection in a spiking neuron via Hebbian learning. Neurocomputing 44–46(C): 133–139
Perrinet L and Samuelides M (2002b) Sparse image coding using an asynchronous spiking neural network. In: Proceedings of ESANN, pp. 313–318
Perrinet L, Samuelides M and Thorpe S (2002) Sparse spike coding in an asynchronous feed-forward multi-layer neural network using Matching Pursuit. Neurocomputing, in press
Perrinet L, Samuelides M and Thorpe S (2003) Coding static natural images using spiking event times: do neurons cooperate?. Submitted
Rodieck RW (1965) Quantitative analysis of cat retinal ganglion cell response to visual stimuli. Vision Research 5: 583–601
Salinas E and Abbott L (2000) Do simple cells in primary visual cortex form a tight frame. Neural Computation 12(2): 313–335
Schwartz O and Simoncelli E (2001) Natural signal statistics and sensory gain control. Nature Neuroscience 4(8): 819–825
Thorpe SJ, Delorme A, Van Rullen R and Paquier W (2000) Reverse engineering of the visual system using networks of spiking neurons. In: IEEE International Symposium on Circuits and Systems, Vol. 4, pp. 405–408
Thorpe SJ and Fabre-Thorpe M (2001) Neuroscience. Seeking categories in the brain. Science 291(5502): 260–263 Comment on: Science, 2001 Jan 12; 291(5502): 312–316
Thorpe SJ, Fize D and Marlot C (1996) Speed of processing in the human visual system. Nature 381: 520–522
Thorpe SJ and Gautrais J (1998) Rank order coding. In: Bower J (ed) Computational Neuroscience: Trends in Research 1998, pp. 113–118. Plenum Press, New York
van Hateren J and van der Schaaf A (1998) Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B 265: 359–366
Van Rullen R and Thorpe SJ (2001) Rate coding versus temporal order coding: what the retina ganglion cells tell the visual cortex. Neural Computation 13(6): 1255–1283
Vapnik V (1995) The Nature of Statistical Learning Theory. Springer, New York
Vinje WE and Gallant JL (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287: 1273–1276
Zibulevsky M and Pearlmutter BA (2001) Blind source separation by sparse decomposition in a signal dictionary. Neural Computation 13(4): 863–882
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Perrinet, L. Finding independent components using spikes: A natural result of hebbian learning in a sparse spike coding scheme. Natural Computing 3, 159–175 (2004). https://doi.org/10.1023/B:NACO.0000027753.27593.a7
Issue Date:
DOI: https://doi.org/10.1023/B:NACO.0000027753.27593.a7