Journal of Computational Neuroscience

, Volume 32, Issue 3, pp 425–441 | Cite as

Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational model

  • Timothée Masquelier


We have built a phenomenological spiking model of the cat early visual system comprising the retina, the Lateral Geniculate Nucleus (LGN) and V1’s layer 4, and established four main results (1) When exposed to videos that reproduce with high fidelity what a cat experiences under natural conditions, adjacent Retinal Ganglion Cells (RGCs) have spike-time correlations at a short timescale (~30 ms), despite neuronal noise and possible jitter accumulation. (2) In accordance with recent experimental findings, the LGN filters out some noise. It thus increases the spike reliability and temporal precision, the sparsity, and, importantly, further decreases down to ~15 ms adjacent cells’ correlation timescale. (3) Downstream simple cells in V1’s layer 4, if equipped with Spike Timing-Dependent Plasticity (STDP), may detect these fine-scale cross-correlations, and thus connect principally to ON- and OFF-centre cells with Receptive Fields (RF) aligned in the visual space, and thereby become orientation selective, in accordance with Hubel and Wiesel (Journal of Physiology 160:106–154, 1962) classic model. Up to this point we dealt with continuous vision, and there was no absolute time reference such as a stimulus onset, yet information was encoded and decoded in the relative spike times. (4) We then simulated saccades to a static image and benchmarked relative spike time coding and time-to-first spike coding w.r.t. to saccade landing in the context of orientation representation. In both the retina and the LGN, relative spike times are more precise, less affected by pre-landing history and global contrast than absolute ones, and lead to robust contrast invariant orientation representations in V1.


Early visual system Continuous vision Saccades Spike time correlations STDP Neural coding 



Contrast Gain Control




Excitatory Post-Synaptic Potential


Inner Plexiform Layer


Inhibitory Post-Synaptic Potential


Lateral Geniculate Nucleus


Long Term Depression


Long Term Potentiation


Outer Plexiform Layer


Post-Stimulus Time Histogram


Receptive Field


Retinal Ganglion Cell


Spike Response Model


Spike Timing-Dependent Plasticity


primary visual cortex (a.k.a. area 17).



This research was supported by the Fyssen Foundation and the FP7 European Project Coronet. We thank Adrien Wohrer for having developed the Virtual Retina simulator (Wohrer and Kornprobst 2009), both user-friendly and highly configurable, and for the quality of his support. We also thank Wolfgang Einhäuser for kindly providing the videos used in (Betsch et al. 2004), and Mario Pannunzi for the numerous insightful brainstorms we had.

Supplementary material

10827_2011_361_MOESM1_ESM.avi (23 mb)
Esm 1 (AVI 23590 kb)
10827_2011_361_MOESM2_ESM.pdf (6 kb)
Esm 2 (PDF 5 kb)
10827_2011_361_MOESM3_ESM.pdf (6 kb)
Esm 3 (PDF 5 kb)


  1. Albrecht, D. G., Geisler, W. S., Frazor, R. A., & Crane, A. M. (2002). Visual cortex neurons of monkeys and cats: temporal dynamics of the contrast response function. Journal of Neurophysiology, 88(2), 888–913.PubMedGoogle Scholar
  2. Babadi, B., Casti, A., Xiao, Y., Kaplan, E., & Paninski, L. (2010). A generalized linear model of the impact of direct and indirect inputs to the lateral geniculate nucleus. Journal of Vision, 10(10), 22.PubMedCrossRefGoogle Scholar
  3. Bacon-Mace, N., Mace, M. J., Fabre-Thorpe, M., & Thorpe, S. J. (2005). The time course of visual processing: Backward masking and natural scene categorisation. Vision Research, 45(11), 1459–69.PubMedCrossRefGoogle Scholar
  4. Barlow, H. (1961). Possible principles underlying the transformation of sensory messages. In Sensory communication (pp. 217–234). Cambridge: MIT. wa rosenblith edition.Google Scholar
  5. Bell, A. J., & Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37(23), 3327–3338.PubMedCrossRefGoogle Scholar
  6. Berkes, P., & Wiskott, L. (2005). Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 5(6), 579–602.PubMedCrossRefGoogle Scholar
  7. Betsch, B., Einhäuser, W., Körding, K., & König, P. (2004). The world from a cat’s perspective—statistics of natural videos. Biological Cybernetics, 90(1), 41–50.PubMedCrossRefGoogle Scholar
  8. Brette, R., & Guigon, E. (2003). Reliability of spike timing is a general property of spiking model neurons. Neural Computation, 15(2), 279–308.PubMedCrossRefGoogle Scholar
  9. Butts, D. A., Weng, C., Jin, J., Yeh, C.-I., Lesica, N. A., Alonso, J.-M., et al. (2007). Temporal precision in the neural code and the timescales of natural vision. Nature, 449(7158), 92–95.PubMedCrossRefGoogle Scholar
  10. Cai, D., DeAngelis, G. C., & Freeman, R. D. (1997). Spatiotemporal receptive field organization in the lateral geniculate nucleus of cats and kittens. Journal of Neurophysiology, 78(2), 1045–1061.PubMedGoogle Scholar
  11. Caporale, N., & Dan, Y. (2008). Spike timing-dependent plasticity: a hebbian learning rule. Annual Review of Neuroscience, 31, 25–46.PubMedCrossRefGoogle Scholar
  12. Carandini, M., Horton, J. C., & Sincich, L. C. (2007). Thalamic filtering of retinal spike trains by postsynaptic summation. Journal of Vision, 7(14), 20.1–2011.CrossRefGoogle Scholar
  13. Celebrini, S., Thorpe, S., Trotter, Y., & Imbert, M. (1993). Dynamics of orientation coding in area V1 of the awake primate. Visual Neuroscience, 10(5), 811–825.PubMedCrossRefGoogle Scholar
  14. Chapman, B., Zahs, K. R., & Stryker, M. P. (1991). Relation of cortical cell orientation selectivity to alignment of receptive fields of the geniculocortical afferents that arborize within a single orientation column in ferret visual cortex. Journal of Neuroscience, 11(5), 1347–1358.PubMedGoogle Scholar
  15. Chase, S. M., & Young, E. D. (2007). First-spike latency information in single neurons increases when referenced to population onset. Proceedings of the National Academy of Sciences of the United States of America, 104(12), 5175–5180.PubMedCrossRefGoogle Scholar
  16. Chung, S., & Ferster, D. (1998). Strength and orientation tuning of the thalamic input to simple cells revealed by electrically evoked cortical suppression. Neuron, 20(6), 1177–1189.PubMedCrossRefGoogle Scholar
  17. Coppola, D., & Purves, D. (1996). The extraordinarily rapid disappearance of entopic images. Proceedings of the National Academy of Sciences of the United States of America, 93(15), 8001–8004.PubMedCrossRefGoogle Scholar
  18. Crouzet, S. M., Kirchner, H., & Thorpe, S. J. (2010). Fast saccades toward faces: face detection in just 100 ms. Journal of Vision, 10(4), 1–17.PubMedCrossRefGoogle Scholar
  19. Delorme, A., Perrinet, L., Thorpe, S., & Samuelides, M. (2001). Networks of integrate-and-fire neurons using rank order coding B: spike timing dependent plasticity and emergence of orientation selectivity. Neurocomputing, 38–40, 539–545.CrossRefGoogle Scholar
  20. Delorme, A., & Thorpe, S. J. (2001). Face identification using one spike per neuron: resistance to image degradations. Neural Networks, 14(6–7), 795–803.PubMedCrossRefGoogle Scholar
  21. Desbordes, G., Jin, J., Weng, C., Lesica, N. A., Stanley, G. B., & Alonso, J.-M. (2008). Timing precision in population coding of natural scenes in the early visual system. PLoS Biology, 6(12), e324.PubMedCrossRefGoogle Scholar
  22. Einhäuser, W., Kayser, C., König, P., & Körding, K. P. (2002). Learning the invariance properties of complex cells from their responses to natural stimuli. European Journal of Neuroscience, 15(3), 475–486.PubMedCrossRefGoogle Scholar
  23. Enroth-Cugell, C., Robson, J. G., Schweitzer-Tong, D. E., & Watson, A. B. (1983). Spatio-temporal interactions in cat retinal ganglion cells showing linear spatial summation. The Journal of Physiology, 341, 279–307.PubMedGoogle Scholar
  24. Fabre-Thorpe, M., Richard, G., & Thorpe, S. J. (1998). Rapid categorization of natural images by rhesus monkeys. NeuroReport, 9(2), 303–8.PubMedCrossRefGoogle Scholar
  25. Ferster, D., Chung, S., & Wheat, H. (1996). Orientation selectivity of thalamic input to simple cells of cat visual cortex. Nature, 380(6571), 249–252.PubMedCrossRefGoogle Scholar
  26. Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3, 194–200.CrossRefGoogle Scholar
  27. Fukushima, K. (1980). Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202.PubMedCrossRefGoogle Scholar
  28. Gawne, T., Kjaer, T., & Richmond, B. (1996). Latency: another potential code for feature binding in striate cortex. Journal of Neurophysiology, 76(2), 1356–1360.PubMedGoogle Scholar
  29. Gerstner, W., Ritz, R., & van Hemmen, J. L. (1993). Why spikes? hebbian learning and retrieval of time-resolved excitation patterns. Biological Cybernetics, 69(5–6), 503–515.PubMedGoogle Scholar
  30. Gilson, M., Masquelier, T., & Hugues, E. (2011). STDP allows fast rate-modulated coding with Poisson-like spike trains. PLoS Computational Biology (in press).Google Scholar
  31. Girard, P., Jouffrais, C., & Kirchner, C. H. (2008). Ultra-rapid categorisation in non-human primates. Animal Cognition, 11(3), 485–493.PubMedCrossRefGoogle Scholar
  32. Gollisch, T., & Meister, M. (2008). Rapid neural coding in the retina with relative spike latencies. Science, 319(5866), 1108–1111.PubMedCrossRefGoogle Scholar
  33. Guyonneau, R., VanRullen, R., & Thorpe, S. (2005). Neurons tune to the earliest spikes through STDP. Neural Computation, 17(4), 859–879.PubMedCrossRefGoogle Scholar
  34. Hubel, D., & Wiesel, T. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology, 160, 106–154.PubMedGoogle Scholar
  35. Hung, C., Kreiman, G., Poggio, T., & DiCarlo, J. (2005). Fast readout of object identity from macaque inferior temporal cortex. Science, 310(5749), 863–866.PubMedCrossRefGoogle Scholar
  36. Hyvärinen, A., & Hoyer, P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41(18), 2413–2423.PubMedCrossRefGoogle Scholar
  37. Johansson, R. S., & Birznieks, I. (2004). First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nature Neuroscience, 7(2), 170–177.PubMedCrossRefGoogle Scholar
  38. Kara, P., Reinagel, P., & Reid, R. C. (2000). Low response variability in simultaneously recorded retinal, thalamic, and cortical neurons. Neuron, 27(3), 635–646.PubMedCrossRefGoogle Scholar
  39. Keat, J., Reinagel, P., Reid, R. C., & Meister, M. (2001). Predicting every spike: a model for the responses of visual neurons. Neuron, 30(3), 803–817.PubMedCrossRefGoogle Scholar
  40. Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Physical Review E, 59(4), 4498–4514.CrossRefGoogle Scholar
  41. Kirchner, H., & Thorpe, S. (2006). Ultra-rapid object detection with saccadic eye movements: visual processing speed revisited. Vision Research, 46(11), 1762–1776.PubMedCrossRefGoogle Scholar
  42. König, P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neurosciences, 19(4), 130–7.PubMedCrossRefGoogle Scholar
  43. Körding, K., Kayser, C., Einhäuser, W., & König, P. (2004). How are complex cell properties adapted to the statistics of natural stimuli? Journal of Neurophysiology, 91(1), 206–212.PubMedCrossRefGoogle Scholar
  44. LeCun, Y., & Bengio, Y. (1998). Convolutional networks for images, speech, and time series. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. 255–258). Cambridge: MIT.Google Scholar
  45. Lichtsteiner, P., Posch, C., & Delbruck, T. (2007). An 128 × 128 120db 15us-latency temporal contrast vision sensor. IEEE Journal of Solid-State Circuits, 43(2), 566–576.CrossRefGoogle Scholar
  46. Liu, H., Agam, Y., Madsen, J. R., & Kreiman, G. (2009). Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron, 62(2), 281–290.PubMedCrossRefGoogle Scholar
  47. Martinez-Conde, S., Macknik, S. L., & Hubel, D. H. (2004). The role of fixational eye movements in visual perception. Nature Reviews Neuroscience, 5(3), 229–240.PubMedCrossRefGoogle Scholar
  48. Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2008). Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PloS One, 3(1), e1377.PubMedCrossRefGoogle Scholar
  49. Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2009). Competitive STDP-based spike pattern learning. Neural Computation, 21(5), 1259–1276.PubMedCrossRefGoogle Scholar
  50. Masquelier, T., Hugues, E., Deco, G., & Thorpe, S. J. (2009). Oscillations, phase-of-firing coding, and spike timing-dependent plasticity: an efficient learning scheme. Journal of Neuroscience, 29(43), 13484–13493.PubMedCrossRefGoogle Scholar
  51. Masquelier, T., Serre, T., Thorpe, S., & Poggio, T. (2007). Learning complex cell invariance from natural videos: a plausibility proof. Massachusetts Institute of Technology, CBCL Paper #269/MIT-CSAIL-TR #2007-060.Google Scholar
  52. Masquelier, T., & Thorpe, S. J. (2007). Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Computational Biology, 3(2), e31.PubMedCrossRefGoogle Scholar
  53. Miller, K. D., & MacKay, D. J. C. (1994). The role of constraints in hebbian learning. Neural Computation, 6, 100–126.CrossRefGoogle Scholar
  54. Montemurro, M. A., Rasch, M. J., Murayama, Y., Logothetis, N. K., & Panzeri, S. (2008). Phase-of-firing coding of natural visual stimuli in primary visual cortex. Current Biology, 18(5), 375–380.PubMedCrossRefGoogle Scholar
  55. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607–609.PubMedCrossRefGoogle Scholar
  56. Oram, M., & Perrett, D. (1992). Time course of neural responses discriminating different views of the face and head. Journal of Neurophysiology, 68(1), 70–84.PubMedGoogle Scholar
  57. Panzeri, S., & Diamond, M. E. (2010). Information carried by population spike times in the whisker sensory cortex can be decoded without knowledge of stimulus time. Frontiers in Synaptic Neuroscience, 2(17), 1–14.Google Scholar
  58. Puchalla, J. L., Schneidman, E., Harris, R. A., & Berry, M. J. (2005). Redundancy in the population code of the retina. Neuron, 46(3), 493–504.PubMedCrossRefGoogle Scholar
  59. Rathbun, D. L., Warland, D. K., & Usrey, W. M. (2010). Spike timing and information transmission at retinogeniculate synapses. Journal of Neuroscience, 30(41), 13558–13566.PubMedCrossRefGoogle Scholar
  60. Rehn, M., & Sommer, F. T. (2007). A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields. Journal of Computational Neuroscience, 22(2), 135–146.PubMedCrossRefGoogle Scholar
  61. Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2(11), 1019–1025.PubMedCrossRefGoogle Scholar
  62. Rolls, E., & Milward, T. (2000). A model of invariant object recognition in the visual system: learning rules, activation functions, lateral inhibition, and information-based performance measures. Neural Computation, 12(11), 2547–2572.PubMedCrossRefGoogle Scholar
  63. Rousselet, G. A., Fabre-Thorpe, M., & Thorpe, S. J. (2002). Parallel processing in high-level categorization of natural images. Nature Neuroscience, 5(7), 629–30.PubMedGoogle Scholar
  64. Serre, T., Oliva, A., & Poggio, T. (2007). A feedforward architecture accounts for rapid categorization. Proc. Nat. Acad. Sci. USA, 104(15).Google Scholar
  65. Singer, W., Tretter, F., & Cynader, M. (1975). Organization of cat striate cortex: a correlation of receptive-field properties with afferent and efferent connections. Journal of Neurophysiology, 38(5), 1080–1098.PubMedGoogle Scholar
  66. Song, S., Miller, K., & Abbott, L. (2000). Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3(9), 919–926.PubMedCrossRefGoogle Scholar
  67. Spratling, M. (2005). Learning viewpoint invariant perceptual representations from cluttered images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5).Google Scholar
  68. Stevenson, I. H., & Kording, K. P. (2011). How advances in neural recording affect data analysis. Nature Neuroscience, 14(2), 139–142.PubMedCrossRefGoogle Scholar
  69. Stewart, N., Brown, G. D. A., & Chater, N. (2005). Absolute identification by relative judgment. Psychological Review, 112(4), 881–911.PubMedCrossRefGoogle Scholar
  70. Stone, J. (1965). A quantitative analysis of the distribution of ganglion cells in the cat’s retina. The Journal of Comparative Neurology, 124(3), 337–352.PubMedCrossRefGoogle Scholar
  71. Stringer, S., & Rolls, E. (2000). Position invariant recognition in the visual system with cluttered environments. Neural Networks, 13(3), 305–315.PubMedCrossRefGoogle Scholar
  72. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381(6582), 520–2.PubMedCrossRefGoogle Scholar
  73. Thorpe, S., & Gautrais, J. (1998). Rank order coding. In J. M. Bower (Ed.), Computational neuroscience: Trends in research (pp. 113–118). New York: Plenum.CrossRefGoogle Scholar
  74. Thorpe, S., & Imbert, M. (1989). Biological constraints on connectionist modelling. In R. Pfeifer, Z. Schreter, F. Fogelman-Soulie, & L. Steels (Eds.), Connectionism in perspective (pp. 63–92). Amsterdam: Elsevier.Google Scholar
  75. Ullman, S., Vidal-Naquet, M., & Sali, E. (2002). Visual features of intermediate complexity and their use in classification. Nature Neuroscience, 5(7), 682–687.PubMedGoogle Scholar
  76. van Hateren, J. H., & Ruderman, D. L. (1998). Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proceedings. Biological sciences / The Royal Society, 265(1412), 2315–2320.PubMedCrossRefGoogle Scholar
  77. van Hateren, J. H., & van der Schaaf, A. (1998). Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings. Biological sciences / The Royal Society, 265(1394), 359–366.PubMedCrossRefGoogle Scholar
  78. van Rossum, M. C., Bi, G. Q., & Turrigiano, G. G. (2000). Stable hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience, 20(23), 8812–8821.PubMedGoogle Scholar
  79. VanRullen, R., Gautrais, J., Delorme, A., & Thorpe, S. (1998). Face processing using one spike per neurone. Biosystems, 48(1–3), 229–239.CrossRefGoogle Scholar
  80. VanRullen, R., & Thorpe, S. (2001). Rate coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex. Neural Computation, 13(6), 1255–1283.CrossRefGoogle Scholar
  81. VanRullen, R., & Thorpe, S. (2002). Surfing a spike wave down the ventral stream. Vision Research, 42(23), 2593–2615.PubMedCrossRefGoogle Scholar
  82. Vinje, W. E., & Gallant, J. L. (2000). Sparse coding and decorrelation in primary visual cortex during natural vision. Science, 287(5456), 1273–1276.PubMedCrossRefGoogle Scholar
  83. Wallis, G., & Rolls, E. (1997). Invariant face and object recognition in the visual system. Progress in Neurobiology, 51(2), 167–194.PubMedCrossRefGoogle Scholar
  84. Williams, P. E., Mechler, F., Gordon, J., Shapley, R., & Hawken, M. J. (2004). Entrainment to video displays in primary visual cortex of macaque and humans. Journal of Neuroscience, 24(38), 8278–8288.PubMedCrossRefGoogle Scholar
  85. Wilson, J. R., & Sherman, S. M. (1976). Receptive-field characteristics of neurons in cat striate cortex: Changes with visual field eccentricity. Journal of Neurophysiology, 39(3), 512–533.PubMedGoogle Scholar
  86. Wiskott, L., & Sejnowski, T. J. (2002). Slow feature analysis: unsupervised learning of invariances. Neural Computation, 14(4), 715–770.PubMedCrossRefGoogle Scholar
  87. Wohrer, A. (2008). Model and large-scale simulator of a biological retina, with contrast gain control. PhD thesis, University of Nice-Sophia Antipolis.Google Scholar
  88. Wohrer, A., & Kornprobst, P. (2009). Virtual retina: a biological retina model and simulator, with contrast gain control. Journal of Computational Neuroscience, 26(2), 219–249.PubMedCrossRefGoogle Scholar
  89. Wörgötter, F., Nelle, E., Li, B., & Funke, K. (1998). The influence of corticofugal feedback on the temporal structure of visual responses of cat thalamic relay cells. The Journal of Physiology, 509(Pt 3), 797–815.PubMedCrossRefGoogle Scholar
  90. Zamarreño-Ramos, C., Camuñas-Mesa, L., Perez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., & Linares-Barranco, B. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosc.—Neuromorph. Eng., 5(26).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Unit for Brain and Cognition, Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations