# A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields

- 590 Downloads
- 92 Citations

## Abstract

Computational models of primary visual cortex have demonstrated that principles of efficient coding and neuronal sparseness can explain the emergence of neurones with localised oriented receptive fields. Yet, existing models have failed to predict the diverse shapes of receptive fields that occur in nature. The existing models used a particular “soft” form of sparseness that limits average neuronal activity. Here we study models of efficient coding in a broader context by comparing soft and “hard” forms of neuronal sparseness.

As a result of our analyses, we propose a novel network model for visual cortex. The model forms efficient visual representations in which the number of active neurones, rather than mean neuronal activity, is limited. This form of hard sparseness also economises cortical resources like synaptic memory and metabolic energy. Furthermore, our model accurately predicts the distribution of receptive field shapes found in the primary visual cortex of cat and monkey.

## Keywords

Biological vision Sparse coding Receptive field learning## Notes

### Acknowledgment

We thank B. A. Olshausen for providing ideas, D. Ringach for the experimental data, T. Bell, J. Culpepper, G. Hinton, J. Hirsch, C. v. d. Malsburg, B. Mel, L. Perrinet, D. Warland, L. Zhaoping and J. Zhu for discussions. Further, we thank the reviewers for helpful suggestions. Financial support was provided by the Strauss-Hawkins trust and KTH.

## References

- Atick JJ (1992) Could information theory provide an ecological theory of sensory processing? Network: Comput. Neural Syst. 3: 213–251.Google Scholar
- Attneave F (1954) Some informational aspects of visual perception. Psychol. Rev. 61: 183–193.Google Scholar
- Baddeley R (1996) An efficient code in V1? Nature 381: 560–561.Google Scholar
- Barlow HB (1983) Understanding natural vision. Springer-Verlag, Berlin.Google Scholar
- Baum EB, Moody J, Wilczek F (1988) Internal representations for associative memory. Biol. Cybern. 59: 217–228.Google Scholar
- Bell AJ, Sejnowski TJ (1997) The independent components of natural images are edge filters. Vis. Res. 37: 3327–3338.Google Scholar
- Bethge M, Rotermund D, Pawelzik K (2003) Second order phase transition in neural rate coding: binary encoding is optimal for rapid signal transmission. Phys. Rev. Lett. 90(8): 088104–0881044.Google Scholar
- Buhmann J, Schulten K (1988) Storing sequences of biased patterns in neural networks with stochastic dynamics. In R Eckmiller, C von der Malsburg, eds. Neural Computers, Berlin. (Neuss 1987), Springer-Verlag, pp. 231–242.Google Scholar
- Chen SS, Donoho DL, Saunders MA (1998) Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1):33–61.Google Scholar
- Dayan P, Abbott L (2003) Theoretical Neuroscience. MIT Press, Cambridge MA.Google Scholar
- Donoho DL, Elad M (2002) Optimally sparse representation in general (nonorthogonal) dictionaries via \(l^{1}\) minimization. PNAS 100(5): 2197–2002.Google Scholar
- Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4: 2379–2394.Google Scholar
- Földiak P (1995) Sparse coding in the primate cortex. In MA Arbib ed. The Handbook of Brain Theory and Neural Networks, MIT Press, Cambridge, MA, pp. 165–170.Google Scholar
- Gardner-Medwin A (1976) The recall of events through the learning of associations between their parts. Proc. R. Soc. London. B 194: 375–402.Google Scholar
- Gibson JJ (1966) The Perception of the Visual World. Houghton Mifflin, Boston, MA.Google Scholar
- Hinton GE, Sallans B, Ghahramani ZJ (1997) A hierarchical community of experts. In M Jordan, ed. Learning in Graphical Models, Kluwer Academic Publishers, pp. 479–494.Google Scholar
- Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. (USA) 79: 2554–2558.Google Scholar
- Hurri J, Hyvärinen A (2003) Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural. Comput. 15: 663–691.Google Scholar
- Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw. 13: 411–430.Google Scholar
- Jones JP, Palmer LA (1987) The two-dimensional spatial structure of simple receptive fields in the striate cortex. J. Neurophysiol. 58: 1187–1211.Google Scholar
- Laughlin SB, Sejnowski TJ (2003) Communication in neuronal networks. Science 301: 1870–1874.Google Scholar
- Lennie P (2003) The cost of cortical computation. Curr. Biol. 13(6): 493–497.Google Scholar
- Li Z, Attick J (1994) Towards a theory of the striate cortex. Neural. Comput. 6: 127–146.Google Scholar
- Mallat S, Zhang Z, (1993) Matching pursuit in time-frequency dictionary. IEEE Trans. Signal Process. 41: 3397–3415.Google Scholar
- Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381: 607–608.Google Scholar
- Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed in V1. Vis. Res. 37: 3311–3325.Google Scholar
- Palm G (1980) On associative memory. Biol. Cybern. 36: 19–31.Google Scholar
- Palm G, Sommer FT (1992) Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states. Network: Comput. Neural Syst. 3: 1–10.Google Scholar
- Palm G, Sommer FT (1995) Associative data storage and retrieval in neural networks. In E Domany, JL van Hemmen, KS Hemmen, eds. Models of Neural Networks III. Springer, New York, pp. 79 –118.Google Scholar
- Pati Y, Reziifar R, Krishnaprasad P (1993) Orthogonal matching pursuits: recursive function approximation with applications to wavelet decomposition. In Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers, pp. 40–44.Google Scholar
- Perrinet L, Samuelides M, Thorpe S (2004) Sparse spike coding in an asynchronous feed-forward multi-layer neural network using matching pursuit. Neurocomputing 57: 125–134.Google Scholar
- Peters A, Paine BR (1994) A numerical analysis of the genicocortical input to striate cortex in the monkey. Cereb. Cortex 4: 215–229.Google Scholar
- Rebollo-Neira L, Lowe D (2002) Optimised orthogonal matching pursuit approach. IEEE Signal Process. Lett. 9: 137–140.Google Scholar
- Rehn M, Sommer FT (2006) Storing and restoring visual input with collaborative rank coding and associative memory. Neurocomputing 69(10–12): 1219–1223.Google Scholar
- Ringach DL (2002) Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. J. Neurophys. 88(1): 455–463.Google Scholar
- Ruderman D (1994) The statistics of natural images. Network: Comput. Neural Syst. 5: 517–548.Google Scholar
- Sallee P (2002) Statistical methods for image and signal processing. PhD Thesis, UC Davis.Google Scholar
- Sallee P, Olshausen BA (2002) Learning sparse multiscale image representations. In TK Leen, TG Dietterich, V Tresp, eds. Advances in Neural Information Processing Systems, vol. 13. Morgan Kaufmann Publishers, Inc., pp. 887–893.Google Scholar
- Sommer FT, Wennekers T (2005) Synfire chains with conductance-based neurons: internal timing and coordination with timed input. Neurocomputing 65–66: 449–454.Google Scholar
- Stroud JM (1956) The fine structure of psychological time. In H Quastler, ed. Information Theory in Psychology. Free Press, pp. 174–205.Google Scholar
- Teh Y, Welling M, Osindero S, Hinton GE (2003) Energy-based models for sparse overcomplete representations. J. Mach. Learn. Res. 4: 1235–1260.Google Scholar
- Treves A (1991) Dilution and sparse coding in threshold-linear nets. J. Phys. A: Math. Gen. 24: 327–335.Google Scholar
- Tsodyks MV, Feigelman MV (1988) The enhanced storage capacity in neural networks with low activity level. Europhys. Lett. 6: 101–105.Google Scholar
- Van Essen DC, Anderson CH (1995) Information processing strategies and pathways in the primate retina and visual cortex. In SF Zornetzer, JL Davis, C Lau, eds. Introduction to Neural and Electronic Networks, 2nd ed. Orlando, Academic Press, pp. 45–76.Google Scholar
- VanRullen R, Koch C (2003) Is perception discrete or continuous? Trends Cogn. Sci. 7: 207–213.Google Scholar
- VanRullen R, Koch C (2005) Attention-driven discrete sampling of motion perception. Proc. Nat. Acad. Sci., USA 102: 5291–5296.Google Scholar
- Willshaw DJ, Buneman OP, Longuet-Higgins HC (1969) Non-holographic associative memory. Nature 222: 960–962.Google Scholar
- Willwacher G (1982) Storage of a temporal sequence in a network. Biol. Cybern. 43: 115–126.Google Scholar
- Zetsche C (1990) Sparse coding: the link between low level vision and associative memory. In R Eckmiller, G Hartmann, G Hauske, eds., Parallel Processing in Neural Systems and Computers. Elsevier Science Publishers B.V. (North Holland).Google Scholar