Neuroinformatics

, Volume 2, Issue 3, pp 275–301 | Cite as

Scaling self-organizing maps to model large cortical networks

  • James A. Bednar
  • Amol Kelkar
  • Risto Miikkulainen
Original Article

Abstract

Self-organizing computational models with specific intracortical connections can explain many functional features of visual cortex, such as topographic orientation and ocular dominance maps. However, due to their computational requirements, it is difficult to use such detailed models to study large-scale phenomenal like object segmentation and binding, object recognition, tilt illusions, optic flow, and fovea-periphery differences. This article introduces two techniques that make large simulations practical. First, we show how parameter scaling equations can be derived for laterally connected self-organizing models. These equations result in quantitatively equivalent maps over a wide range of simulation sizes, making it possible to debug small simulations and then scale them up only when needed. Parameter scaling also allows detailed comparison of biological maps and parameters between individuals and species with different brain region sizes. Second, we use parameter scaling to implement a new growing map method called GLISSOM, which dramatically reduces the memory and computational requirements of large self-organizing networks. With GLISSOM, it should be possible to simulate all of human V1 at the single-column level using current desktop workstations. We are using these techniques to develop a new simulator Topographica, which will help make it practical to perform detailed studies of large-scale phenomena in topographic maps.

Index Entries

Self-organization cortical modeling vision orientation maps growing networks computational techniques simulator development visual areas comparative anatomy 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amari, S. I. (1980) Topographic organization of nerve fields. Bulletin Math. Bio. 42, 339–364.Google Scholar
  2. Anderson, J. A. and Rosenfeld, E., eds. (1988) Neurocomputing: Foundations of Research. MIT Press, Cambridge, MA.Google Scholar
  3. Bednar, J. A. (2002) Learning to See: Genetic and Environmental Influences on Visual Development. PhD thesis, Department of Computer Sciences, The University of Texas at Austin. Technical Report AI-TR-02-294.Google Scholar
  4. Bednar, J. A., Kelkar, A., and Miikkulainen, R. (2002) Modeling large cortical networks with growing self-organizing maps. Neurocomputing 44–46, 315–321. (Special issue containing the proceedings of the CNS*01 conference.)Google Scholar
  5. Bednar, J. A. and Miikkulainen, R. (2000) Tilt after-effects in a self-organizing model of the primary visual cortex. Neural Computation 12, 1721–1740.CrossRefGoogle Scholar
  6. Bednar, J. A. and Miikkulainen, R. (2003a) Learning innate face preferences. Neural Computation 15, 1525–1557.CrossRefGoogle Scholar
  7. Bednar, J. A. and Miikkulainen, R. (2003b) Selforganization of spatiotemporal receptive fields and laterally connected direction and orientation maps. Neurocomputing 52–54, 473–480.CrossRefGoogle Scholar
  8. Bednar, J. A. and Miikkulainen, R. (2004) Prenatal and postnatal development of laterally connected orientation maps. Neurocomputing, in press.Google Scholar
  9. Blasdel, G. G. (1992) Orientation selectivity, preference, and continuity in monkey striate cortex. J. Neurosc. 12, 3139–3161.Google Scholar
  10. Bosking, W. H., Zhang, Y., Schofield, B., and Fitzpatrick, D. (1997) Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J. Neurosc. 17, 2112–2127.Google Scholar
  11. Bower, J. M. and Beeman, D. (1998) The Book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System. Telos, Santa Clara, CA.Google Scholar
  12. Burger, T. and Lang, E.W. (1999) An incremental Hebbian learning model of the primary visual cortex with lateral plasticity and real input patterns. Zeitschrift für Naturforschung C—A J. Biosc., 54, 128–140.Google Scholar
  13. Chang, L. C. and Chang, F. J. (2002) An efficient parallel algorithm for LISSOM neural network. Parallel Computing 28, 1611–1633.CrossRefGoogle Scholar
  14. Chapman, B., Stryker, M. P., and Bonhoeffer, T. (1996) Development of orientation preference maps in ferret primary visual cortex. J. Neurosc. 16, 6443–6453.Google Scholar
  15. Choe, Y. and Miikkulainen, R. (1998) Self-organization and segmentation in a laterally connected orientation map of spiking neurons. Neurocomputing 21, 139–157.CrossRefGoogle Scholar
  16. Cover, T. M. and Thomas, J. (1991). Elements of Information Theory. Wiley, pp. 247–249.Google Scholar
  17. Dittenbach, M., Merkl, D., and Rauber, A. (2000). The growing hierarchical self-organizing map. In Amari S., Giles C.L., Gori M., and Puri V., eds., Proc of the International Joint Conference on Neural Networks (IJCNN 2000), volume VI, 15–19. IEEE Computer Society.Google Scholar
  18. Erwin, E., Obermayer, K., and Schulten, K. (1992) Self-organizing maps: Ordering, convergence properties and energy functions. Biological Cybernetics 67, 47–55.CrossRefGoogle Scholar
  19. Erwin, E., Obermayer, K., and Schulten, K. (1995) Models of orientation and ocular dominance columns in the visual cortex: A critical comparison. Neural Computation 7(3), 425–468.Google Scholar
  20. Fritzke, B. (1995) Growing grid: A self-organizing network with constant neighborhood range and adaptation strength. Neural Proc. Lett. 2, 9–13.CrossRefGoogle Scholar
  21. Gilbert, C. D., Das, A., Ito, M., Kapadia, M., and Westheimer, G. (1996) Spatial integration and cortical dynamics. Proc. Nat. Acad. Sci., USA 93, 615–622.CrossRefGoogle Scholar
  22. Gilbert, C. D., Hirsch, J. A., and Wiesel, T. N. (1990) Lateral interactions in visual cortex. In Cold Spring Harbor Symposia on Quantitative Biology, Volume LV, pp.663–677. Cold Spring Harbor Laboratory Press, Cold Spring Harbor, NY.Google Scholar
  23. Gould, E., Reeves, A. J., Graziano, M. S. A., and Gross, C. G. (1999) Neurogenesis in the neocortex of adult primates. Science 286, 548–552.CrossRefGoogle Scholar
  24. Grinvald, A., Lieke, E. E., Frostig, R. D., and Hildesheim, R. (1994) Cortical point-spread function and long-range lateral interactions revealed by real-time optical imaging of macaque monkey primary visual cortex. J. Neurosc. 14, 2545–2568.Google Scholar
  25. Grossberg, S. (1976) On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems. Biol. Cybernet. 21, 145–159.CrossRefGoogle Scholar
  26. Hines, M. L. and Carnevale, N. T. (1997) The NEURON simulation environment. Neural Computation 9, 1179–1209.CrossRefGoogle Scholar
  27. Hirsch, J. A. and Gilbert, C. D. (1991) Synaptic physiology of horizontal connections in the cat’s visual cortex. J. Neurosc. 11, 1800–1809.Google Scholar
  28. Kaas, J. H. (2000) Why is brain size so important: Design problems and solutions as neocortex gets bigger or smaller. Brain and Mind 1, 7–23.CrossRefGoogle Scholar
  29. Kalarickal, G. J. and Marshall, J. A. (2002) Rearrangement of receptive field topography after intracortical and peripheral stimulation: The role of plasticity in inhibitory pathways. Network: Comput. Neural Sys. 13, 1–40.CrossRefGoogle Scholar
  30. Kohonen, T. (1989) Self-Organization and Associative Memory. Springer, Berlin; New York, Third edition.Google Scholar
  31. Kolen, J. F. and Pollack, J. B. (1990) Scenes from exclusive-OR: Back propagation is sensitive to initial conditions. In Proceedings of the 12th Annual Conference of the Cognitive Science Society, 868–875. Hillsdale, NJ: Erlbaum.Google Scholar
  32. Luttrell, S. P. (1988) Self-organizing multilayer topographic mappings. In Proceedings of the IEEE International Conference on Neural Networks (San Diego, CA). Piscataway, NJ: IEEE.Google Scholar
  33. Marshall, J. A. (1995) Adaptive perceptual pattern recognition by self-organizing neural networks: Context, uncertainty, multiplicity, and scale. Neural Networks 8, 335–362.CrossRefGoogle Scholar
  34. Miikkulainen, R., Bednar, J. A., Choe, Y., and Sirosh, J. (1997) Self-organization, plasticity, and low-level visual phenomena in a laterally connected map model of the primary visual cortex. In Goldstone R.L., Schyns P.G., and Medin D.L., eds., Perceptual Learning, volume 36 of Psychology of Learning and Motivation, 257–308. Academic Press, San Diego, CA.Google Scholar
  35. Mundel, T., Dimitrov, A., and Cowan, J. D. (1997) Visual cortex circuitry and orientation tuning. In Mozer M. C., Jordan M. I., and Petsche T., eds., Advances in Neural Information Processing Systems 9, 887–893. Cambridge, MA: MIT Press.Google Scholar
  36. Obermayer, K., Ritter, H. J., and Schulten, K. J. (1990) A principle for the formation of the spatial structure of cortical feature maps. Pro. Nat. Acad. Sci., USA 87, 8345–8349.CrossRefGoogle Scholar
  37. Osan, R. and Ermentrout, B. (2002) Development of joint ocular dominance and orientation selectivity maps in a correlation-based neural network model. In Bower J.M., ed., Computational Neuroscience: Trends in Research, 2002. New York: Elsevier.Google Scholar
  38. Purves, D. (1988) Body and Brain: A Trophic Theory of Neural Connections. Harvard University Press, Cambridge, MA.Google Scholar
  39. Rall, W. (1962) Theory of physiological properties of dendrites. Ann. NY Acad. Sci. 96, 1071–1092.CrossRefGoogle Scholar
  40. Rockel, A. J., Hiorns, R. W., and Powell, T. P. S. (1980) The basic uniformity in structure of the neocortex. Brain 103, 221–244.CrossRefGoogle Scholar
  41. Rodriques, J. S. and Almeida, L. B. (1990) Improving the learning speed in topological maps of patterns. In Proceedings of the International Neural Networks Conference (Paris, France), 813–816. Kluwer, Dordrecht; Boston.Google Scholar
  42. Roque Da Silva Filho, A. C. (1992) Investigation of a Generalized Version of Amari’s Continuous Model for Neural Networks. PhD thesis, University of Sussex at Brighton, Brighton, UK.Google Scholar
  43. Sirosh, J. (1995) A Self-Organizing Neural Network Model of the Primary Visual Cortex. PhD thesis, Department of Computer Sciences, The University of Texas at Austin, Austin, TX. Technical Report A195-237.Google Scholar
  44. Sirosh, J. and Miikkulainen, R. (1994) Cooperative self-organization of afferent and lateral connections in cortical maps. Biol. Cybernet. 71, 66–78.Google Scholar
  45. Sirosh, J., Miikkulainen, R., and Bednar, J. A. (1996) Self-organization of orientation maps, lateral connections, and dynamic receptive fields in the primary visual cortex. In Sirosh J., Miikkulainen R., and Choe Y., eds., Lateral Interactions in the Cortex: Structure and Function. The UTCS Neural Networks Research Group, Austin, TX. Electronic book, ISBN 0-9647060-0-8, http://www.cs.utexas.edu/users/nn/webpubs/htmlbook96.Google Scholar
  46. Swindale, N. V. (1992) A model for the coordinated development of columnar systems in primate striate cortex. Biol. Cybernet. 66, 217–230.CrossRefGoogle Scholar
  47. Swindale, N. V. (1996) The development of topography in the visual cortex: A review of models. Network — Comput. Neural Sys. 7, 161–247.CrossRefGoogle Scholar
  48. Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., and Nelson, S. B. (1998). Activity-dependent scaling of quantal amplitude in neocortical neurons. Nature 391, 845–846.CrossRefGoogle Scholar
  49. van Praag, H., Schinder, A. F., Christie, B. R., Toni, N., Palmer, T. D., and Gage, F. H. (2002) Functional neurogenesis in the adult hippocampus. Nature 415, 1030–1034.CrossRefGoogle Scholar
  50. von der Malsburg, C. (1973) Self-organization of orientation-sensitive cells in the striate cortex. Kybernetik 15,85–100. Reprinted in Anderson and Rosenfeld, 1988.Google Scholar
  51. Wandell, B. A. (1995) Foundations of Vision. Sinauer Associates, Inc., Sunderland, MA.Google Scholar
  52. Weliky, M., Kandler, K., Fitzpatrick, D., and Katz, L.C. (1995) Patterns of excitation and inhibition evoked by horizontal connections in visual cortex share a common relationship to orientation columns. Nature 15, 541–552.Google Scholar

Copyright information

© Humana Press Inc 2004

Authors and Affiliations

  • James A. Bednar
    • 1
  • Amol Kelkar
    • 1
  • Risto Miikkulainen
    • 1
  1. 1.Department of Computer SciencesThe University of Texas at AustinAustin

Personalised recommendations