Skip to main content
SpringerLink
Log in

The InfoMin Criterion: An Information Theoretic Unifying Objective Function for Topographic Mappings

  • Conference paper
  • First Online:
Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003 (ICANN 2003, ICONIP 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2714))

Abstract

In this paper, we propose a new objective function for forming topographic mappings, named the “InfoMin” criterion. This criterion is defined as the average of the information transferred through small neighbor areas over a mapping, and its closed form is derived by use of the Edgeworth expansion. If the second-order statistics (namely, normal correlations among neurons) are not zero, the InfoMin criterion is consistent with the C measure (a unifying objective function for topographic mapping proposed by Goodhill and Sejnowski [1]). In addition, the higher-order correlations are dominant in this criterion only if the second-order ones are negligible. So, it can explain many previous models comprehensively, and is applicable to uncorrelated signals such that ZCA or ICA generates as well. Numerical experiments on natural scenes verify that the InfoMin criterion gives a strong unifying framework for topographic mappings based on information theory.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Goodhill, G. J. & Sejnowski, T. (1997). A Unifying Objective Function for Topographic Mappings Neural Computation, 9, 1291–1303.

    Article  Google Scholar 

  2. Tootell, R. B. H., Switkes, E., & Silverman, M. S. (1988). Functional anatomy of macaque striate cortex, II, Retinotopic organization. The Journal of Neuroscience, 8(5), 1531–1568.

    Google Scholar 

  3. Hubel, D. H., & Wiesel, T. N. (1977). Functional Architecture of macaque monkey visual cortex. Proceedings of the Royal Society of London B, 198, 1–59.

    Google Scholar 

  4. Swindale, N. V. (1996). The development of topography in the visual cortex: A review of models. Network: Computation in Neural Systems, 7(2), 161–247.

    Article  MATH  Google Scholar 

  5. Erwin, E., Obermayer, K., & Schulten, K. (1995). Models of orientation and ocular dominance columns in the visual cortex: A critical comparison. Neural Computation, 7, 425–468.

    Article  Google Scholar 

  6. Wiskott, L. & Sejnowski, T. J. (1998). Constrained optimization for neural map formation: A unifying framework for weight growth and normalization. Neural Computation, 10, 671–716.

    Article  Google Scholar 

  7. Li, Z. & Atick, J. J. (1994). Toward a theory of the striate cortex. Neural Computation 6(1), 127–146.

    Article  Google Scholar 

  8. Hyvärinen, A., Hoyer, P. O., & Inki, M. (2001). Topographic independent component analysis. Neural Computation, 13(7), 1527–1558.

    Article  MATH  Google Scholar 

  9. Linsker, R. (1988). Self-organization in a Perceptual Network. Computer, 21(3), 105–117.

    Article  Google Scholar 

  10. Plumbley, M. (1993). Efficient information transfer and anti-Hebbian neural networks. Neural Networks, 6, 823–833.

    Article  Google Scholar 

  11. Bell, A. J. & Sejnowski, T. J. (1997). The independent components of natural scenes are edge filters. Vision Research 37(23), 3327–3338.

    Article  Google Scholar 

  12. Linsker, R. (1989). How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Computation, 1, 402–411.

    Article  Google Scholar 

  13. Luttrell, S. P. (1994). A Bayesian analysis of self-organizing maps. Neural Computation, 6, 767–794.

    Article  MATH  Google Scholar 

  14. Bishop, C. M., Svensen, M., & Williams, C. K. I. (1998). GTM: The generative topographic mapping. Neural Computation, 10(1), 215–234.

    Article  Google Scholar 

  15. Matsuda, Y., & Yamaguchi, K. (2001). The InfoMin principle: A unifying information-Based criterion for forming topographic Mappings. in Proceedings of ICONIP, Shanghai, China, 14–19.

    Google Scholar 

  16. Barndorff-Nielsen, O. E. & Cox, D. R. (1989). Asymptotic techniques for use in statistics. London; New York: Chapman and Hall.

    MATH  Google Scholar 

  17. Durbin, R. & Mitchson, G. (1990). A dimension reduction framework for understanding cortical maps. Nature, 343, 644–647.

    Article  Google Scholar 

  18. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.

    Article  MathSciNet  Google Scholar 

  19. Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10, 626–634.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kazunori Yamaguchi Laboratory, Department of General Systems Studies, Graduate School of Arts and Sciences, The University of Tokyo, 3-8-1, Komaba, Meguro-ku, Tokyo, 153-8902, Japan

    Yoshitatsu Matsuda & Kazunori Yamaguchi

Authors
  1. Yoshitatsu Matsuda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kazunori Yamaguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Bogazici University, Bebek, 34342, Istanbul, Turkey

    Okyay Kaynak  & Ethem Alpaydin  & 

  2. Laboratory of Computer and Information Science, Helsinki University of Technology, P.O.B. 5400, 02015, Finland

    Erkki Oja

  3. Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong

    Lei Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Matsuda, Y., Yamaguchi, K. (2003). The InfoMin Criterion: An Information Theoretic Unifying Objective Function for Topographic Mappings. In: Kaynak, O., Alpaydin, E., Oja, E., Xu, L. (eds) Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003. ICANN ICONIP 2003 2003. Lecture Notes in Computer Science, vol 2714. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44989-2_48

Download citation

  • DOI: https://doi.org/10.1007/3-540-44989-2_48

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40408-8

  • Online ISBN: 978-3-540-44989-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation