Skip to main content
SpringerLink
Log in

Evolving Distributed Representations for Language with Self-Organizing Maps

  • Conference paper
Symbol Grounding and Beyond (EELC 2006)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4211))

Abstract

We present a neural-competitive learning model of language evolution in which several symbol sequences compete to signify a given propositional meaning. Both symbol sequences and propositional meanings are represented by high-dimensional vectors of real numbers. A neural network learns to map between the distributed representations of the symbol sequences and the distributed representations of the propositions. Unlike previous neural network models of language evolution, our model uses a Kohonen Self-Organizing Map with unsupervised learning, thereby avoiding the computational slowdown and biological implausibility of back-propagation networks and the lack of scalability associated with Hebbian-learning networks. After several evolutionary generations, the network develops systematically regular mappings between meanings and sequences, of the sort traditionally associated with symbolic grammars. Because of the potential of neural-like representations for addressing the symbol-grounding problem, this sort of model holds a good deal of promise as a new explanatory mechanism for both language evolution and acquisition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Harnad, S.: Grounding symbols in the analog world with neural nets. Think 2(1), 12–78 (1993)

    Google Scholar 

  2. Searle, J.: Minds, brains, and programs. Behavioral and Brain Sciences 3 (1980)

    Google Scholar 

  3. Batali, J.: Computational simulations of the emergence of grammar. In: Hurford, J., Studdert-Kennedy, M., Knight, C. (eds.) Approaches to the Evolution of Language: Social and Cognitive Bases. Cambridge University Press, Cambridge (1998)

    Google Scholar 

  4. Rumelhart, D., Hinton, G., Williams, R.: Learning internal representation by error propagation. In: Rumelhart, D., McClelland, J. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1. MIT Press, Cambridge (1986)

    Google Scholar 

  5. Smith, K.: The cultural evolution of communication in a population of neural networks. Connection Science 14(1), 65–84 (2002)

    Article  Google Scholar 

  6. Fodor, J.: The Language of Thought. Crowell, New York (1975)

    Google Scholar 

  7. Grossberg, S.: Competitive learning: from interactive activation to adaptive resonance. In: Connectionist models and their implications: readings from cognitive science, pp. 243–283. Ablex Publishing Corp., Norwood (1988)

    Google Scholar 

  8. McClelland, J., Rumelhart, D., Hinton, G.: The appeal of parallel distributed processing. In: Rumelhart, D., McClelland, J. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, MIT Press, Cambridge (1986)

    Google Scholar 

  9. Smith, K., Brighton, H., Kirby, S.: Complex systems in language evolution: the cultural emergence of compositional structure. Advances in Complex Systems 6(4), 537–558 (2003)

    Article  Google Scholar 

  10. Hinton, G.: Distributed representations. Technical Report CMU-CS-84-157, Computer Science Department, Carnegie Mellon University (1984)

    Google Scholar 

  11. Elman, J.: Finding structure in time. Cognitive Science 14, 179–211 (1990)

    Article  Google Scholar 

  12. Landauer, T.K., Dumais, S.T.: A solution to plato’s problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review 104, 211–240 (1997)

    Article  Google Scholar 

  13. Gärdenfors, P.: Conceptual Spaces: The Geometry of Thought. MIT Press, Cambridge (2000)

    Google Scholar 

  14. Steedman, M.: Connectionist sentence processing in perspective. Cognitive Science 23(4), 615–634 (1999)

    Article  Google Scholar 

  15. Plate, T.A.: Holographic Reduced Representation: Distributed Representation for Cognitive Science. CSLI Publications (2003)

    Google Scholar 

  16. Kanerva, P.: The binary spatter code for encoding concepts at many levels. In: Marinaro, M., Morasso, P. (eds.) ICANN 1994: Proceedings of International Conference on Artificial Neural Networks, vol. 1, pp. 226–229. Springer, London (1994)

    Google Scholar 

  17. Rachkovskij, D.A., Kussul, E.M.: Binding and normalization of binary sparse distributed representations by context-dependent thinning. Neural Computation 13(2), 411–452 (2001)

    Article  MATH  Google Scholar 

  18. Gayler, R.: Multiplicative binding, representation operators, and analogy. In: Holyoak, K., Gentner, D., Kokinov, B. (eds.) Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences. New Bulgarian University, Sofia, Bulgaria, p. 405 (1998)

    Google Scholar 

  19. Pollack, J.: Recursive distributed representations. Artifical Intelligence 36, 77–105 (1990)

    Article  Google Scholar 

  20. Smolensky, P.: Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence 46, 159–216 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  21. Kohonen, T.: Self-Organizing Maps, 3rd edn. Springer, Secaucus (2001)

    MATH  Google Scholar 

  22. VanHulle, M.: Faithful Representations and Topographic Maps. Wiley-Interscience, New York (1990)

    Google Scholar 

  23. Brighton, H., Kirby, S.: Understanding linguistic evolution by visualizing the emergence of topographic mappings. Artificial Life 12(2), 229–242 (2006)

    Article  Google Scholar 

  24. Kirby, S.: Learning, bottlenecks and the evolution of recursive syntax. In: Briscoe, T. (ed.) Linguistic Evolution through Language Acquisition: Formal and Computational Models, Cambridge University Press, Cambridge (2002)

    Google Scholar 

  25. MacLennan, B.: Synthetic ethology: An approach to the study of communication. In: Langton, C., Taylor, C., Farmer, D., Rasmussen, S. (eds.) Artificial Life II, pp. 631–658. Addison-Wesley, Redwood City (1992)

    Google Scholar 

  26. Werner, G., Dyer, M.: Evolution of communication in artificial organisms. In: Langton, C., Taylor, C., Farmer, D., Rasmussen, S. (eds.) Artificial Life II, pp. 659–687. Addison-Wesley, Redwood City (1992)

    Google Scholar 

  27. Briscoe, T.: Grammatical acquisition: Inductive bias and coevolution of language and the language acquisition device. Language 76(2), 245–296 (2000)

    Article  Google Scholar 

  28. Chomsky, N.: Language and Mind. Harcourt Brace Jovanovich, San Diego (1972)

    Google Scholar 

  29. Smith, K.: Natural selection and cultural selection in the evolution of communication. Adaptive Behavior 10(1), 25–44 (2002)

    Article  Google Scholar 

  30. Chomsky, N.: Rules and Representations. Basil Blackwell, Oxford (1980)

    Google Scholar 

  31. Croft, W.: Explaining language change: an evolutionary approach. Longman, Harlow, Essex (2000)

    Google Scholar 

  32. Lewandowsky, S., Murdock, B.: Memory for serial order. Psychological Review 96(1), 25–27 (1989)

    Article  Google Scholar 

  33. Hauser, M.D., Chomsky, N., Fitch, W.T.: The faculty of language: What is it, who has it, and how did it evolve? Science 298, 1569–1579 (2002)

    Article  Google Scholar 

  34. Chomsky, N.: Three models for the description of language. IRE Transactions on information theory 2, 113–124 (1956)

    Article  Google Scholar 

  35. Murdock, B.B.: Serial order effects in a distributed-memory model. In: Gorfein, D.S., Hoffman, R.R. (eds.) MEMORY AND LEARNING: The Ebbinghaus Centennial Conference, pp. 277–310. Lawrence Erlbaum Associates, Mahwah (1987)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, Washington and Lee University, Lexington, VA, 24450, USA

    Simon D. Levy

  2. Language Evolution and Computation Research Unit, School of Philosophy, Psychology and Language Sciences, University of Edinburgh, 40, George Square, Edinburgh, UK

    Simon Kirby

Authors
  1. Simon D. Levy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Kirby
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Vrije Universiteit Amsterdam, The Netherlands

    Paul Vogt

  2. RIKEN Brain Science Institute, 2-1 Hirosawa, 3510198, Wako-shi, Saitama, Japan

    Yuuya Sugita

  3. IRIDIA, CoDE, Université Libre de Bruxelles, Brussels, Belgium

    Elio Tuci

  4. Adaptive Systems Research Group, University of Hertfordshire, College Lane, AL10 9AB, Hatfield, Hertfordshire, U.K.

    Chrystopher Nehaniv

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Levy, S.D., Kirby, S. (2006). Evolving Distributed Representations for Language with Self-Organizing Maps. In: Vogt, P., Sugita, Y., Tuci, E., Nehaniv, C. (eds) Symbol Grounding and Beyond. EELC 2006. Lecture Notes in Computer Science(), vol 4211. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11880172_5

Download citation

  • DOI: https://doi.org/10.1007/11880172_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-45769-5

  • Online ISBN: 978-3-540-45771-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation