Self-Organizing Maps in Symbol Processing

  • Timo Honkela
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1778)


A symbol as such is disassociated from the world. In addition, as a discrete entity a symbol does not mirror all the details of the portion of the world that it is meant to refer to. Humans establish the association between the symbols and the referenced domain – the words and the world – through a long learning process in a community. This paper studies how Kohonen self-organizing maps can be used for modeling the learning process needed in order to create a conceptual space based on a relevant context with which the symbols are associated. The categories that emerge in the self-organizing process and their implicitness are considered as well as the possibilities to model contextuality, subjectivity and intersubjectivity of interpretation.


Natural Language Processing Predicate Logic Model Vector Conceptual Space Word Sense Disambiguation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amari, S.-I.: A theory of adaptive pattern classifiers. IEEC 16, 299–307 (1967)zbMATHGoogle Scholar
  2. 2.
    de Boer, B.: Investigating the Emergence of Speech Sounds. In: Dean, T. (ed.) Proceedings of IJCAI 1999, International Joint Conference on Artificial Intelligence, vol. 1, pp. 364–369. Morgan Kaufmann, San Francisco (1999)Google Scholar
  3. 3.
    Carpenter, G., Grossberg, S.: A massively parallel architecture for a self- organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing 37, 54–115 (1987)CrossRefGoogle Scholar
  4. 4.
    Charniak, E.: Statistical Language Learning. MIT Press, Cambridge (1993)Google Scholar
  5. 5.
    de Sa, V.: Unsupervised Classification Learning from Cross-Modal Environmental Structure. PhD thesis, University of Rochester, Department of Computer Science, Rochester, New York (1994)Google Scholar
  6. 6.
    Finch, S., Chater, N.: Unsupervised methods for finding linguistic categories. In: Aleksander, I., Taylor, J. (eds.) Artificial Neural Networks, vol. 2, pp. II-1365–1368. North-Holland, Amsterdam (1992)Google Scholar
  7. 7.
    von Foerster, H.: Notes on an epistemology for living things. In: Observing Systems, pp. 258–271. Intersystems publications (1981)Google Scholar
  8. 8.
    Gallant, S.I.: A practical approach for representing context and for performing word sense disambiguation using neural networks. ACM SIGIR Forum 3(3), 293–309 (1991)Google Scholar
  9. 9.
    Gärdenfors, P.: Mental representation, conceptual spaces and metaphors. Synthese 106, 21–47 (1996)CrossRefGoogle Scholar
  10. 10.
    Gärdenfors, P.: Philosophy and Cognitive Science. In: chapter Conceptual spaces as a framework for cognitive semantics, pp. 159–180. Kluwer, Dordrecht (1996)Google Scholar
  11. 11.
    Honkela, T., Vepsäläinen, A.M.: Interpreting imprecise expressions: experiments with Kohonen’s Self-Organizing Maps and associative memory. In: Kohonen, T., Mäkisara, K., Simula, O., Kangas, J. (eds.) Proceedings of ICANN-1991, International Conference on Artificial Neural Networks, vol. 1, pp. 897–902. North-Holland, Amsterdam (1991)Google Scholar
  12. 12.
    Honkela, T.: Neural Nets that Discuss: A General Model of Communication Based on Self-Organizing Maps. In: Gielen, S., Kappen, B. (eds.) Proceedings of ICANN-1993, International Conference on Artificial Neural Networks, Amsterdam, pp. 408–411. Springer, London (1993)Google Scholar
  13. 13.
    Honkela, T., Pulkki, V., Kohonen, T.: Contextual relations of words in Grimm tales analyzed by self-organizing map. In: Fogelman-Soulié, F., Gallinari, P. (eds.) Proceedings of ICANN-1995, International Conference on Artificial Neural Networks, EC2 et Cie, Paris, vol. 2, pp. 3–7 (1995)Google Scholar
  14. 14.
    Honkela, T., Kaski, S., Lagus, T., Kohonen, T.: Newsgroup exploration with WEBSOM method and browsing interface. Technical report A32, Helsinki University of Technology, Laboratory of Computer and Information Science, Espoo, Finland (1996)Google Scholar
  15. 15.
    Honkela, T.: Self-Organizing Maps in Natural Language Processing. PhD Thesis, Helsinki University of Technology, Espoo, Finland (1997), See
  16. 16.
    Hyötyniemi, H.: On mental images and computational semantics. In: Proceedings of Finnish Artificial Intelligence Conference, pp. 199–208. Finnish Artificial Intelligence Society, Espoo (1998)Google Scholar
  17. 17.
    Nissinen, A.S., Hyötyniemi, H.: Evolutionary Self-Organizing Map. In: Proceedings of EUFIT 1998: European Congress on Intelligent Techniques and Soft Computing, pp. 1596–1600 (1998)Google Scholar
  18. 18.
    Hörmann, H.: Meaning and Context. Plenum Press, New York (1986)Google Scholar
  19. 19.
    Kaski, S., Honkela, T., Lagus, K., Kohonen, T.: Creating an order in digital libraries with self-organizing maps. In: Proceedings of WCNN 1996, World Congress on Neural Networks (1996)Google Scholar
  20. 20.
    Kaski, S., Honkela, T., Lagus, K., Kohonen, T.: WEBSOM–Self-Organizing Maps of Document Collections. Neurocomputing 21, 101–117 (1998)zbMATHCrossRefGoogle Scholar
  21. 21.
    Kaski, S.: Dimensionality reduction by random mapping: Fast similarity compu- tation for clustering. In: Proceedings of IJCNN 1998, International Joint Conference on Neural Networks (1998)Google Scholar
  22. 22.
    Kohonen, T.: Self-organizing formation of topologically correct feature maps. Bio- logical Cybernetics 43(1), 59–69 (1982)zbMATHCrossRefGoogle Scholar
  23. 23.
    Kohonen, T.: The Adaptive-Subspace SOM (ASSOM) and its use for the imple- mentation of invariant feature detection. In: Fogelman-Soulié, F., Gallinari, P. (eds.) Proceedings of ICANN 1995, International Conference on Artificial Neural Networks, Nanterre, France. EC2, vol. 2, pp. 3–10 (1995)Google Scholar
  24. 24.
    Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (1995)Google Scholar
  25. 25.
    Kohonen, T., Hynninen, J., Kangas, J., Laaksonen, J.: SOM_ PAK: The Self- Organizing Map program package. Report A31, Helsinki University of Technology, Laboratory of Computer and Information Science (1996)Google Scholar
  26. 26.
    Kohonen, T., Kaski, S., Lappalainen, H., Salojärvi, J.: The adaptive- subspace self-organizing map (ASSOM). In: Proceedings of WSOM 1997, Workshop on Self-Organizing Maps, Espoo, Finland, June 4-6, pp. 191–196. Helsinki University of Technology, Neural Networks Research Centre, Espoo, Finland (1997)Google Scholar
  27. 27.
    Kohonen, T., Hari, R.: Where the abstract feature maps of the brain might come fromo. Trends In Neurosciences 22(3), 135–139 (1999)CrossRefGoogle Scholar
  28. 28.
    Kohonen, T.: Fast Evolutionary Learning with Batch-Type Self-Organizing Maps. Neural Processing Letters 9, 153–162 (1999)CrossRefGoogle Scholar
  29. 29.
    Lakoff, G.: Women, Fire and Dangerous Things. University of Chicago Press, Chi- cago (1987)Google Scholar
  30. 30.
    Lin, X., Soergel, D., Marchionini, G.: A self-organizing semantic map for infor- mation retrieval. In: Proceedings of 14th. Ann. International ACM/SIGIR Conference on Research & Development in Information Retrieval, pp. 262–269 (1991)Google Scholar
  31. 31.
    MacLennan, B.: Continuous formal systems: A unifying model in language and cognition. In: Proceedings of the IEEE Workshop on Architectures for Semiotic Modeling and Situation Analysis in Large Complex Systems, Monterey, CA, August 27-29 (1995)Google Scholar
  32. 32.
    MacWhinney, B.: Competition and Lexical Categorization. Linguistic categorization. Benjamins, New York (1989)Google Scholar
  33. 33.
    MacWhinney, B.: Lexical Connectionism. Cognitive approaches to language learning. MIT Press, Cambridge (1997)Google Scholar
  34. 34.
    Mayberry, M.R., Miikkulainen, R.: SardSrn: A Neural Network Shift-Reduce Parser. In: Dean, T. (ed.) Proceedings of IJCAI 1999, International Joint Conference on Artificial Intelligence, vol. 2, pp. 820–825. Morgan Kaufmann, San Francisco (1999)Google Scholar
  35. 35.
    McCawley, J.D.: Everything that Linguists have always Wanted to Know about Logic but were ashemed to ask. Basil Blackwell, London (1981)Google Scholar
  36. 36.
    Merkl, D.: Self-Organization of Software Libraries: An Artificial Neural Network Approach. PhD thesis, Institut für Angewandte Informatik und Informationssy- steme, Universität Wien (1994)Google Scholar
  37. 37.
    Miikkulainen, R.: DISCERN: A Distributed Artificial Neural Network Model of Script Processing and Memory. PhD thesis, Computer Science Department, University of California, Los Angeles, Tech. Rep. UCLA-AI-90-05 (1990)Google Scholar
  38. 38.
    Miikkulainen, R.: Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon, and Memory. MIT Press, Cambridge (1993)Google Scholar
  39. 39.
    Miikkulainen, R.: Self-organizing feature map model of the lexicon. Brain and Language 59, 334–366 (1997)CrossRefGoogle Scholar
  40. 40.
    Miikkulainen, R., Dyer, M.G.: Natural language processing with modular neural networks and distributed lexicon. Cognitive Science 15, 343–399 (1991)CrossRefGoogle Scholar
  41. 41.
    Mitra, S., Pal, S.: Self-organizing neural network as a fuzzy classifier. IEEE Transactions on Systems, Man and Cybernetics 24(3), 385–399 (1994)CrossRefGoogle Scholar
  42. 42.
    Mitra, S., Pal, S.: Fuzzy self-organization, inferencing, and rule generation. IEEE Transactions on Systems, Man & Cybernetics, Part A [Systems & Humans] 26(5), 608–620 (1996)CrossRefGoogle Scholar
  43. 43.
    Nenov, V.I., Dyer, M.G.: Perceptually grounded language learning: Part 1 - A neural network architecture for robust sequence association. Connection Science 5(2), 115–138 (1993)CrossRefGoogle Scholar
  44. 44.
    Nenov, V.I., Dyer, M.G.: Perceptually grounded language learning: Part 2 - DETE: a neural/procedural model. Connection Science 6(1), 3–41 (1994)CrossRefGoogle Scholar
  45. 45.
    Pulkki, V.: Data averaging inside categories with the self-organizing map. Report A27, Helsinki University of Technology, Laboratory of Computer and Information Science, Espoo, Finland (1995)Google Scholar
  46. 46.
    Regier, T.: A model of the human capacity for categorizing spatial relations. Cognitive Linguistics 6(1), 63–88 (1995)CrossRefGoogle Scholar
  47. 47.
    Ritter, H., Kohonen, T.: Self-organizing semantic maps. Biological Cybernetics 61(4), 241–254 (1989)CrossRefGoogle Scholar
  48. 48.
    Rosch, E.: Human categorization. Studies in cross-cultural psychology, vol. 1, pp. 3–49. Academic Press, New York (1977)Google Scholar
  49. 49.
    Scholtes, J.C.: Neural Networks in Natural Language Processing and Information Retrieval. PhD thesis, Universiteit van Amsterdam, Amsterdam, Netherlands (1993)Google Scholar
  50. 50.
    Schütze, H.: Dimensions of meaning. In: Proceedings of Supercomputing, pp. 787–796 (1992)Google Scholar
  51. 51.
    Ultsch, A.: Self-organizing neural networks for visualization and classification. In: Opitz, O., Lausen, B., Klar, R. (eds.) Information and Classification, London, UK, pp. 307–313. Springer, Springer (1993)Google Scholar
  52. 52.
    Ultsch, A., Siemon, H.: Kohonen’s self organizing feature maps for exploratory data analysis. In: Proceedings of INNC 1990, International Neural Network Conference, Dordrecht, Netherlands, pp. 305–308. Kluwer, Dordrecht (1990)Google Scholar
  53. 53.
    Varela, F.J., Thompson, E., Rosch, E.: The Embodied Mind: Cognitive Science and Human Experience. MIT Press, Cambridge (1993)Google Scholar
  54. 54.
    von der Malsburg, C.: Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14, 85–100 (1973)CrossRefGoogle Scholar
  55. 55.
    Wermter, S., Riloff, E., Scheler, G.: Connectionist, Statistical and Symbolic Approaches to Learning for Natural Language Processing. Springer, New York (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Timo Honkela
    • 1
  1. 1.Media LabUniversity of Art and DesignHelsinkiFinland

Personalised recommendations