From simple associations to the building blocks of language: Modeling meaning in memory with the HAL model

  • Curt BurgessEmail author
Invited Address


This paper presents a theoretical approach of how simple, episodic associations are transduced into semantic and grammatical categorical knowledge. The approach is implemented in the hyperspace analogue to language (HAL) model of memory, which uses a simple global co-occurrence learning algorithm to encode the context in which words occur. This encoding is the basis for the formation of meaning representations in a high-dimensional context space. Results are presented, and the argument is made that this simple process can ultimately provide the language-comprehension system with semantic and grammatical information required in the comprehension process.


Lexical Decision Word Meaning Latent Semantic Analysis Word Association Vector Element 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Au, T. K. (1986). A verb is worth a thousand words: The causes and consequences of interpersonal events implicit in language.Journal of Memory & Language,25, 104–122.CrossRefGoogle Scholar
  2. Berwick, R. C. (1989). Learning word meanings from examples. In D. L. Waltz (Ed.),Semantic structures: Advances in natural language processing (pp. 89–124). Hillsdale, NJ: Erlbaum.Google Scholar
  3. Burgess, C., Livesay, K., &Lund, K. (1998). Explorations in context space: Words, sentences, discourse.Discourse Processes,25, 211–257.CrossRefGoogle Scholar
  4. Burgess, C., &Lund, K. (1997a). Parsing constraints and highdimensional semantic space.Language & Cognitive Processes,12, 177–210.CrossRefGoogle Scholar
  5. Burgess, C., &Lund, K. (1997b). Representing abstract words and emotional connotation in high-dimensional memory space.Proceedings of the Cognitive Science Society (pp. 61–66). Hillsdale, NJ: Erlbaum.Google Scholar
  6. Cairns, P., Shillcock, R., Chater, N., &Levy, J. (1997). Bootstrapping word boundaries: A bottom-up corpus-based approach to speech segmentation.Cognitive Psychology,33, 111–153.CrossRefPubMedGoogle Scholar
  7. Chiarello, C., Burgess, C., Richards, L., &Pollock, A. (1990). Semantic and associative priming in the cerebral hemispheres: Some words do, some words don’t … sometimes, some places.Brain & Language,38, 75–104.CrossRefGoogle Scholar
  8. Cushman, L.,Burgess, C., &Maxfield, L. (1993, February).Semantic priming effects in patients with left neglect. Paper presented at the meeting of the International Neuropsychological Society, Galveston, TX.Google Scholar
  9. Deese, J. (1965).The structure of associations in language and thought (pp. 97–119). Baltimore: Johns Hopkins University Press.Google Scholar
  10. Elman, J. L. (1990). Finding structure in time.Cognitive Science,14, 179–211.CrossRefGoogle Scholar
  11. Ervin, S. M. (1963). Correlates of associative frequency.Journal of Verbal Learning & Verbal Behavior,1, 422–431.CrossRefGoogle Scholar
  12. Ervin-Tripp, S. M. (1970). Substitution, context, and association. In L. Postman & G. Keppel (Eds.),Norms of word association (pp. 383–395). New York: Academic Press.Google Scholar
  13. Finch, S., &Chater, N. (1992). Bootstrapping syntactic categories by unsupervised learning. InProceedings of the Fourteenth Annual Meeting of the Cognitive Science Society (pp. 820–825). Hillsdale, NJ: Erlbaum.Google Scholar
  14. Fischler, I. (1977). Semantic facilitation without association in a lexical decision task.Memory & Cognition,5, 335–339.CrossRefGoogle Scholar
  15. Foltz, P. W. (1996). Latent semantic analysis for text-based research.Behavior Research Methods, Instruments, & Computers,28, 197–202.Google Scholar
  16. Garnham, A. (1996). The other side of mental models: Theories of language comprehension. In J. Oakhill & A. Garnham (Eds.),Mental models in cognitive science (pp. 35–52). Hove, U.K.: Psychology Press.Google Scholar
  17. Hinton, G. E., McClelland, J. L., &Rumelhart, D. E. (1986). Distributed representations. In D. E. Rumelhart, J. L. McClelland, & the PDP Research Group (Eds.),Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 1: Foundations (pp. 77–109). Cambridge, MA: MIT Press.Google Scholar
  18. Johnson-Laird, P. N. (1983).Mental models: Towards a cognitive science of language, inference, and consciousness. Cambridge: Cambridge University Press.Google Scholar
  19. Komatsu, L. K. (1992). Recent views of conceptual structure.Psychological Bulletin,112, 500–526.CrossRefGoogle Scholar
  20. Landauer, T. K., &Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction and representation of knowledge.Psychological Bulletin,104, 211–240.Google Scholar
  21. Lund, K., &Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence.Behavior Research Methods, Instruments, & Computers,28, 203–208.Google Scholar
  22. Lund, K., &Burgess, C. (1997).Recurrent neural networks and global co-occurrence models: Developing contextual representations of word-meaning. Paper presented at the NIPS*97 (Neural Information Processing Systems) Neural Models of Concept Learning Postconference Workshop, Breckenridge, CO.Google Scholar
  23. Lund, K., Burgess, C., &Atchley, R. A. (1995). Semantic and associative priming in high-dimensional semantic space.Proceedings of the Cognitive Science Society (pp. 660–665). Hillsdale, NJ: Erlbaum.Google Scholar
  24. Lund, K., Burgess, C., &Audet, C. (1996). Dissociating semantic and associative word relationships using high-dimensional semantic space.Proceedings of the Cognitive Science Society (pp. 603–608). Hillsdale, NJ: Erlbaum.Google Scholar
  25. Lupker, S. J. (1984). Semantic priming without association: A second look.Journal of Verbal Learning & Verbal Behavior,23, 709–733.CrossRefGoogle Scholar
  26. McRae, K., &Boisvert, S. (in press). Automatic semantic similarity priming.Journal of Experimental Psychology: Learning, Memory, & Cognition.Google Scholar
  27. Miller, G. (1969). The organization of lexical memory: Are word associations sufficient? In G. A. Talland & N. C. Waugh (Eds.),The pathology of memory (pp. 223–237). New York: Academic Press.Google Scholar
  28. Morris, W. (Ed.) (1971).The American Heritage dictionary of the English language. Boston: American Heritage.Google Scholar
  29. Neely, J. H. (1991). Semantic priming effects in visual word recognition: A selective review of current findings and theories. In D. Besner & G.W. Humphreys (Eds.),Basic processes in reading: Visual word recognition (pp. 264–336). Hillsdale, NJ: Erlbaum.Google Scholar
  30. Nelson, K. (1977). The syntagmatic-paradigmatic shift revisited: A review of research and theory.Psychological Bulletin,84, 93–116.CrossRefPubMedGoogle Scholar
  31. Ogden, C. K., &Richards, I. A. (1923).The meaning of meaning. New York: Harcourt, Brace.Google Scholar
  32. Osgood, C. E. (1971). Exploration in semantic space: A personal diary.Journal of Social Issues,27, 5–64.CrossRefGoogle Scholar
  33. Osgood, C. E., Suci, G. J., &Tannenbaum, P. H. (1957).The measurement of meaning. Urbana: University of Illinois Press.Google Scholar
  34. Palermo, D. S., &Jenkins, J. J. (1964).Word association norms grade school through college. Minneapolis: University of Minnesota Press.Google Scholar
  35. Rips, L. J., Shoben, E. J., &Smith, E. E. (1973). Semantic distance and the verification of semantic relations.Journal of Verbal Learning & Verbal Behavior,12, 1–20.CrossRefGoogle Scholar
  36. Shelton, J. R., &Martin, R. C. (1992). How semantic is automatic semantic priming?Journal of Experimental Psychology: Learning, Memory, & Cognition,18, 1191–1210.CrossRefGoogle Scholar
  37. Spence, D. P., &Owens, K. C. (1990). Lexical co-occurrence and association strength.Journal of Psycholinguistic Research,19, 317–330.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 1998

Authors and Affiliations

  1. 1.Psychology DepartmentUniversity of CaliforniaRiverside

Personalised recommendations