Skip to main content

An Instance Theory of Semantic Memory

Computational Brain & Behavior volume 1, pages 119–136 (2018)Cite this article

A Correction to this article was published on 29 July 2019

This article has been updated

Abstract

Distributional semantic models (DSMs) specify learning mechanisms with which humans construct a deep representation of word meaning from statistical regularities in language. Despite their remarkable success at fitting human semantic data, virtually all DSMs may be classified as prototype models in that they try to construct a single representation for a word’s meaning aggregated across contexts. This prototype representation conflates multiple meanings and senses of words into a center of tendency, often losing the subordinate senses of a word in favor of more frequent ones. We present an alternative instance-based DSM based on the classic MINERVA 2 multiple-trace model of episodic memory. The model stores a representation of each language instance in a corpus, and a word’s meaning is constructed on-the-fly when presented with a retrieval cue. Across two experiments with homonyms in both an artificial and natural language corpus, we show how the instance-based model can naturally account for the subordinate meanings of words in appropriate context due to nonlinear activation over stored instances, but classic prototype DSMs cannot. The instance-based account suggests that meaning may not be something that is created during learning or stored per se, but may rather be an artifact of retrieval from an episodic memory store.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Change history

  • 29 July 2019

    Figure 9 in the original version of the article contained an error. The corrected Fig.��9 is presented below. Conclusions from the Instance Theory of Semantics (ITS) are preserved. However, conclusions from LSA and BEAGLE are not.

  • 29 July 2019

    Figure 9 in the original version of the article contained an error. The corrected Fig.��9 is presented below. Conclusions from the Instance Theory of Semantics (ITS) are preserved. However, conclusions from LSA and BEAGLE are not.

Notes

  1. Ignore the misspelling of break in the vehicular sense (i.e., brake). If the language is auditory, then the phonology of the break-brake homophone is identical, and so we use a single spelling (break) here so the word has an identical input to the model in either verb sense.

  2. We could have used a substantially smaller dimensionality for the word vectors, but very high dimensionality vectors allowed us to derive stable semantic representations later in the paper when we apply the theory to a large corpus of natural language.

References

  • Armstrong, B. C., Tokowicz, N., & Plaut, D. C. (2012). eDom: norming software and relative meaning frequencies for 544 English homonyms. Behavior Research Methods, 44, 1015–1027.

    Article  Google Scholar 

  • Arndt, J., & Hirshman, E. (1998). True and false recognition in MINERVA2: explanations from a global matching perspective. Journal of Memory and Language, 39, 371–391.

    Article  Google Scholar 

  • Aujla, H., Jamieson, R. K., & Cook, M. T. (2018). A psychologically inspired search engine. In Lecture notes in computer science: high performance computing systems and applications. Springer, Berlin (in press).

  • Bartlett, F. C. (1932). Remembering. Cambridge.

  • Bedi, G., Carrillo, F., Cecchi, G. A., Slezak, D. F., Sigman, M., Mota, N. B., Ribeiro, S., Javitt, D. C., Copelli, M., & Corcoran, C. M. (2015). Automated analysis of free speech predicts psychosis onset in high-risk youths. npj Schizophrenia.

  • Benjamin, A. S. (2010). Representational explanations of “process” dissociations in recognition: the DRYAD theory of aging and memory judgments. Psychological Review, 117, 1055–1079.

    Article  Google Scholar 

  • Brooks, L. R. (1978). Nonanalytic concept formation and memory for instances. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization (pp. 169–211). Hillsdale: Erlbaum.

    Google Scholar 

  • Brooks, L. R. (1987). Decentralized control of categorization: the role of prior processing episodes. In U. Neisser (Ed.), Concepts and conceptual development: ecological and intellectual factors in categorization (pp. 141–174). Cambridge: Cambridge University Press.

    Google Scholar 

  • Clark, S. E. (1997). A familiarity-based account of confidence–accuracy inversions in recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 232–238.

    Google Scholar 

  • Cohen, T., & Widdows, D. (2016). Embedding probabilities in predication space with Hermitian holographic reduced representations. In H. Atmanspacher, T. Filk, & E. Pothos (Eds.), Quantum interaction. QI 2015. Lecture notes in computer science (Vol. 9535, pp. 245–257). Cham: Springer.

    Google Scholar 

  • Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning & Verbal Behavior, 8, 240–247.

    Article  Google Scholar 

  • Curtis, E. T., & Jamieson, R. K. (2018). Computational and empirical simulations of selective memory impairments: converging evidence for a single-system account of memory dissociations. Quarterly Journal of Experimental Psychology (in press).

  • Dennis, S. (2005). A memory-based theory of verbal cognition. Cognitive Science, 29, 145–193.

    Article  Google Scholar 

  • Dougherty, M. R. P., Gettys, C. F., & Ogden, E. E. (1999). MINERVA-DM: a memory processes model for judgments of likelihood. Psychological Review, 106, 180–209.

    Article  Google Scholar 

  • Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179–211.

    Article  Google Scholar 

  • Erk, K., & Padó, S. (2008). A structured vector space model for word meaning in context. In Proceedings of the conference on empirical methods in natural language processing (pp. 897–906). Association for Computational Linguistics.

  • Estes, W. K. (1994). Classification and cognition. Oxford University Press.

  • Feldman-Stewart, D., & Mewhort, D. J. K. (1994). Learning in small connectionist networks does not generalize to large networks. Psychological Research, 56, 99–103.

    Article  Google Scholar 

  • Firth, J. R. (1957). A synopsis of linguistic theory. Studies in Linguistic Analysis, 1930–1955.

  • Foltz, P. W., Laham, D., & Landauer, T. K. (1999). The intelligent essay assessor: applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1, 939–944.

    Google Scholar 

  • Griffiths, T. L., Steyvers, M., Blei, D. M., & Tenenbaum, J. B. (2005). Integrating topics and syntax. In Advances in Neural Information Processing Systems (pp. 537–544).

  • Griffiths, T. L., Steyvers, M., & Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114, 211–244.

    Article  Google Scholar 

  • Godden, D., & Baddeley, A. (1975). Context dependent memory in two natural environments. British Journal of Psychology, 66, 325–331.

    Article  Google Scholar 

  • Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105, 251–279.

    Article  Google Scholar 

  • Günther, F., Dudschig, C., & Kaup, B. (2015). LSAfun—an R package for computations based on latent semantic analysis. Behavior Research Methods, 47, 930–944.

    Article  Google Scholar 

  • Hintzman, D. L. (1984). MINERVA-2: a simulation model of human memory. Behavior Research Methods, Instruments & Computers, 16, 96–101.

    Article  Google Scholar 

  • Hintzman, D. L. (1986). “Schema abstraction” in a multiple-trace memory model. Psychological Review, 93, 411–428.

    Article  Google Scholar 

  • Hintzman, D. L. (1988). Judgments of frequency and recognition memory in a multiple-trace memory model. Psychological Review, 95, 528–551.

    Article  Google Scholar 

  • Jamieson, R. K., Crump, M. J. C., & Hannah, S. D. (2012). An instance theory of associative learning. Learning & Behavior, 40, 61–82.

    Article  Google Scholar 

  • Jamieson, R. K., Hannah, S. D., & Crump, M. J. C. (2010b). A memory-based account of retrospective revaluation. Canadian Journal of Experimental Psychology, 64, 153–164.

    Article  Google Scholar 

  • Jamieson, R. K., & Hauri, B. (2012). An exemplar model of performance in the artificial grammar task: holographic representation. Canadian Journal of Experimental Psychology, 66, 98–105.

    Article  Google Scholar 

  • Jamieson, R. K., Holmes, S., & Mewhort, D. J. K. (2010a). Global similarity predicts dissociation of classification and recognition: evidence questioning the implicit/explicit learning distinction in amnesia. Journal of Experimental Psychology: Learning, Memory and Cognition, 36, 1529–1535.

    Google Scholar 

  • Jamieson, R. K., & Mewhort, D. J. K. (2005). The influence of grammatical, local, and organizational redundancy on implicit learning: an analysis using information theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 9–23.

    PubMed  Google Scholar 

  • Jamieson, R. K., & Mewhort, D. J. K. (2009a). Applying an exemplar model to the artificial-grammar task: inferring grammaticality from similarity. Quarterly Journal of Experimental Psychology, 62, 550–575.

    Article  Google Scholar 

  • Jamieson, R. K., & Mewhort, D. J. K. (2009b). Applying an exemplar model to the serial reaction time task: anticipating from experience. Quarterly Journal of Experimental Psychology, 62, 1757–1783.

    Article  Google Scholar 

  • Jamieson, R. K., & Mewhort, D. J. K. (2010). Applying an exemplar model to the artificial-grammar task: string-completion and performance for individual items. Quarterly Journal of Experimental Psychology, 63, 1014–1039.

    Article  Google Scholar 

  • Jamieson, R. K., & Mewhort, D. J. K. (2011). Grammaticality is inferred from global similarity: a reply to Kinder (2010). Quarterly Journal of Experimental Psychology, 64, 209–216.

    Article  Google Scholar 

  • Jamieson, R. K., Mewhort, D. J. K., & Hockley, W. E. (2016a). A computational account of the production effect: still playing twenty questions with nature. Canadian Journal of Experimental Psychology, 70, 154–164.

    Article  Google Scholar 

  • Jamieson, R. K., Nevzorova, U., Lee, G., & Mewhort, D. J. K. (2016b). Information theory and artificial grammar learning: inferring grammaticality from redundancy. Psychological Research, 80, 195–211.

    Article  Google Scholar 

  • Jamieson, R. K., Vokey, J. R., & Mewhort, D. J. K. (2017). Implicit learning is order dependent. Psychological Research, 81, 204–218.

    Article  Google Scholar 

  • Jones, M. N. (2017). Big data in cognitive science. United Kingdom: Psychology Press, Taylor & Francis.

  • Johns, B. T., & Jones, M. N. (2015). Generating structure from experience: A retrieval-based model of language processing. Canadian Journal of Experimental Psychology , 69, 233–251.

    Article  Google Scholar 

  • Jones, M. N., Kintsch, W., & Mewhort, D. J. K. (2006). High-dimensional semantic space accounts of priming. Journal of Memory and Language, 55, 534–552.

    Article  Google Scholar 

  • Jones, M. N., & Mewhort, D. J. K. (2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114, 1–37.

    Article  Google Scholar 

  • Johns, B. T., Taler, V., Pisoni, D. B., Farlow, M. R., Hake, A. M., Kareken, D. A., & Jones, M. N. (2013). Using cognitive models to investigate the temporal dynamics of semantic memory impairments in the development of Alzheimer’s disease. In Proceedings of the 12th international conference on cognitive modeling (pp. 23–28).

  • Kintsch, W. (2001). Predication. Cognitive Science, 25, 173–202.

    Article  Google Scholar 

  • Kintsch, W., & Mangalath, P. (2011). The construction of meaning. Topics in Cognitive Science, 3(2), 346-370.

    Article  Google Scholar 

  • Kwantes, P. J. (2005). Using context to build semantics. Psychonomic Bulletin & Review, 12, 703–710.

    Article  Google Scholar 

  • Kwantes, P., & Neal, A. (2006). Why people underestimate y when extrapolating in linear functions. Journal of Experimental Psychology: Learning Memory, and Cognition, 32, 1019–1030.

    Google Scholar 

  • Kwantes, P. J., Derbentseva, N., Lam, Q., Vartanian, O., & Marmurek, H. H. (2016). Assessing the Big Five personality traits with latent semantic analysis. Personality and Individual Differences, 102, 229–233.

    Article  Google Scholar 

  • Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: the latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240.

    Article  Google Scholar 

  • Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28, 203–208.

    Article  Google Scholar 

  • Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (pp. 3111–3119).

  • Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Memory and Language, 16(5), 519

    Article  Google Scholar 

  • Murdock, B. B. (1982). A theory for the storage and retrieval of item and associative information. Psychological Review, 89, 609–626.

    Article  Google Scholar 

  • Murdock, B. B. (1983). A distributed memory model for serial-order information. Psychological Review, 90, 316–338.

    Article  Google Scholar 

  • Murdock, B. B. (1995). Developing TODAM: three models for serial-order information. Memory & Cognition, 23, 631–645.

    Article  Google Scholar 

  • Murdock, B. B. (1997). Context and mediators in a theory of distributed associative memory (TODAM2). Psychological Review, 104, 839–862.

    Article  Google Scholar 

  • Newell, A. (1973). You can’t play 20 questions with nature and win: projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing (pp. 283–308). New York: Academic.

    Chapter  Google Scholar 

  • Newell, A. (1994). Unified theories of cognition. Harvard University Press.

  • Nosofsky, R. M. (1984). Choice, similarity, and the context theory of classification. Journal of Experimental Psychology: Learning, Memory, & Cognition, 10, 104–114.

    Google Scholar 

  • Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57.

    Article  Google Scholar 

  • Recchia, G. L., Jones, M. N., Sahlgren, M., & Kanerva, P. (2010). Encoding sequential information in vector space models of semantics: comparing holographic reduced representation and random permutation. In S. Ohisson & R. Catrambone (Eds.), Cognition in flux: Proceedings of the 32nd annual cognitive science society (pp. 865–870). Austin: Cognitive Science Society.

    Google Scholar 

  • Reisinger, J., & Mooney, R. J. (2010, June). Multi-prototype vector-space models of word meaning. In Human language technologies: the 2010 annual conference of the North American Chapter of the Association for Computational Linguistics (pp. 109–117). Association for Computational Linguistics.

  • Rosch, E., & Mervis, C. B. (1975). Family resemblances: Studies in the internal structure of categories. Cognitive psychology, 7(4), 573–605.

    Article  Google Scholar 

  • Rubin, T. N., Koyejo, O., Gorgolewski, K. J., Jones, M. N., Poldrack, R. A., & Yarkoni, T. (2016a). Decoding brain activity using a large-scale probabilistic functional-anatomical atlas of human cognition. bioRxiv, 059618.

  • Rubin, T., Koyejo, O., Jones, M. N., & Yarkoni, T. (2016b). Generalized correspondence-LDA models (GC-LDA) for identifying functional regions in the brain. Advances in Neural Information Processing Systems.

  • Schvaneveldt, R. W., Meyer, D. E., & Becker, C. A. (1976). Lexical ambiguity, semantic context, and visual word recognition. Human perception and performance. Journal of experimental psychology 2(2), 243.

    PubMed  Google Scholar 

  • Shepard, R. N. (1980). Multidimensional scaling, tree-fitting, and clustering. Science, 210, 390–398.

    Article  Google Scholar 

  • Simon, H. A. (1969). The sciences of the artificial. Cambridge: MIT Press.

    Google Scholar 

  • Smits, T., Storms, G., Rosseel, Y., & De Boeck, P. (2002). Fruits and vegetables categorized: an application of the generalized context model. Psychonomic Bulletin and Review, 9, 836–844.

    Article  Google Scholar 

  • Stanton, R. D., Nosofsky, R. M., & Zaki, S. R. (2002). Comparisons between exemplar similarity and mixed prototype models using a linearly separable category structure. Memory & Cognition, 30, 934–944.

    Article  Google Scholar 

  • Stone, B., Dennis, S., & Kwantes, P. J. (2011). Comparing methods for single paragraph similarity analysis. Topics in Cognitive Science, 3, 92–122.

    Article  Google Scholar 

  • Storms, G., De Boeck, P., & Ruts, W. (2000). Prototype and exemplar based information in natural language categories. Journal of Memory and Language, 42, 51–73.

    Article  Google Scholar 

  • Surprenant, A. M., & Neath, I. (2013). Principles of memory. Psychology Press.

  • Thomas, R. P., Dougherty, M. R., Sprenger, A. M., & Harbison, J. I. (2008). Diagnostic hypothesis generation and human judgment. Psychological Review, 115, 155–185.

    Article  Google Scholar 

  • Tulving, E. (1972). Episodic and semantic memory. Organization of Memory, 1, 381–403.

    Google Scholar 

  • Tulving, E., & Pearlstone, Z. (1966). Availability versus accessibility of information in memory for words. Journal of Verbal Learning & Verbal Behavior, 5, 381–391.

    Article  Google Scholar 

  • Tulving, E., & Thomson, D. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352–373.

    Article  Google Scholar 

  • Tulving, E., & Watkins, M. J. (1973). Continuity between recall and recognition. The American Journal of Psychology, 739–748.

    Article  Google Scholar 

  • Verbeemen, T., Vanpaemel, W., Pattyn, S., Storms, G., & Verguts, T. (2007). Beyond exemplars and prototypes as memory representations of natural concepts: a clustering approach. Journal of Memory and Language, 56, 537–554.

    Article  Google Scholar 

  • Voorspoels, W., Vanpaemel, W., & Storms, G. (2008). Exemplars and prototypes in natural language concepts: a typicality-based evaluation. Psychonomic Bulletin and Review, 15, 630–637.

    Article  Google Scholar 

  • Voorspoels, W., Vanpaemel, W., & Storms, G. (2011). A formal ideal-based account of typicality. Psychonomic Bulletin and Review, 18, 1006–1014.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Manitoba, Winnipeg, Canada

    Randall K. Jamieson

  2. Department of Psychological and Brain Sciences, Indiana University, Bloomington, USA

    Johnathan E. Avery & Michael N. Jones

  3. University at Buffalo, Buffalo, USA

    Brendan T. Johns

Authors
  1. Randall K. Jamieson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johnathan E. Avery
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brendan T. Johns
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael N. Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael N. Jones.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jamieson, R.K., Avery, J.E., Johns, B.T. et al. An Instance Theory of Semantic Memory. Comput Brain Behav 1, 119–136 (2018). https://doi.org/10.1007/s42113-018-0008-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42113-018-0008-2

Keywords

  • Semantic memory
  • Computational model
  • Exemplar-based model
  • Episodic memory