Skip to main content

An Instance Theory of Semantic Memory

A Correction to this article was published on 29 July 2019

This article has been updated

Abstract

Distributional semantic models (DSMs) specify learning mechanisms with which humans construct a deep representation of word meaning from statistical regularities in language. Despite their remarkable success at fitting human semantic data, virtually all DSMs may be classified as prototype models in that they try to construct a single representation for a word’s meaning aggregated across contexts. This prototype representation conflates multiple meanings and senses of words into a center of tendency, often losing the subordinate senses of a word in favor of more frequent ones. We present an alternative instance-based DSM based on the classic MINERVA 2 multiple-trace model of episodic memory. The model stores a representation of each language instance in a corpus, and a word’s meaning is constructed on-the-fly when presented with a retrieval cue. Across two experiments with homonyms in both an artificial and natural language corpus, we show how the instance-based model can naturally account for the subordinate meanings of words in appropriate context due to nonlinear activation over stored instances, but classic prototype DSMs cannot. The instance-based account suggests that meaning may not be something that is created during learning or stored per se, but may rather be an artifact of retrieval from an episodic memory store.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Change history

  • 29 July 2019

    Figure 9 in the original version of the article contained an error. The corrected Fig.��9 is presented below. Conclusions from the Instance Theory of Semantics (ITS) are preserved. However, conclusions from LSA and BEAGLE are not.

  • 29 July 2019

    Figure 9 in the original version of the article contained an error. The corrected Fig.��9 is presented below. Conclusions from the Instance Theory of Semantics (ITS) are preserved. However, conclusions from LSA and BEAGLE are not.

Notes

  1. 1.

    Ignore the misspelling of break in the vehicular sense (i.e., brake). If the language is auditory, then the phonology of the break-brake homophone is identical, and so we use a single spelling (break) here so the word has an identical input to the model in either verb sense.

  2. 2.

    We could have used a substantially smaller dimensionality for the word vectors, but very high dimensionality vectors allowed us to derive stable semantic representations later in the paper when we apply the theory to a large corpus of natural language.

References

  1. Armstrong, B. C., Tokowicz, N., & Plaut, D. C. (2012). eDom: norming software and relative meaning frequencies for 544 English homonyms. Behavior Research Methods, 44, 1015–1027.

    Article  Google Scholar 

  2. Arndt, J., & Hirshman, E. (1998). True and false recognition in MINERVA2: explanations from a global matching perspective. Journal of Memory and Language, 39, 371–391.

    Article  Google Scholar 

  3. Aujla, H., Jamieson, R. K., & Cook, M. T. (2018). A psychologically inspired search engine. In Lecture notes in computer science: high performance computing systems and applications. Springer, Berlin (in press).

  4. Bartlett, F. C. (1932). Remembering. Cambridge.

  5. Bedi, G., Carrillo, F., Cecchi, G. A., Slezak, D. F., Sigman, M., Mota, N. B., Ribeiro, S., Javitt, D. C., Copelli, M., & Corcoran, C. M. (2015). Automated analysis of free speech predicts psychosis onset in high-risk youths. npj Schizophrenia.

  6. Benjamin, A. S. (2010). Representational explanations of “process” dissociations in recognition: the DRYAD theory of aging and memory judgments. Psychological Review, 117, 1055–1079.

    Article  Google Scholar 

  7. Brooks, L. R. (1978). Nonanalytic concept formation and memory for instances. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization (pp. 169–211). Hillsdale: Erlbaum.

    Google Scholar 

  8. Brooks, L. R. (1987). Decentralized control of categorization: the role of prior processing episodes. In U. Neisser (Ed.), Concepts and conceptual development: ecological and intellectual factors in categorization (pp. 141–174). Cambridge: Cambridge University Press.

    Google Scholar 

  9. Clark, S. E. (1997). A familiarity-based account of confidence–accuracy inversions in recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 232–238.

    Google Scholar 

  10. Cohen, T., & Widdows, D. (2016). Embedding probabilities in predication space with Hermitian holographic reduced representations. In H. Atmanspacher, T. Filk, & E. Pothos (Eds.), Quantum interaction. QI 2015. Lecture notes in computer science (Vol. 9535, pp. 245–257). Cham: Springer.

    Google Scholar 

  11. Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning & Verbal Behavior, 8, 240–247.

    Article  Google Scholar 

  12. Curtis, E. T., & Jamieson, R. K. (2018). Computational and empirical simulations of selective memory impairments: converging evidence for a single-system account of memory dissociations. Quarterly Journal of Experimental Psychology (in press).

  13. Dennis, S. (2005). A memory-based theory of verbal cognition. Cognitive Science, 29, 145–193.

    Article  Google Scholar 

  14. Dougherty, M. R. P., Gettys, C. F., & Ogden, E. E. (1999). MINERVA-DM: a memory processes model for judgments of likelihood. Psychological Review, 106, 180–209.

    Article  Google Scholar 

  15. Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179–211.

    Article  Google Scholar 

  16. Erk, K., & Padó, S. (2008). A structured vector space model for word meaning in context. In Proceedings of the conference on empirical methods in natural language processing (pp. 897–906). Association for Computational Linguistics.

  17. Estes, W. K. (1994). Classification and cognition. Oxford University Press.

  18. Feldman-Stewart, D., & Mewhort, D. J. K. (1994). Learning in small connectionist networks does not generalize to large networks. Psychological Research, 56, 99–103.

    Article  Google Scholar 

  19. Firth, J. R. (1957). A synopsis of linguistic theory. Studies in Linguistic Analysis, 1930–1955.

  20. Foltz, P. W., Laham, D., & Landauer, T. K. (1999). The intelligent essay assessor: applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1, 939–944.

    Google Scholar 

  21. Griffiths, T. L., Steyvers, M., Blei, D. M., & Tenenbaum, J. B. (2005). Integrating topics and syntax. In Advances in Neural Information Processing Systems (pp. 537–544).

  22. Griffiths, T. L., Steyvers, M., & Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114, 211–244.

    Article  Google Scholar 

  23. Godden, D., & Baddeley, A. (1975). Context dependent memory in two natural environments. British Journal of Psychology, 66, 325–331.

    Article  Google Scholar 

  24. Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105, 251–279.

    Article  Google Scholar 

  25. Günther, F., Dudschig, C., & Kaup, B. (2015). LSAfun—an R package for computations based on latent semantic analysis. Behavior Research Methods, 47, 930–944.

    Article  Google Scholar 

  26. Hintzman, D. L. (1984). MINERVA-2: a simulation model of human memory. Behavior Research Methods, Instruments & Computers, 16, 96–101.

    Article  Google Scholar 

  27. Hintzman, D. L. (1986). “Schema abstraction” in a multiple-trace memory model. Psychological Review, 93, 411–428.

    Article  Google Scholar 

  28. Hintzman, D. L. (1988). Judgments of frequency and recognition memory in a multiple-trace memory model. Psychological Review, 95, 528–551.

    Article  Google Scholar 

  29. Jamieson, R. K., Crump, M. J. C., & Hannah, S. D. (2012). An instance theory of associative learning. Learning & Behavior, 40, 61–82.

    Article  Google Scholar 

  30. Jamieson, R. K., Hannah, S. D., & Crump, M. J. C. (2010b). A memory-based account of retrospective revaluation. Canadian Journal of Experimental Psychology, 64, 153–164.

    Article  Google Scholar 

  31. Jamieson, R. K., & Hauri, B. (2012). An exemplar model of performance in the artificial grammar task: holographic representation. Canadian Journal of Experimental Psychology, 66, 98–105.

    Article  Google Scholar 

  32. Jamieson, R. K., Holmes, S., & Mewhort, D. J. K. (2010a). Global similarity predicts dissociation of classification and recognition: evidence questioning the implicit/explicit learning distinction in amnesia. Journal of Experimental Psychology: Learning, Memory and Cognition, 36, 1529–1535.

    Google Scholar 

  33. Jamieson, R. K., & Mewhort, D. J. K. (2005). The influence of grammatical, local, and organizational redundancy on implicit learning: an analysis using information theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 9–23.

    PubMed  Google Scholar 

  34. Jamieson, R. K., & Mewhort, D. J. K. (2009a). Applying an exemplar model to the artificial-grammar task: inferring grammaticality from similarity. Quarterly Journal of Experimental Psychology, 62, 550–575.

    Article  Google Scholar 

  35. Jamieson, R. K., & Mewhort, D. J. K. (2009b). Applying an exemplar model to the serial reaction time task: anticipating from experience. Quarterly Journal of Experimental Psychology, 62, 1757–1783.

    Article  Google Scholar 

  36. Jamieson, R. K., & Mewhort, D. J. K. (2010). Applying an exemplar model to the artificial-grammar task: string-completion and performance for individual items. Quarterly Journal of Experimental Psychology, 63, 1014–1039.

    Article  Google Scholar 

  37. Jamieson, R. K., & Mewhort, D. J. K. (2011). Grammaticality is inferred from global similarity: a reply to Kinder (2010). Quarterly Journal of Experimental Psychology, 64, 209–216.

    Article  Google Scholar 

  38. Jamieson, R. K., Mewhort, D. J. K., & Hockley, W. E. (2016a). A computational account of the production effect: still playing twenty questions with nature. Canadian Journal of Experimental Psychology, 70, 154–164.

    Article  Google Scholar 

  39. Jamieson, R. K., Nevzorova, U., Lee, G., & Mewhort, D. J. K. (2016b). Information theory and artificial grammar learning: inferring grammaticality from redundancy. Psychological Research, 80, 195–211.

    Article  Google Scholar 

  40. Jamieson, R. K., Vokey, J. R., & Mewhort, D. J. K. (2017). Implicit learning is order dependent. Psychological Research, 81, 204–218.

    Article  Google Scholar 

  41. Jones, M. N. (2017). Big data in cognitive science. United Kingdom: Psychology Press, Taylor & Francis.

  42. Johns, B. T., & Jones, M. N. (2015). Generating structure from experience: A retrieval-based model of language processing. Canadian Journal of Experimental Psychology , 69, 233–251.

    Article  Google Scholar 

  43. Jones, M. N., Kintsch, W., & Mewhort, D. J. K. (2006). High-dimensional semantic space accounts of priming. Journal of Memory and Language, 55, 534–552.

    Article  Google Scholar 

  44. Jones, M. N., & Mewhort, D. J. K. (2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114, 1–37.

    Article  Google Scholar 

  45. Johns, B. T., Taler, V., Pisoni, D. B., Farlow, M. R., Hake, A. M., Kareken, D. A., & Jones, M. N. (2013). Using cognitive models to investigate the temporal dynamics of semantic memory impairments in the development of Alzheimer’s disease. In Proceedings of the 12th international conference on cognitive modeling (pp. 23–28).

  46. Kintsch, W. (2001). Predication. Cognitive Science, 25, 173–202.

    Article  Google Scholar 

  47. Kintsch, W., & Mangalath, P. (2011). The construction of meaning. Topics in Cognitive Science, 3(2), 346-370.

    Article  Google Scholar 

  48. Kwantes, P. J. (2005). Using context to build semantics. Psychonomic Bulletin & Review, 12, 703–710.

    Article  Google Scholar 

  49. Kwantes, P., & Neal, A. (2006). Why people underestimate y when extrapolating in linear functions. Journal of Experimental Psychology: Learning Memory, and Cognition, 32, 1019–1030.

    Google Scholar 

  50. Kwantes, P. J., Derbentseva, N., Lam, Q., Vartanian, O., & Marmurek, H. H. (2016). Assessing the Big Five personality traits with latent semantic analysis. Personality and Individual Differences, 102, 229–233.

    Article  Google Scholar 

  51. Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: the latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240.

    Article  Google Scholar 

  52. Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28, 203–208.

    Article  Google Scholar 

  53. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (pp. 3111–3119).

  54. Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Memory and Language, 16(5), 519

    Article  Google Scholar 

  55. Murdock, B. B. (1982). A theory for the storage and retrieval of item and associative information. Psychological Review, 89, 609–626.

    Article  Google Scholar 

  56. Murdock, B. B. (1983). A distributed memory model for serial-order information. Psychological Review, 90, 316–338.

    Article  Google Scholar 

  57. Murdock, B. B. (1995). Developing TODAM: three models for serial-order information. Memory & Cognition, 23, 631–645.

    Article  Google Scholar 

  58. Murdock, B. B. (1997). Context and mediators in a theory of distributed associative memory (TODAM2). Psychological Review, 104, 839–862.

    Article  Google Scholar 

  59. Newell, A. (1973). You can’t play 20 questions with nature and win: projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing (pp. 283–308). New York: Academic.

    Chapter  Google Scholar 

  60. Newell, A. (1994). Unified theories of cognition. Harvard University Press.

  61. Nosofsky, R. M. (1984). Choice, similarity, and the context theory of classification. Journal of Experimental Psychology: Learning, Memory, & Cognition, 10, 104–114.

    Google Scholar 

  62. Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57.

    Article  Google Scholar 

  63. Recchia, G. L., Jones, M. N., Sahlgren, M., & Kanerva, P. (2010). Encoding sequential information in vector space models of semantics: comparing holographic reduced representation and random permutation. In S. Ohisson & R. Catrambone (Eds.), Cognition in flux: Proceedings of the 32nd annual cognitive science society (pp. 865–870). Austin: Cognitive Science Society.

    Google Scholar 

  64. Reisinger, J., & Mooney, R. J. (2010, June). Multi-prototype vector-space models of word meaning. In Human language technologies: the 2010 annual conference of the North American Chapter of the Association for Computational Linguistics (pp. 109–117). Association for Computational Linguistics.

  65. Rosch, E., & Mervis, C. B. (1975). Family resemblances: Studies in the internal structure of categories. Cognitive psychology, 7(4), 573–605.

    Article  Google Scholar 

  66. Rubin, T. N., Koyejo, O., Gorgolewski, K. J., Jones, M. N., Poldrack, R. A., & Yarkoni, T. (2016a). Decoding brain activity using a large-scale probabilistic functional-anatomical atlas of human cognition. bioRxiv, 059618.

  67. Rubin, T., Koyejo, O., Jones, M. N., & Yarkoni, T. (2016b). Generalized correspondence-LDA models (GC-LDA) for identifying functional regions in the brain. Advances in Neural Information Processing Systems.

  68. Schvaneveldt, R. W., Meyer, D. E., & Becker, C. A. (1976). Lexical ambiguity, semantic context, and visual word recognition. Human perception and performance. Journal of experimental psychology 2(2), 243.

    PubMed  Google Scholar 

  69. Shepard, R. N. (1980). Multidimensional scaling, tree-fitting, and clustering. Science, 210, 390–398.

    Article  Google Scholar 

  70. Simon, H. A. (1969). The sciences of the artificial. Cambridge: MIT Press.

    Google Scholar 

  71. Smits, T., Storms, G., Rosseel, Y., & De Boeck, P. (2002). Fruits and vegetables categorized: an application of the generalized context model. Psychonomic Bulletin and Review, 9, 836–844.

    Article  Google Scholar 

  72. Stanton, R. D., Nosofsky, R. M., & Zaki, S. R. (2002). Comparisons between exemplar similarity and mixed prototype models using a linearly separable category structure. Memory & Cognition, 30, 934–944.

    Article  Google Scholar 

  73. Stone, B., Dennis, S., & Kwantes, P. J. (2011). Comparing methods for single paragraph similarity analysis. Topics in Cognitive Science, 3, 92–122.

    Article  Google Scholar 

  74. Storms, G., De Boeck, P., & Ruts, W. (2000). Prototype and exemplar based information in natural language categories. Journal of Memory and Language, 42, 51–73.

    Article  Google Scholar 

  75. Surprenant, A. M., & Neath, I. (2013). Principles of memory. Psychology Press.

  76. Thomas, R. P., Dougherty, M. R., Sprenger, A. M., & Harbison, J. I. (2008). Diagnostic hypothesis generation and human judgment. Psychological Review, 115, 155–185.

    Article  Google Scholar 

  77. Tulving, E. (1972). Episodic and semantic memory. Organization of Memory, 1, 381–403.

    Google Scholar 

  78. Tulving, E., & Pearlstone, Z. (1966). Availability versus accessibility of information in memory for words. Journal of Verbal Learning & Verbal Behavior, 5, 381–391.

    Article  Google Scholar 

  79. Tulving, E., & Thomson, D. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352–373.

    Article  Google Scholar 

  80. Tulving, E., & Watkins, M. J. (1973). Continuity between recall and recognition. The American Journal of Psychology, 739–748.

    Article  Google Scholar 

  81. Verbeemen, T., Vanpaemel, W., Pattyn, S., Storms, G., & Verguts, T. (2007). Beyond exemplars and prototypes as memory representations of natural concepts: a clustering approach. Journal of Memory and Language, 56, 537–554.

    Article  Google Scholar 

  82. Voorspoels, W., Vanpaemel, W., & Storms, G. (2008). Exemplars and prototypes in natural language concepts: a typicality-based evaluation. Psychonomic Bulletin and Review, 15, 630–637.

    Article  Google Scholar 

  83. Voorspoels, W., Vanpaemel, W., & Storms, G. (2011). A formal ideal-based account of typicality. Psychonomic Bulletin and Review, 18, 1006–1014.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michael N. Jones.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jamieson, R.K., Avery, J.E., Johns, B.T. et al. An Instance Theory of Semantic Memory. Comput Brain Behav 1, 119–136 (2018). https://doi.org/10.1007/s42113-018-0008-2

Download citation

Keywords

  • Semantic memory
  • Computational model
  • Exemplar-based model
  • Episodic memory