Distributional semantic models (DSMs) specify learning mechanisms with which humans construct a deep representation of word meaning from statistical regularities in language. Despite their remarkable success at fitting human semantic data, virtually all DSMs may be classified as prototype models in that they try to construct a single representation for a word’s meaning aggregated across contexts. This prototype representation conflates multiple meanings and senses of words into a center of tendency, often losing the subordinate senses of a word in favor of more frequent ones. We present an alternative instance-based DSM based on the classic MINERVA 2 multiple-trace model of episodic memory. The model stores a representation of each language instance in a corpus, and a word’s meaning is constructed on-the-fly when presented with a retrieval cue. Across two experiments with homonyms in both an artificial and natural language corpus, we show how the instance-based model can naturally account for the subordinate meanings of words in appropriate context due to nonlinear activation over stored instances, but classic prototype DSMs cannot. The instance-based account suggests that meaning may not be something that is created during learning or stored per se, but may rather be an artifact of retrieval from an episodic memory store.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Ignore the misspelling of break in the vehicular sense (i.e., brake). If the language is auditory, then the phonology of the break-brake homophone is identical, and so we use a single spelling (break) here so the word has an identical input to the model in either verb sense.
We could have used a substantially smaller dimensionality for the word vectors, but very high dimensionality vectors allowed us to derive stable semantic representations later in the paper when we apply the theory to a large corpus of natural language.
Armstrong, B. C., Tokowicz, N., & Plaut, D. C. (2012). eDom: norming software and relative meaning frequencies for 544 English homonyms. Behavior Research Methods, 44, 1015–1027.
Arndt, J., & Hirshman, E. (1998). True and false recognition in MINERVA2: explanations from a global matching perspective. Journal of Memory and Language, 39, 371–391.
Aujla, H., Jamieson, R. K., & Cook, M. T. (2018). A psychologically inspired search engine. In Lecture notes in computer science: high performance computing systems and applications. Springer, Berlin (in press).
Bartlett, F. C. (1932). Remembering. Cambridge.
Bedi, G., Carrillo, F., Cecchi, G. A., Slezak, D. F., Sigman, M., Mota, N. B., Ribeiro, S., Javitt, D. C., Copelli, M., & Corcoran, C. M. (2015). Automated analysis of free speech predicts psychosis onset in high-risk youths. npj Schizophrenia.
Benjamin, A. S. (2010). Representational explanations of “process” dissociations in recognition: the DRYAD theory of aging and memory judgments. Psychological Review, 117, 1055–1079.
Brooks, L. R. (1978). Nonanalytic concept formation and memory for instances. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization (pp. 169–211). Hillsdale: Erlbaum.
Brooks, L. R. (1987). Decentralized control of categorization: the role of prior processing episodes. In U. Neisser (Ed.), Concepts and conceptual development: ecological and intellectual factors in categorization (pp. 141–174). Cambridge: Cambridge University Press.
Clark, S. E. (1997). A familiarity-based account of confidence–accuracy inversions in recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 232–238.
Cohen, T., & Widdows, D. (2016). Embedding probabilities in predication space with Hermitian holographic reduced representations. In H. Atmanspacher, T. Filk, & E. Pothos (Eds.), Quantum interaction. QI 2015. Lecture notes in computer science (Vol. 9535, pp. 245–257). Cham: Springer.
Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning & Verbal Behavior, 8, 240–247.
Curtis, E. T., & Jamieson, R. K. (2018). Computational and empirical simulations of selective memory impairments: converging evidence for a single-system account of memory dissociations. Quarterly Journal of Experimental Psychology (in press).
Dennis, S. (2005). A memory-based theory of verbal cognition. Cognitive Science, 29, 145–193.
Dougherty, M. R. P., Gettys, C. F., & Ogden, E. E. (1999). MINERVA-DM: a memory processes model for judgments of likelihood. Psychological Review, 106, 180–209.
Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179–211.
Erk, K., & Padó, S. (2008). A structured vector space model for word meaning in context. In Proceedings of the conference on empirical methods in natural language processing (pp. 897–906). Association for Computational Linguistics.
Estes, W. K. (1994). Classification and cognition. Oxford University Press.
Feldman-Stewart, D., & Mewhort, D. J. K. (1994). Learning in small connectionist networks does not generalize to large networks. Psychological Research, 56, 99–103.
Firth, J. R. (1957). A synopsis of linguistic theory. Studies in Linguistic Analysis, 1930–1955.
Foltz, P. W., Laham, D., & Landauer, T. K. (1999). The intelligent essay assessor: applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1, 939–944.
Griffiths, T. L., Steyvers, M., Blei, D. M., & Tenenbaum, J. B. (2005). Integrating topics and syntax. In Advances in Neural Information Processing Systems (pp. 537–544).
Griffiths, T. L., Steyvers, M., & Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114, 211–244.
Godden, D., & Baddeley, A. (1975). Context dependent memory in two natural environments. British Journal of Psychology, 66, 325–331.
Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105, 251–279.
Günther, F., Dudschig, C., & Kaup, B. (2015). LSAfun—an R package for computations based on latent semantic analysis. Behavior Research Methods, 47, 930–944.
Hintzman, D. L. (1984). MINERVA-2: a simulation model of human memory. Behavior Research Methods, Instruments & Computers, 16, 96–101.
Hintzman, D. L. (1986). “Schema abstraction” in a multiple-trace memory model. Psychological Review, 93, 411–428.
Hintzman, D. L. (1988). Judgments of frequency and recognition memory in a multiple-trace memory model. Psychological Review, 95, 528–551.
Jamieson, R. K., Crump, M. J. C., & Hannah, S. D. (2012). An instance theory of associative learning. Learning & Behavior, 40, 61–82.
Jamieson, R. K., Hannah, S. D., & Crump, M. J. C. (2010b). A memory-based account of retrospective revaluation. Canadian Journal of Experimental Psychology, 64, 153–164.
Jamieson, R. K., & Hauri, B. (2012). An exemplar model of performance in the artificial grammar task: holographic representation. Canadian Journal of Experimental Psychology, 66, 98–105.
Jamieson, R. K., Holmes, S., & Mewhort, D. J. K. (2010a). Global similarity predicts dissociation of classification and recognition: evidence questioning the implicit/explicit learning distinction in amnesia. Journal of Experimental Psychology: Learning, Memory and Cognition, 36, 1529–1535.
Jamieson, R. K., & Mewhort, D. J. K. (2005). The influence of grammatical, local, and organizational redundancy on implicit learning: an analysis using information theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 9–23.
Jamieson, R. K., & Mewhort, D. J. K. (2009a). Applying an exemplar model to the artificial-grammar task: inferring grammaticality from similarity. Quarterly Journal of Experimental Psychology, 62, 550–575.
Jamieson, R. K., & Mewhort, D. J. K. (2009b). Applying an exemplar model to the serial reaction time task: anticipating from experience. Quarterly Journal of Experimental Psychology, 62, 1757–1783.
Jamieson, R. K., & Mewhort, D. J. K. (2010). Applying an exemplar model to the artificial-grammar task: string-completion and performance for individual items. Quarterly Journal of Experimental Psychology, 63, 1014–1039.
Jamieson, R. K., & Mewhort, D. J. K. (2011). Grammaticality is inferred from global similarity: a reply to Kinder (2010). Quarterly Journal of Experimental Psychology, 64, 209–216.
Jamieson, R. K., Mewhort, D. J. K., & Hockley, W. E. (2016a). A computational account of the production effect: still playing twenty questions with nature. Canadian Journal of Experimental Psychology, 70, 154–164.
Jamieson, R. K., Nevzorova, U., Lee, G., & Mewhort, D. J. K. (2016b). Information theory and artificial grammar learning: inferring grammaticality from redundancy. Psychological Research, 80, 195–211.
Jamieson, R. K., Vokey, J. R., & Mewhort, D. J. K. (2017). Implicit learning is order dependent. Psychological Research, 81, 204–218.
Jones, M. N. (2017). Big data in cognitive science. United Kingdom: Psychology Press, Taylor & Francis.
Johns, B. T., & Jones, M. N. (2015). Generating structure from experience: A retrieval-based model of language processing. Canadian Journal of Experimental Psychology , 69, 233–251.
Jones, M. N., Kintsch, W., & Mewhort, D. J. K. (2006). High-dimensional semantic space accounts of priming. Journal of Memory and Language, 55, 534–552.
Jones, M. N., & Mewhort, D. J. K. (2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114, 1–37.
Johns, B. T., Taler, V., Pisoni, D. B., Farlow, M. R., Hake, A. M., Kareken, D. A., & Jones, M. N. (2013). Using cognitive models to investigate the temporal dynamics of semantic memory impairments in the development of Alzheimer’s disease. In Proceedings of the 12th international conference on cognitive modeling (pp. 23–28).
Kintsch, W. (2001). Predication. Cognitive Science, 25, 173–202.
Kintsch, W., & Mangalath, P. (2011). The construction of meaning. Topics in Cognitive Science, 3(2), 346-370.
Kwantes, P. J. (2005). Using context to build semantics. Psychonomic Bulletin & Review, 12, 703–710.
Kwantes, P., & Neal, A. (2006). Why people underestimate y when extrapolating in linear functions. Journal of Experimental Psychology: Learning Memory, and Cognition, 32, 1019–1030.
Kwantes, P. J., Derbentseva, N., Lam, Q., Vartanian, O., & Marmurek, H. H. (2016). Assessing the Big Five personality traits with latent semantic analysis. Personality and Individual Differences, 102, 229–233.
Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: the latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240.
Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28, 203–208.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (pp. 3111–3119).
Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Memory and Language, 16(5), 519
Murdock, B. B. (1982). A theory for the storage and retrieval of item and associative information. Psychological Review, 89, 609–626.
Murdock, B. B. (1983). A distributed memory model for serial-order information. Psychological Review, 90, 316–338.
Murdock, B. B. (1995). Developing TODAM: three models for serial-order information. Memory & Cognition, 23, 631–645.
Murdock, B. B. (1997). Context and mediators in a theory of distributed associative memory (TODAM2). Psychological Review, 104, 839–862.
Newell, A. (1973). You can’t play 20 questions with nature and win: projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing (pp. 283–308). New York: Academic.
Newell, A. (1994). Unified theories of cognition. Harvard University Press.
Nosofsky, R. M. (1984). Choice, similarity, and the context theory of classification. Journal of Experimental Psychology: Learning, Memory, & Cognition, 10, 104–114.
Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57.
Recchia, G. L., Jones, M. N., Sahlgren, M., & Kanerva, P. (2010). Encoding sequential information in vector space models of semantics: comparing holographic reduced representation and random permutation. In S. Ohisson & R. Catrambone (Eds.), Cognition in flux: Proceedings of the 32nd annual cognitive science society (pp. 865–870). Austin: Cognitive Science Society.
Reisinger, J., & Mooney, R. J. (2010, June). Multi-prototype vector-space models of word meaning. In Human language technologies: the 2010 annual conference of the North American Chapter of the Association for Computational Linguistics (pp. 109–117). Association for Computational Linguistics.
Rosch, E., & Mervis, C. B. (1975). Family resemblances: Studies in the internal structure of categories. Cognitive psychology, 7(4), 573–605.
Rubin, T. N., Koyejo, O., Gorgolewski, K. J., Jones, M. N., Poldrack, R. A., & Yarkoni, T. (2016a). Decoding brain activity using a large-scale probabilistic functional-anatomical atlas of human cognition. bioRxiv, 059618.
Rubin, T., Koyejo, O., Jones, M. N., & Yarkoni, T. (2016b). Generalized correspondence-LDA models (GC-LDA) for identifying functional regions in the brain. Advances in Neural Information Processing Systems.
Schvaneveldt, R. W., Meyer, D. E., & Becker, C. A. (1976). Lexical ambiguity, semantic context, and visual word recognition. Human perception and performance. Journal of experimental psychology 2(2), 243.
Shepard, R. N. (1980). Multidimensional scaling, tree-fitting, and clustering. Science, 210, 390–398.
Simon, H. A. (1969). The sciences of the artificial. Cambridge: MIT Press.
Smits, T., Storms, G., Rosseel, Y., & De Boeck, P. (2002). Fruits and vegetables categorized: an application of the generalized context model. Psychonomic Bulletin and Review, 9, 836–844.
Stanton, R. D., Nosofsky, R. M., & Zaki, S. R. (2002). Comparisons between exemplar similarity and mixed prototype models using a linearly separable category structure. Memory & Cognition, 30, 934–944.
Stone, B., Dennis, S., & Kwantes, P. J. (2011). Comparing methods for single paragraph similarity analysis. Topics in Cognitive Science, 3, 92–122.
Storms, G., De Boeck, P., & Ruts, W. (2000). Prototype and exemplar based information in natural language categories. Journal of Memory and Language, 42, 51–73.
Surprenant, A. M., & Neath, I. (2013). Principles of memory. Psychology Press.
Thomas, R. P., Dougherty, M. R., Sprenger, A. M., & Harbison, J. I. (2008). Diagnostic hypothesis generation and human judgment. Psychological Review, 115, 155–185.
Tulving, E. (1972). Episodic and semantic memory. Organization of Memory, 1, 381–403.
Tulving, E., & Pearlstone, Z. (1966). Availability versus accessibility of information in memory for words. Journal of Verbal Learning & Verbal Behavior, 5, 381–391.
Tulving, E., & Thomson, D. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352–373.
Tulving, E., & Watkins, M. J. (1973). Continuity between recall and recognition. The American Journal of Psychology, 739–748.
Verbeemen, T., Vanpaemel, W., Pattyn, S., Storms, G., & Verguts, T. (2007). Beyond exemplars and prototypes as memory representations of natural concepts: a clustering approach. Journal of Memory and Language, 56, 537–554.
Voorspoels, W., Vanpaemel, W., & Storms, G. (2008). Exemplars and prototypes in natural language concepts: a typicality-based evaluation. Psychonomic Bulletin and Review, 15, 630–637.
Voorspoels, W., Vanpaemel, W., & Storms, G. (2011). A formal ideal-based account of typicality. Psychonomic Bulletin and Review, 18, 1006–1014.
About this article
Cite this article
Jamieson, R.K., Avery, J.E., Johns, B.T. et al. An Instance Theory of Semantic Memory. Comput Brain Behav 1, 119–136 (2018). https://doi.org/10.1007/s42113-018-0008-2
- Semantic memory
- Computational model
- Exemplar-based model
- Episodic memory