Skip to main content

Evocation: analyzing and propagating a semantic link based on free word association


Studies of lexical–semantic relations aim to understand the mechanism of semantic memory and the organization of the mental lexicon. However, standard paradigmatic relations such as “hypernym” and “hyponym” cannot capture connections among concepts from different parts of speech. WordNet, which organizes synsets (i.e., synonym sets) using these lexical–semantic relations, is rather sparse in its connectivity. According to WordNet statistics, the average number of outgoing/incoming arcs for the hypernym/hyponym relation per synset is 1.33. Evocation, defined as how much a concept (expressed by one or more words) brings to mind another, is proposed as a new directed and weighted measure for the semantic relatedness among concepts. Commonly applied semantic relations and relatedness measures do not seem to be fully compatible with data that reflect evocations among concepts. They are compatible but evocation captures MORE. This work aims to provide a reliable and extendable dataset of concepts evoked by, and evoking, other concepts to enrich WordNet, the existing semantic network. We propose the use of disambiguated free word association data (first responses to verbal stimuli) to infer and collect evocation ratings. WordNet aims to represent the organization of mental lexicon, and free word association which has been used by psycholinguists to explore semantic organization can contribute to the understanding. This work was carried out in two phases. In the first phase, it was confirmed that existing free word association norms can be converted into evocation data computationally. In the second phase, a two-stage association-annotation procedure of collecting evocation data from human judgment was compared to the state-of-the-art method, showing that introducing free association can greatly improve the quality of the evocation data generated. Evocation can be incorporated into WordNet as directed links with scales, and benefits various natural language processing applications.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. Link to the evocation network created from the Free Association Norm data described in Sect. 4:

  2. The pertainym and derivationally related form relations in WordNet 3.1 connect adverbs and adjectives to their noun forms. These relations do not capture other semantic associations across parts of speech.


  4. Evocation datasets available on


  • Amazon’s Mechanical Turk (MTurk). (2009).

  • Banerjee, S., & Pedersen, T. (2002). An adapted Lesk algorithm for word sense disambiguation using WordNet. In Proceedings of the 3rd international conference on intelligent text processing and computational linguistics.

  • Batagelj, V., & Mrvar, A. (2006). Pajek dataset: USF free association norms.

  • Boyd-Graber, J., Fellbaum, C., Osherson, D., & Schapire, R. (2006). Adding dense, weighted connections to WordNet. In Proceedings of the thirds international WordNet conference. Jaeju, Korea.

  • British National Corpus. (2008).

  • Buss, D. M. (1987). Selection, evocation, and manipulation. Journal of Personality and Social Psychology, 53, 1214–1221.

    Article  Google Scholar 

  • Church, K., & Hanks, P. (1991). Word association norms, mutual information and lexicography. Computational Linguistics, 16(1), 22–29.

    Google Scholar 

  • Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behavior, 8, 240–247.

    Article  Google Scholar 

  • Cruse, D. A. (1986). Lexical semantics. New York: Cambridge University Press.

    Google Scholar 

  • Döllein, J. J. (1976). On non-related alternations. Linguistics, 9(1–2), 177–181.

    Google Scholar 

  • Ernst, A. (2010). A relational version of WordNet with evocation ratings and its application to word sense disambiguation. Unpublished Senior Thesis, Department of Computer Science, Princeton University.

  • Fellbaum, C. (1998). WordNet: An electronic Lexical Database. Cambridge, MA: MIT Press.

    Google Scholar 

  • Fellbaum, C., & Miller, G. A. (2003). Morphosemantic links in WordNet. Traitement automatique de langue.

  • Grice, G. R. (1968). Stimulus intensity and response evocation. Psychological Review, 75, 359–373.

    Article  Google Scholar 

  • Ipeirotis, P.G., Provost, F., & Wang, J. (2010). Quality management on Amazon Mechanical Turk. In Proceedings of the ACM SIGKDD workshop on human computation. pp. 64–67.

  • Jiang, J. J., & Conrath, D. W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of 10th international conference on research in computational linguistics.

  • Kilgarriff, A., Rychly, P., Smrz, P., & Tugwell, D. (2004). The sketch engine. In Proceedings of EURALEX 2004, Lorient, France, pp. 105–116.

  • Larsen, R. J., & Buss, D. M. (2002). Personality psychology: Domains of knowledge about human nature. NewYork: McGraw-Hill.

    Google Scholar 

  • Leacock, C., & Chodorow, M. (1998). Combining local context and WordNet similarity for word sense identification. In C. Fellbaum (Ed.), WordNet: An electronic lexical database. Cambridge, MA: MIT Press.

  • Lee, B., Parr, C. S., Plaisant, C., Bederson, B. B., Veksler, V. D., Gray, W. D., et al. (2006). TreePlus: Interactive exploration of networks with enhanced tree layouts. IEEE TVCG Special Issue on Visual Analytics, 12(6), 1414–1426.

    Google Scholar 

  • Lin, D. (1997). Using syntactic dependency as a local context to resolve word sense ambiguity. In Proceedings of the 35th annual meeting of the association for computational linguistics, pp. 64–71.

  • Lyons, J. (1977). Semantics (Vol. 2). New York: Cambridge University Press.

  • Ma, X., Fellbaum, C., & Cook, P. (2010). A multimodal vocabulary for augmentative and alternative communication from sound/image label datasets. In NAACL human language technologies (HLT’2010) workshop of speech and language processing for assistive technologies.

  • Miller, G. (1969). The organization of lexical memory: Are word association sufficient? In G. A. Talland & N. C. Waugh (Eds.), The pathology of memory. New York: Academic Press.

    Google Scholar 

  • Moss, H., & Older, L. (1996). Birkbeck word association norms. UK: Psychology Press.

    Google Scholar 

  • Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (1998). The University of South Florida word association, rhyme, and word fragment norms.

  • Nikolova, S., Boyd-Graber, J., & Fellbaum, C. (2011). Collecting semantic similarity ratings to connect concepts in assistive communication tools. Modelling, Learning and Processing of Text-Technological Data Structures, Springer Studies in Computational Intelligence, pp. 81–93.

  • Nikolova, S., Ma, X., Tremaine, M., & Cook, P. (2010). Vocabulary navigation made easier. 2010. In Proceedings of IUI’10.

  • Patwardhan, S., & Pedersen, T. (2006). Using WordNet based context vectors to estimate the semantic relatedness of concepts. In Proceedings of the EACL 2006 workshop making sense of sensebringing computational linguistics and psycholinguistics together, pp. 1–8.

  • Resnik, P. (1995). Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th international joint conference on artificial intelligence.

  • Rosch, E. (1978). Principles of categorization. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization (pp. 27–48). Publishers: Lawrence Erlbaum Associates.

    Google Scholar 

  • Schulte im Walde and Melinger. (2008). An In-depth look into the co-occurrence distribution of semantic associates. Italian Journal of Linguistics Special Issue on From Context to Meaning: Distributional Models of the Lexicon in Linguistics and Cognitive Science, 20(1), 89–128.

    Google Scholar 

  • Shapiro, S., & Palermo, D. (1968). An atlas of normative free association data. Psychonomic Monograph Supplements, 2, 219–250.

    Google Scholar 

  • Spence, D. P., & Owens, K. C. (1990). Lexical co-occurrence and association strength. Journal of Psycholinguistic Research, 19, 317–330.

    Article  Google Scholar 

  • Wu, Z., & Palmer, M. (1994). Verb semantics and lexical selection. In Proceedings of ACL, pp. 133–138.

Download references


I thank Dr. Christiane Fellbaum and Prof. Perry Cook for their precious guidance and support for this work, Ahti Lohk for WordNet statistics, and the Kimberley and Frank H. Moss’71 Princeton SEAS Research Fund for sponsorship.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Xiaojuan Ma.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ma, X. Evocation: analyzing and propagating a semantic link based on free word association. Lang Resources & Evaluation 47, 819–837 (2013).

Download citation

  • Published:

  • Issue Date:

  • DOI:


  • Evocation
  • Free association
  • WordNet relations