Advertisement

Psychonomic Bulletin & Review

, Volume 22, Issue 6, pp 1717–1724 | Cite as

Examining assortativity in the mental lexicon: Evidence from word associations

  • Bram Van RensbergenEmail author
  • Gert Storms
  • Simon De Deyne
Brief Report
  • 1.1k Downloads

Abstract

Words are characterized by a variety of lexical and psychological properties, such as their part of speech, word-frequency, concreteness, or affectivity. In this study, we examine how these properties relate to a word’s connectivity in the mental lexicon, the structure containing a person’s knowledge of words. In particular, we examine the extent to which these properties display assortative mixing, that is, the extent to which words in the lexicon are more likely to be connected to words that share these properties. We investigated three types of word properties: 1) subjective word covariates: valence, dominance, arousal, and concreteness; 2) lexical information: part of speech; and 3) distributional word properties: age-of-acquisition, word frequency, and contextual diversity. We assessed which of these factors exhibit assortativity using a word association task, where the probability of producing a certain response to a cue is a measure of the associative strength between the cue and response in the mental lexicon. Our results show that the extent to which these aspects exhibit assortativity varies considerably, with a high cue-response correspondence on valence, dominance, arousal, concreteness, and part of speech, indicating that these factors correspond to the words people deem as related. In contrast, we find that cues and responses show only little correspondence on word frequency, contextual diversity, and age-of-acquisition, indicating that, compared to subjective and lexical word covariates, distributional properties exhibit only little assortativity in the mental lexicon. Possible theoretical accounts and implications of these findings are discussed.

Keywords

Word meaning Concepts Psycholinguistics Semantics 

The mental lexicon, defined by Jackendoff (2002) as the store of words in long-term memory from which the grammar constructs phrases and sentences, contains information such as part of speech (house is a noun), denotation (a dog is an animal), pronunciation (balloon is pronounced bə-lōōn’), affective meaning (cake is something I like), and so forth. When studying aspects of word meaning, the mental lexicon is sometimes portrayed as a semantic network, in which nodes correspond to words and connections indicate a meaningful relation between them (Collins & Loftus, 1975; Collins & Quillian, 1969).

While connections between concepts often reflect semantic relationships (e.g., synonymy, hyponomy, meronomy; Murphy, 2003), research suggests that the properties of a word itself correlate with connectivity as well. In particular, a small corpus of studies indicates that the probability that two words are connected correlates with the presence of similar lexical or psychological properties. In network terms, this tendency for connected nodes to exhibit similar covariates is called assortativity or assortative mixing 1 (Newman, 2010; Vitevitch, 2008; Vitevitch, Chan, & Goldstein, 2014).

To study assortative mixing, word association data are often used. In a word association task, the probability of producing a certain response to a cue is a measure of the associative strength between the cue and response in the lexicon (De Deyne, Navarro, & Storms, 2015; Nelson, McEvoy, & Schreiber, 2004). As such, a cue-response correspondence on some factor would be indicative of that factor displaying assortative mixing in the mental lexicon. Using this approach, word association research has identified several factors that exhibit assortativity, that is, several properties that tend to be shared between connected concepts.

First, there is evidence for assortative mixing by syntax: in a word association task, cues tend to elicit responses with the same syntactic properties (Cramer, 1968; Deese, 1962, 1966). These results are corroborated by the finding that processing an utterance with a specific syntactic form facilitates processing utterances with a similar syntax (a phenomenon named syntactic priming; Bock, 1986; Pickering & Branigan, 1998, 1999), by the finding that word selection errors frequently preserve part of speech (Hotopf, 1980), and by noun- or verb-specific deficits in patient studies (Mätzig, Druks, Masterson, & Vigliocco, 2009).

There also is evidence that valence (i.e., how positive a word is considered, cfr. Osgood, Suci, & Tannenbaum, 1957) exhibits assortativity, as research shows a positive cue-response correlation on this dimension (Cramer, 1968; Pollio, 1964; Staats & Staats, 1959) and activation of a specific evaluative attitude (e.g., good) facilitates processing of information that shares that evaluation (a concept called affective priming; see Klauer, 1997, for an overview).

Similarly, word association studies show evidence for assortative mixing by dominance (whether a word refers to a strong or dominant concept, e.g., power) and arousal (whether a word refers to an active or aroused concept, e.g., explosion), again evidenced by positive cue-response correlations on these aspects (Pollio, 1964; Staats & Staats, 1959).

Finally, research on concreteness (the extent to which words are imageable, i.e., refer to something perceptible) suggests this factor may exhibit assortativity as well, as processing a concept with a specific degree of concreteness facilitates processing of concepts with similar imageability (Bleasdale, 1987).

Research on the structure of the mental lexicon has not been limited to assessments of assortativity. A separate line of inquiry has focused on uncovering which word properties contribute to the overall number of connections a word has, that is, what aspects determine which nodes are highly connected or central in the mental lexicon, and which are not. Some of this research examined the same word properties described above—observing, for example, that words with a high valence show increased connectivity (Cramer, 1968; Johnson & Lim, 1964; Matlin & Stang, 1978; Pollio, 1964), as do highly imageable words (de Groot, 1989). Other researchers investigated the role of statistical word properties that are not related directly to meaning but are inferred from the environment in which a word is acquired. They find that concepts that are learned at a young age show higher network connectivity (Barabási & Albert, 1999; Steyvers & Tenenbaum, 2005) and that a person’s exposure to a particular word is involved as well: words with a high word frequency show higher network connectivity (Steyvers & Tenenbaum, 2005), as do words with a high contextual diversity (the number of different contexts in which a word is seen; Hills, Maouene, Riordan, & Smith, 2010). Clearly, these distributional word properties are linked to the structure of the mental lexicon, yet to our knowledge, no research has assessed whether they exhibit assortative mixing, which considers similarity of connected concepts and is distinct from a relation between these factors and overall connectivity.

Current Study

As indicated above, a number of studies have identified several word covariates that display assortativity in the mental lexicon: part of speech, valence, dominance, arousal, and concreteness. Yet, none of these studies have investigated these factors simultaneously, which makes it very hard to evaluate whether they exert an independent contribution. Potentially, these factors depend on one another; it is conceivable, for example, that after controlling for one factor, the effects of some other factor(s) disappear. In the same vein, the lack of common ground between these studies makes it hard to estimate the relative importance of each factor.

A second problem is that part of the research that looked into these factors made use of very small sample sizes, mostly due to technical limitations of their time, making generalizations towards the entire mental lexicon somewhat unfeasible. For example, the study of Staats & Staats (1959) was based on 10 words, and the study of Pollio (1964) comprised 52 words; these small stimulus sets are likely to misrepresent the variability captured by a combination of the investigated factors.

In this study, we use word association data to investigate the linguistic and subjective factors that underlie the configuration of the mental lexicon by examining the extent to which cue word and their associative responses exhibit similar properties. We investigate part of speech, valence, dominance, arousal, and concreteness—five factors that have been established previously to display assortativity. We also will examine word frequency, contextual diversity, and age-of-acquisition—three aspects that have been found to be involved with the structure of the mental lexicon, but for which assortativity has not yet been assessed.

Our main goal was to (a) establish which of these factors display assortativity in the mental lexicon, (b) investigate their relative contribution, and (c) examine whether these findings uphold for a large variability of cue stimuli.

Method

Materials

Word association corpus

To derive the associative strength for a large set of items, we made use of the Dutch Small World of Words project,2 which comprises 3.8 million cue-response pairs (see De Deyne, Navarro, & Storms, 2013, for full details). Briefly, these associations were gathered in response to more than 12,571 cues; each cue was presented to 100 participants, who gave up to three responses to a number of cues in a continued word association task.

Lexical and psycho-affective variables

Three norming databases were used to gather lexical and psycho-affective measures of a large set of words. Word frequency, contextual diversity, and syntactic form (part of speech) for 437,000 Dutch words was obtained from Keuleers, Brysbaert, and New (2010). Word frequency was derived from the raw word count in the subtitles of 8,070 films and television show episodes, contextual diversity was based on the number of films or episodes a word occurred in, and part of speech was estimated using an integrated Dutch morphosyntactic analyzer and part of speech tagger (Tadpole: Van Den Bosch, Busser, Canisius, & Daelemans, 2007).

Age-of-acquisition estimates and concreteness ratings for 30,000 Dutch words were taken from the dataset by Brysbaert, Stevens, De Deyne, Voorspoels, and Storms (2014). Age-of-acquisition was estimated in years, while concreteness was rated on a 5-point Likert scale, where a value of 1 corresponded to “very abstract” and a value of 5 to “very concrete.”

Valence, arousal, and dominance ratings for 4,300 Dutch words were available through Moors et al. (2013). Each dimension was rated on a 7-point Likert scale, where a value of 1 corresponded to “very negative/unpleasant,” “very passive/calm,” and “very weak/submissive,” respectively, and a value of 7 to “very positive/pleasant,” “very active/aroused,” and ”very strong/dominant.” Cues in this database were selected from various sources and consisted of mostly nouns, adjectives, and verbs.

Procedure

Of the 3.8 million cue-response pairs in the Dutch Small World of Words project, 665,461 consist of a cue and response both present in all three norming databases described above. These word pairs contain 4,151 unique words (2,472 nouns, 764 verbs, 814 adjectives, and 101 other words types, based on the dominant syntactical role described by Keuleers, Brysbaert, & New, 2010).

Results

To investigate the extent to which part of speech, valence, arousal, dominance, concreteness, word frequency, contextual diversity, and age-of-acquisition display assortativity in the mental lexicon, we assessed how cues and associative responses correspond on these factors. Our main objective was to inspect correspondence within one dimension—that is, how much of the variance in associative responses’ values on some factor is explained by cue values on that factor. A secondary goal was to examine the extent to which the different factors depend on each other.

To this end, we fitted seven multiple linear regression models, each of which predicts response values on one factor using cue values on all seven measures. The relative contribution of each predictor in the regression model was assessed using the metric lmg in the R package relaimpo (Grömping, 2006), which takes into account predictor collinearity and handles the issue of predictor order by averaging across all possible orders. The resulting R 2 values are described in Table 1.
Table 1

Proportion of variance (> .001) in response values on various psychological and lexical dimensions explained by cue values on those dimensions

Observed variable

Predictors

Cue valence

Cue arousal

Cue dominance

Cue concreteness

Cue log10 of word frequency per million words

Cue log10 of contextual diversity

Cue age-of-acquisition

Response valence

.31

 

.01

    

Response arousal

 

.17

.05

.01

   

Response dominance

.01

.04

.15

    

Response concreteness

   

.20

.01

.01

.03

Response log10 of word frequency per million words

    

.01

.01

 

Response log10 of contextual diversity

    

.01

.01

 

Response age-of-acquisition

.01

  

.04

  

.04

Note: n = 665,461. Cells contain R 2 values derived from a multiple linear regression model with all seven predictors, analyzed using the lmg metric found in R package relaimpo (Grömping, 2006).

For affective dimensions, we find that response values are by far best predicted by cue values on that same measure, as one might expect if these aspects display assortativity. Cues and responses correspond most strongly on valence, with cue valence explaining 31 % of the variance in response valence. We found a smaller but still considerable cue-response correspondence on arousal, dominance, and concreteness, with cue properties explaining between 15 % and 20 % of variance in response values.

We find almost no cue-response correspondence on word frequency and contextual diversity, with cue properties explaining at most 1 % of variance in response values. Lastly, we find a small effect-size of age-of-acquisition, with cue age-of-acquisition explaining 4 % of variance in response age-of-acquisition.

Scatterplots of cue and response values reveal distributions that are somewhat skewed, at least for some of the examined variables (Fig. 1). As such, it is possible that the cue-response correspondence displayed in Table 1 is the result of the distributional properties of the used data, instead of being indicative of assortative mixing. To investigate this alternate explanation, we performed the above regression analysis after permuting the cue-association pairs (so responses are not matched to ”their” cue but to a random cue). This approach yields R 2 values less than .001 for all predictors in all seven models, which indicates that the R 2 values reported in Table 1 are not a result of the properties of the used dataset, but rather indicate that when presented with a cue, people tend to respond with associations of similar valence, arousal, dominance, and concreteness, and to a small extent, age-of-acquisition.3
Fig. 1

Regression lines and scatterplots (with semitransparent markers) of cue-response correspondence on various psychological and linguistic ratings (n = 665,461)

Finally, to investigate cue-response correspondence on part of speech, we included a part of speech contingency table (Table 2). Overall, 57.50 % of responses match the syntactical role of their corresponding cue. Combining the six smallest categories into one (adverbs, pronouns, prepositions, interjections, determiners, and numerals) allows us to perform a chi-squared test on the contingencies, which indicates that part of speech of responses is significantly related to part of speech of their corresponding cue (χ2 = 82,469, df = 9, p < .001, Cramér’s C = .205).
Table 2

Contingency table denoting part of speech of 654,484 cue-response pairs

Cue part of speech

Response part of speech

 

Noun

Adjective

Verb

Adverb

Pronoun

Preposition

Interjection

Determiner

Numeral

n

Noun

283,415

81,511

40,926

2,218

2,284

1,415

107

61

2

411,939

Adjective

43,680

65,314

9,279

1,866

820

206

27

2

0

121,194

Verb

65,616

20,293

27,254

955

573

214

5

1

0

114,911

Adverb

1,437

1,031

320

242

118

5

1

0

1

3,155

Pronoun

910

507

128

86

39

2

0

0

0

1,672

Preposition

400

194

60

14

2

18

1

0

0

689

Interjection

366

56

65

2

67

15

31

0

0

602

Determiner

166

31

1

1

0

0

0

0

0

199

Numeral

55

24

41

0

0

0

0

0

3

123

n

396,045

168,961

78,074

5,384

3,903

1,875

172

64

6

654,484

Note: Cues and responses with unknown or unclear part of speech were omitted (items that were tagged as SPEC in the database of Keuleers, Brysbaert, & New, 2010).

Discussion

The present research used word association data to assess the assortativity of various linguistic and psycho-affective factors. Using an approach that allows us to compare the relative importance of each factor, we examined valence, arousal, dominance, concreteness, word frequency, contextual diversity, age-of-acquisition, and part of speech.

In investigating cue-response correspondence on these dimensions, we find a very strong assortative effect of valence. This pivotal role of evaluative attitude is in line with existing word association research; for example, Deese (1966) identified valence as the dominant factor in determining which concepts people consider related, and a study of our own found valence to account for over 83 % of the variance in a spatial representation of the mental lexicon (De Deyne et al., 2013). The vital importance of evaluative attitude is corroborated in other domains as well, such as in word recognition research (Kuperman, Estes, Brysbaert, & Warriner, 2014), categorization tasks (Niedenthal, Halberstadt, & Innes-Ker, 1999), or affective priming (Klauer, 1997).

We also find a high cue-response correspondence on dominance and arousal, again in line with existing research (Pollio, 1964; Staats & Staats, 1959). This seminal role of the affective dimensions valence, dominance, and arousal is in agreement with the traditional view on semantic meaning. In an attempt to quantify connotative meaning, Osgood and colleagues performed a factor analysis on ratings of concepts on a large number of semantic dimensions (Osgood, Suci, & Tannenbaum, 1957). They found that evaluation (valence), potency (dominance), and activity (arousal) are by far the most powerful aspects in differentiating subjective meaning. Moreover, the importance of these dimensions seems to be near universal, as follow-up studies have replicated these results across dozens of cultures (see Heise, 2010, or Osgood, 1975, for an overview).

In examining concreteness, we find that the level of abstractness of cues is highly predictive of that of its corresponding responses, indicating that this factor, too, is involved with the structure of the mental lexicon. Some research on concreteness-based priming reports similar findings (Bleasdale, 1987), although in general, this factor has received little attention in literature on the mental lexicon. Considering the strong effect we report, inclusion of this factor in future research on the structure of the lexicon might be merited.

Overall, we find that all investigated subjective dimensions show a high cue-response correspondence, indicative of assortative mixing. This is clear evidence for the idea that subjective/affective dimensions are involved with the structure of the mental lexicon and likely play an important role in shaping chain of thought overall.

We also examined the role of syntactic information. We found that cues tend to elicit associative responses with similar syntactic properties, in concordance with existing research (see Deese, 1966, for an overview). This effect was highly significant; in fact, we find that more than half of all associations share the part of speech of their corresponding cue, evidence that syntax exhibits network assortativity as well. We also assessed whether the effects of the psycho-affective and statistical word properties that we investigated were mediated by cue part of speech. In comparing results for verb cues, adjective cues, and noun cues, we find some small baseline differences, although all correspondence patterns described above remained true in all three cases.

As described above, existing research also reports evidence for assortative mixing by valence, arousal, dominance, concreteness, and part of speech. Most of these aspects were studied separately; as such, these existing studies cannot rule out the possibility that some of these factors depend on one another. By investigating all aspects simultaneously, we were able to establish that the assortativity effects reported both by us and in this previous literature cannot be explained by any codependence between the different factors; rather, each of these investigated aspects displays assortative mixing independently of any relation to the remaining factors.

A separate concern with existing research on assortativity in the mental lexicon is that these studies often made use of stimulus sets of (very) limited size, making generalizations towards the entire lexicon somewhat unfeasible. The current study employs a much larger dataset, comprising 4,151 unique words (contained in 665,461 word-pairs). With this, we were able to ascertain that the assortativity effects reported in existing research hold up for a large variability of cue stimuli.

Finally, we investigated word frequency, contextual, and age-of-acquisition, three factors that are not related directly to the meaning of concepts, but rather reflect how a word is acquired by a speaker. Existing research reports that these aspects are all involved with connectivity in the mental lexicon: concepts that are learned at a young age show higher connectivity (Barabási & Albert, 1999; Steyvers & Tenenbaum, 2005), as are words with a high word frequency (Steyvers & Tenenbaum, 2005) and words with a high contextual diversity (Hills, Maouene, Riordan, & Smith, 2010). Note that while this indicates that these aspects are involved with the structure of the mental lexicon, we do not necessarily expect them to exhibit assortativity, which considers similarity between connected concepts and is distinct from overall connectivity. Indeed, our results show only a small cue-response correspondence for age-of-acquisition and virtually no correspondence on word frequency and contextual diversity, indicating that these aspects do not display assortativity in the mental lexicon.

From the previous discussion, it should be clear that assortivity describes how the mental lexicon is structured, but in itself does not directly inform us about causality. This raises the question whether factors that display assortativity actually influence response tendencies or whether they simply co-vary with the type of responses made in an association task. In other words, do we produce a negative response to a negative cue because of their congruency in valence, or because they have similar (negative) meanings? It often is assumed that semantic similarity is the strongest determinant of response tendencies (Mollin, 2009), yet this does not necessarily rule out any influence of the psycho-affective properties of a word: these properties could correspond to semantic features, in which case the likelihood that the response depends on similarity to the cue would increase.

An alternative is to consider the word association process as reflecting learned co-occurrences derived from the linguistic environment. In this view, valence assortativity reflects negative or positive words co-occurring in language. The validity of this perspective could be addressed easily by examining assortativity in text corpora and should be part of future investigations. However, we are very cautious at presenting this as a comprehensive explanation, as it has been pointed out on several occasions that by virtue of not being propositional, word associations capture different information than what can be inferred from a linguistic environment that conveys communicative constraints, such as pragmatics (McRae, Khalkhali, & Hare, 2012; Szalay & Deese, 1978; De Deyne, Verheyen, & Storms, 2014).

Assortativity effects have implications for studies in other domains, such as in research on priming. First, assortativity as measured through word associations can be used to predict which factors will exhibit prime-target congruency effects, and which factors do not. For example, our findings are in line with the affective priming effect, where an affectively congruent prime facilitates processing more than an affectively incongruent prime (Fazio, 2001; Klauer, 1997; Spruyt, Hermans, De Houwer, Vandekerckhove, & Eelen, 2007). However, the current findings also point towards the fact that not all types of congruencies are equally strong and that other factors can enhance or diminish these effects. For example, our findings suggest a larger congruency effect for valence than for concreteness; while these factors have been investigated separately in the priming literature, to our knowledge, they have not been compared directly. Moreover, our results also suggest strong effects for part of speech, which suggests that this factor should be controlled for when investigating congruency effects of other factors, such as in affective priming. Conversely, this relation between cue-target assortativity and congruency effects in priming research also might lead to new factors being included in future investigations of assortativity; for example, because a congruency effect of modality has been established in the priming literature (Pecher, Zeelenberg, & Barsalou, 2003), one might expect cue-target pairs to correspond on this dimension, too.

Common to all these cases is the idea that affectivity, modality, and concreteness might be part of a hierarchy of semantic properties, where valence is relevant to most words in the lexicon, while modality (visual, haptic) applies only to a subset of word, and specific semantic properties (e.g., “is an animal”) to even smaller regions of the lexicon.

In summary, the present research investigated the extent to which various word covariates exhibit assortativity in the mental lexicon. We find assortative mixing by valence, dominance, arousal, concreteness, and part of speech, but not by word frequency, contextual diversity, and age-of-acquisition.

Footnotes

  1. 1.

    Note that this term indicates a specific type of mixing, as it only refers to the tendency of nodes to attach to others that are similar in some way. The opposite situation, where attachment is driven by dissimilarity, is referred to as disassortative mixing. Since throughout this study no evidence is found for disassortative mixing, we only discuss the positive case.

  2. 2.
  3. 3.

    We also investigated whether this cue-response correspondence is mediated by part of speech. We ran the same multiple linear regression models, this time limited to cue-response pairs where the cue is a noun, a verb, or an adjective. Because the findings were very similar to those derived from the entire dataset, with large effect-sizes for affective and lexical dimensions, and minimal correspondence on distributional properties, they are not repeated here.

Notes

Authors Note

Bram Van Rensbergen, Laboratory of Experimental Psychology, University of Leuven, Belgium; Gert Storms, Laboratory of Experimental Psychology, University of Leuven, Belgium; Simon De Deyne, Laboratory of Experimental Psychology, University of Leuven, Belgium, and University of Adelaide, School of Psychology, Adelaide, Australia.

This research was supported by Research Grant OT/10/024 from the University of Leuven Research Council.

Author Contributions

BVR conceived and designed the study, analyzed the data, and wrote the manuscript. GS and SDD conceived and designed the study, and contributed to the writing of the manuscript.

References

  1. Barabási, A.-L., & Albert, R. (1999). Emergence of Scaling in Random Networks. Science, 286(5439), 509–512. doi: 10.1126/science.286.5439.509 CrossRefPubMedGoogle Scholar
  2. Bleasdale, F. A. (1987). Concreteness-dependent associative priming: Separate lexical organization for concrete and abstract words. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13(4), 582–594. doi: 10.1037//0278-7393.13.4.582 Google Scholar
  3. Bock, J. (1986). Syntactic persistence in language production. Cognitive Psychology, 18(3), 355–387.CrossRefGoogle Scholar
  4. Brysbaert, M., Stevens, M., De Deyne, S., Voorspoels, W., & Storms, G. (2014). Norms of age of acquisition and concreteness for 30,000 Dutch words. Acta Psychologica, 150, 80–4. doi: 10.1016/j.actpsy.2014.04.010 CrossRefPubMedGoogle Scholar
  5. Collins, A., & Loftus, E. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82(6), 407.CrossRefGoogle Scholar
  6. Collins, A., & Quillian, M. (1969). Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behavior, 8, 240–247.CrossRefGoogle Scholar
  7. Cramer, P. (1968). Word association. Academic Press New York.Google Scholar
  8. De Deyne, S., Navarro, D. J., & Storms, G. (2015). Associative strength and semantic activation in the mental lexicon: evidence from continued word associations. In M. Knauff, M. Pauen, N. Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th Annual Conference of the Cognitive Science Society.Google Scholar
  9. De Deyne, S., Navarro, D. J., & Storms, G. (2013). Better explanations of lexical and semantic cognition using networks derived from continued rather than single-word associations. Behavior Research Methods, 45(2), 480–98. doi: 10.3758/s13428-012-0260-7 CrossRefPubMedGoogle Scholar
  10. De Deyne, S., Verheyen, S., & Storms, G. (2014). The role of corpus size and syntax in deriving lexico-semantic representations for a wide range of concepts. Quarterly Journal of Experimental Psychology.Google Scholar
  11. De Groot, A. M. (1989). Representational aspects of word imageability and word frequency as assessed through word association. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(5), 824–845. doi: 10.1037//0278-7393.15.5.824 Google Scholar
  12. Deese, J. (1962). Form class and the determinants of association. Journal of Verbal Learning and Verbal Behavior, 1(2), 79–84. doi: 10.1016/S0022-5371(62)80001-2
  13. Deese, J. (1966). The structure of associations in language and thought. Baltimore, MD: Johns Hopkins University Press.Google Scholar
  14. Fazio, R. H. (2001). On the automatic activation of associated evaluations: An overview. Cognition & Emotion, 15(2), 115–141. doi: 10.1080/02699930125908
  15. Grömping, U. (2006). Relative importance for linear regression in R: the package relaimpo. Journal of Statistical Software, 17(1).Google Scholar
  16. Heise, D. R. (2010). Surveying cultures: Discovering shared conceptions and sentiments. Hoboken, NJ: John Wiley & Sons.Google Scholar
  17. Hills, T., Maouene, J., Riordan, B., & Smith, L. (2010). The associative structure of language: Contextual diversity in early word learning. Journal of Memory and Language, 63(3), 259–273.PubMedCentralCrossRefPubMedGoogle Scholar
  18. Hotopf, W. H. N. (1980). Semantic similarity as a factor in whole-word slips of the tongue. In V. A. Fromkin (Ed.), Errors in linguistic performance: Slips of the tongue, ear, pen, and hand (pp. 97–110). New York, NY: Academic Press.Google Scholar
  19. Jackendoff, R. (2002). Foundations of language: Brain, meaning, grammar, evolution. New York, NY: Oxford University Press.CrossRefGoogle Scholar
  20. Johnson, R. C., & Lim, D. (1964). Personality variables in associative production. Journal of General Psychology, 71(2), 349–350.PubMedGoogle Scholar
  21. Keuleers, E., Brysbaert, M., & New, B. (2010). SUBTLEX-NL: a new measure for Dutch word frequency based on film subtitles. Behavior Research Methods, 42(3), 643–50. doi: 10.3758/BRM.42.3.643 CrossRefPubMedGoogle Scholar
  22. Klauer, K. C. (1997). Affective Priming. European Review of Social Psychology, 8(1), 67–103. doi: 10.1080/14792779643000083 CrossRefGoogle Scholar
  23. Kuperman, V., Estes, Z., Brysbaert, M., & Warriner, A. (2014). Emotion and Language: Valence and Arousal Affect Word Recognition. Journal of Experimental Psychology. General, 143(3), 1065–1081.Google Scholar
  24. Matlin, M. W., & Stang, D. J. (1978). The Pollyanna principle: Selectivity in language, memory, and thought. Cambridge, MA: Schenkman.Google Scholar
  25. Mätzig, S., Druks, J., Masterson, J., & Vigliocco, G. (2009). Noun and verb differences in picture naming: Past studies and new evidence. Cortex, 45, 738–758. doi: 10.1016/j.cortex.2008.10.003 CrossRefPubMedGoogle Scholar
  26. McRae, K., Khalkhali, S., & Hare, M. (2012). Semantic and associative relations in adolescents and young adults: Examining a tenuous dichotomy.Google Scholar
  27. Mollin, S. (2009). Combining corpus linguistic and psychological data on word co-occurrences: Corpus collocates versus word associations. Corpus Linguistics and Linguistic Theory, 5(2), 175–200. doi: 10.1515/CLLT.2009.008 CrossRefGoogle Scholar
  28. Moors, A., De Houwer, J., Hermans, D., Wanmaker, S., van Schie, K., Van Harmelen, A.-L., & Brysbaert, M. (2013). Norms of valence, arousal, dominance, and age of acquisition for 4,300 Dutch words. Behavior Research Methods, 45(1), 169–77. doi: 10.3758/s13428-012-0243-8 CrossRefPubMedGoogle Scholar
  29. Murphy, M. L. (2003). Semantic relations and the lexicon. Cambridge University Press.Google Scholar
  30. Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004). The University of South Florida free association, rhyme, and word fragment norms. Behavior Research Methods, Instruments, & Computers: A Journal of the Psychonomic Society, Inc, 36(3), 402–7.CrossRefGoogle Scholar
  31. Newman, M. (2010). Networks: an introduction. New York, NY: Oxford University Press.CrossRefGoogle Scholar
  32. Niedenthal, P., Halberstadt, J. B., & Innes-Ker, A. H. (1999). Emotional response categorization. Psychological Review, 106(2), 336–361.CrossRefGoogle Scholar
  33. Osgood, C. E. (1975). Cross-cultural universals of affective meaning. Urbana: University of Illinois Press.Google Scholar
  34. Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. Urbana: University of Illinois Press.Google Scholar
  35. Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2003). Verifying different madality properties for concepts produces swtiching costs. Psychological Science, 14(2), 119–124.CrossRefPubMedGoogle Scholar
  36. Pickering, M., & Branigan, H. (1999). Syntactic priming in language production. Trends in Cognitive Sciences, 3(4), 136–141.CrossRefPubMedGoogle Scholar
  37. Pickering, M. J., & Branigan, H. P. (1998). The Representation of Verbs: Evidence from Syntactic Priming in Language Production. Journal of Memory and Language, 39(4), 633–651. doi: 10.1006/jmla.1998.2592 CrossRefGoogle Scholar
  38. Pollio, H. (1964). Some semantic relations among word-associates. The American Journal of Psychology, 77(2), 249–256.CrossRefPubMedGoogle Scholar
  39. Spruyt, A., Hermans, D., De Houwer, J., Vandekerckhove, J., & Eelen, P. (2007). On the predictive validity of indirect attitude measures: Prediction of consumer choice behavior on the basis of affective priming in the picture-picture naming task. Journal of Experimental Social Psychology, 43, 599–610. doi: 10.1016/j.jesp.2006.06.009 CrossRefGoogle Scholar
  40. Staats, A. W., & Staats, C. K. (1959). Meaning and m: correlated but separate. Psychological Review, 66(2), 136–44.CrossRefPubMedGoogle Scholar
  41. Steyvers, M., & Tenenbaum, J. B. (2005). The large-scale structure of semantic networks: statistical analyses and a model of semantic growth. Cognitive Science, 29(1), 41–78. doi: 10.1207/s15516709cog2901_3 CrossRefPubMedGoogle Scholar
  42. Szalay, L. B., & Deese, J. (1978). Subjective meaning and culture: An assessment through word associations. Hillsdale: Lawrence Erlbaum.Google Scholar
  43. Van Den Bosch, A., Busser, B., Canisius, S., & Daelemans, W. (2007). An efficient memory-based morphosyntactic tagger and parser for Dutch. LOT Occasional Series, 7, 191–206.Google Scholar
  44. Vitevitch, M. (2008). What can graph theory tell us about word learning and lexical retrieval? Journal of Speech, Language, and Hearing Research, 51(2), 408–422. doi: 10.1044/1092-4388(2008/030) PubMedCentralCrossRefPubMedGoogle Scholar
  45. Vitevitch, M. S., Chan, K. Y., & Goldstein, R. (2014). Insights into failed lexical retrieval from network science. Cognitive Psychology, 68, 1–32. doi: 10.1016/j.cogpsych.2013.10.002 PubMedCentralCrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2015

Authors and Affiliations

  • Bram Van Rensbergen
    • 1
    Email author
  • Gert Storms
    • 1
  • Simon De Deyne
    • 1
    • 2
  1. 1.University of LeuvenLeuvenBelgium
  2. 2.University of AdelaideAdelaideAustralia

Personalised recommendations