A sharp image or a sharp knife: norms for the modality-exclusivity of 774 concept-property items
According to recent embodied cognition theories, mental concepts are represented by modality-specific sensory-motor systems. Much of the evidence for modality-specificity in conceptual processing comes from the property-verification task. When applying this and other tasks, it is important to select items based on their modality-exclusivity. We collected modality ratings for a set of 387 properties, each of which was paired with two different concepts, yielding a total of 774 concept-property items. For each item, participants rated the degree to which the property could be experienced through five perceptual modalities (vision, audition, touch, smell, and taste). Based on these ratings, we computed a measure of modality exclusivity, the degree to which a property is perceived exclusively through one sensory modality. In this paper, we briefly sketch the theoretical background of conceptual knowledge, discuss the use of the property-verification task in cognitive research, provide our norms and statistics, and validate the norms in a memory experiment. We conclude that our norms are important for researchers studying modality-specific effects in conceptual processing.
KeywordsConceptual knowledge Grounded cognition Property-verification task Modality exclusivity Flexibility of meaning
People understand and interact with their environment by using conceptual knowledge about objects and their properties. According to various models of conceptual representation (Caramazza & Shelton, 1998; Farah & McClelland, 1991; Humphreys & Forde, 2001), concepts can be described as distributed patterns of activation across units representing semantic features. Several of these models propose that at least a subset of the features are modality-specific (e.g., Cree & McRae, 2003; Plaut, 2002; Rogers et al., 2004; Vigliocco, Vinson, Lewis, & Garrett, 2004). This is consistent with the embodied cognition view on mental representation (e.g., Barsalou, 1999, 2008a; Grush, 2004), which proposes that concepts are grounded in sensory-motor processing. According to this view, object concepts are learned through recurrent experiences of perceiving and interacting with a particular object. During those experiences, the patterns of activity in sensory-motor brain areas are captured and stored in memory to form elaborated, multimodal knowledge structures, called simulators. Representation is achieved by reactivating a subset of this stored knowledge to construct a specific simulation. The exact content of a particular simulation depends on the individual’s experience with the simulated concept, as well as on situational factors such as current goals and task demands. As a result, each simulation is unique.
A method often used to assess conceptual knowledge about objects and their properties is the property-verification task. In this task, the word for a concept (e.g., banana) is presented together with the word for a property that might be true (e.g., yellow) or false (e.g., blue) for the concept. Participants have to judge whether the property is true or false as quickly as possible. According to the simulation theory, the response time reflects the effort required to form an adequate simulation of the concept and property. Since simulation is analogous to sensory-motor processing, this theory predicts that the response time is affected by factors that also influence perception and action. For example, Solomon and Barsalou (2004) found that property-verification response times depend on perceptual factors such as the size and perceptual salience of a property.
Given that representations are flexible and task-dependent (e.g., Barclay, Bransford, Franks, McCarrel, & Nitsch, 1974; Barsalou, 1993), a simulation may be dominated by one sensory modality, with other modalities engaged to a lesser extent. For example, when performing a property-verification task, one may form different simulations of SAXOPHONE,1 depending on the property to be verified. In order to verify that a SAXOPHONE is shiny, a predominantly visual simulation is created, whereas to verify that a SAXOPHONE sounds loud, a predominantly auditory simulation is created. Evidence for such flexible and task-dependent recruitment of modality-specific features in representations is provided by the modality-switch effect. If two representations are simulated by different modality-specific systems, then constructing them consecutively will involve a switch of attention from one modality to another. This will result in a longer reaction time and/or a lower accuracy, similar to modality-switching costs in perceptual tasks (Spence, Nicholls, & Driver, 2001). Conceptual modality-switch effects have been found in various property-verification studies (e.g., Marques, 2006; Pecher, Zeelenberg, & Barsalou, 2003; Pecher, Van Dantzig, & Schifferstein, 2009; Vermeulen, Corneille & Niedenthal, 2008). Other studies demonstrated direct interactions between conceptual processing and perceptual processing, providing even stronger evidence for the theory that the sensory-motor system is involved in conceptual representation (Van Dantzig, Pecher, Zeelenberg, & Barsalou, 2008; Vermeulen et al., 2008). Brain imaging studies have provided corroborative evidence. Goldberg, Perfetti, and Schneider (2006) applied functional magnetic resonance imaging (fMRI) to scan participants during a property-verification task that involved properties from four different sensory modalities (vision, sound, touch, and taste). When participants verified properties from these modalities, activity was recorded in corresponding sensory areas. Thus, verification of visual properties caused activation of visual brain areas, whereas verification of auditory properties caused activation of auditory brain areas. Similar results were found by Simmons et al. (2007) and Kan, Barsalou, Solomon, Minor, and Thompson-Schill (2003). In addition, Cree and McRae (2003), see also Plaut (2002) showed that damage to modality-specific sensory areas may lead to semantic deficits. In sum, multiple studies provide evidence that conceptual representation takes place in the modality-specific brain areas that are also used during actual experience of instances of the represented concepts.
As has become clear, the property-verification task is well suited to the assessment of conceptual knowledge. When applying the task to study modality-specific processing, one should realize that property-verification items vary widely in their degree of modality-exclusivity. Whereas some properties are perceived exclusively by one specific sensory modality (e.g., PIANO-black is purely visual), others may be perceived by multiple modalities (e.g., SHOWER-splashing may be perceived by vision, audition, and touch). It is therefore important to carefully select items based on their modality-exclusivity. Lynott and Connell (2009) found that controlling property-verification items for modality-exclusivity results in a markedly larger modality-switch effect (in terms of effect size). During pilot studies in our own lab, we also found that the size of the modality-switch effect depended on the modality-exclusivity of the items used (e.g., Van Dantzig et al., 2008).
In the present study, in order to select concept-property items on modality-exclusivity, we obtained modality ratings for a large set of items. For each item, participants rated the degree to which the property could be perceived through five different senses (vision, audition, touch, smell, and taste). The modality ratings form a significant extension of those collected by Lynott and Connell (2009). In particular, while they obtained ratings for isolated properties (e.g., yellow), we obtained ratings for concept-property combinations (e.g., BANANA-yellow). It is important to realize that a property is never experienced in isolation, but is always connected to a particular object. In representation, independent simulators may exist for properties (Barsalou, 2008b). However, since simulation is a dynamic process, property-verification does not involve the activation of invariant representations of concepts and properties (Barsalou, 2009). Instead, the representations of the concept and property mutually constrain each other’s content. Thus, the concept WINE is represented differently when one has to verify the property white than when one has to verify the property sweet (Pecher, Zanolie, & Zeelenberg, 2007; Pecher, Zeelenberg, & Barsalou, 2004). Conversely, the property white is represented differently for the concept WINE than for the concept SNOW (Halff, Ortony, & Anderson, 1976; Solomon & Barsalou, 2001). In an experiment by Murphy and Andrew (1993) participants provided synonyms or antonyms for adjectives that were presented in conjunction with a concept, e.g., fresh WATER, fresh AIR or fresh BREAD. The concept strongly influenced which synonyms and antonyms were provided for a particular adjective. The degree to which a specific sensory modality is involved in perceiving a particular property may also differ across concepts. For example, the property hot for the concept LIGHTBULB is perceived almost exclusively by touch, as is well known by anyone who has ever burnt their fingers unscrewing a lamp. Fortunately, in other objects the property hot may also be seen (as in a red-hot GRILL or a steaming mug of COFFEE) or heard (as in a sizzling-hot PAN), so that we do not always have to find out the hard way. Strictly speaking, in these situations it is not the actual property hot that is perceived visually or auditorily, but another visual (steaming, red) or auditory (sizzling) property that merely indicates the hotness of the object. However, according to the simulation theory, property-verification involves simulating a (partial and schematic) sensory-motor experience, by reactivating patterns laid down during earlier experiences. As a result, properties (steaming, sizzling) that frequently co-occurred with the target property (hot) during those experiences may also be included in the simulation and may be used to indicate the presence of the target property. Thus, whereas the property hot in itself is exclusively tactile, in a property-verification task it may be associated with other-modality properties, just as in real experience. The degree to which a particular property is perceived by one or (indirectly) by multiple sensory modalities may vary across concepts.
To capture this variability in property meaning, we paired each property with two concepts, resulting in two different concept-property items. For example, the property plain was paired with the concept FOOD and with the concept FABRIC. For each concept-property item, participants rated the degree to which the property is experienced through five different sensory modalities (vision, audition, touch, smell, and taste). This resulted in a specific modality profile for each item, a five-dimensional vector representing the mean strength ratings on all five modalities. The modality with the highest rating was defined as the dominant modality for that item. Item pairs could be unidominant, with both items having the same dominant modality (e.g., pink FLAMINGO and pink COTTON CANDY are both predominantly visual) or bidominant, with the two items having different dominant modalities (e.g., plain FABRIC and plain FOOD are predominantly visual and gustatory, respectively). Additionally, we computed the angle between the two vectors of each item pair, as well as the cosine of this angle. These measures indicate how similar the modality profiles of the two items of a pair are. Finally, we computed modality-exclusivity scores for each concept-property item, indicating to what extent properties are perceived exclusively by one modality. When comparing our data with those of Lynott and Connell (2009), some interesting patterns arose. In the remainder of this paper, we describe how the norms were collected and we provide the obtained ratings and statistics. Furthermore, the norms were validated in a memory experiment. During the study phase of this experiment, property words were presented together with one of their two normed concepts (e.g., “plain food”). Subsequently, memory for the property words was tested using a cued recognition test in which the property was presented either with the same concept word as in the study phase (“plain food”) or with the other concept of the item pair (“plain fabric”). When the concept was repeated, recognition of the property word was better than when the concept was switched. More interestingly, this effect was influenced by the similarity of the modality profiles of an item pair; the switch cost was larger for items with very different modality profiles (e.g., plain fabric – plain food) than for items with highly similar modality profiles (e.g., grey elephant – grey pebble). This result indicates that our norms reflect actual differences in the representation of properties and concepts.
We selected 387 properties that could be experienced through one or more sensory modalities. Of these 387 items, 364 properties were obtained from the set of Lynott and Connell (2009) and the remaining 23 properties had been used in earlier studies in our lab. Each of the properties was paired with two different concepts, yielding a total number of 774 concept-property items. The set of items was divided into ten different lists that consisted of 77 items (six lists) or 78 items (four lists) each. Properties and concepts were never repeated within a list. Participants received only one list, such that they never rated the same property or concept twice. The complete set of items is available in Electronic supplementary material.
Four hundred thirty-one undergraduate students from the University of California, San Diego participated in the experiment in return for course credits. Given that there were 10 different lists of items, each item was rated by at least 40 participants.
The procedure was similar to that used by Lynott and Connell (2009). The E-Prime 2.0 software package (Psychology Software Tools, Pittsburgh, PA) was used to present the items and collect the participants’ responses. Concept-property items were presented in random order. During each trial, a screen was presented with the text “To what extent do you experience [concept] being [property]?” (e.g., “To what extent do you experience a banana being yellow?”). This text was followed by five separate rating scales for each perceptual modality, labeled “by feeling through touch”, “by hearing”, “by seeing”, “by smelling”, and “by tasting”. To avoid any order effects, the order of the rating scales was randomly determined for each participant. Participants rated the extent to which they would experience the property through each of the five senses from 0 (not at all) to 5 (greatly), by clicking on the corresponding number with the computer mouse. Once the property had been rated on all five modalities, the participant clicked on the “Next Item” button to proceed to the subsequent item. If participants were unfamiliar with a concept or property, they were instructed to skip the item by pressing the “Next Item” button. The instruction emphasized that there were no right or wrong answers and that participants could give a high rating for as many or as few of the modalities as they wished, and a low rating for as many or as few of the modalities as they wished. Participants were encouraged to use their first intuitions about the items, and not to think too long about their answers. The experiment was self-paced and lasted approximately 20–30 min. The complete instructions are provided in Appendix.
Results and discussion
The data from fourteen participants were excluded from the analysis because they were non-native English speakers. One other participant was excluded because he/she failed to comply with the instructions, leaving a total of 416 participants.
The ratings provided by the participants were used to compute mean ratings for each concept-property item. The complete set of items and their ratings can be found in Electronic supplementary material.
Number of properties per dominant modality, with modality exclusivity scores and mean strength ratings (0–5) on the five perceptual modalities
Correlation matrix for mean strength ratings per item on the five perceptual modalities
Properties are never experienced in isolation, but are always connected to a particular object. These objects influence the way in which properties are perceived and represented. As a result, a property may receive different modality ratings depending on the concept that it is combined with. This variance can be investigated in our dataset, because each property occurs twice, in combination with two different concepts. The two concept-property items sharing the same property formed an item pair, which could be analyzed in several ways.
Frequencies and examples of unidominant and bidominant item pairs by modality combination
Howling WIND - howling WOLF
Sour LEMON - sour VINEGAR
Damp FOG - damp TOWEL
Eggy BREATH - eggy HOT SPRING
Red FIRETRUCK - red STRAWBERRY
Honeyed VOICE - honeyed COOKIES
Thumping DRYER - thumping HEADACHE
Boiling KETTLE - boiling SOUP
Bitter COFFEE - bitter COLD
Coconutty PINA COLADA - coconutty SUN CREAM
Dry BISCUIT - dry RIVERBED
Light FEATHER - light COLOGNE
Hairy COCONUT - hairy MAMMOTH
Burning TOAST - burning CANDLE
Similarity of item pairs
Each concept-property item has a specific modality profile, a five-dimensional vector representing the mean strength ratings on the five modalities. In order to determine how similar the two modality profiles of a particular item pair are, we computed the angle between their two vectors. If two items have highly similar modality profiles (e.g., spiky HEDGEHOG – spiky SEA ANEMONE), the angle between the vectors will be small. If the profiles are very different (e.g., honeyed COOKIES – honeyed VOICE), the angle between the vectors will be large. Overall, unidominant item pairs had smaller angles (M = 15.8 degrees, SD = 10.6) than bidominant pairs (M = 40.6 degrees, SD = 21.0), t(385) = 15.47, p < .001. Note, however, that the two measures do not map completely. The two items of a unidominant pair, while having the same dominant modality, could have very different scores on the remaining modalities, rendering them very dissimilar. Conversely, the two items of a bidominant pair could have very similar modality profiles, while their dominant modalities differ by a very small margin. Because the similarity measure is based on the full modality profile, it is more sensitive to subtle differences between the items of a pair than when one only takes into account the dominant modalities. The angle and the cosine of the angle of each item pair are reported in the data file (see Electronic supplementary material).
The majority of properties used in our dataset (364 out of 387 properties, 94%) were also normed by Lynott and Connell (2009). By comparing our modality exclusivity scores for this subset with the scores of Lynott and Connell (furthermore referred to as L&C exclusivity scores), some interesting patterns become apparent. First, our modality exclusivity scores are slightly higher (M = 0.51, SD = 0.14) than the L&C exclusivity scores (M = 0.47, SD = 0.17), t(363) = 7.50, p < .001. This difference might be because properties were paired with specific concepts in the current study, whereas they were presented in isolation in the study of Lynott and Connell. The latter may have caused participants to represent different concepts for the same property. For example, when asked to rate the property rotten, one participant might form a representation of an apple with brown spots and a mushy texture that indicate its rottenness. Thus, this participant will rate rotten as predominantly visual and haptic. Another participant might instead form a representation of a rotten egg, distinguished by its sulfuric smell. Thus, this participant will rate rotten as predominantly olfactory. It is also possible that a participant may represent multiple concepts for a particular property. This participant may choose to provide modality ratings that average across the represented concepts. The variance in represented concepts, either between participants or within the same participant, will result in lower modality exclusivity scores. Alternatively, participants may form a generic, multimodal representation of the property rotten, based on earlier experience but abstracted from the specific objects with which it was experienced. Such generic representations will result in lower modality exclusivity scores than the more specific representation that is formed when the property is presented together with a particular object.
Our norms show that when properties are paired with different concepts, the resulting modality profiles can vary widely in similarity. Next, we wanted to investigate whether the similarity in modality profiles in our norms reflects similarity in the mental representations of the properties. For example, the norms indicate that the representation of mild might be very different in the context of CURRY than in the context of WINTER, whereas the representation of grey might not differ so much between ELEPHANT and PEBBLE. Previous studies showed that both implicit (Zeelenberg, Pecher, Shiffrin, & Raaijmakers, 2003) and explicit (Barclay et al., 1974; Tulving & Thomson, 1973; Zeelenberg, 2005) memory for concept nouns are sensitive to the overlap in contextual cues at study and test. Furthermore, such contextual effects might be due to differences in modality specific features of the representations at study and test (Pecher et al., 2004, 2007; Pecher, Van Dantzig, Zwaan, & Zeelenberg, 2009). In the present study, we investigated how memory of property words was influenced by context (i.e., a concept word). During the study phase, property words were presented together with one of their two concepts (e.g., “grey elephant”). Subsequently, memory for the property words was tested using a cued recognition test, in which the property was presented together with either the same concept word as in the study phase (“grey elephant”), or with the other concept of the item pair (“grey pebble”). We selected item pairs that had either highly similar modality profiles (e.g., grey ELEPHANT – grey PEBBLE) or very different modality profiles (e.g., mild CURRY – mild WINTER). We expected a main effect of concept switch; recognition of the property would be worse if the concept was switched than if the concept was repeated. Moreover, we expected the switch effect to be influenced by the similarity of the modality profiles; it should be larger for items with different modality profiles than for items with similar modality profiles. After all, for items with different modality profiles switching the concept would result in very different representations of the property during study and test, whereas this difference would be much less pronounced for items with similar modality profiles.
Twenty-five undergraduate students from the University of California, San Diego, participated in the experiment in return for course credits. None had participated in the norming study. One participant did not perform the memory task as instructed and was removed from the analysis.
Characteristics of the items of the similar modality profile items and different modality profile items (standard deviations in parentheses) used in the memory experiment
Similar modality profile items
Different modality profile items
Property word length
Log frequency of property word
Log frequency of concept words
Cosine of modality profiles
Angle of modality profiles (in degrees)
LSA cosine (concept 1 to concept 2)
LSA cosine (property-concept item 1 to property-concept item 2)
Examples of the design and counterbalancing in the memory experiment
Different modality profile – repeat concept
Different modality profile – switch concept
Similar modality profile – repeat concept
Similar modality profile – switch concept
Participants were instructed to study the properties (presented in uppercase) for a later memory test. They were told that they should also read the concepts (presented in lowercase) because they might be provided at test to help them remember the properties. Each property-concept pair was presented for 3,500 ms in the center of the computer screen. The property was always the left hand item of the pair and the concept the right hand item (e.g., MILD curry). Between items, a 500-ms blank screen was presented. The 80 items were presented in a different random order for each participant. The test phase started immediately after the study phase. Participants were instructed that they would see property-concept pairs, and their task was to indicate whether they had studied the property in the previous phase. They were told that the concept might help their memory, but that they should indicate “old” to all studied properties, whether the concept was studied or not. An item was presented at the center of the screen and remained there until the participant indicated “old” by pressing the m key or “new” by pressing the z key. All studied items and distractors were presented in different random order for each participant.
Hit rates (standard errors in parentheses) in the memory experiment
Similar modality profile
Different modality profile
We collected modality-exclusivity norms for 387 properties, each paired with two different concepts. The results show that modality profiles and modality exclusivity vary widely for a substantial proportion of properties. In a following experiment, we showed that such variability affects recognition memory for the properties. These results indicate that the norms reflect something essential about the representation of properties. When applying the property verification task or a different verbal task to study modality-specific effects, such as the modality-switch effect, it is important to select items carefully based on their modality-exclusivity. This may reduce the variability attributed to error, yielding cleaner data and larger effect sizes, as demonstrated by the study of Lynott and Connell (2009). Because our norms incorporate the variance of properties across concepts, they are more specific than the norms collected by Lynott and Connell. In order to select items for an experiment, the two data sets can be used in conjunction. We recommend our norms for selecting items for a property-verification task or other verbal tasks using specific concept-property combinations (such as the memory task described in this paper). The norms of Lynott and Connell might be preferred for tasks that address more general meanings of properties.
Throughout this article, we use uppercase italics to represent concepts, lowercase italics to represent properties, and quotes to represent linguistic forms (words, sentences).
We are grateful to Jennifer Duong, Ana Flores, and Dan Killoran for assistance with the data collection.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
- Baayen, R. H., Piepenbrock, Rv, & van Rijn, H. (1993). The CELEX lexical database [CDROM]. Philadelphia: University of Pennsylvania, Linguistic Data Consortium.Google Scholar
- Barsalou, L. W. (1993). Flexibility, structure, and linguistic vagary in concepts: Manifestations of a compositional system of perceptual symbols. In A. C. Collins, S. E. Gathercole, M. A. Conway, & P. E. M. Morris (Eds.), Theories of memory (pp. 29–101). Hillsdale, NJ: Erlbaum.Google Scholar
- Barsalou, L. W. (2008b). Grounding symbolic operations in the brain’s modal systems. In G. R. Semin & E. R. Smith (Eds.), Embodied grounding: Social, cognitive, affective, and neuroscientific approaches (pp. 9–42). New York: Cambridge University Press.Google Scholar
- Cree, G. S., & McRae, K. (2003). Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). Journal of Experimental Psychology: General, 132, 163–201. doi:10.1037/0096-3418.104.22.168 CrossRefGoogle Scholar
- Kan, I. P., Barsalou, L. W., Solomon, K. O., Minor, J. K., & Thompson-Schill, S. L. (2003). Role of mental imagery in a property verification task: fMRI evidence for perceptual representations of conceptual knowledge. Cognitive Neuropsychology, 20, 525–540. doi:10.1080/02643290244000257 PubMedCrossRefGoogle Scholar
- Rogers, T. T., Lambon Ralph, M. A., Garrard, P., Bozeat, S., McClelland, J. L., Hodges, J. R., et al. (2004). Structure and deterioration of semantic memory: A neuropsychological and computational investigation. Psychological Review, 111, 205–235. doi:10.1037/0033-295X.111.1.205 PubMedCrossRefGoogle Scholar