Psychonomic Bulletin & Review

, Volume 23, Issue 4, pp 941–958

Arguments about the nature of concepts: Symbols, embodiment, and beyond

Theoretical Review

Abstract

How are the meanings of words, events, and objects represented and organized in the brain? This question, perhaps more than any other in the field, probes some of the deepest and most foundational puzzles regarding the structure of the mind and brain. Accordingly, it has spawned a field of inquiry that is diverse and multidisciplinary, has led to the discovery of numerous empirical phenomena, and has spurred the development of a wide range of theoretical positions. This special issue brings together the most recent theoretical developments from the leaders in the field, representing a range of viewpoints on issues of fundamental significance to a theory of meaning representation. Here we introduce the special issue by way of pulling out some key themes that cut across the contributions that form this issue and situating those themes in the broader literature. The core issues around which research on conceptual representation can be organized are representational format, representational content, the organization of concepts in the brain, and the processing dynamics that govern interactions between the conceptual system and sensorimotor representations. We highlight areas in which consensus has formed; for those areas in which opinion is divided, we seek to clarify the relation of theory and evidence and to set in relief the bridging assumptions that undergird current discussions.

Keywords

Concept representation Concept organization Embodied cognition Grounded cognition Neural basis of concepts Symbols Semantic categories Word meaning Actions and objects 

Scope of the special issue

The contributions that form this issue of Psychonomic Bulletin & Review are unified around the objective of developing and clarifying theories of how meaning is represented and organized, at both the cognitive and neural levels. Collectively, this issue represents the “next generation” of theories of meaning representation in the brain. As such, it provides a new vantage point from which to survey the ground that has been covered in recent years, and to scout the new directions in which the field will likely move. Thus, and as a way of framing the impetus for many of the original theories that are presented herein, we start with a brief and curated overview of the major developments over the last several decades. We then signal the dominant themes of debate that resonate across the contributions in this issue.

Every project has its limits. With respect to this introductory article, theoretical neutrality in our exposition is an aspiration difficult to practice: What one takes to be the substantive issues that are at stake is in part theory-dependent. We at least hope to frame the issues squarely and to be forthright about our presuppositions. In fact, the core argument we develop is that only by being explicit about the bridging assumptions implicit in current debates will the field move forward. With respect to the organization of this special issue, the limits of this collection of articles are conspicuous when set against the breadth of research on concept representation. We therefore note the restricted scope of this issue: The present issue does not include contributions on many areas of research in conceptual processing, including conceptual development (e.g., Baillargeon, 1998; Carey, 1987; Carey & Spelke, 1994; Keil, 1981; Wellman & Gelman, 1992), comparative work on concept acquisition and structure (e.g., Fabre-Thorpe, 2003; Thompson & Oden, 2000; Zentall, Wasserman, Lazareva, Thompson, & Rattermann, 2008), evolutionary perspectives on human conceptual capacities (e.g., Cosmides & Tooby, 1994), social cognition (e.g., Ellis, Young, & Critchley, 1989; Lyons, Kay, Hanley, & Haslam, 2006; Miceli et al., 2000; Mitchell, Heatherton, & Macrae, 2002), numerical and spatial reasoning (e.g., Cantlon, 2012; Dehaene, 2011; Feigenson, Dehaene, & Spelke, 2004; Hermer & Spelke, 1994), concept learning (e.g., Gallistel, 1990; Gelman, 1990), or computational modeling (e.g., Chen & Rogers, 2014; Devlin, Gonnerman, Andersen, & Seidenberg, 1998; Farah & McClelland, 1991; Jones & Mewhort, 2007; Landauer, Foltz, & Laham, 1998; Plaut, 2002), among other areas. Finally, we must recognize and apologize for an error during the production of this issue: David Kemmerer’s article (Kemmerer, 2015) was mistakenly published ahead of schedule in a regular issue of Psychonomic Bulletin & Review. His article is among those invited to appear in this special issue.

This issue consists of 15 original theoretical articles; it is bookended by this introduction and by a synthesis of the entire issue by Lawrence Barsalou (2016a), as viewed through the lens of that author’s influential theoretical framework on concept representation.

Background and introduction to the question

The question of how word, object, event, and action concepts are represented and organized in the brain has a rich and dynamic history. An important aspect of that history was initiated by neuropsychological studies of patients with acquired brain lesions. More recently, noninvasive imaging approaches, in particular functional magnetic resonance imaging (fMRI), but also positron emission tomography, magnetoencephalography, electroencephalography, and electrocorticography, as well as transcranial magnetic stimulation (TMS), transcranial direct current stimulation, and psychophysical studies, have opened up new theoretical ground and proven instrumental for adjudicating among extant hypotheses.

A number of neuropsychological phenomena were critical in laying the basis for the modern understanding of the neural systems that support conceptual knowledge. First, it was observed that patients could present with disproportionate deficits to a category or domain of knowledge: category-specific semantic deficits. Those deficits are conceptual impairments that are selective or disproportionate for living animate entities (animals), living inanimate entities (fruit/vegetables), conspecifics (knowledge of people), or nonliving things (Warrington & McCarthy, 1983, 1987; Warrington & Shallice, 1984; for earlier work, see Hécaen & De Ajuriaguerra, 1956; see also Caramazza & Shelton, 1998; Hart, Berndt, & Caramazza, 1985; Sacchett & Humphreys, 1992). Category-specific semantic deficits are distinct from category-specific visual deficits (e.g., Duchaine, Yovel, Butterworth, & Nakayama, 2006; Moscovitch, Winocur, & Behrmann, 1997) because the level of impairment is to conceptual knowledge (for reviews, see Capitani, Laiacona, Mahon, & Caramazza, 2003, 2009; Caramazza & Mahon, 2003; Gainotti, 2000; Humphreys & Forde, 2001; Tyler & Moss, 2001). At the same time as that early formative work by Warrington and her collaborators was underway on category-specific deficits, Goodale and Milner and colleagues (Goodale & Milner, 1992; Goodale, Milner, Jakobson, & Carey, 1991; Milner et al., 1991) described the consequences of a focal lesion to what came to be termed the ventral object-processing pathway: Bilateral lesions to lateral occipital–temporal cortex could lead to impairments in recognizing objects but no difficulty performing grasping and reaching movements to the same objects. Other patients presented with the reverse behavioral dissociation after lesions to posterior parietal cortex: impaired object-directed reaching and grasping, despite intact object identification (e.g., Marotta, Behrmann, & Goodale, 1997; Pisella, Binkofski, Lasek, Toni, & Rossetti, 2006; see also Aglioti, DeSouza, & Goodale, 1995; Desmurget et al., 1999; for a review, see Milner & Goodale, 2007). Those and other data formed the basis for a new understanding of functionally and anatomically dissociable channels of visual processing, what has come to be referred to as the two-visual- (or ventral/dorsal) systems model (see also Jeannerod & Jacob, 2005; Livingstone & Hubel, 1988; Merigan & Maunsell, 1993; Rizzolatti & Matelli, 2003; Ungerleider & Mishkin, 1982). Around the same time, it was also established that neurodegenerative diseases, such as semantic dementia, could lead to broad deterioration of semantic knowledge, sometimes with category-specific dissociations (e.g., Garrard, Patterson, Watson, & Hodges, 1998; Gonnerman, Andersen, Devlin, Kempler, & Seidenberg, 1997; Lambon Ralph, Graham, Ellis, & Hodges, 1998; Mesulam, 1982; see also Brambati et al., 2008). Another key piece of neuropsychological evidence was the observation that the ability to produce nouns (typically objects) or verbs (typically actions) was doubly dissociable, in some cases attributable to a semantic-level dissociation, but in other cases attributable to a lexical and/or syntactic-level dissociation (e.g., Daniele, Giustolisi, Silveri, Colosimo, & Gainotti, 1994; Goodglass, Klein, Carey, & Jones, 1966; Caramazza & Hillis, 1991).

Thus, when the first functional neuroimaging studies on conceptual processing in the human brain were being reported (Damasio, Grabowski, Tranel, & Hichwa, 1996; Martin, Haxby, Lalonde, Wiggs, & Ungerleider, 1995; Petersen, Fox, Posner, Mintun, & Raichle, 1988; Posner, Petersen, Fox, & Raichle, 1988), established efforts were well underway to understand (a) how conceptual knowledge could be organized in the brain such that it could dissociate after brain damage along (seemingly) categorical lines; (b) how different channels within the cortical visual-processing network differentially contributed to visual object identification and object-directed action and spatial processing; and (c) how object and action knowledge, and nouns and verbs, might be dissociably represented by distinct neural systems.

The first neuroimaging studies to look at the neural organization of concepts seemed to largely substantiate the idea that semantic category or domain is an organizing principle in the brain: Martin and colleagues (e.g., Chao, Haxby, & Martin, 1999; Chao, Weisberg, & Martin, 2002) observed that subregions of the ventral object-processing pathway exhibited differential activation for animals and tools, and Kanwisher and colleagues (Kanwisher, McDermott, & Chun, 1997; see also Allison et al., 1994; Puce, Allison, Bentin, Gore, & McCarthy, 1998) reported neural specificity for faces. Subsequent investigations have documented stimulus specificity for places (Epstein & Kanwisher, 1998; see also Bar & Aminoff, 2003), body parts (e.g., Downing, Jiang, Shuman, & Kanwisher, 2001), and written words (e.g., Cohen et al., 2000; Glezer, Jiang, & Riesenhuber, 2009) in the ventral visual pathway. Another series of studies led by Martin and colleagues (e.g., Chao & Martin, 2000; Mahon et al., 2007) described regions in the dorsal visual pathway, such as posterior parietal cortex, that were differentially engaged when participants viewed manipulable objects such as tools and utensils. It is also now well described that semantic knowledge of actions, as opposed to semantic knowledge of entities, has specific loci of representation in the brain, widely claimed to involve frontal lobe motor-related areas, with closer examination suggesting the left posterior middle temporal gyrus (see, e.g., Hickok, 2014; Kemmerer, 2015; Kemmerer, Rudrauf, Manzel, & Tranel, 2012; Tranel, Kemmerer, Adolphs, Damasio, & Damasio, 2003; see also Beauchamp, Lee, Haxby, & Martin, 2002, 2003; Kable, Lease-Spellmeyer, & Chatterjee, 2002; Brambati et al., 2006). More recently, it has been discovered that many regional biases by category or stimulus type, although present in what are principally “visual” or “visually responsive” regions of the brain, are nonetheless also present in individuals without any visual experience, and in remarkably high anatomical correspondence with the biases observed in sighted individuals (Bedny, Caramazza, Pascual-Leone, & Saxe, 2012; Büchel, Price, & Friston, 1998; He et al., 2013; Mahon, Anzellotti, Schwarzbach, Zampini, & Caramazza, 2009; Mahon, Schwarzbach, & Caramazza, 2010; Striem-Amit, Cohen, Dehaene, & Amedi, 2012; Striem-Amit, Dakwar, Reich, & Amedi, 2012; Strnad, Peelen, Bedny, & Caramazza, 2013); the data from congenitally blind participants suggest that although vision clearly plays a critical role in shaping neural responses in high-level visual areas, the broad scaffolding of the visual system by “semantic domain” originates in constraints that operate independent of visual experience. Those constraints may be hardwired anatomical connectivity between regions of the “visual system” and other regions of the brain that process nonvisual information about the same classes of stimuli (Mahon et al., 2007, 2009; Mahon & Caramazza, 2009, 2011; see also Dehaene, Cohen, Sigman, & Vinckier, 2005; Plaut & Behrmann, 2011; Riesenhuber, 2007). The idea that connectivity-based constraints drive category specificity in high-level visual areas has been gaining momentum (see, e.g., Hutchison, Culham, Everling, Flanagan, & Gallivan, 2014; Osher et al., 2016; Stevens, Tessler, Peng, & Martin, 2015; see also Polk, Park, Smith, & Park, 2007; Wilmer et al., 2010).

Hearty discussion ensued, and continues, about the constraints and principles of neural organization that give rise to category specificity, where “category” in this context is understood broadly, to capture objects, actions, nouns, verbs, faces, places, body parts, and so on (e.g., Caramazza & Mahon, 2003; Downing, Chan, Peelen, Dodds, & Kanwisher, 2006; Gauthier, Skudlarski, Gore, & Anderson, 2000; Grill-Spector & Malach, 2004; Hasson, Levy, Behrmann, Hendler, & Malach, 2004; Humphreys & Forde, 2001; Konkle & Oliva, 2012; Levy, Hasson, Avidan, Hendler, & Malach, 2001; Lewis, 2006; Mahon et al., 2009; Mahon & Caramazza, 2011; Martin, 2007, 2009, 2016; McClelland & Rogers, 2003; Op de Beeck, Haushofer, & Kanwisher, 2008; Patterson, Nestor, & Rogers, 2007; Rogers, Hocking, Mechelli, Patterson, & Price, 2005; Rumiati & Foroni, 2016; Sartori & Lombardi, 2004; Tarr & Gauthier, 2000; Tranel, Logan, Frank, & Damasio, 1997; Tyler & Moss, 2001). In the last decade, multivariate techniques for analyzing distributed patterns of neural activity (e.g., Anderson, Zinszer, & Raizada, 2016; Haxby et al., 2001; Kriegeskorte, Goebel, & Bandettini, 2006; Kriegeskorte et al., 2008; Spiridon & Kanwisher, 2002) have grown in prominence as a means to understand how object concepts are coded in cortex. The widespread adoption of multivoxel pattern classification approaches has led to a shift in emphasis from “which regions” of the brain support which “types” or “classes” of concepts, to studying the representational space within those regions. Another important and recent development has been an emphasis on using functional and anatomical connectivity to probe the network structure of anatomically distinct regions that exhibit congruent category preferences (Almeida, Fintzi, & Mahon, 2013; Bouhali et al., 2014; Garcea & Mahon, 2014; Hutchison et al., 2014; Mahon et al., 2007; Mahon, Kumar, & Almeida, 2013; Osher et al., 2016; Simmons & Martin, 2012; Stevens et al., 2015; Thomas et al., 2009).

Along with reporting the first comprehensively described cases of category specificity, Warrington and her collaborators (Warrington & McCarthy, 1983, 1987; Warrington & Shallice, 1984) also articulated a proposal that has dramatically shaped theoretical approaches for understanding the organization and representation of concepts in the brain. The sensory/functional theory refers to the idea that dissociations among different classes or categories of information arise because of damage to modality-specific brain systems, and that there are high correlations between certain categories and certain modality-specific systems. The original formulation of this proposal in the context of category-specific semantic deficits suggested that living things differentially depend on visual perceptual properties for their identification, whereas nonliving things differentially depend on functional/associative properties for their identification. Subsequent developments of the theory have emphasized a finer granularity of the relevance of different types of visual information for different categories (e.g., color and motion for fruits and animals, respectively; e.g., Borgo & Shallice, 2001; 2003; Cree & McRae, 2003; Crutch & Warrington, 2003; Vinson, Vigliocco, Cappa, & Siri, 2003; for an early discussion of “different” semantic systems, see Beauvois, 1982; Lhermitte & Beauvois, 1973).

Largely in parallel to research investigating the granularity of neural specificity for different categories, the broad discovery was made that the sensorimotor systems are engaged during conceptual processing (e.g., Glenberg & Kaschak, 2002; Pecher, Zeelenberg, & Barsalou, 2003; Zwaan, 2004; for reviews, see Barsalou, 1999, 2008a; Simmons & Barsalou, 2003). The reckoning that sensorimotor systems are engaged during conceptual processing led to a resurgence of interest in the idea that conceptual content is distributed in the sensorimotor systems—the embodied cognition hypothesis (also referred to as grounded cognition and situated cognition; herein we refer to this class of proposals as embodied cognition). Where prior theories of concept representation had emphasized the organization of conceptual content (i.e., what concepts are about, as in a concept is about the color red, or about dogs, or about peace), the embodied framework emphasizes the format in which conceptual information is represented (i.e., in a modality-specific or sensorimotor format, a multisensory format, as opposed to in an amodal format). Allport (1985), one of the pioneers of this perspective, put it thus:

The essential idea is that the same neural elements that are involved in coding the sensory attributes of a (possibly unknown) object presented to eye or hand or ear also make up the elements of the auto-associated activity-patterns that represent familiar object-concepts in “semantic memory.” This model is, of course, in radical opposition to the view, apparently held by many psychologists, that “semantic memory” is represented in some abstract, modality-independent, “conceptual” domain remote from the mechanisms of perception and motor organization. (Allport, 1985, p. 53, emphasis in original)

The “embodied” theory of semantic representation has been enormously influential over the last decade, particularly in the context of investigations of the neural underpinnings of conceptual knowledge. In a now-classic theoretical treatment of this issue, Barsalou (1999, 2008a) laid out what is perhaps the most complete framework to date. Within the perceptual symbol system framework, conceptual processing is embodied or grounded because conceptual content is, at least in part, reaccessed sensorimotor information—concepts are not an additional level of representation that is abstracted away from sensorimotor systems and represented in a stand-alone manner from those sensorimotor systems. As will be discussed below, Barsalou (2016a; see also Barsalou, 1999, 2008a) has argued that the critical issue is one of “neural reuse”—the same neural regions that are involved in perception and action are involved in conceptual processing. Barsalou (1999, 2008a) was diligent in his framing of the theory to argue how key phenomena of human conceptual processing (e.g., compositionality, productivity) could arise from the quasi-symbolic operation of perceptual symbols (for other proposals within the embodied or grounded framework, see Gallese & Lakoff, 2005; Glenberg, 2015a, 2015b; Glenberg & Kaschak, 2002; Kiefer & Pulvermüller, 2011; Martin, Ungerleider, & Haxby, 2000; Prinz, 2002; Pulvermüller, 2005, 2013; Zwaan, 2004; for broader discussions, see Chatterjee, 2010; Dove, 2009; Goldinger, Papesh, Barnhart, Hansen, & Hout, 2016; Hickok, 2014; Machery, 2007; Mahon, 2015a, 2015b, 2015c).

Another important discovery that has shaped the climate in which recent theoretical accounts of meaning representation have been developed is that of so-called “mirror neurons”: neurons (and neural regions) in or adjacent to the motor system that are activated when observing others’ actions (di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992; Gallese, Fadiga, Fogassi, & Rizzolatti, 1996; see also Buccino et al., 2001; Fadiga, Craighero, Buccino, & Rizzolatti, 2002). The discovery of mirror neurons reinvigorated interest in motor theories of the perception and recognition of actions. The basic idea of motor theories of action recognition is that to recognize and identify an action, it is necessary to covertly simulate the motor movements that would be required in order to produce that action. In relatively recent times, this theory was most prominently proposed in the domain of speech perception (e.g., Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967; Liberman & Mattingly, 1985; for a review, see Galantucci, Fowler, & Turvey, 2006), and has since been extended into many other domains on the basis of the putative mirror properties of motor responses. An active discussion ensued, and continues, about what “mirror” neuron (and neural) activity is really telling us about action perception and recognition (e.g., Caramazza, Anzellotti, Strnad, & Lingnau, 2014; Dinstein, Thomas, Behrmann, & Heeger, 2008; Hickok, 2009, 2010, 2014; Hickok, Costanzo, Capasso, & Miceli, 2011; Lingnau, Gesierich, & Caramazza, 2009; Lotto, Hickok, & Holt, 2009; Mahon & Caramazza, 2005; Rizzolatti & Craighero, 2004; Stasenko, Garcea, & Mahon, 2013).

It should be noted that although there is no necessary relation between embodied theories of meaning representation and motor theories of action perception, the two accounts have nonetheless become intertwined in many discussions because they are proposals in the same spirit. One’s concept “kick” is constituted (at least in some important ways) by information stored in a motor format in that individual’s motor system and pertaining to that individual’s foot; and similarly, one’s ability to recognize the action of “kicking” depends on covert simulation (by that individual’s motor system) of kicking (with that individual’s actual foot motor representation). Thus, the representation of “kicking” by one’s motor system serves both to support action recognition and constitutes, in part, the concept “kick.”

One obvious test of mirror-neuron and embodied accounts is to ask what happens to action recognition and concept representation when a brain lesion has disrupted sensorimotor information. The general finding that has emerged is that despite lesions that cause sensorimotor impairments, and presumably high levels of redaction of sensorimotor information, patients can exhibit little to no impairment of conceptual processing (for reviews, see, e.g., Hickok, 2014; Mahon & Caramazza, 2005; 2008). Those neuropsychological data seem to be influencing the theoretical discussion more and more. As Barsalou (2016a) emphasizes, a key issue is the degree to which a given concept may be distributed over multiple modality-specific systems: For instance, damaging the motor system may not affect the concept “kick” if that concept can change the “posture” of its reliance on modality-specific information to lean more heavily on representations in the visual system. In this regard, the patient data have formed an important part of the motivation for “hybrid” accounts: According to those accounts, although conceptual processing reaches into the sensorimotor systems, the center of gravity of conceptual processing is abstracted away from specific sensorimotor information, and distributed across many systems. A number of contributions to this special issue fall into this “hybrid” class of proposals (Binder, 2016; Jamrozik, McQuire, Cardillo, & Chatterjee, 2016; Kemmerer, 2015; Reilly, Peelle, Garcia, & Crutch, 2016; Zwaan, 2016; see also Dove, 2016; Hickok, 2014; Mahon & Caramazza, 2008). One generalization that emerges across this issue—which is a good trope for the broader field—is a general recognition that simple oppositions, such as “are concepts this or that”, do not capture the subtleties that are demanded of any reasonably sufficient theory.

The need to distinguish claims of representational format from claims of representational content was front and center in the first wave of theoretical discussion of the sensory/functional theory (Caramazza, Hillis, Rapp, & Romani, 1990; see also Riddoch, Humphreys, Coltheart, & Funnell, 1988; Shallice, 1988, 1993; see the parallel discussion in the context of the mental imagery debate—e.g., Pylyshyn, 2003). What is new about current discussions of content and format in conceptual representation is that we now have (many) methods that probe in vivo neural activity during conceptual processing. Throughout our summary below, we seek to ask two questions of any given theoretical assumption: First, is it a hypothesis about the nature of conceptual content or about the nature of conceptual format? Second, given the answer to the first question, what would an alternative hypothesis look like? These two questions set in relief the bridging assumptions that connect theories of the organization and representation of concepts to the empirical phenomena, and thus serve to reappraise the substantive issues that need to be adjudicated.

Some major themes from this issue and their broader implications

Are there isomorphisms between representational format and content?

Theories of meaning representation may be usefully organized into two (roughly hewn) groups based on whether or not it is assumed that the format of concepts is constrained by what those concepts are about. The core idea of embodied theories is that there is an isomorphism between the format of conceptual representation and conceptual content. According to those theories, conceptual format is not arbitrary, as it is determined by the content of the concept: For instance, sensory concepts have a sensory format, and action concepts have a motor format. In the second group are theories that assume that concepts are represented as amodal symbols, also sometimes referred to as “classic” accounts or “disembodied” theories. The core idea of this class of theories is that the format of conceptual representations is not constrained by what those concepts are about; concepts are amodal in their format, regardless of what those concepts are about. Thus the concepts “kick” and “bitter,” or for that matter “magnanimity” and “shame,” have a common format (for discussion, see Chatterjee, 2010; Dove, 2009; Hickok, 2014; Leshinskaya & Caramazza, 2016; Machery, 2016; Mahon & Caramazza, 2008; Mahon, 2015a, 2015c);

One argument that has been offered as to why concepts are unlikely to be represented in an amodal symbolic format is that, if they were, this would mean there is an arbitrary relation between concepts and what they represent. This is presented as a difficulty for theories that assume amodal symbols, although why it should be a problem has received less attention. We believe there is some fuzziness about such assertions. An analogy illustrates the reason why: In the domain of language processing, the form of word representations bears little relation to their meaning, a point famously made by Saussure (for discussion, see Harris, 1988; for recent critical discussion and research on sound symbolism, see Casasanto & Lupyan, 2015; Dingemanse, Blasi, Lupyan, Chistiansen, & Monaghan, 2015). For instance, the word form “dog” means DOG in English, and the word form “perro” means DOG in Spanish. The forms of the words “dog” and “perro” (or “chien,” “gǒu,” etc.) bear no isomorphism to the concepts to which those words refer. This type of “arbitrary relation” between representational form and reference works because there are systematic mappings, or connections, between word forms and the meanings of those words, on the one hand, and the phonological/articulatory information corresponding to those words, on the other. Thus, although the form of words may be arbitrarily related (ontologically) to their referents, the words themselves are not arbitrarily connected to their referents (i.e., in terms of a processing model). The broader point applies to concepts: Just as words are connected in highly systematic and constrained ways to specific semantic information and specific phonological/articulatory information, so too are concepts connected in systematic ways to modality-specific input and output representations.

The amodal symbolic account would thus not deny that many of one’s distinguishing experiences with telephones may have been auditory or tactile, nor that specific auditory and tactile information (by virtue of experience) is systematically connected to one’s concept “telephone.” But, according to the amodal account, the format in which the concept “telephone” is represented will not be “auditory” or “tactile” by virtue of the fact that one has had auditory and tactile experiences with telephones (for a discussion, see Machery, 2016). Thus, the proposal that the format of concept representations is amodal should not be caricatured as the claim that concepts are deracinated, or disconnected, or even in some way “remote” from the modality-specific information that pertains to their referents, as Allport’s (1985) quote above suggests (for recent versions of that argument, see Pulvermüller, 2013). How, one might ask, can a representation of “arbitrary” conceptual content be connected to the correct and specific sensorimotor information? The answer is the same as how an “arbitrary” word form can be connected to the correct and specific phonological information: A metaphorical line is drawn (Mahon, 2015c); in neural terms, one might conceptualize it as a form of Hebbian association, perhaps implemented by spike-timing-dependent plasticity (e.g., Caporale & Dan, 2008). It is true that the format of amodal symbols is arbitrarily related to that which they refer—this is what gives amodal symbols their computational alacrity, as they can be about anything and yet their form remains the same. An amodal symbol can be “about” the number 2, or cheese, or the color blue—and it can participate in the same way in abstract computations because it is a “cog of the same shape” in those processes. But that does not disqualify that “amodal symbol” from being systematically connected up with the right input and output representations.

Another argument as to why concepts are unlikely to be represented in an amodal format is based on the intuition that if concepts are represented by modality-specific systems, then the problem of concept representation has somehow been simplified. Specifically, the intuition is that if the format of concepts is isomorphic with the content of concepts, then no “transduction” problem must be solved by the system (or accounted for by a theory of how the system works). That intuition, however, belies the complexity of sensorimotor representations, which are themselves already a transduction (many times over) of the parameters of the physical stimulus that impinges on our sensory organs. Even in the earliest stages of sensory processing (and the latest in motor planning), information is abstractly represented and way more complicated than a simple resonance with the physical world. Pushing concepts into sensorimotor systems does not simplify those representations, or make them “less abstracted away from the world.”

Consider the retina. One can conceptualize patterns of activity in retinal ganglion cells as representing the light reflectance patterns in the environment. The pattern of activity elicited by a tree will be different than that elicited by a hammer. But the formats of the representations—that is, the patterns of neural activity—are the same. In fact, one could imagine each retinal ganglion cell as a binary switch, a 0 or a 1, and the pattern of activity across the array as a matrix of values. And one could further imagine that the retinal matrix is transformed into a secondary or n-ary (cortical) matrix depending on the arrangement of values in the retinal matrix. On one level the representation of the tree or the hammer has lost all of its tree-ness or hammer-ness, even at the level of the retina (it’s just a matrix of values), and even more so as it is transformed by further operations. Indeed, under this conceptualization the format of the representation is something quite suitable for digital computer processing and highly abstract, symbolic, and amodal representation. On another level, there is a systematic mapping between the matrix and the pattern of light reflectance that, in the words of the embodied cognition movement, grounds the representation. The point is there is no inherent tension between abstract, symbolic, amodal representations and grounding.

To flip the issue around, it might be asked: Is it inherently problematic to suppose that the format of amodal concepts is the same regardless of what those concepts are about? As some of the contributions to this issue make clear, intuitions diverge on this question, and the answer (in either direction) does not follow with the force of logic. It depends very much on one’s broader views and intuitions about how the human mind/brain works. And critically, it largely depends on whether one takes conceptual content or conceptual format to be at stake in the debate about whether concepts are embodied (for discussion, see Barsalou, 2016a).

In summary, the principal evidence for the view that concepts are represented (at least in part) by sensorimotor information—the embodied cognition hypothesis—is the observation that conceptual processing leads to the activity of modality-specific systems. “Activity” here is construed as capturing both behavioral and neural signatures (e.g., Buccino et al., 2005; Hauk, Johnsrude, & Pulvermüller, 2004; Pecher et al., 2003). As we noted above, Barsalou (2016a) emphasizes that the key claim of the embodied cognition hypothesis is one of “neural reuse”—that the neural regions involved in perception and action are also involved in conceptual processing (for discussion, see also Martin, 2016). What bridging assumptions are presupposed when one argues that demonstrations of neural reuse support the embodied cognition hypothesis over alternative theories? To address this question, we need to clearly articulate the theoretical alternatives to the embodied approach. Broadly speaking, there are two. The first, and perhaps most obvious, theoretical alternative to the embodied approach is the proposal that the representational format of concepts is amodal and symbolic (as opposed to, at least in part, sensorimotor), and that systematic connections exist between amodal symbols and the corresponding sensorimotor representations. In the context of this alternative, the discussion shifts toward understanding how the observed patterns of sensorimotor activation during conceptual processing might have implications for how activation spreads between amodal symbolic representations and sensorimotor systems. We unpack these issues further in the next section, entitled “Representation versus processing—Toward a theory of dynamics.” The second alternative to the embodied framework sublimates the distinction between “embodied” and “amodal” representations by emphasizing that sensorimotor representations are already abstractions. To the extent that sensorimotor representations are already abstractions, and thus not “sensorimotor” in their format, the available evidence does not suggest that the format of concepts is sensorimotor, but rather that the content of concepts and conceptual processing may be (at least, in part) sensorimotor (cf. Pylyshyn, 2003). This second alternative thus shifts the discussion toward the distinction between conceptual format and content. We address these issues in section 3, entitled “Drawing inferences from neural data to cognitive organization: A special role for patient evidence.”

Representation versus processing—Toward a theory of dynamics

Much recent research on the nature of conceptual representation has worked within a theoretical framework that juxtaposes amodal concept representations with embodied representations. Thus, the theoretical issue under debate ostensibly concerns the format of concept representation. In this context, it is useful to understand the theoretical position argued to be rejected in favor of an embodied account. It turns out that what has been rejected in favor of an embodied perspective is a rather listless and static version of the proposal that concepts are amodal representations. The goal in this section is to underline how a more viable explanation of the available data within an amodal framework can motivate new inferences about the dynamics of information exchange among concepts and sensorimotor representations.

Taking a step back, when sensorimotor representations are activated during conceptual processing, (at least) two inferences could be drawn:
  • Inference i.Concepts are represented, at least in part, in a sensorimotor format, and thus the activity of sensorimotor representations during conceptual processing is a direct reflection of “conceptual access” itself.

  • Inference ii.Concepts are represented in an amodal format, and activation spreads (perhaps automatically, perhaps in a task-defined manner) from amodal symbolic concepts to the sensorimotor representations to which they are connected; thus, sensorimotor activity is not evidence about the format of conceptual representation, but rather about the dynamics of information flow in the system.

As an example, it is well established that motor systems are engaged when processing lexical knowledge that refers (in its meaning) to a motor action (e.g., “kick”; Hauk et al., 2004; Glenberg & Kaschak, 2002; for a review, see Pulvermüller, 2013; but see Papesh, 2015; Postle, McMahon, Ashton, Meredith, & de Zubicaray, 2008). An embodied interpretation of those findings takes motor activity to indicate that at least a part of the concept in question (e.g., the concept “kick”) is represented in a motoric format. The alternative interpretation argues that motor activity is contingent on access to an amodal conceptual representation, and that the processing dynamics of the system are such that input and output systems are engaged during conceptual processing (for an attempt to adjudicate these possibilities, see Papeo et al., 2015).

Of course, none of these considerations are special with respect to motor system activity. Similar phenomena have been observed in the context of visual motion processing (e.g., Dils & Boroditsky, 2010; Pirog Revill, Aslin, Tanenhaus, & Bavelier, 2008; but see Pavan & Baggio, 2010), color processing (e.g., Simmons et al., 2007), visual form processing (e.g., Zwaan, Stanfield, & Yaxley, 2002; but see Rommers, Meyer, & Huettig 2013), gustatory processing (Simmons et al., 2013), and auditory processing (Kiefer, Sim, Herrnberger, Grothe, & Hoenig, 2008)—and similar considerations apply as to whether the phenomena reflect the format of conceptual representations or rather the processing dynamics that obtain between amodal symbolic representations and input/output systems. What this means, concretely, is that we need explicit bridging hypotheses about the dynamics of activation flow in the system, and those bridging assumptions would need to be investigated prior to deriving inferences about representational format. For instance, for motor activity during action-word processing to be taken as support for an embodied theory over an amodal account (e.g., Pulvermüller, 2013), the bridging assumption would have to be attributed to the amodal framework that “activity does not spread between amodal representations and input/output systems.” However, we have independent evidence that is contrary to this hypothesis (e.g., Dehaene et al., 1998; Navarrete & Costa, 2005; Peterson & Savoy, 1998). This implies that the activity of motor systems during action-word processing does not “support” the embodied hypothesis—such empirical phenomena are, at best, ambiguous with respect to whether action concepts have a motoric or an amodal format. The substantive issue that requires adjudication is no longer the format of concept representations, but rather how activity may or may not spread in the system.

These considerations suggest that, in an important way, current arguments about whether or not concepts are embodied have outstripped the inferential ground afforded by the phenomena the theories seek to explain. Theories of the neural basis of concepts have to this point focused almost exclusively on issues of representational format and neural and cognitive organization. Further progress now requires us to tackle the issue of the processing dynamics among concepts and between concepts and input and output systems (for further discussion, see Gotts, 2016; Hauk, 2016).

Drawing inferences from neural data to cognitive organization: A special role for patient evidence

As we noted above, a major issue that has recently (and within this issue) become a dominant theme is whether sensorimotor activity indexes the retrieval of information that is stored in a sensorimotor format. When a study concludes from sensorimotor activity that conceptual processing involves modality-specific information, that conclusion presupposes that the observed modality-specific activity is in fact modality-specific in its format. That supposition, or bridging hypothesis, is an empirical matter that merits direct evaluation; recognition of this point is gaining momentum, and there are some excellent examples of studies seeking independent evidence for those bridging hypotheses (e.g., Postle et al., 2008; Simmons et al., 2007; Simmons et al., 2013). Reverse inference over neuroimaging data can go some of the way toward addressing these issues. For instance, Pulvermüller (2013) has constructed the following argument, based largely on reverse inference: (i) Motor systems are activated during conceptual processing (e.g., when reading the word “kick”); (ii) we know from other studies that those are “motor regions” because they are activated during overt movements; (iii) this implies that the representation of some concepts (e.g., “kick”) is, at least in part, “motoric.” However, as has been widely discussed, arguments based on reverse inference can be misleading (Poldrack, 2006; see also Henson, 2006). It could be that a given region carries out multiple, and qualitatively distinct computations. For instance, if it were the case that sensorimotor representations were already abstractions, then would the activation of “abstract” sensorimotor representations be evidence for the embodied framework? This depends on whether the embodied cognition hypothesis is a claim about representational format or about representational content.

To put a finer point on this, imagine that a study uses Task A to define “primary” sensorimotor regions, and then Task B to test whether those regions are engaged during conceptual processing. One would want to know whether Task A (the localizer task) activates not only the “primary” regions, but also secondary areas; if the localizer task that identifies primary cortex also activates a secondary area, then the “overlap” between Tasks A and B could be in the secondary area (rather than the primary area). This matters greatly for whether those data support the hypothesis that the format of conceptual knowledge, rather than only the content of conceptual knowledge, is modality-specific. An example of this situation is the study by Simmons et al. (2007), in which they found that judgments about object color activated a region in the ventral stream that overlapped with a region independently identified as being involved in color perception. The study is notable for the real effort on the part of the authors to independently verify the nature of the information that was represented in the modality-specific region activated during the conceptual task. The color perception task activated both posterior (presumably primary) and anterior (presumably secondary) areas in the ventral stream; however, the task testing color knowledge activated the anterior (but not the posterior) region (for relevant comparative work in macaques, see Tootell, Nelissen, Vanduffel, & Orban, 2004; for a discussion, see Simmons et al., 2007; Leshinskaya & Caramazza, 2016).

In this context, patient evidence can play a pivotal—perhaps decisive—role, as it provides a means to address whether damage to a given region of the brain impairs conceptual processing and/or affects sensorimotor processing. For instance, if the motor representation of the foot is damaged, and a patient has lost the ability to move that foot but can still use the concept “kick” correctly (e.g., in conversation or naming the action), it then follows that the concept “kick” is not constituted by the information that has been lost due to the lesion. Since at least an aspect of the patient’s impairment is an inability to use the foot, one can conclude that the motor representations necessary for moving the foot are not part of the concept (see also Vannuscorps & Caramazza, 2016). Note that this conclusion follows regardless of whether that same region (i.e., motor representation) is activated when processing the concept “kick.” Because of this asymmetry between interference evidence (e.g., lesion data) and activation evidence (e.g., fMRI), the neuropsychological approach has a special role to play in adjudicating claims about the format of concept representation.

The approach of using neuropsychological data to infer the format of conceptual representations has been pursued from several angles. As we alluded to above, patient evidence has been used to construct an argument against the idea that concepts are stored in a sensorimotor format. For instance, patients with impairments in using tools can have no discernible impairments for naming the same objects; those data indicate that concepts of tools are not represented in terms of the motor information that supports their use, even though naming and identifying tools leads to activity in the same structures of the brain that are damaged in those patients (for patient evidence, see Buxbaum, Veramonti, & Schwartz, 2000; Garcea, Dombovy, & Mahon, 2013; Negri et al., 2007; Ochipa, Rothi, & Heilman, 1989; Rapcsak, Ochipa, Anderson, & Poizner, 1995; Rosci, Chiesa, Laiacona, & Capitani, 2003; Rumiati, Zanini, Vorano, & Shallice, 2001; for reviews, see Cubelli, Marchetti, Boscolo, & Della Sala, 2000; Johnson-Frey, 2004; Mahon & Caramazza, 2005; Rothi, Ochipa, & Heilman, 1991; for imaging evidence, see Chao & Martin, 2000; Mahon et al., 2007; 2013; Noppeney, Price, Penny, & Friston, 2006; for reviews of the imaging evdence, see Lewis, 2006; Martin, 2007, 2009). Or, stated more precisely, the conceptual information that is involved in naming or recognizing pictures does not involve the sensorimotor information that is damaged in patients who can no longer effectively use tools. Similarly, patients with motor lesions are not necessarily impaired in recognizing actions (even though their own ability to produce actions is compromised)—those patient data suggest that mirror-neuron-type accounts are overinterpreting the activation evidence (for evidence and discussion, see, Hickok, 2009, 2014; Negri et al., 2007; Rumiati et al., 2001; Vannuscorps & Caramazza, 2016; but see Avenanti, Candidi, & Urgesi, 2013; Pazzaglia, Pizzamiglio, Pes, & Aglioti, 2008). In the domain of language, patients have been described with difficulties with speech motor production, but intact recognition of speech sounds (e.g., Hickok et al., 2011; Rogalsky, Love, Driscoll, Anderson, & Hickok, 2011; Stasenko et al., 2015). In the domain of color, patients can be impaired in their color knowledge (“this is the color yellow,” “this is green,” etc.) but retain intact knowledge of the typical colors of objects (“bananas are typically yellow,” “grass is typically green”), or the reverse: patients can have spared color perception but impaired knowledge of the typical colors of objects (e.g., Bouvier & Engel, 2006; Luzzatti & Davidoff, 1994; Miceli et al., 2001; Stasenko, Garcea, Dombovy, & Mahon, 2014).

Just as importantly, patient evidence has increasingly been used as an argument for the view that sensorimotor systems participate in conceptual representation, and that the motor system participates in action recognition. For instance, it has been argued that patients with logopenic variant of primary progressive aphasia, and atrophy of cortical regions in and around primary auditory cortex, can have a conceptual deficit for knowledge of object sounds (Bonner & Grossman, 2012; Kiefer et al., 2008). Some prior work with patients with motor impairments has suggested that their performance on tasks tapping concepts of actions may not be normal (Bak, O’Donovan, Xuereb, Boniface, & Hodges, 2001; Boulenger et al., 2006; but see Kemmerer, Miller, MacPherson, Huber, & Tranel, 2013). In the domain of language processing, repetitive TMS to frontal motor areas can disrupt the ability to produce certain types of sounds, and there is also a drop in recognition of the same sounds when auditory stimuli are degraded (e.g., Meister, Wilson, Deblieck, Wu, & Iacoboni, 2007; Möttönen & Watkins, 2012).

These issues have gained increased attention recently, and will undoubtedly continue to attract more interest as emphasis shifts toward evaluating hypotheses about the representational format of conceptual representations and the attendant bridging assumptions. Certainly, past are the days when sensorimotor activation during conceptual processing can be taken to sanction, in a straightforward way, the inference that the format of conceptual representations is sensorimotor. The new and hard work of the field will be to continue to improve the quality of inferences that are derived from novel experimental effects.

Flexibility in conceptual representation

Another important theme that cuts across multiple contributions to this special issue is the idea that conceptual processing is flexible. The claim of flexibility in conceptual representation is intended to go beyond the observation that human conceptual abilities themselves are flexible. In other words, it is true by casual observation that human conceptual processing can flexibly recombine information essentially ad infinitum—the properties of the human mind that make this possible have been the topic of much theoretical work (e.g., Barsalou, 1999; Chomsky, 1959; Fodor, 1975; Pinker, 1994). The recent emphasis on “flexibility in concept representation” is focused instead on the idea that concepts (i.e., the “representations themselves”) are dynamic with dissociable components, such that one aspect of a “concept” may be used in one context or task, but another aspect of the concept may be used in another (van Dam, van Dijk, Bekkering, & Rueschemeyer, 2012; Willems & Casasanto, 2011; for discussions, see Dove, 2016; Kemmerer, 2015; Yee & Thompson-Schill, 2016). In other words, concepts do not have cores that are retrieved each time a concept is tokened: Overlapping subsets of conceptual information, which collectively form the “full” concept, can be solicited in a flexible manner according to the task constraints. And, critically, the retrieval of a given task- or context-dependent subset of a concept is sufficient for successful application of that concept.

It is clear that what counts as a “useful concept” will be task-dependent. If one needs to figure out whether a perceived animal is dangerous, one must access information about predatory animals, but if one needs to figure out whether that picture of a lion is Cecil or not, then one needs to access information about differential features of individual lions. This highlights the fact that there is no single notion of a concept, only clusters of information that are called upon in the service of the task at hand, and different tasks will dictate the utility of different types of information (see also the related discussion in Murphy, 2016). Such information may be high-level (is it a predator?) or low-level (is that Cecil’s coloring pattern or face shape?) and will probably be represented in different brain networks according to the regions that support the different distinctions being made. Similarly, according to whether the task is to act on a thing (e.g., grasp it) or evaluate its behavioral relevance or value, the necessary abstractions one needs to compute are different. A trophy is different from a goblet, and they need to be treated differently when deciding into what to pour the dinner wine; but the two objects may be analyzed equivalently when the task is to grasp the object to move it.

Where to draw the line around a concept, which relates to whether the same conceptual information is used across tasks, is not something that can be decided a priori. Assuming a theory of the processes involved in a given task, empirical scrutiny of those processes is required. A theory is required, at least as an initial hypothesis, as to which contexts or tasks will result in one aspect of a concept being invoked rather than another. Otherwise, the theory of flexibility in concept representation becomes no more than an ad hoc redescription of the data. For instance, imagine that in Task A it is observed that processing the concept “kick” leads to activity in the motor system, whereas processing “kick” in Task B leads to activity in the visual system (e.g., van Dam et al., 2012). To conclude from those data that the concept “kick” is distributed over both motor and visual information and has a dynamic representation that is modulated by context would amount to no more than a redescription of the data. Do the variable patterns of sensorimotor activity during conceptual processing indicate the irrelevance of sensorimotor processing for conceptual processing, or rather flexibility in the sensorimotor underpinnings of concept representation? To make progress on this question, we need independent theoretical or empirical motivation as to why Tasks A and B should lead to activity in the motor and visual systems, respectively.

As a concrete example, the motor system is more activated for affirmative sentences (“kick the ball!”) than for negative sentences (“Do not kick the ball”; Tettamanti et al., 2008). But, “kick” still means “kick,” regardless of the degree to which motor information is engaged during the computation of the meaning of that word—the meaning of “kick” is the same in both the affirmative and negative sentence contexts. Thus, the conclusion would seem to follow that the meaning of the word “kick” does not map one-to-one to how that meaning is reflected in the sensorimotor system. This conclusion, in and of itself, does not support the idea of flexibility in concept representation; it could as well be taken to support the idea that sensorimotor processing is not constitutive of meaning (see Mahon, 2015c). This ambiguity becomes clear when considering an alternative to the hypothesis of flexibility in concept representation. If one assumed that concepts are represented as “amodal cores” that are representationally static in terms of their content—the so-called “classic” view, with roots that are arguably traceable to an Aristotelian ontology of mental representations—one could still explain the phenomena that are argued to support the idea of flexibility in concept representation. The classic view could argue that the variable sensorimotor mappings of the concept are a reflection of how activity spreads in a task- or context-dependent manner from an amodal “core” to peripheral input and output systems. For this reason, theories of concept flexibility need to explain why there is variability in the sensorimotor correlates of meaning. Without a renewed focus to develop those criteria, the intriguing idea that conceptual representations are flexible representations risks devolving into a collection of ad hoc accounts of each “exception” to “what would otherwise be a pattern” (for discussion of such criteria and a framework for approaching these issues, see Barsalou, 2003, 2008b, 2016b).

Other data that have motivated the hypothesis of conceptual flexibility are the same neuropsychological data, reviewed above, that challenge strong forms of the embodied hypothesis. For instance, if damage to motor-relevant structures does not seem to substantially affect the processing of concepts thought to depend (representationally) on motor information, then there are two conclusions: (i) Those concepts do not constitutively include motor information, or (ii) those concepts sometimes constitutively involve motor information, but in a task-dependent manner, and not in the tasks on which the patients were tested and shown to be unimpaired (e.g., picture naming). As Barsalou (2016a) adroitly frames it:

grounded views generally predict that significant damage to a modality should have consequences for conceptual processing. If conceptual processing relies on a modality—as argued later for neural reuse—then it should change in some way after the modality is damaged. To assess such consequences, however, it is first necessary to rule out uninteresting and uncontrolled confounding factors. If, for example, a double dissociation doesn’t control for the compensatory use of other modalities, association areas, and/or distributed linguistic representations, it is impossible to assess the consequences of a damaged modality. Furthermore, assessments of the damaged modality should be sufficiently sensitive and well-designed to establish these consequences. Without designing double dissociation research around these basic principles, it is impossible to assess the theoretical implications of damage to a modality rigorously.

For instance, if an fMRI experiment is run and motor activity is observed during conceptual processing, then it would be concluded that part of the meaning in question is distributed over motor information; if motor activity is not observed, on the other hand, it would be concluded that meaning representation is flexible, so motor activity is not always a criterion for concluding that motor information is part of the concept. Similarly, if a patient with a sensorimotor impairment does not have a conceptual impairment for the relevant concepts, then it would be concluded that those concepts are flexible, and thus not affected by redaction of that sensorimotor information. If it were observed that the patient did have a conceptual deficit, it would be concluded that those sensorimotor representations were critical substrates of the conceptual representation. What are needed are well-articulated bridging hypotheses that specify why motor activity (or an impairment) is predicted in one situation and not another. Critical to this enterprise will be consistency between arguments that sensorimotor activity supports the thesis of embodiment and patient data that challenge the thesis of embodiment. One way to ensure such consistency would be to pair activation studies with patient studies: If an fMRI study indicates motor activity for a given type of concept in a given task, and those data are taken as support for the embodied perspective, the prediction then follows that damage to that region of the motor system should lead to an impairment for that task. If damage to that region of the motor system does not result in a conceptual impairment, then it can be concluded that the activation of that region during conceptual processing in fact has little (or nothing) to do with meaning.

The view that concepts are dynamic representations that are flexibly deployed according to task demands is new, interesting, and exciting. On one level of analysis, it is almost certainly true—the key moving forward will be to develop concrete proposals within this new framework so that we can begin the work of evaluating in exactly what ways concept representations are flexible. Certainly, our predictions can no longer be simple—the predictions that must be evaluated are complex, because they have embedded within them a theory of the processes that are engaged by a given task.

Summary and beyond

The areas within the cognitive and brain sciences that study how concepts are represented, organized, and processed in the mind and brain have made tremendous progress over the last several decades. That progress has been accelerating in recent years with the widespread use of new methods that permit detailed investigation of the neural basis of conceptual processing. New areas of investigation are opening up, and with new populations, including whole-brain connectonomic approaches to understanding the cognitive and neural bases of concept organization, multivariate analysis of the “neural code” of information “storage” in specialized regions of the brain, and attempts to test claims about the format of concept representation. The first generation of theories argued for unidimensional schemes, such as that some concepts are organized by semantic category, or that concepts are organized by modality, or that concepts are amodal, or that concepts are distributed over sensorimotor representations. Those initial theories were largely shown to be inadequate in accounting for the diversity of findings that we now have as a field. The new wave of theories, a cross-section of which are captured in this issue, are hybrid models that draw individual assumptions from what had been opposing theories. As a result, new questions of fundamental significance to any theory of concept representation have come into sharper focus: What are the [ontogenetic | phylogenetic | anatomical | experiential | statistical | cognitive | perceptual | etc.] constraints that lead to specialization of function in the brain for a given type of conceptual knowledge? Does specialization of function impose a hard limit on the brain’s ability to recover conceptual knowledge after injury, or is significant plasticity within the conceptual system possible? What is/are the representational format/s of concepts? What are the processing dynamics that interface concepts with sensorimotor systems? To what degree are concepts task-defined entities, such that full task analyses must be embedded within any sufficient explanation of how concepts are represented and deployed? How can we use neural data to constrain cognitive theories, and vice versa?

One of the great debates in cognitive science—whether mental imagery involves “pictures” in the brain—bears a number of instructive similarities to current discussions about the nature of concepts (Pylyshyn, 1973; for a discussion, see Hauk & Tschentscher, 2013). The theory championed by Kosslyn, and many others, argued that visual imagery occurs over a representation that is visual in its format—that a mental image is a “picture in the brain”; as such (and only as such), would visual mental images inherit and exhibit the metrical properties of an actual physical image. For instance, the time taken to scan an image is a function of the distance traversed (Kosslyn, 1973; for relevant findings, see also Behrmann, Winocur, & Moscovitch, 1992; Kerr, 1983), and “big” visual images use more retinotopic cortex than “little” mental images (e.g., Kosslyn, Thompson, Kim, & Alpert, 1995; for a review, see Kosslyn, Ganis, & Thompson, 2001). The alternative theory (e.g., Pylyshyn, 2003) argues that the format of mental images is not imagistic or pictorial, or at least that we have no reason to suppose that it is; for that matter, the format of cortical visual processing is likely not itself imagistic or pictorial, and we also have no compelling reasons to suppose that it is. The fact that vision, and perhaps visual mental imagery, is laid out in a retinotopic manner on the manifold of the cortex doesn’t mean that the format of retinotopic visual processing is in any interesting sense “imagistic.” No more does the fact that the motor representation is laid out as a homunculus mean that motor representations are “homuncular” (whatever that would mean). In this context, Pylyshyn (2003, p. 113) observed that “the difference between pictorial and other forms of reasoning rests primarily in what different thoughts are about, rather than the form that they take, and that contemporary discussions of mental imagery often confound questions of form with questions of content.” Theoretical discussions of the nature of concept representation also at times too easily glide between issues of representational content and issues of representational format.

Unlike other areas in cognitive science, the study of concepts has fewer a priori structures available to constrain the nature of the problem that needs to be solved. The study of language processing can look to linguistics for formal accounts of the scope of the issues that need explanation; the study of perception and motor processes can look to objective definitions of the physical parameters of stimulation or movement to define the scope of the problem that computations must solve. The study of concepts can and does usefully draw on insights from other fields, such as analytic philosophy, artificial intelligence, and linguistics—but those inputs offer only soft constraints. Such is the study of concepts. Continued progress will come with a renewed and sustained attention to the bridging assumptions that undergird our theories. Turning our theories “inside out” and empirically studying their bridging assumptions promises to be one of the next great adventures in the study of concept representation in the brain.

Copyright information

© Psychonomic Society, Inc. 2016

Authors and Affiliations

  1. 1.Department of Brain and Cognitive SciencesUniversity of RochesterRochesterUSA
  2. 2.Department of NeurosurgeryUniversity of RochesterRochesterUSA
  3. 3.Center for Visual ScienceUniversity of RochesterRochesterUSA
  4. 4.Center for Language SciencesUniversity of RochesterRochesterUSA
  5. 5.Department of Cognitive SciencesUniversity of CaliforniaIrvineUSA

Personalised recommendations