On Staying Grounded and Avoiding Quixotic Dead Ends

The study of concepts offers a significant set of challenges. First, conceptual processing proceeds largely unconsciously, such that it is difficult to observe concepts and their effects. Indeed, perhaps the dominant view on the relation between concepts and consciousness is that concepts operate completely unconsciously (for a review, see Kemmerer, 2015b). Second, as a consequence of operating unconsciously, concepts are difficult to discern in content, structure, and operation. By and large, researchers must use indirect methods to study them. Third, concepts operate across an unusually broad spectrum of intelligent activities, ranging from perception, to language and thought, to social and affective processes. How researchers view concepts in each of these areas can vary widely. Approaches to concepts in language, for example, often differ considerably from approaches in nonverbal domains, such as object and scene recognition. Fourth, as a result of the preceding challenges, concepts are associated with an unusually wide variety of theoretical proposals and positions. Indeed, it is difficult to think of a domain characterized by so many different views and so much disagreement (Barsalou, 2003b, 2012; Margolis & Laurence, 1999; McRae & Jones, 2013; Murphy, 2002).

Because of these challenges, there is the potential to head off in the wrong direction. At least many researchers in the area believe this, often accusing each other of being completely misguided in their approach. In this spirit, I begin with candidates for what I will refer to as Quixotic dead ends. In my opinion, pursuing these approaches is unlikely to yield success in understanding concepts, specifically, and cognition, more generally. Because these approaches often are principled, inspired, idealistic, and optimistic, while being impractical and unrealistic, I refer to them as Quixotic.

I do appreciate that what one researcher perceives as a Quixotic dead end is another’s ultimate solution. Thus, I recognize that I may be wrong about my predictions to follow. After presenting my predictions for approaches unlikely to succeed in this section, I turn to approaches that I believe are more likely to succeed in the next.

I begin with two approaches associated with grounded cognition that are unlikely to succeed as accounts of concepts (and of cognition more generally). I refer to these approaches and related ones later as “grounded,” not “embodied,” because “grounded” better captures the central focus of the general perspective by including other forms of grounding beside embodiment, such as multimodal simulation, physical situations, and social situations (Barsalou 2008a, Barsalou 2010; Kiefer & Barsalou, 2013).

Machery (2016) refers to grounded cognition with the endearing moniker of “neo-empiricism” (also see Machery, 2007). I agree that purely empiricist accounts, incorporating no biological constraints, are unlikely to be successful. Indeed, some grounded cognition researchers have explicitly expressed a commitment to nativist factors (Barsalou, 1999, 2008a; Simmons & Barsalou, 2003). Many classic nativists, such as Kant and Reid, centrally included sensory-motor processes, such as imagery, in their accounts (Barsalou, 1999, 2008a, 2010). Nevertheless, grounded cognition researchers continue to be called neo-empiricists, even though none of us to my knowledge explicitly rejects important native contributions. Strong biological constraints certainly exist on the cognitive system, including on conceptual processing. Consistent with Caramazza and Shelton (1998), for example, Simmons and Barsalou (2003) argued that feature and association areas in the brain have been shaped by evolution to anticipate important categories, such as foods, tools, and agents.

Throughout later sections, Barsalou (2008a) continues to describe how both language and internal states contribute to the representation of concepts above and beyond the sensory-motor modalities. Analogously, Pulvermüller and Fadiga (2010) never claimed that cognition can be reduced to sensory-motor representations but instead focused on how action supports perception and how perception-action relations contribute to language.

Second, both Barsalou and Pulvermüller clearly claim throughout their published articles that other mechanisms besides sensory-motor representations are essential for explaining concepts. Pulvermüller (2013), for example, argued that disembodied mechanisms in the brain’s hub regions contribute to semantic meaning (Pulvermüller, 2012; Pulvermüller & Garagnani, 2014). Similarly, Barsalou (1999, pp. 585, 600-603) went to considerable lengths in proposing that internal states play central roles in conceptual processing, especially for abstract concepts (also see Barsalou & Wiemer-Hastings, 2005). In a different vein, Barsalou, Santos, Simmons, and Wilson (2008) reviewed a program of research showing that distributed linguistic representations play central roles in representing and processing concepts. Given the large numbers of publications from both groups going beyond sensory-motor mechanisms, it is difficult to understand how Leshinskaya and Caramazza could have missed them.

In concluding their article, Leshinskaya and Caramazza make a rather ironic proposal about how the field should be “Moving forward.” Specifically, they propose that future research should focus on nonsensory attributes of knowledge, such as intention, belief, and function. To provide background for their recent research, Leshinskaya and Caramazza cite Wilson-Mendenhall, Simmons, Martin, and Barsalou (2013) as having previously illustrated this approach (i.e., by establishing non-sensory-motor attributes associated with the concepts of convince and arithmetic). Notably, Wilson-Mendenhall et al. (p. 921) cited four previous articles going back to Barsalou (1999) as background for the hypothesis that “the lexical representation for “convince” is associated with much nonlinguistic semantic content that supports meaningful understanding of the concept, including the intentions, beliefs, internal states, affect, and actions of self and others that unfold in a spatio-temporal context.” Using neuroimaging, Wilson-Mendehall et al. established the neural areas that underlie this abstract non-sensory-motor content. Given that Leshinskaya and Caramazza cited this work, their earlier claim that Barsalou (2008a) and Barsalou et al. (2003) reduce concepts to “modality-specific sensory-motor representations” is puzzling. It’s been clear for some time that our group views abstract representations as essential.

Goldinger et al. (2016) offer a host of common misconceptions about grounded approaches, warping them into Quixotic dead ends. I address only a few of the more salient misconceptions. First, Goldinger et al. focus on embodiment in embodied cognition. Although embodiment is an important theme of grounded cognition, it is certainly not the only theme or arguably the most important one. Most, if not all, grounded cognition researchers would agree that embodiment is far from sufficient for explaining cognition and believe that it may be irrelevant during much cognitive processing (Barsalou, 2008a, 2010). The crux of the grounded approach is understanding how the modalities, the physical environment, the social environment, and the body contribute to cognition, playing central roles in the diverse forms it takes (Barsalou, Breazeal, & Smith, 2007).

Second, Goldinger et al. state that “modeling is currently impossible in EC [embodied cognition].” Without a doubt, any approach that can’t be modeled is a non-starter. Review articles, such as Barsalou (2008, pp. 634-635), however, often have cited computational models of grounded cognition, which continue to accumulate (Adams et al. 2014; Blouw, Solodkin, Thagard, & Eliasmith, 2015; Caligiore, Borghi, Parisi, & Baldassarre, 2010; Eliasmith, 2013; Pulvermüller & Garagnani, 2014; Schrodt, Layher, Neumann, & Butz, 2015; Thagard & Stewart, 2011). Given the existing literature, how could anyone responsibly claim that computational models aren’t possible?

Goldinger et al. further propose that grounded cognition is not sufficiently specified to produce concrete models. Notably, all other approaches to cognitive theory must address this issue as well, including the classic information processing perspective, neural nets, Bayesian modeling, and so forth. For each of these general approaches, it is necessary to make assumptions that constrain the space of models, thereby making the construction of specific models possible. Information processing, for example, is so general that a huge space of models exists, many of which are implausible in humans, such as many models in machine learning, other areas of computer science, and mathematics. Only when assumptions relevant to human information processing are made, does it become possible to construct meaningful models about human cognition. Grounded cognition is no exception, with the models developed so far reflecting various theoretical commitments that made them possible.

Third, Goldinger et al. suggest that grounded cognition aims to replace standard theories of cognition but is unable to do so. In my experience, re-explaining all basic cognitive phenomena is not something to which grounded cognition researchers aspire at all. Again, most, if not all, grounded cognition researchers would agree that grounded cognition, on its own, is far from capable of explaining everything that we know about cognition. To the contrary, grounded cognition complements traditional approaches. Rather than aiming to replace traditional approaches, grounded cognition develops their relations with the modalities, the body, and the environment (Barsalou, 2008a, p. 635-636). Similarly, an integration of grounded approaches with classic symbolic and statistical approaches is likely and necessary (Barsalou, 1999, p. 652; Barsalou, 2010, p. 721).

Finally, Goldinger et al. present a long list of basic cognitive phenomena, such as word frequency, short-term memory scanning, and serial recall, arguing that the grounded approach hasn’t informed our understanding of them. Goldinger et al.’s surprisingly lengthy (and elementary) descriptions of these phenomena give the “copious theoretical writing” of grounded cognition a run for its money. No grounded cognition researcher would argue that grounded cognition should or could re-explain all cognitive phenomena. Again, grounded cognition aims to complement and incorporate theoretical constructs and empirical findings in the field, such as those on Goldinger et al.’s arbitrary list.

In summary, I completely agree that the approaches to grounded cognition covered in this section are dead ends, including approaches that have no biological constraints, approaches that assume concepts are reducible to sensory-motor systems, approaches that focus completely on the body, and approaches that aim to replace existing cognitive theories rather than complement them.

I turn next to another class of theories that I believe are equally unlikely to succeed, namely, theories proposing that amodal symbols constitute the central form of conceptual representation. I will assume that the amodal symbols discussed in this section have two properties: (1) they are arbitrarily related to their corresponding categories in the world and experience; and (2) they can stand alone without grounding to perform the basic computations underlying conceptual processing. In a later section, I entertain other types of abstract representations having different properties that strike me as more plausible.

Interestingly, the interaction hypothesis often (but not necessarily) assumes that conceptual areas lie near related sensory-motor areas. To explain why conceptual processing often activates regions within sensory-motor processing streams, the interaction hypothesis proposes that conceptual processing areas utilize amodal symbols that reside within these streams. When processing color conceptually activates areas in the color perception pathway (Hsu, Frankland, & Thompson-Schill, 2012; Simmons et al., 2007), the interaction hypothesis proposes that these activated areas contain amodal symbols. In this way, it becomes possible to argue that activations for conceptual processing in sensory-motor streams don’t actually offer support for grounded views. Instead, these activated areas implement conceptual processing using amodal symbols that interact with nearby sensory-motor areas when necessary.²

My pessimism about some approaches represented in the special issue is more than tempered by my optimism about others. The large majority of contributions point to future directions that are likely to be productive, leading to greater understanding of concepts, specifically, and of cognition, in general (including contributions of which I’ve been critical so far). I turn next to themes raised in these contributions that, in my opinion, are likely to be useful elements of future research, including grounding, abstraction, flexibility, temporal dynamics, the ability to explain classic conceptual phenomena, and making contact with real-world situations.

Many of the contributions in the special issue acknowledge the importance of grounding conceptual processes and further acknowledge the considerable evidence for grounding that has accumulated (Binder, 2016; Dove, 2016; Hauk, 2016; Jamrozik et al., 2016; Kemmerer, 2015a; Martin, 2016; Reilly et al., 2016; Yee & Thompson-Schill, 2016; Zwaan, 2016). Because some contributions raise issues about what is meant by grounding (Leshinskaya & Caramazza, 2016; Martin, 2016; Mahon, 2015), I begin with what I mean by it.

To a large extent, grounding concerns itself with the grounding problem raised initially by Searle (1980) and Harnad (1990), which asks how amodal symbols, specifically, and cognition, more generally, are linked to the modalities, body, and environment. In a review of research on grounding, Barsalou (2008a) argued that researchers have attempted to ground concepts and cognition by establishing their relations with modality-specific systems, the body, the physical environment, and the social environment (also see Barsalou, 2010; Barsalou et al., 2007; Kiefer & Barsalou, 2013). Thus, at a general level, grounding simply refers to programmatically studying cognition in new ways. Rather than studying cognitive mechanisms in isolation, establish their relations with the contexts in which they are embedded and on which they depend. At more specific levels, grounding refers to establishing specific accounts of how cognitive processes in the brain utilize the modalities, the body, and the environment. It does not mean reducing concepts and cognition to anything, including sensory-motor mechanisms. As described next, one important form of grounding is neural reuse, where cognition partially utilizes the modalities to implement its basic functions (but does not reduce to them).

An alternative view is that conceptual processing depends on modality-specific systems, utilizing the same—not different—systems that support perception, action, and internal states, what Anderson (2010) called neural reuse. As Martin (2016) describes for the domains of color and taste, conceptual processing in these domains partially utilizes the same systems used for visually perceiving color and for gustatorally perceiving taste (see Wang et al., 2013, for another thoughtful discussion of this issue). Obviously, the neural systems in conceptual processing and perception differ, given that the brain is implementing two different processes. Conceptual processing, for example, is more likely to draw on integrative and abstractive mechanisms in association areas (Binder, 2016; Simmons & Barsalou, 2003). As a consequence, conceptual processing does not reduce to modality-specific processing but utilizes many other systems as well. Nevertheless, depending on task conditions, conceptual processing, in part, often reuses systems that underlie perception, action, and internal states.

Neural reuse differs significantly from the idea that amodal symbols describe modality-specific information in stand-alone conceptual processing areas (e.g., amodal symbols that describe color and taste features; Leshinskaya & Caramazza, 2016; Mahon, 2015). Instead, neural reuse makes the stronger commitment that modality-specific information is represented conceptually by partially reusing the same brain areas that represent this information during perception and action. Thus, representing color and taste features conceptually requires reusing some of the same systems active during vision and eating.

Neural reuse offers a natural account of what is meant by simulation (Barsalou, 1999, 2003b, 2009): Reusing a modality-specific pathway during conceptual processing simulates the kind of processing that this pathway performs during perception, action, and/or internal states. Again, reuse may not be complete and may vary considerably across tasks and contexts. Nevertheless, the point remains that higher cognitive processes are grounded in more basic processes by virtue of reusing their neural resources. From this perspective, cognition cannot be performed by only using independent amodal symbols. Instead, it depends on modality-specific processes to some extent (along with other mechanisms such compressed multimodal representations, distilled abstractions, and distributed linguistic representations, as discussed later).

To the extent that neural reuse plays central roles in conceptual processing, much work must be performed to better understand it. Specifically, how are conceptual uses of modality-specific pathways related to uses of these pathways during perception and action? Are common areas used in similar manners, or in different manners? What computational functions do these areas perform, and how are they related to modality-specific and conceptual processing? What are the roles of different laminar layers for bottom-up and top-down input? In general, we need more sophisticated theoretical accounts of neural reuse that go beyond the simple proposal that neural reuse occurs. Analogously, we need to go beyond simple demonstration experiments and develop more systematic empirical investigations that test and inform these theories.

For all of these reasons, arguing that brain regions with amodal symbols reside in what have traditionally viewed as modality-specific processing streams is unusual and counterintuitive. Yet, as we saw earlier, this is the claim of the interaction hypothesis (Leshinskaya & Caramazza, 2016; Mahon, 2015). According to this account, amodal regions in modality-specific processing streams can perform stand-alone conceptual processing, only interacting with adjacent modality-specific areas as needed.

From this perspective, capturing records of neural states allows later reusing them for a wide variety of cognitive purposes, without making any commitment to their format (for discussion of reusing captured neural states, see Barsalou, 1999 pp. 603-608, 2008a, 2009).

Ironically, Leshinskaya and Caramazza (2016) and Mahon (2015) don’t follow their own admonitions about the inability to specify format, proposing that conceptual processing areas in modality-specific streams have an amodal format (or a non-sensory-motor format). Again, the default explanation has its privileges. Regardless, these proposed amodal regions continue to have the character of black holes in conceptual space. No accounts are provided of why these regions contain representations in an amodal format, much less hypothesis-driven empirical evidence for these claims. Instead, complex elaborate explanations based on questionable assumptions are used to justify them.

Consistent with neural reuse, conceptual processing utilizes modality-specific resources, such that conceptual processing overlaps with modality-specific processing. It is not the format that matters but the fact that conceptual processing utilizes modality-specific resources—whatever their formats happen to be. Conceptual processing does not operate in a stand-alone manner but instead relies on regions of modality-specific pathways. Martin (2016), together with many others in the special issue issue and elsewhere, review considerable evidence consistent with the reuse perspective (for some recent examples, see Schwiedrzik, Bernstein, & Melloni, 2016; Waldhauser, Braun, & Hanslmayr, 2016).

One final point about format: Regions of a modality-specific pathway, such as the ventral stream, have probably evolved to perform specific types of computations on relevant information. As a consequence, the representations and processes in these regions are likely to reflect such constraints. As a further consequence, representations and processes are likely to differ across modalities (e.g., vision vs. audition vs. action vs. affect vs. taste). If so, then representations and processes within a particular modality-specific processing stream are modality-specific, not amodal. The fact that cytoarchitectonics vary within and across modalities is consistent with this proposal (Amunts & Zilles, 2015), as is the fact that different conceptual information can be represented in different cytoarchitectonic areas (Grill-Spector & Weiner, 2014). Thus, if the neural reuse hypothesis is correct, it follows that when a conceptual process utilizes the resources of a modality-specific processing stream, the resultant conceptual representations have a modality-specific character, not an amodal one.

Bringing cytoarchitectonics into the discussion may make it possible to address the format issue in a much more sophisticated way than simply being concerned with whether a brain area contains amodal or iconic (sensory-motor) representations. The key issues are likely to be, first, establishing the neural computations that the cytoarchitectonics of an area make possible, and second, establishing the cognitive functions these computations realize. I suspect that once we understand these computations, they will have little to do with the issue of amodal vs. iconic format. Instead, understanding these computations is likely to play central roles in understanding how conceptual processing reuses modality-specific pathways.

Alternatively, concern exists that cognitive psychology is not evolving in important new ways, but has become stuck in an approach that hasn’t changed much in decades. Traditionally, cognitive psychologists focus on cognitive processes per se, ignoring how they’re grounded (e.g., the classic processes of information processing, such as attention, working memory, lexical access, etc.). Studying these processes typically occurs in highly idealized laboratory paradigms designed for the sake of establishing the properties of a particular process, not because these paradigms reflect important behaviors in the world that we need to understand better. When modeling findings from these paradigms, the resultant theories typically have little relevance for explaining how the cognitive processes of interest manifest themselves in real-world behavior. Thus, we end up with theories of laboratory paradigms whose relations to the world are tenuous. As a further consequence, we accumulate a long list of processes, like those reviewed in Goldinger et al.’s article, with little understanding of how they relate to anything, including the modalities, the body, and the environment. One might worry that this poverty of riches is leading the field down a Quixotic dead end.

Grounding offers a natural direction for developing accounts of basic cognitive processes. Again, grounded cognition primarily aims to complement existing accounts, not to replace them. Not only does grounded cognition benefit from incorporating basic cognitive processes, it offers new opportunities for studying them, and for understanding how they operate when embedded in the modalities, body, and environment. As Barsalou (in press-a) suggests, cognition can be defined as the mediating processes between stimuli and responses that make adaptive action in goal-directed situations possible. As Barsalou (in press-c) further suggests, cognition is organized around a situation processing architecture that underlies a wide variety of cognitive, social, affective, and appetitive behaviors, with neural reuse and simulation playing central roles. Examining basic cognitive processes from grounded perspectives like these offers new opportunities for understanding, studying, and explaining their underlying mechanisms and their roles in human intelligence.

Perhaps new insights into basic cognitive processes will develop, despite Goldinger et al.’s skepticism. Several of the topics Goldinger et al. cover—priming, face processing, and sentence processing—have received considerable attention in grounded cognition, with researchers believing that something new has been learned (Lebois et al., 2015; Niedenthal, Mermillod, Maringer, & Hess, 2010; Santiago et al., 2011; Zwan, 2016).

As researchers have become increasingly convinced that concepts are grounded, they have simultaneously become increasingly aware of how extensively abstraction is associated with conceptual processing. Indeed, abstraction appears to be a hallmark of human cognition and an important source of its computational power. Thus, a current challenge is explaining how grounding and abstraction emerge together.

It’s perhaps first worth noting that abstraction takes many forms. Barsalou (2003a), for example, listed six forms of abstraction often associated with conceptual processing: (1) categorical knowledge, (2) the ability to generalize, (3) summary representation, (4) schematic representation, (5) flexible representation, and (6) abstract concepts. Dove (2016) similarly addresses various forms of abstraction. Contributors to the special issue bring up all these forms at one point or another across articles (Binder, 2016; Dove, 2016; Jamrozik et al., 2016; Leshinskaya & Caramazza, 2016; Martin, 2016; Murphy, 2016; Reilly et al., 2016; Yee & Thompson-Schill, 2016; Zwaan, 2016; also see Mahon, 2015).

More significantly, these contributors suggest several mechanisms that could potentially implement abstraction, which differ in important ways from amodal symbols: multimodal compression, distilled abstraction, and distributed linguistic representation. Although Mahon (2015) reduces these different mechanisms into a single construct of amodal symbols, this move does not offer a useful way forward. Rather than seeing these mechanisms as the same, it is important to understand the differences between them and to establish their consequences for explaining results, motivating future research, and for informing our understandings of the brain and cognition. Certainly, amodal symbols offer one way of implementing abstraction. Nevertheless, multimodal compression, distilled abstraction, and distributed linguistic representation offer important alternatives that appear easier to define theoretically and that enjoy stronger support empirically. They also all appear highly compatible with grounded approaches, as we will see.

Binder reviews much literature that bears on the localization of multimodal compressions, such as CCRs, in the brain. Specifically, this literature suggests that the associative processes underlying conceptual knowledge consistently engage brain areas outside modality-specific areas, including medial prefrontal cortex (dorsal and ventral), posterior cingulate cortex, angular gyrus/supramarginal gyrus, lateral middle temporal gyrus, inferior frontal gyrus, and ventromedial temporal cortex (also see Legrand & Ruby, 2009). In general, these areas tend to become more active for stimuli associated with many abstractions than for stimuli associated with fewer abstractions. Specifically, these areas tend to become more active for meaningful words than for pseudo-words (and for meaningful sentences and texts than for anomalous ones). The same areas are more active for familiar proper names and high-frequency words than for unfamiliar proper names and low-frequency words, and when highly related and associated concepts are processed together than when less related and associated concepts are processed together. Finally, processing concepts deeply activates these areas more than when the same concepts are processed shallowly.

Based on these findings, Binder concludes that these association areas process abstractions. As the abstractions associated with a stimulus increase, these areas become more active, because they represent and process the increasing numbers of abstractions. Additional findings from Fernandino et al. (in press) further implicate compressed multimodal representations in association areas, showing that association areas all over the brain track the modality-specific content of concepts. Rather than being arbitrarily related to modality-specific content in concepts, association areas represent it.

Compressed representations, such as CCRs, are essentially the same kind of representations as prototypes in cognitive theories of concepts (Murphy, 2016). According to prototype theory, statistically likely features are extracted from category exemplars and conjoined in a prototype that represents the category conceptually (Hampton, 2006; McRae, Cree, Seidenberg, & McNorgan, 2005; Rosch & Mervis, 1975; Smith & Medin, 1981). Notably, prototypes are not amodal symbols arbitrarily linked to examplars. Instead, the features of exemplars appear in the prototype that covers exemplars, following various possible forms of data compression, as for CCRs.

An alternative form of data compression that could underlie multimodal abstractions reduces exemplar information to a new set of dimensions (as in PCA, ICA, NMF, and so forth). Rather than using features from exemplars to represent a prototype, new dimensions are abstracted that capture correlated sets of features. As a result, different vectors across these new dimensions would represent concepts in Binder’s association areas. Again, these compressions aren’t amodal symbols, because they carry information about the information in exemplars. For examples of how this type of compression can be used to represent different kinds of concepts, see Reilly et al. (2016), Blouw et al. (2015), and Mitchell et al. (2008) . Many neural net models similarly use dimensional reduction to represent knowledge (Ganguli & Sompolinsky, 2012; Hinton, 2006).

As Murphy (2016) notes, exemplar theories offer an alternative to prototype theories, with exemplar theories assuming that individual exemplar memories represent a category instead of an aggregate prototype. Although Murphy has strong inclinations to view exemplar theories as a Quixotic dead end, the mechanisms that integrate exemplars into a category representation could potentially reside in the areas that Binder (2016) associates with abstraction, if these theories turned out to be accurate.

Murphy further suggests that exemplar effects offer a plausible alternative to exemplar theories, with salient exemplars sometimes controlling categorization (Allen & Brooks, 1991). A class of accounts that naturally explains both prototype and exemplar effects assumes that abstractions include statistical information about correlations of features, not just about individual features (Barsalou, 1990; Barsalou & Hale, 1993; McClelland & Rumelhart, 1985; also see Love, Medin, & Gureckis, 2004). Because exemplars can be viewed as patterns of correlated features, storing correlational information from them in abstractions implements exemplar effects (e.g., using hidden units; Gotts, 2016). Factorization approaches, such as PCA, ICA, and NMF, also are readily capable of capturing exemplar effects, representing salient exemplars with individual components. Again, such patterns constitute data compression that could plausibly be implemented in Binder’s (2016) association areas. As all these approaches indicate, it is possible to capture the exemplar effects that Murphy endorses, without further endorsing exemplar models that posit individual exemplar representations in memory.

What sorts of abstract features exist that could be distilled into abstractions? As discussed earlier, recent research has already established that the brain represents a variety of abstract features associated with the concepts of convince, arithmetic, goal, belief, and function (Leshinskaya & Caramazza, 2016; Wilson-Mendenhall et al., 2013). The abstract features of reward and magnitude offer further examples of important abstract features, given that dedicated brain areas represent them to some extent, namely, the orbital frontal cortex and intraparietal sulcus, respectively (Rudebeck & Murray, 2014; Walsh, 2003; Wilson, Takahashi, Schoenbaum, & Niv, 2014). Potentially, many diverse abstract features structure experience extensively and thus play central roles in distilled abstraction.

Linguistic analyses of syntactic and semantic primitives offer another possible source for discovering abstract features. Talmy (1985), for example, offered numerous examples of abstract features that are lexicalized across languages, playing a wide variety of important roles in syntax and semantics, including features for cause, aspect, valence, together with more concrete features such as path, motion, and manner (also see Langacker, 1986, 2008, and many other cognitive linguists). Increasing research establishes the neural systems that implement these features (Kemmerer, 2006; Kemmerer, Castillo, Talavage, Patterson, & Wiley, 2008).

What is the status of abstract features in the brain? Because these features represent abstract information present in a situation, perhaps they are similar in status to sensory-motor features, such as color, motion, pitch, taste, touch, and movement that represent features in perception and action. As instances of these features are experienced, the corresponding feature areas become active to represent them. Alternatively, amodal symbols could represent abstract features, a common assumption that many researchers make. Nevertheless, it seems important and potentially useful to consider the possibility that abstract features have a similar status in the brain as sensory-motor features, coding aspects of experience directly, rather than being represented by amodal symbols that have been somehow transduced from experience (Barsalou, 1999; Barsalou & Wiemer-Hastings, 2005; Wilson-Mendenhall et al., 2013).

On subsequent occasions, when abstract features of entities and events are situations are represented conceptually, the areas associated with processing abstract features are reactivated, simulating what those situations were like abstractly. Similar to sensory-motor features, neural reuse supports the conceptual processing of abstract features. Many of these features are likely to have strong biological origins (e.g., magnitude, reward, mental state).

A key issue is whether abstract features reside in the association areas that Binder (2016) reviews or elsewhere. Preliminary evidence suggests that they can reside in both. Consider findings from Wilson-Mendenhall et al. (2013). For the abstract concept, arithmetic, they found activations in the intraparietal sulcus that overlapped with activations for a numerical localizer task. Because this brain area does not fall within Binder’s association areas, and because it is often associated with the representation of magnitude, it follows that some abstract features are processed by dedicated feature areas. Again, it seems important to enumerate these kinds of features, to establish where they reside in the brain, and how they contribute to abstractions.

Wilson-Mendenhall et al., however, observed a different finding for the activations associated with the abstract concept, convince. In general, activations for these areas fell largely into Binder’s association areas (with some exceptions). Because these activations also overlapped with activations for a mental states localizer task, it is not clear whether the activations for convince reflect data compression versus distilled abstraction for features of mental states. Thus, another important issue for future research is to establish whether Binder’s association areas (and association areas more generally) perform data compression, represent abstract features, or both.

In general, this approach proposes that computations over distributed linguistic representations can be used to perform cognitive tasks effectively, although many researchers suggest that the effectiveness is less than when using multimodal simulations and other forms of conceptual knowledge (Barsalou et al., 2008; Connell & Lynott, 2013; Glaser, 1992; Louwerse & Connell, 2011; Zwaan, 2016). In other words, distributed linguistic representations offer a heuristic for performing a task quickly, when conditions permit, but may not be sufficient for sophisticated task performance. Again, they can be viewed as abstractions that carry information about more complex conceptual representations.

If this account of concepts is correct, then establishing the distributed networks that implement neural reuse, data compression, distilled abstraction, and distributed linguistic representations presents a number of research challenges. One important challenge is to establish the association areas that play central roles in abstraction. Because a variety of conflicting proposals exist, assessing their relative merits is essential to producing a compelling account of these distributed networks.

Hub-and-spoke theories constitute one important class of theories. According to these theories, the most important association areas for concepts reside in the anterior temporal lobes (ATL) of both the left and right hemispheres (Reilly et al., 2016; also see Lambon Ralph, Sage, Jones, & Mayberry, 2010; Patterson, Nestor, & Rogers, 2007; Rogers & McClelland, 2004). From this perspective, the ATLs contain stand-alone amodal symbols that can perform conceptual processing on their own, but that project to sensory-motor areas when fleshing out amodal symbols is useful (similar to the interaction hypothesis). Although hub-and-spoke theories assume that the abstractions in the ATLs are amodal, the ATLs also could also potentially implement multimodal data compressions and/or distilled abstraction.

Perhaps a more serious problem for hub-and-spoke theories than its assumptions about amodal symbols is that many researchers do not view the ATL as a single homogeneous region for representing all concepts in general, but rather as a highly differentiated region where distinct concepts, such as those for social information and individuals, are represented separately from other concepts (Drane et al., 2008, 2009, 2013; Martin, Simmons, Beauchamp, & Gotts, 2014; Wong & Gallate, 2012). Furthermore, when Fernandino et al. (in press) used conjunction analysis to establish common association areas active across diverse concepts, the ATLs were not present. Binder (2016) suggests that other association areas in the temporal and parietal lobes are most likely to perform “hub” functions (also see Binder, Desai, Graves, & Conant, 2009). Martin (2016) similarly suggests that other temporal and parietal areas operate as hubs.

Once the association areas for conceptual processing are established, it then becomes important to establish whether they support data compression, distilled abstraction, distributed linguistic representation, or some other function (including amodal symbols). Once the operative representations in an associative area are established, another critical issue is determining whether these representations can operate in a stand-alone manner or can only operate together with modality-specific representations. When abstractions and sensory-motor simulations operate together, the nature of their interactions becomes important, along with how these interactions accomplish various conceptual functions (similar to issues associated with the interaction hypothesis).

Following Jamrozik et al. (2016), distilled abstractions can probably operate as stand-alone representations, given that they sometimes become active in metaphor without the support of activations in modality-specific areas (which can accompany them optionally). As Zwaan (2016) points out, however, abstract representations often become much easier to understand, once they have been embellished with modality-specific information (also see Akpinar & Berger, 2015; Schwanenflugel, 1991).

Distributed linguistic representations may also be capable of functioning as stand-alone representations, following many recent empirical demonstrations (Zwaan, 2016; also see Barsalou et al., 2008; Connell & Lynott, 2013; Louwerse & Jeuniaux, 2010). Again, they may only provide heuristic information about a concept, with more definitive representation requiring the addition of multimodal simulations.

Finally, it is not clear that multimodal compressions in association areas can function as stand-alone representations during conceptual processing, or whether they require support from modality-specific areas. One proposal is that compressed representations primarily serve to index, integrate, and control distributed representations across multimodal systems rather than to represent them in a stand-alone manner (Damasio, 1989; Simmons & Barsalou, 2003). Nevertheless, because multimodal compressions carry information about distributed multimodal representations, they could potentially be used in a stand-alone manner, at least for heuristic purposes.

Another possibility—consistent with research on flexibility discussed shortly—is that the networks underlying concepts can be accessed in diverse ways, such that no strict time course characterizes their operation. In this spirit, Zwaan (2016) illustrates how the processing of an abstract concept could differ when used cataphorically versus anaphorically (i.e., when an abstract concept is processed before or after a relevant situation in which it applies). When processed cataphorically, a sparse representation is accessed initially (perhaps a distributed linguistic representation), which is then fleshed out in modality-specific detail with the text that follows. Conversely, when processed anaphorically, relevant modality-specific detail is established before the abstract concept is presented, such that a rich conceptual representation is available immediately. To the extent that such flexibility exists, taking tasks and contexts into account is essential for making predictions about temporal dynamics.

Predictions about temporal dynamics offer an important tool for confirming and disconfirming hypotheses about the distributed networks that underlie concepts. Localization of these networks spatially is obviously important but limited in the information provided. Assessing temporal dynamics not only offers additional means of discriminating theories, it also forces researchers to articulate their theories more clearly. Developing detailed models of the types that Gotts (2016) explores is likely to support making sophisticated predictions about temporal dynamics.

Abstract concepts seem like a particularly hard nut to crack. Our lack of understanding doesn’t reflect a lack of effort but more likely reflects what a hard problem this is. If we want to understand human concepts—and indeed human cognition—it is essential that we establish some traction in understanding abstract concepts. I suspect that all the abstraction mechanisms described earlier play important roles. Because abstract concepts often appear to integrate information across situations, they may utilize multimodal compression. Because they often include abstract features, they may utilize distilled abstraction. Because they appear highly dependent on language (Binder, Westbury, McKiernan, Possing, & Medler, 2005), they may utilize distributed linguistic representations. Consistent with the multifaceted character of abstract concepts, both Dove (2016) and Jamrozik et al. (2016) propose that multiple mechanisms are likely to underlie them.

Moving beyond our modest understanding of abstract concepts may require studying a few of them in detail, rather than studying them in the aggregate, especially given their diversity (Wilson-Mendenhall et al., 2013). Because abstract concepts appear dependent on background situations (Akpinar & Berger, 2015; Schwanenflugel, 1991; Zwaan, 2016), examining their operation in specific situations may provide leverage—and perhaps be necessary—for making significant progress.

Another major theme across contributions to the special issue is that concepts operate flexibly, with their representations and effects depending on tasks and contexts (Binder, 2016; Dove, 2016; Kemmerer, 2015a; Yee & Thompson-Schill, 2016; Zwaan, 2016; also see Mahon, 2015). In their review of context effects on concepts, Yee and Thompson-Schill illustrate that the processing of a concept varies as a function of long-term context, recent context, immediate context, ongoing context, and individual abilities. They further argue that (1) concepts cannot be separated from the contexts in which they occur (consistent with context availability theory; Schwanenflugel, 1991), (2) concepts are not rigidly fixed in content, and (3) concepts do not have conceptual cores. Yee and Thompson-Schill further suggest that recurrent neural networks offer a natural way of explaining how all the different levels of context shape the knowledge underlying a concept and its changing content over a processing episode.

Based on a review of automaticity in the semantic processing literature, Lebois et al. (2015) similarly argue that concepts don’t have conceptual cores that are activated immediately across all contexts. Instead, even the most central features of a concept can vary widely in activation. They further propose that the flexible activation of information in a concept reflects Bayesian sampling: On a given occasion, the probability that a given feature of a concept is active reflects (1) the overall frequency with which the feature has been processed in previous contexts (its prior) and (2) its relevance in the current context (its likelihood). Although frequent features have a high chance of becoming active, context can override them, conferring an advantage on contextually relevant features. As context becomes increasingly salient and specified, contextually relevant features may become increasingly dominant, similar to well-known context effects in the Visual World Paradigm (Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995; for a recent review see, Huettig, Rommers, & Meyer, 2011) Based on a wide variety of such findings, Hauk (2016), Yee and Thompson-Schill (2016), and Lebois et al. (2015) suggest that understanding the dynamic activation of features for a concept in context over time should be a central topic of future research (for a recent example of such work, see van Dam, Brazil, Bekkering, & Rueschemeyer, 2014).

Finally, Gotts (2016) provides elegant and insightful reviews of repetition priming in behavior, together with the corresponding repetition suppression in neural activity, exploring the implications for incremental learning. Although the models that he reviews and develops primarily address the processing of individual stimuli, the underlying mechanisms have clear relevance for the representation of conceptual knowledge, as he notes. In addition, these mechanisms provide a natural way to understand and model the kinds of conceptual flexibility that Kemmerer (2015a), Yee and Thompson-Schill (2016), and Lebois et al. (2015) review. Our understanding of conceptual flexibility will be enhanced significantly to the extent that we can explain the phenomena reviewed in these articles with the kinds of models that Gotts develops. Implementing such models using the network architectures motivated earlier in the sections on grounding and abstraction offers one direction for model development. Because these architectures implement multiple forms of representation across association areas and modalities, a given concept can be flexibly activated in many different ways.

In the special issue, Binder (2016) and Murphy (2016) remind us that theories of concepts must explain classic conceptual phenomena (Barsalou, 2003b, 2012; Margolis & Laurence, 1999; McRae & Jones, 2013; Murphy, 2002). As Binder reviews, concepts play important roles in taxonomies, thematic relations, propositions, and schemata. Murphy also addresses the importance of taxonomies, along with basic level categories, exemplar effects, and category-based induction. In particular, Murphy emphasizes the importance of causal reasoning and background knowledge in conceptual processing. Typicality and all its effects would be another basic phenomenon that I’d add to this list.

I’d further add basic symbolic operations, such as type-token binding, predication, propositions, productivity, and concept composition (Fodor, 1975; Fodor & Pylyshyn, 1988; Pylyshyn, 1984). Although it might seem that symbolic operations are only relevant to amodal theories, and can only be explained by them, I continue to believe that they fundamentally structure human cognition, and that grounded theories have potential to explain them in novel and insightful ways (Barsalou, 1999, 2008b, in press-b). As Binder (2016) suggests, another important function of association areas is to integrate conceptual information (also see Humphries, Binder, Medler, & Liebenthal, 2006; Legrand & Ruby, 2009).

Although these conceptual phenomena may seem dated, the only things dated are our theories of them. As current theories evolve, it’s essential that they continue to offer compelling accounts of foundational phenomena associated with conceptual processing.

Cognitive psychologists typically assume that the kinds of phenomena listed in Goldinger et al. (2016) generalize readily to real-world situations. As a consequence, no attempt is made to assess these mechanisms in richer, more natural situations outside the idealized laboratory environments in which they were established. As Henrich, Heine, and Norenzayan (2010) found, however, such assumptions are not always warranted, given that a given cognitive mechanism can take different forms outside the laboratory.

The Visual World Paradigm offers another good example of how a basic cognitive phenomenon can change significantly when embedded in more realistic situations (Huettig et al., 2011; Tanenhaus et al., 1995). Prior to the visual world paradigm, what we knew about the processing of words and syntax was based on paradigms that largely examined them in isolation. Once researchers began studying them in the context of visual worlds, understanding evolved considerably to reflect substantial contributions from context.

Because concepts are typically studied in relatively idealized laboratory tasks, accounts of them may similarly be limited; we may be missing important phenomena and experiencing a variety of distortions. Although many of the basic conceptual phenomena established so far are likely to remain, it wouldn’t be surprising if new findings and insights followed as well.

As Rumiati and Foroni (2016) illustrate, food concepts offer an example of how concepts can be studied in a rich real-world domain (also see Papies, 2013; Papies & Barsalou, 2015; Ross & Murphy, 1999). Because eating is a multifaceted phenomenon that utilizes many cognitive processes, including attention, categorization, learning, decision making, goal pursuit, and affect, it offers the opportunity to examine how concepts operate in a rich cognitive context that also happens to be of social significance.

Many other real-world domains similarly offer useful contexts for increasing our understanding of how concepts operate in the real world. Barsalou et al. (2007) suggested two general types of situations that are central in human activity: (1) A single individual performs situated action in the physical environment to achieve a goal, as when building an object, navigating to a location, or gathering resources; (2) Multiple individuals perform coordinated social interaction, as when performing knowledge transfer, group decision making, or distributed operation of complex equipment. Studying how concepts support intelligent activity in situations like these might lead to new insights about their content, structure, and function, which wouldn’t emerge from studying them in the laboratory.

I’m not suggesting that basic cognitive research become applied. Instead, I’m suggesting that doing a better job of linking basic research to the real world is likely to have significant benefits for basic research. Bringing cognitive theory and results to bear on real-world problems will not only produce social benefits, it will increase the visibility of cognitive research and perhaps bring it greater appreciation and resources. Furthermore, examining basic cognitive processes in real world situations offers opportunities to assess whether our basic accounts are correct and complete, and if not, where further development is needed. To the extent that an account of a cognitive process is inadequate, assessing its behavior in the real world is likely to inform how the account could be revised, leading to important new basic research opportunities in the laboratory. In these manners, connecting basic research to important real world situations has significant potential to keep basic research from becoming insular, and to promote its productive growth and evolution.