Introduction

Our environment is filled with information, some of it is useful to our goals, and some of it is not. Selective attention is the process through which we scrutinize items in our environment (Pashler 1997). Visual search is an especially effective task in investigating how items are selected for directed attention, a process known as attentional selection. Experimental work on selective attention began in the mid-twentieth century (Broadbent, 1958; Cherry, 1953) as did research on visual search (Carpenter, 1948; Ellis, 1947; Mackworth, 1948, 1952). However, not until later were they discussed together. Some of the earliest influential work integrating visual search and selective attention was performed by Charles W. Eriksen, whom this special issue memorializes.

Pop-out visual search, perhaps the simplest form of visual search and what we focus on in this review, is characterized by identification of a conspicuous target. That is, pop-out refers to search conditions where an object is noticeably different in one feature relative to the objects surrounding it, as for example is a red berry among green leaves. In the early 1950s, Eriksen performed the first study investigating the speed at which items in visual search displays are selected through careful manipulation of the features making up the objects participants searched through (Eriksen, 1952). He found that targets distinguished by single features (i.e., pop-outs) were identified faster than conjunctions of features. In arrays of items comprised of multiple features, a conjunction of unique features (i.e., pop-out in more than one feature) made search faster (Eriksen, 1953) and more accurate (Eriksen & Hake, 1955). Furthermore, in these heterogenous displays, search time increased with the number of distractors (Eriksen, 1955). This early work laid the foundation for further investigation into the factors influencing human visual search performance and the role of attention in searching these arrays.

The observed behavioral differences between single-feature pop-outs and conjunction search, originally reported in the work of Eriksen and colleagues, gave rise to the idea of two interactive processes for visual search: those processes being characterized by being “bottom-up,” where features are processed in parallel, and those characterized by being “top-down,” where features are integrated. Within this framework, pop-out only requires the bottom-up process as features don’t need to be bound for target identification to occur. If single features are processed in parallel across space and pop-out is defined by a difference in a single feature, then surely pop-out search is already as fast as possible. However, further investigation would prove otherwise.

Investigators showed that when participants repeatedly perform the pop-out search with the same stimuli, search times become faster. The task took the form of repetitive pop-out visual search where the search contingencies varied throughout the entire experiment (e.g., multiple features distinguished target and distractors), but remained consistent in blocks of trials (e.g., the features distinguishing target and distractor did not change for several trials) (Maljkovic & Nakayama, 1994, 1996, 2000). Under these conditions, although the visual search task was as efficient as it could be (e.g., single-feature, homogenous distractors), repetitive performance in fact speeded behavioral response times (Maljkovic & Nakayama, 1994, 1996, 2000). This speeding has been replicated across multiple research groups (e.g., Huang, Holocombe, & Pashler, 2004; Kristjánsson 2006; Lamy, Antebi, Aviani, & Carmel, 2008) and species (Bichot & Schall, 1999, McPeek & Keller, 2001) (Fig. 1). This finding suggests that pop-out search, in isolation, is not always as fast as it can be. Changes can be implemented that make pop-out search faster, search history permitting.

Fig. 1
figure 1

Priming of pop-out task and response time profiles. Top: Ten example trials organized in a row showing each of the trial epochs (fixation, array onset, target identification/saccade, and reward/inter-trial interval [ITI]). Priming occurs when search conditions remain consistent (e.g., first five and last five example trials). Bottom: Response-time profiles adapted from the original Maljkovic and Nakayama study (1994) (left) as well as examples from early monkey studies (Bichot & Schall, 2002; McPeek & Keller, 2001) (right). Response time is plotted against trial since the pop-out feature change. As search conditions repeat, response times speed up

Full size image

Subsequently, the mechanism underlying the behavioral improvements associated with repetition of performance in pop-out in visual search has been a matter of some debate in psychology (Kristjánsson & Campana, 2010; Kristjánsson & Asgeirsson, 2019). While it was initially hypothesized that priming of pop-out is mediated by memory mechanisms (Maljkovic & Nakayama, 1994, 1996, 2000) and there is some psychophysical evidence suggesting this (Hillstrom, 2000; Huang, Holocombe, & Pashler, 2004; Huang & Pashler, 2005; but see Asgeirsson & Kristjánsson, 2011), other psychophysical evidence largely supports the idea that it is mediated through attentional priming. That is, speeding of behavioral responses observed in priming of pop-out is accomplished through speeding the process of attentional selection. In interleaving choice trials following priming of pop-out sequences, participants are more likely to freely choose, and are faster to attend to stimuli that were the targets of the preceding priming block (Brascamp, Blake, & Kristjánsson, 2011). Furthermore, evidence for priming of pop-out as priming of attentional selection can be found beyond the psychophysics literature.

Electrophysiological (Eimer, Kiss, & Cheung, 2010) and, as detailed below, neurophysiological (Bichot & Schall, 2002; Westerberg, Maier, & Schall, 2020a) evidence also exists to support the priming of attentional selection hypothesis. In each of these studies, signals in or originating from brain areas associated with attentional selection during visual search, rather than areas associated with memory reactivation, were the ones to change. While other cognitive phenomena may modulate as a function of priming in this task, such as performance monitoring, which is explored later in this review (Westerberg, Maier, Woodman, & Schall, 2020b), they are likely secondary to the changes in attentional selection (Kristjánsson & Asgeirsson, 2019).

While a substantial contribution to the psychophysical understanding of priming of pop-out has been made over the years, a coherent picture of the neural mechanism supporting these changes is incomplete. That is not to say that the pieces do not exist. Several studies have identified brain areas contributing to, or at least affected by, priming in visual search, but these results have not been synthesized. Hence, the purpose of this review was to introduce and discuss the brain areas involved in priming of pop-out to elucidate a more integrated conception of the neural mechanism and formulate testable hypotheses for future investigation. More generally, in understanding the neural mechanism of priming of pop-out, we gain a greater understanding of how brain areas coordinate to select behaviorally relevant stimuli in the visual environment. To achieve these objectives, we begin by introducing how attentional selection is measured in the brain, specifically in brain areas where priming has been investigated. Next, we survey those neurophysiological studies of priming in visual search and bridge them to noninvasive studies in humans. We conclude the review by generating and providing support for three rival hypotheses for the neural mechanism of priming during visual search.

Attentional selection during visual search at the neural level

Behavioral work of early pioneers in attention research, like Eriksen, has been ongoing for decades. This work has attracted neuroscientists to investigate the neural underpinnings of the behavioral phenomenon of these seminal behavioral studies. Since the genesis of neurophysiological investigation into attention began in the late twentieth century, study into visual selective attention at the neural level has blossomed and produced more work on the neural basis of selective attention than can be reasonably reviewed here (e.g., Carrasco, 2011; Moore & Zirnsak, 2017). Instead, we focus on introducing attentional selection during visual search in the brain areas implicated in priming of pop-out, which is discussed later. Frontal and visual cortex are perhaps two of the most well-studied regions with respect to visual search. Specifically, the frontal eye field (FEF) in the frontal cortex and area V4 in the visual cortex.

FEF is a distinct frontal cortical area found in all primates (Schall, Zinke, Cosman, Schall, Pare, & Pouget, 2017). It contributes to the representation of visual information and the coordination of eye movements including saccades and smooth pursuit eye movements (Schall, 2015). Early work also identified several populations of functional cell types in FEF involved in the representation of visual information, fixation, and motor signaling. More recent reports suggest FEF functional populations might lie on a spectrum of more visual to more motor responses (Lowe & Schall, 2018) and there might even be more complex functional types such as those that track previously viewed stimuli (Mirpour, Bolandnazar, & Bisley, 2019). The responses of visual cells in FEF modulate during visual search (Schall & Hanes, 1993), which suggests implications for attentional selection. Specifically, FEF neurons discriminate between targets and distractors in search arrays. Subsequent work has investigated whether this modulation is related to the motor preparation in FEF or if it is the process of attentional selection. Even though investigation in FEF has been ongoing for over a century, it was not considered an “attention area” until the mid-1990s. Earlier work suggested that modulation of responses to stimuli in FEF only occurred when gaze was to be directed to the stimulus (Bruce & Goldberg, 1985; Goldberg & Bushnell, 1981). Later work showed that FEF activity during the selection epoch indicated an attentional selection process, as the motor process could be distinguished from the selection process at the single-unit level (Juan, Shorter-Jacobi, & Schall, 2004; Kodaka, Mikami, & Kubota, 1997; Monosov, Trageser, & Thompson, 2008; Thompson, Bichot, & Sato, 2005). For example, in an antisaccade task where the saccadic endpoint can be dissociated from the location of the attentional target, selectivity in FEF neurons representing the space where the target is can still be observed when the saccade must be made elsewhere (Juan, Shorter-Jacobi, & Schall, 2004).

Investigation into selectivity of FEF neurons in visual search tasks has uncovered additional insights important in understanding the neural underpinnings of behavioral changes associated with attentional selection. While FEF neurons predominantly do not show selectivity for features of visual stimuli, under certain circumstances, visual cells in FEF can develop feature selectivity. By repeatedly performing a visual search task where the target feature of the search array remains consistent across days, FEF neurons can develop selectivity for the behaviorally relevant feature (Bichot, Schall, & Thompson, 1996). This has been observed for colors (Bichot, Schall, & Thompson, 1996) as well as for shapes (Lowe & Schall, 2019). This experience-driven change in the function of FEF neurons demonstrates their plasticity in the representation of features. Furthermore, FEF neurons appear to be sensitive to salient colors present in a search array, even when they are not the target, perhaps through the initial feedforward sweep of visual information from V4 (Cosman, Lowe, Zinke, Woodman, & Schall, 2018). As priming of pop-out seems to be driven through the potentiation of features, this might be important to this process, albeit these reports show this effect at a much longer timescale than in priming of pop-out.

Area V4 is a mid-level visual cortical area in the ventral visual-processing stream that was originally thought to be the “color area” (Zeki, 1973, 1978). It has since been shown to show selectivity for other features such as shapes and contours, among others (Roe, Chelazzi, Connor, Conway, Fujita, Gallant, Lu, & Vanduffel, 2012). It also shows robust attentional modulation (Moran & Desimone, 1985), to a greater degree than earlier visual cortical areas (Buffalo, Fries, Landman, Liang, & Desimone, 2010; Ghose & Maunsell, 2002; Luck, Chelazzi, Hillyard, & Desimone, 1997). Removal of area V4 through lesions impacts performance in attention tasks (De Weerd, Desimone, & Ungerleider, 1996, 1999, 2003; Schiller & Lee, 1991). Given V4’s selectivity for shapes, colors, contours, and other features as well as its robust attentional modulation, it has become a highly relevant area in the study of neural processing during visual search. The earliest work investigated the modulation of V4 neurons during tasks where spatial selective attention was directed towards or away from an item in a search array co-localized with the neuronal response field (Motter, 1993, 1994). In arrays containing competing stimuli, the spiking responses of V4 neurons modulated as a function of attentional state. Specifically, attention directed to the response field of the neurons enhanced their responses. This modulation of activity was also found during memory-guided visual search where the timing of the target cue and the array onset was interrupted by a delay epoch (Chelazzi, Miller, Duncan, & Desimone, 2001). Further work has since delved into the nuances of this modulation.

Ogawa and Komatsu (2004) had monkeys perform a multidimensional feature-search task to investigate the role of V4 feature selectivity when the feature may or may not be relevant to the search. In this task, monkeys were shown search arrays with feature conjunctions (e.g., shape × color) and identified the oddball along one of those feature dimensions (e.g., red circle or square among green circles and squares). They found that when V4 responses were aligned to the presentation of the array, individual neurons did not reliably indicate the presence of a target in their response field. However, a shift in the population response was observed when a target was present. This suggests that while individual V4 neurons can’t discriminate behavioral targets in the initial visual response, at the population level, this can be evaluated. A similar study investigating the feature selectivity during visual search in V4 replicated this result (Mirabella, Bertini, Samengo, Kilavik, Frilli, Della Libera, & Chelazzi, 2007). However, they found that individual neurons in V4 could distinguish behaviorally relevant stimuli. This discrepancy might be explained by a difference in the time epochs where selection was measured (Ogawa & Komatsu, 2006). Measuring the selectivity nearer to the time of saccade showed selection in individual V4 neurons in the multidimensional search task. Nonetheless, it was known from work prior to these studies that V4 modulates with feature-based attention (McClurkin & Optican, 1996) and both of these studies indicate feature-based attentional selectivity in V4 during visual search. However, evidence exists that this feature-based attentional selectivity is limited to goal-directed behaviors (Hayden & Gallant, 2005) such as visual search (Ipata, Gee, & Goldberg, 2012).

From the studies detailed above we know that FEF and V4, independently, are important for attentional selection enabling visual search. Notably, they show similar modulation of visual responses to target stimuli in visual search whereby following the initial visual response and their activity distinguishes between targets and distractors in their visual response fields (Ipata, Gee, & Goldberg, 2012; Schall & Hanes 1993). Coupled with previous work demonstrating these areas are interconnected (Ninomiya, Sawamura, Inoue, & Takada, 2012; Schall, Morel, King, & Bullier, 1995; Stanton, Bruce, & Goldberg, 1995; Ungerleider, Galkin, Desimone, & Gattass, 2008), investigation has been conducted into the relationship between these areas during attention tasks. Simultaneous recordings in area V4 and FEF during an attentional task has shown that they are indeed functionally coupled, and that coupling changes with attentional state (Gregoriou, Gotts, Zhou, & Desimone, 2009; Gregoriou, Gotts, & Desimone, 2012; Zhou & Desimone, 2011). While we do not know the impact of V4 lesioning on FEF responses during an attention task, lesioning FEF limits attention-driven modulation in V4 (Gregoriou, Rossi, Ungerleider, & Desimone, 2014). Together, these studies imply a role for the FEF-V4 circuit in attention modulation and attentional selection during visual search.

To summarize, FEF and V4 have been implicated in the process of selecting behaviorally relevant stimuli during visual search. Anatomically, they have been shown to be interconnected, and functionally, the activity of each of these areas affects the other. Additionally, their processing during search is not identical, indicating there are neural computations being performed between these two areas that might be important in the selection process. This circuit gives us a target to investigate how attentional selection might change through priming and, as is detailed next, we know that these areas are implicit in priming of pop-out.

Brain areas contributing to priming of pop-out: Frontal and visual cortex

Investigators have begun to causally study neural populations and record activity during priming of pop-out tasks. Priming of pop-out is hypothesized to speed neural processes underpinning attentional selection of targets during visual search. Researchers have investigated this hypothesis by probing the areas of the brain highlighted above that are involved in visual selective attention.

Visual cortical areas were the first to be implicated in priming of pop-out. Walsh and colleagues investigated the roles of areas V4 and TEO in priming through a lesion study (Walsh, Le Mare, Blaimire, & Cowey, 2000). To do so, they trained control monkeys and monkeys with bilateral V4 or TEO lesions to a priming of pop-out visual search task. Ultimately, they found that although baseline search performance was not impacted significantly, the effects of priming were diminished in the lesion monkeys. However, the priming effect was diminished rather than eliminated, suggesting there is not a single source for priming. No monkeys included had bilateral V4 and TEO lesions. Alternatively, priming effects could have been recovered partially following the lesion as neural plasticity can lead to recovery from deficits following non-reversible lesioning (Finger & Stein, 1982; Newsome & Pare, 1988; Raisman, 1969; Yamasaki & Wurtz, 1991), making it possible that V4 and TEO are the source of priming in the normal brain. Regardless, this finding suggests a role for visual cortex in priming of pop-out. Further work has since delved into that role.

We have begun to investigate the underlying change in neural processing that is occurring in area V4 related to the diminished priming of the lesion study (Westerberg, Maier, & Schall, 2020a). Monkeys trained to a pop-out task showed speeded attentional selection in area V4 with priming (Fig. 2). This was mediated by greater enhancement in the neural responses to targets and suppression of distractors following priming as compared to the unprimed condition. This suggests the mechanism is not solely mediated by changes in target enhancement or distractor suppression, but rather a combination. Area V4 is not the only brain region investigated with respect to the attentional selection changes associated with priming of pop-out. Area FEF has also been shown to reflect changes in a very similar yet subtly different manner. Bichot and Schall (2002) trained monkeys to perform a priming of pop-out search task and recorded the activity of neurons in FEF. Much like the results of the V4 study, they found that visuomovement neurons in FEF showed speeded target selection with priming of pop-out mediated by target enhancement and distractor suppression (Fig. 2). However, the relationship between the activity in FEF and the observed behavior was distinctly different than that same comparison between V4 and behavior. This provides some insight into the mechanism of priming of pop-out.

Fig. 2
figure 2

Changes in neural responses associated with priming of pop-out. Neural responses from areas FEF (left) and V4 (right) when the target (black) of a search array is in the response field and when a distractor (gray) is in the response field (RF) when the search is unprimed (top) relative to primed (bottom). Data are adapted from Bichot and Schall (2002) and Westerberg, Maier, and Schall (2020a). Neurons discriminate between targets and distractors at some point in time (target selection time (TST), red arrows). The TST occurs earlier in primed trials as compared to unprimed. Additionally, the distinguishability between targets and distractors is enhanced as indicated above by a greater difference between the target and distractor responses with priming (blue fills)

Full size image

In comparing the change in response time associated with priming of pop-out to the change in neural target selection times, we drew inferences about the roles of FEF and V4 in priming of pop-out. In area V4, the neural target selection times speeded more than the associated behavior. This contrasts the one-to-one relationship between FEF and behavior. If anything, this highlights the role of FEF in motor output as behavior more closely follows changes in this area than changes in visual cortex (Bruce & Goldberg, 1985; Bruce, Goldberg, Bushnell, & Stanton, 1985; Lowe & Schall, 2018).

In addition to the work done in FEF, another frontal area has been investigated with respect to its role in priming of pop-out. Investigation has been done into the role of supplementary eye field (SEF), a medial frontal cortical area involved in sensory and internally guided eye movements (Schall, 1991; Schlag & Schlag-Rey, 1987). Purcell et al. (2012b) investigated whether area SEF contributes to saccade target selection during visual search and associated changes with priming of pop-out. They found no target selectivity in the visual response across their sample of SEF neurons and no change in target selectivity with priming of pop-out. This indicates no role for SEF in the selection of targets or the speeding of attentional selection during visual search. However, SEF not only has visual responses, but is implicated in other cognitive processes that can be examined with respect to priming of pop-out.

One additional study has been performed in SEF investigating a potential neural process to instantiate the change in neural selection seen in FEF. SEF has been implicated in the process of performance monitoring (Sajad, Godlove, & Schall, 2019; Stuphorn, Taylor, & Schall, 2000). Performance monitoring is the process by which outcomes during behavioral tasks are overseen by a “supervisory attentional system” to alter behavior on subsequent trials (Norman & Shallice, 1986). Neurons in SEF signal errors during visual search (Purcell, Weigand, & Schall, 2012b). We hypothesized that this supervisory control could initiate the changes observed in attentional selection (Westerberg, Maier, Woodman, & Schall, 2020b). SEF is interconnected with area FEF (Schall, Morel, & Kaas, 1993), and thus could enact changes in activity that could be related to the changes in attentional selection associated with priming of pop-out (Bichot & Schall, 2002). To investigate the potential role of performance monitoring in SEF during priming of pop-out, we measured the activity of SEF neurons that showed differential activity when a correct response versus an incorrect response was made relative to the position in the priming sequence. We found that the neural activity of this performance-monitoring population does not modulate with priming of pop-out.

To summarize, physiological and lesion studies of priming of pop-out have been ongoing for the last couple of decades. These studies have identified three areas, and rejected one, in contributing to priming of pop-out (Fig. 3). Areas FEF (Bichot & Schall, 2002), V4, and TEO (Walsh, Le Mare, Blaimire, & Cowey, 2000; Westerberg, Maier, & Schall, 2020a) all contribute to priming of pop-out. The physiology in V4 (Westerberg, Maier, & Schall, 2020a) and FEF (Bichot & Schall, 2002) indicates changes in the timing of attentional selection with priming of pop-out. Area SEF did not show any changes with priming of pop-out, neither in its visual processing (Purcell, Weigand, & Schall, 2012b) nor in its performance monitoring (Westerberg, Maier, Woodman, & Schall, 2020b).

Fig. 3
figure 3

Brain areas involved in priming of pop-out (PoP). Neurophysiological recordings in the frontal eye field (FEF) (Bichot & Schall, 2002) in the frontal cortex and area V4 (Westerberg, Maier, & Schall, 2020a) in the visual cortex have shown modulation of neural activity in these areas with priming of pop-out. No modulation was found in the supplementary eye fields (SEFs) during priming of pop-out (Purcell, Weigand, & Schall, 2012b; Westerberg, Maier, Woodman, & Schall, 2020b). Lesions of areas TEO and V4 reduce the magnitude of the priming effects with repetitive pop-out search (Walsh, Le Mare, Blaimire, & Cowey, 2000), indicating a causal role for these areas

Full size image

Noninvasive indicators for neural changes with priming of pop-out

The first evidence for changes in neural activation found in humans associated with priming of pop-out was through neuroimaging (Kristjánsson, Vuilleumier, Schwartz, Macaluso, & Driver, 2007). The blood oxygen level-dependent (BOLD) signal was measured while participants performed a color-based priming of pop-out visual search task. Changes associated with priming were found in both frontal and occipito-parietal regions of the brain, mirroring the findings described in the neurophysiology literature. Specifically, investigators observed repetition suppression of the BOLD response with priming. Notably, FEF showed changes. Changes with priming of pop-out were also observed in the area surrounding the fusiform gyrus (FG), nearby to the location where the human homologue of macaque area V4 (human V4, hV4) resides (Lueck, Zeki, Friston, Deiber, Cope, & Cunningham, 1989; McKeefry & Zeki, 1997). Additionally, changes were observed in the intraparietal sulcus (IPS), an area where neurophysiology has not been performed.

What is repetition suppression and how does it tell us anything about the neural mechanism of priming of pop-out? Repetition suppression is the diminishing response that is often associated with repetition of visual stimuli (Grill-Spector & Malach, 2001). Not limited to BOLD responses, it can also be observed in electroencephalogram (EEG; Sambeth, Maes, Quian Quiroga, & Coenen, 2004) and electrocorticography (ECOG) (Puce, Allison, & McCarthy, 1999), as well as responses at the single neuron (Baylis & Rolls, 1987) and neural population levels (Brunet, Bosman, Vinck, Roberts, Oostenveld, Desimone, De Weerd, & Fries, 2014). Originally described in the inferotemporal cortex when complex (multidimensional) stimuli were presented repeatedly (Baylis & Rolls, 1987; see also Gross, Schiller, Wells, & Gerstein, 1967), repetition suppression has since been described across many visual cortical areas. For example, V4 shows repetition suppression of gamma responses (Brunet, Bosman, Vinck, Roberts, Oostenveld, Desimone, De Weerd, & Fries, 2014). Repetition suppression can be observed also at the earliest stage of visual cortical processing in V1 and is hypothesized to come about due to changes in feedback activation (Westerberg, Cox, Dougherty, & Maier, 2019). Furthermore, repetition suppression is hypothesized to come about through more efficient encoding of visual stimuli, perhaps through predictive coding (Aukztulewicz & Friston, 2016; Summerfield, Tritttschuh, Monti, Musalem, & Egner, 2008). However, recent work suggests changes in encoding efficiency and the repetition suppression reflect different processes (Tang, Smout, Arabzadeh, & Mattingley, 2018). Regardless, repetition suppression is associated with changes in the processing of visual stimuli, seemingly in a top-down fashion and in order to more efficiently process stimuli. In relating these properties to repetition suppression in priming of pop-out, this would suggest that priming results in more efficient processing of visual stimuli perhaps through top-down modulation of visual processing.

Following the work in humans (Kristjánsson, Vuilleumier, Schwartz, Macaluso, & Driver, 2007) and macaques (Bichot & Schall, 2002) identifying FEF as a relevant area in priming of pop-out, investigators sought to understand whether TMS of FEF in humans altered behavioral performance in the task (O’Shea, Muggleton, Cowey, & Walsh, 2007). While stimulation of FEF during the intertrial interval had no effect on behavioral performance (Taylor, Muggleton, Kalla, Walsh, & Eimer, 2011), stimulation during the period of array presentation had mixed effects. Stimulation during feature (color)-priming blocks had no effect on behavioral performance. However, stimulation during spatial-priming blocks did affect behavioral performance, but only stimulation of left FEF. These results suggest that FEF is not the source of priming for features but is important in the priming of spatial location. Coupled with the lesion studies of attention in macaques (Gregoriou, Rossi, Ungerleider, & Desimone, 2014; Rossi, Bichot, Desimone, & Ungerleider, 2007), including priming of pop-out (Walsh, Le Mare, Blaimire, & Cowey, 2000), these findings might suggest a source for feature priming in the visual cortex and a source for spatial priming in the frontal cortex.

The EEG in humans has also been a powerful noninvasive tool in investigating priming of pop-out. Visual selective attention has a robust electrophysiological index known as the N2pc (Eimer, 1996; Luck & Hillyard, 1990, 1994; Woodman & Luck, 1999). The N2pc is an event-related potential (ERP) measured in the EEG that indexes whether selective spatial attention has been allocated contra- or ipsilaterally. Part of the history of the N2pc involved investigating how it changes with temporal manipulation of attentional selection. Cuing the location of an attentional target elicits an earlier N2pc than when the target position remains non-cued (Foster, Bsales, & Awh, 2020). Priming of pop-out also affects the timing of the N2pc. Eimer and colleagues found the N2pc speeds with priming of pop-out (Eimer, Kiss, & Cheung, 2010). This was the first noninvasive electrophysiological evidence for a temporal change in attentional selection associated with priming of pop-out. More recent work has investigated the timing of attentional selection in visual search by decoding the raw EEG (Ort, Fahrenfort, ten Cate, Eimer, & Olivers, 2019). In a visual search task where the target remains the same trial-to-trial versus trials where the target switches, they found that evidence accumulation for the target was equivalent, but the onset of that accumulation was delayed for switches. This finding complements that of the earlier N2pc findings by showing changes in temporal processing of visual search with priming through noninvasive, high temporal resolution EEG. However, how these changes map to those observed through invasive electrophysiology remains an open question.

Other non-attention-related EEG ERPs have since been shown to change with priming of pop-out. Investigators have found ERPs in the EEG associated with performance monitoring (error-related negativity [ERN] – Gehring, Goss, Coles, Meyer, & Donchin, 1993; error positivity [Pe] – Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991) did modulate with priming of pop-out (Westerberg, Maier, Woodman, & Schall, 2020b). While this may suggest changes associated with priming of pop-out manifest as a result of performance monitoring, alternatively, the observed changes may be related to a surprise signal. That is, a change in the target feature during priming of pop-out could generate a greater surprise signal in the EEG. Recall, neurophysiological recordings in area SEF, a known contributor to the ERN (Sajad, Godlove, & Schall, 2019), did not show any changes at the neural level (Westerberg, Maier, Woodman, & Schall, 2020b).

Reconciling the fMRI repetition suppression with the neurophysiology

Functional magnetic resonance imaging (fMRI) findings indicate that the BOLD response in brain areas implicated in priming of pop-out (FEF, FG, IPS) show repetition suppression (Kristjánsson, Vuilleumier, Schwartz, Macaluso, & Driver, 2007). However, the neurophysiology results indicate a mixture of enhanced (when the target is in the response field) and suppressed (when a distractor is in the response field) responses (Bichot & Schall, 2002; Westerberg, Maier, & Schall, 2020a). Why might there be this dissociation between the measures?

One hypothesis suggests that the changes observed during priming of pop-out in the neurophysiology, namely the enhanced processing of targets and suppression of distractors, lead to overall enhanced processing efficiency at the mesoscopic scale. That is, while the processing of targets is indeed enhanced, seemingly opposite to the fMRI finding, that only applies to the retinotopic region of the cortex representing the target stimulus. It should follow that the remainder of the same retinotopic map (i.e., the remainder of the V4 or FEF map of visual space) has overall less neural activation. Therefore, on the whole, the activation across one retinotopic map will be less for the efficient primed case as opposed to the relatively inefficient unprimed case. At this scale (swaths of cortex rather than individual neurons) the neural activation would be less with priming. This change in population dynamics would be congruent with the fMRI finding.

An alternative hypothesis explains the dissociation through distinction between the signal generators. It may be that the BOLD measure is not a good indicator of activity at the neural level. BOLD is a measure of change in blood flow rather than the electrical activity of neural populations. Additionally, the time course of the BOLD signal is an order of magnitude slower than neurophysiological responses. Therefore, BOLD may not be an accurate indicator of activity at the single-unit level. Indeed, this may be the case as fMRI seems to be more tightly correlated to fluctuation in the local field potentials (LFPs) (Logothetis & Wandell, 2004). This would suggest that fMRI and single-unit results need not parallel each other. However, it would suggest that changes in the LFP might manifest as diminished responses with priming. It is important to note that this may not be the case as, again, LFPs do not cause the BOLD, they merely correlate better than other neural measures. All in all, there are several alternative hypotheses as to why we see the dissociation between the neurophysiology and the functional imaging. Therefore, these results need not be seen as opposing, just as different indices that provide distinct insight into the neural mechanism.

Relating N2pc and neurophysiology

The speeding of attentional selection in V4 (Westerberg, Maier, & Schall, 2020a) parallels the finding of the speeding of the N2pc (Eimer, Kiss, & Cheung, 2010). Since its discovery, the N2pc has been described as having greatest magnitude over the posterior brain (Luck & Hillyard, 1994). It was therefore hypothesized that its neural source lies in occipital or parietal cortex. Evidence from magnetoencephalography (MEG) supports this hypothesis (Hopf et al., 2000). Given that the N2pc is largest over the occipital and parietal cortex, V4 lies in the occipital cortex, and both V4 and the N2pc show evidence of speeding of attentional selection with priming of pop-out, the logical hypothesis is that V4 contributes to the generation of the N2pc. This hypothesis can be tested in the macaque monkey.

Woodman et al. (2007) demonstrated that the monkeys performing visual search show a homologue of the human N2pc (see Woodman, 2012, for review). The N2pc has been found across several search tasks and across monkeys (Cohen, Heitz, Schall, & Woodman, 2009; Cosman, Lowe, Woodman, & Schall, 2018; Heitz, Cohen, Woodman, & Schall, 2010; Purcell, Schall, & Woodman, 2013). The N2pc homologue allows for the investigation of the neural origins for the N2pc through concurrent EEG and invasive neurophysiological recordings that would otherwise be off limits in human participants. Examining the relationship between V4 and the N2pc during a priming of pop-out task would provide further evidence for or against V4 being a neural contributor to the N2pc. If V4 contributes to the N2pc, the changes in attentional selection observed in V4 during priming should be consistent with the changes in the N2pc.

How does FEF contribute to the generation of the N2pc? FEF activity changes with priming of pop-out (Bichot & Schall, 2002). Additionally, the time course of attentional selection measured in FEF and the N2pc correlates well and perhaps suggests a causal relationship (Cohen, Heitz, Schall, & Woodman, 2009; Purcell, Schall, & Woodman, 2013). Are the changes in FEF secondary to those in V4? Or are FEF projections to V4 driving this change? The latter would suggest that although V4 is generating the N2pc signal, FEF may actually be the signal source. Further investigation through concurrent EEG and neurophysiological recordings of FEF and V4 is necessary to answer these questions. This too may provide insight into the source of the priming changes and how those changes flow through attention networks to produce the speeded neural and behavioral selection measures.

Neural mechanism for priming of pop-out: Three hypotheses

Physiological, imaging, and lesion studies have identified areas in the brain that contribute to or are affected by priming of pop-out. However, those identifications are not enough to generate a complete hypothesis regarding the neural mechanism. It is also important to know the primary source of change and what is secondary. For example, changes may come about in visual cortex that are inherited in a feedforward, or bottom-up, manner by the frontal cortex, or the changes arise in the frontal cortex and are fed back to the visual cortex in a top-down manner (Kinchla & Wolfe, 1979). Here we propose three hypotheses to explain the temporal relationships measured between neural selection and response times across studies of priming of pop-out (Fig. 4).

Fig. 4
figure 4

Predictions regarding the relationship between neural and behavioral selection times for each mechanistic hypothesis. (A) Relationship between the behavioral response time and neural target selection time found for FEF (top, Bichot & Schall, 2002) and V4 (bottom, Westerberg, Maier, & Schall, 2020a). (B) Bottleneck hypothesis: V4 identifies the target, which is fed forward to FEF. Visual cortex is the source of priming. FEF accumulates the information from V4 but is not as efficient as the V4 priming. (C) Feedback hypothesis: FEF identifies the target location and feeds back that information to V4. Frontal cortex is the source of priming. FEF primes and the magnitude of the priming effect is amplified when fed back to V4. (D) Selection tradeoff hypothesis: FEF identifies the target position in unprimed trials and V4 on primed. The source of priming is not localized to one area and arises in the interaction between areas

Full size image

Bottom-up mechanism

The automatic nature of priming of pop-out suggests a bottom-up mechanism (Maljkovic & Nakayama, 1994), consistent with the automaticity of feedforward visual processing (Kinchla & Wolfe, 1979). Psychophysical models assert lower-order feature extraction is integral to pop-out visual search (Cave, 1999; Cave & Wolfe, 1990; Treisman & Gelade, 1980; Wolfe, Cave, & Franzel, 1989). Feature contrasts across space can generate salience maps (Itti & Koch 2000, 2001; Koch & Ullman 1985) that are biologically feasible (Boehler, Tsotsos, Schoenfeld, Heinze, & Hopf, 2009, 2011; Bruce, Wloka, Frosst, Rahman, & Tsotsos, 2015; Hopf, Boehler, Luck, Tsotsos, Heinze, & Schoenfeld, 2006; Hopf, Boehler, Schoenfeld, Heinze, & Tsotsos, 2010; Mazer & Gallant, 2003; Thompson & Bichot, 2005; Tsotsos, Culhane, Wai, Davis, & Nuflo, 1995). Priming of bottom-up feature extraction or salience-map generation would manifest as V4 selecting the search target and also the source of priming. These would be fed forward to FEF. This seems plausible as the axons of many V4 neurons terminate in FEF (Schall, Morel, King, & Bullier, 1995; Stanton, Bruce, & Goldberg, 1995; Ungerleider, Galkin, Desimone, & Gattass, 2008) and physiology during attentional processing supports this (Gregoriou, Gotts, & Desimone, 2012; Gregoriou, Gotts, Zhou, & Desimone, 2009), importantly during visual search (Zhou & Desimone, 2011). V4 has also been shown to be sensitive to pop-out stimuli in the absence of controlled attention (Burrows & Moore, 2009). It is worth noting that sensitivity disappeared when selective attention was directed elsewhere, suggesting the salience of the pop-out item alone is not sufficient. Even so, V4 seems sensitive to pop-out stimuli, diminishes priming when lesioned, and is affected by priming as measured through the changes in neural activation.

However, the change in neural selection time as a function of priming in V4 is about twice the magnitude of the change in response time (RT). As noted above, FEF is closer to the behavioral output through connections to the brainstem saccade generator. Thus, the incongruencies in timing measures can arise through FEF acting as a bottleneck to the overall priming effect through its accumulation of information from various visual inputs to generate responses (Purcell, Heitz, Cohen, Schall, Logan, & Palmeri, 2010; Purcell, Schall, Logan, & Palmeri, 2012a). Eriksen and colleagues also presented evidence investigating the relationship of evidence accumulation and response preparation (Coles, Gratton, Bashore, Eriksen, & Donchin, 1985; Gratton, Coles, Sirevaag, Eriksen, & Donchin, 1988). Priming of pop-out has not been investigated from the perspective of evidence-accumulation and perceptual decision-making; nonetheless, it seems a fruitful avenue to evaluate the validity of the “bottom-up” hypothesis. Still, it is not the only framework in which to consider this hypothesis. The bottleneck could also be a result of the stimulus-response mapping necessary for accurate behavior. That is, while the speeding of selection might be greatest in earlier sensory areas, the translation of the visual information into the motor selection signal can be a distinct process that speeds at a different rate. Human psychophysical evidence shows that response mapping contributes significantly to priming of pop-out (Meeter & Olivers, 2006; Olivers & Meeter, 2006). Stimulus-response mapping and continuous evidence accumulation need not be considered mutually exclusive as models of perceptual decision-making are compatible with both (Purcell, Heitz, Cohen, Schall, Logan, & Palmeri, 2010; Purcell, Schall, Logan, & Palmeri, 2012a), and neurophysiology supports such compatibility (Bichot, Rao, & Schall, 2001). Regardless of the relationship between stimulus-response mapping and evidence accumulation, evidence supports a bottom-up mechanism, albeit further neurophysiological research should investigate the biologically feasibility of such bottlenecks.

When considering a bottom-up mechanism, it is also interesting to speculate about functional localization of feature processing in the visual cortex. That is, different brain areas in the visual cortex tend to represent different visual features, such as color, shape, motion, or orientation, among others. From the human psychophysical work, we know that pop-out occurs with features other than shape and color, and judging the presence of multiple features occurs more quickly than judging the presence of a single feature (Fournier, Eriksen, & Bowd, 1998). While an area like V4 is ideal for the investigation of priming with respect to colors or shapes (Roe, Chelazzi, Connor, Conway, Fujita, Gallant, Lu, & Vanduffel, 2012), it might not be ideal for motion, for example. Perhaps, priming of spatial frequency could occur in V1 or V2 (Foster, Gaska, Nagler, & Pollen, 1985), priming of color or shape in V4 or TEO (Conway, Moeller, & Tsao, 2007; Conway & Tsao, 2009; Tootell, Nelissen, Vanduffel, & Orban, 2004; Zeki 1973), and priming of motion in MT (Mikami, Newsome & Wurtz, 1986a, 1986b; Newsome & Pare, 1988). This might suggest that the process of priming, if instantiated early in visual processing through a bottom-up mechanism, might be distributed across areas. It would potentially explain the compounded priming effect found in the human psychophysics where changes in RT are greater when multiple features are primed (Kristjánsson, 2006, 2009). To investigate this possibility, one could employ the same techniques of prior studies of priming of pop-out on data collected from V4 using a feature it does not strongly represent, like motion. Complementarily, recording from area MT during a motion-priming search task (Kristjánsson, 2009) would also shed light on this question. If V4 does not show changes associated with motion priming of pop-out while MT does, this might suggest a common bottom-up cortical mechanism for changes in attentional selection with priming. However, it could suggest that the feedback from the frontal cortex is highly specific. We know that feedback pathways from FEF to V4 and MT are distinct (Ninomiya, Sawamura, Inoue, & Takada, 2012). Should both V4 and MT show changes associated with priming, it would be indicative of a diffuse top-down mechanism, which would be evidence for our next hypothesis.

Top-down mechanism

An alternative hypothesis predicts FEF neural selection of the target always precedes V4 neural selection. This comes about if FEF selects the target and delivers that information to V4. Therefore, a change in selection time in FEF, elicited through priming or any other similar manipulation, would lead to a change in the timing of selection in V4. FEF is considered an attentional control structure, thereby coordinating attentional selection across brain areas. Anatomical work has demonstrated dedicated projection pathways from FEF to V4, as projections from FEF to area MT were distinct from those to V4 (Ninomiya, Sawamura, Inoue, & Takada, 2012) and different neurons projecting to V4 and to superior colliculus (Pouget et al., 2009). Additionally, previous work has demonstrated that electrical stimulation of FEF can exert an attention-like modulation on V4 (Armstrong, Fitzgerald, & Moore, 2006; Armstrong & Moore, 2007; Moore & Armstrong, 2003), and physiological work reaffirms this (Gregoriou, Gotts, & Desimone, 2012; Gregoriou, Gotts, Zhou, & Desimone, 2009; Zhou & Desimone, 2011). Furthermore, lesions to the prefrontal cortex diminish attentional modulation in V4 (Gregoriou, Rossi, Ungerleider, & Desimone, 2014; Rossi, Bichot, Desimone, & Ungerleider, 2007). Together, these findings demonstrate the capacity for a top-down mechanism that lends support for a feedback hypothesis for priming of pop-out.

Further evidence for a top-down mechanism can perhaps be drawn from the processing of distractors in the frontal cortex. Previous work has identified the frontal cortex, specifically FEF, as a source for the processing of distractor stimuli in the brain (Cosman, Lowe, Woodman, & Schall, 2018). We know that a primed target feature becoming the distractor on subsequent trials will lead to slowed responses on those trials (Kristjánsson & Driver, 2008). This might result from the now distractor feature being facilitated (e.g., negative priming). That is, the distractor feature that has previously been ignored has become the target, leading to slowed responses to that target (Dalrymple-Alford & Budayr, 1966; Greenwald, 1972; see Milliken, Joordens, Merikle, & Seiffert, 1998, for review). With FEF as the source for distractor processing, it is likely that changes in that processing also come about in FEF, which could be taken as evidence for FEF as the source, thereby implying it as the source of changes associated with priming of pop-out. However, this is speculative. In the neurophysiological studies of FEF and V4, target and distractor features were not independent. When a target feature became a distractor, the distractor became the target. There were instances where both target and distractor changed in the study of FEF (Bichot & Schall, 2002); however, comparisons were not made between those switches and the target-distractor swaps. This makes direct evaluation of the independence target and distractor-processing changes impossible with current neurophysiological reports. It is worth mentioning that both enhancement of neural responses to targets and suppression of neural responses to distractors were observed in V4 and FEF (Bichot & Schall, 2002; Westerberg, Maier, & Schall, 2020a), hinting at perhaps the existence of multiple mechanisms. However, psychophysical evidence exists indicating priming affects processing of targets and distractors independently (Kristjánsson & Driver, 2008). To summarize, psychophysics indicates distinct processes instantiating changes in target and distractor processing, and neurophysiology indicates distractor processing is instantiated in the frontal cortex. Together this suggests changes in distractor processing with priming of pop-out might originate in the frontal cortex and the findings in the visual cortex are secondary.

Selection tradeoff hypothesis

The final hypothesis to consider is a switch in the brain area that first selects the target as a function of priming. In the context of FEF and V4, this could manifest as the frontal cortex selecting the target first in trials early in the priming sequence, when attentional demand is perhaps greatest and visual cortex selecting first once the target feature has been established (e.g., later in the priming sequence). The rationale behind the switch could also be that the frontal cortex must override a sort of attentional selection template that is driving selection in the visual cortex when the search conditions change, thus necessitating earlier selection in the frontal cortex in the priming sequence, which could then switch once the template is established. This would also potentially explain the higher error rate early in the priming sequence (Bichot & Schall, 2002; Westerberg, Maier, & Schall, 2020a), as perhaps the frontal cortex does not inhibit the selection of the visual cortex for the incorrect item quickly enough. Regardless, the most general assertion of this hypothesis is that attentional selection of the target first occurs in different brain areas depending on the priming state. Neurophysiologically, this would be observed as a crossover in selection times between FEF and V4 as a function of priming. The selection times for area V4 seem to speed roughly twice as much as the selection times in FEF when comparing across studies (Bichot & Schall, 2002; Westerberg, Maier, & Schall, 2020a). Therefore, it is plausible that the selection first occurs in FEF in the unprimed state but occurs first in V4 following priming. While perhaps a bit more abstract than the other hypotheses, this could be interpreted as persistent top-down modulation of visual processing. In other words, the frontal cortex identifies novel features in the visual environment that are useful for directing behavior. Once that feature is identified (e.g., the first trial in the priming sequence), the frontal cortex imposes modulatory activity on the visual cortex to change processing in such a way that it promotes the activation in response to that feature (e.g., trials later in the priming sequence). While such a change did not manifest as modulation of baseline activity in V4 (Westerberg, Maier, & Schall, 2020a), there are a number of other mechanisms through which top-down control could affect visual processing. Oscillatory activity could change with priming through modifications in the power of certain frequency bands related to attention, such as alpha (see Klimesch, 2012, for review), in cortical regions representing the feature of interest. An alternative might be through phase-resetting of neural oscillations, which have shown the capacity to coordinate large-scale networks during attentional processing (Voloh & Womelsdorf, 2016). These are only a couple of possibilities for the adaptations that might exist to generate the selection tradeoff. This hypothesis is perhaps the least likely given the complexity of the changes that would be necessary to elicit the priming effect. Nonetheless, it is important to recognize its feasibility.

Which of these hypotheses is correct? When considering the evidence from the lesion study where priming diminished with lesions of visual cortical areas (Walsh, Le Mare, Blaimire, & Cowey, 2000), it seems the source of priming lies prior to FEF. FEF cannot be the sole source of priming as it is nearest to the production of the behavior, and lesioning areas V4 or TEO attenuate priming. This would suggest that the feedforward bottleneck or selection tradeoff is more likely the neural mechanism than feedback amplification. The sensitivity of area V4 to pop-out stimuli even in the absence of selective attention might provide further evidence for the feedforward bottleneck hypothesis (Burrows & Moore, 2009). Furthermore, the evidence from human TMS studies suggests FEF is not the source for these feature-based priming effects (Taylor, Muggleton, Kalla, Walsh, & Eimer, 2011). It may also be the case that the underlying mechanism is even something different than what is proposed here as the evidence is limited to only several studies. Additionally, we are only considering the effect of priming of spike-rate changes. It may also be the case that changes associated with priming of pop-out are driven through changes in multivariate representations of stimulus features beyond simple spike-rates (Goddard, Solomon, & Carlson, 2017; Tovar, Westerberg, Cox, Dougherty, Carlson, Wallace, & Maier, 2019) or through changes in synchrony between frontal and visual cortical areas (Gregoriou, Gotts, & Desimone, 2012; Gregoriou, Gotts, Zhou, & Desimone, 2009).

Conclusions

We set out to review the neural mechanism at the heart of priming of pop-out and provide insight into how the brain automatically adjusts visual selective attention, primarily with respect to the timing of selection. The dynamics of timing in attentional selection during visual search stems from the work of Eriksen, who first reported how differences in the composition of visual arrays influences the speed to which we can search. These fundamental observations led to studies of how search times change with search history. We introduced the psychophysics of priming of pop-out and the changes that have been observed in neural activity associated with priming of pop-out to capture a perspective on where visual search investigation has gone in the time since Eriksen’s early work. This included review of studies in functional imaging, human electrophysiology, and macaque neurophysiology, among others. We discussed the regions of the brain implicated in the production of the priming effect including areas FEF and V4. While work up to this point has identified potential sources for priming of selective attention, further work is needed for a complete understanding of the neural mechanism. For one, a gap exists in the breadth of brain areas investigated with respect to attentional priming. Areas like the parietal cortex, superior colliculus, and visual pulvinar are highly involved in attention and have not been investigated at the neural level, just to name a few. Perhaps the most notable gap is how priming flows through the attentional processing brain structures. To that end, we proposed three rival hypotheses regarding the neural pathway supporting changes in attentional selection. Specifically, these hypotheses suggest changes in attentional selection are driven: (1) bottom-up and therefore instantiated in the early visual processing pathway and inherited by frontal attentional control structures, (2) top-down from attentional control structures to earlier visual areas, or (3) interactively whereby attentional control structures recognize switches in targets and adjust attentional selection before earlier visual areas take control once a pattern is established. These represent potential avenues for future investigation into the neural basis of priming of pop-out, which in turn will inform visual selective attention more generally.