Visual search is an everyday part of the human experience. Most readers have likely had the frustrating experience of searching seemingly everywhere without success for their keys before leaving the house. In these situations, it would be beneficial to be able to keep track of where you have already searched to prevent multiple evaluations of the same object. In fact, most models of visual search assume that each distractor is evaluated only once per search, which implies a strong role for memory in visual search (Treisman & Gelade, 1980; Wolfe, Cave, & Franzel, 1989; though see Wolfe, 2007). However, behavioral evidence on this topic has been surprisingly mixed, and it remains unclear if rejected distractors are reliably maintained during visual search.

The strongest candidate for a mechanism that biases attention toward new locations during search is inhibition of return, or IOR (Klein, 1988), which refers to the delay in response time (RT) toward probes in recently attended locations relative to novel locations (Posner & Cohen, 1984). This phenomenon has been replicated in visual search by probing both previous fixations and distractor locations (Dodd, van der Stigchel, & Hollingworth, 2009; Klein & MacInnes, 1999; MacInnes & Klein, 2003; Thomas et al., 2006). Researchers have suggested the function of IOR is to serve as a “foraging facilitator” (Klein & MacInnes, 1999). Similarly, eye-tracking evidence indicates that observers appear to have some memory for previously fixated locations (Bays & Husain, 2012; Klein & MacInnes, 1999; Peterson, Kramer, Wang, Irwin, & McCarley, 2001; Vaughan, 1984). However, IOR is too limited in capacity and duration to support memory for previously rejected distractors in a meaningful sense outside of searches with very small set sizes (Wolfe, 2003). Furthermore, search is not impaired when items in the array are moved to new locations every ~100 ms, which should restrict the ability of any mnemonic mechanism to operate during search (Horowitz & Wolfe, 1998). As a result of these findings, IOR is thought to discourage perseveration on salient items, but is likely insufficient to result in visual search with perfect “memory” for rejected distractors (Itti & Koch, 2001; Wolfe, 2003).

Regardless of the mechanism, it is apparent that the number of tagged distractors is necessarily limited by the amount of cognitive resources available for storing information (i.e., working memory [WM] capacity). In support of this view, the number of previous fixations remembered during visual search is roughly equivalent to WM capacity estimates of ~4 items (McCarley, Wang, Kramer, Irwin, & Peterson, 2003). WM is thought to serve multiple functions in visual search, including the maintenance of the target template and the comparison site between the target template and candidate items from the search array (Bundesen, 1990; Desimone & Duncan, 1995; Duncan & Humphreys, 1989).

One common method for investigating the role of WM in visual search is the dual-task paradigm. If WM is heavily used in visual search tasks, filling WM to capacity with information from an irrelevant task should impair search performance. Early investigations using this approach found that search efficiency was unaffected by a concurrent WM load, but overall RTs were longer (Downing & Dodds, 2004; Woodman, Vogel, & Luck, 2001). Later research found that search performance was impaired only when the WM task is spatial in nature, which would be expected if rejected distractors could no longer be “tagged” in spatial WM when it is filled to capacity (Oh & Kim, 2004; Woodman & Luck, 2004). However, it should be noted that IOR during search appears to be severely disrupted when the search display is removed, which suggests this particular mechanism is object based rather than location based (Klein & MacInnes, 1999; Takeda & Yagi, 2000). Furthermore, it is possible that interference from the spatial WM task is due to an impairment of overall search strategy (e.g., top to bottom) or a more general disruption of commonly shared resources between the two tasks (Woodman & Luck, 2004). In addition, Emrich, Al-Aidroos, Pratt, and Ferber (2010) demonstrated that at set sizes above WM capacity, distractor maintenance theories would predict the effect of a concurrent WM load to be on overall RT rather than search slope. At larger set sizes, WM should already be filled to capacity due to the maintenance of the distractor items, thus diminishing the effect of an additional load on search efficiency.

If WM is important for maintaining rejected distractors, it stands to reason that WM capacity should explain some of the variance in search performance. However, WM capacity did not predict search efficiency at a wide range of search difficulty levels (Kane, Poole, Tuholski, & Engle, 2006). Similarly, Williams and Drew (2018) found that WM capacity was weakly associated with search accuracy but was not predictive of search efficiency or overall RT. The observed correlation between WM capacity and search accuracy is consistent with the proposal that the target template is maintained in WM. However, it does not support the idea that rejected distractors are stored in WM, which would be better captured by RT and search efficiency rather than accuracy.

One key limitation of existing research is that it can be challenging to assess the role of WM behaviorally without disrupting the search process itself, particularly when using dual-task paradigms or dynamic displays (e.g., von Mühlenen, Müller, & Müller, 2003). Therefore, ongoing measures of WM activity, such as the contralateral delay activity (CDA), are particularly valuable for better understanding these processes. The CDA is a lateralized event-related potential (ERP) component that indexes the number of objects maintained in visual WM, such that it increases in amplitude until an individual’s WM capacity is reached (Vogel & Machizawa, 2004). The CDA is typically used in tasks with an explicit WM requirement, such as maintaining colored squares over a retention interval in a change detection task (Vogel & Machizawa, 2004). However, the CDA is also present in tasks that require manipulating information while the stimulus remains visible, such as multiple object tracking (Drew, Horowitz, & Vogel, 2013; Drew, Horowitz, Wolfe, & Vogel, 2012; Drew & Vogel, 2008) and mental rotation (Prime & Jolicoeur, 2010).

These findings highlight the potential of the CDA to help answer questions that have been difficult to address behaviorally, such as the maintenance of distractors in visual search. For example, Emrich, Al-Aidroos, Pratt, and Ferber (2009) asked participants to complete a lateralized WM task and a lateralized visual search task (find the upright T amongst rotated Ts). The CDA was present during the search task and the authors reasoned that the CDA elicited during search reflects the maintenance of rejected distractors in WM. This idea was supported by the observed delay in peak amplitude in the search task relative to the WM task, which suggests a slow accumulation of information over time. However, it is challenging to precisely know what information is reflected in the CDA without a search task manipulation. For example, the CDA could solely reflect information about the target rather than the distractors. However, the researchers minimized WM requirements for the maintenance of the target template by repeating the target across trials (Woodman, Luck, & Schall, 2007). In addition, the amplitude of the CDA was similar between the WM task, which displayed four items on the screen, and the search task, which displayed 10 items. This suggests a similar number of object representations were encoded into WM for both tasks, which would be unexpected if the CDA only indexed the target representation. Finally, a key property of the CDA is its sensitivity to WM load (e.g., number of objects), and it is important to consider whether the CDA elicited during search truly reflects the number of encoded objects in WM rather than spatial attention to the cued side of the display. For example, observers had a larger CDA amplitude during a multiple object tracking task than when observing the same stimulus and attending a lateralized random dot kinematogram, which suggests the CDA reflects ongoing WM processes, rather than spatial attention, in tasks without explicit WM requirements (Drew, Horowitz, Wolfe, & Vogel, 2011). Similarly, Emrich et al. (2009) demonstrated that the increase in CDA amplitude over time explained unique variance in search performance beyond an index of attentional selection, the N2pc (Luck & Hillyard, 1994).

Luria and Vogel (2011) also used the CDA to investigate the role of WM in visual search. They found that the amplitude of the CDA increased as it became more difficult to distinguish targets from distractors, suggesting WM is involved in filtering distractors. Although this group also attributed the presence of the CDA to the storage of distractors in WM, their results were interpreted using a different theoretical framework. Emrich et al. (2009) found that both WM capacity and the change in CDA amplitude over time correlated with search efficiency, such that individuals with higher capacity had better search performance and relied more on WM during search. Presumably, individuals with higher WM capacity can maintain more distractors in WM and therefore spend less time reevaluating distractors. In contrast to Emrich et al. (2009), Luria and Vogel (2011) found that higher WM capacity predicted smaller CDA amplitude during search, which suggests reduced WM activity in high-capacity individuals. WM capacity predicts the ability to successfully filter out irrelevant information (Vogel, McCollough, & Machizawa, 2005). Using this framework, the authors reasoned that individuals with higher WM capacity should be better able to ignore distractors and locate the target, consequently storing fewer items in WM during search. Finally, it is also possible that distractors are not maintained in WM indefinitely during search. Rather, WM capacity might determine the rate that distractors can be shuttled through WM for evaluation. If so, high-capacity individuals should reach peak CDA amplitude relatively quickly. However, Emrich et al. (2009) found the opposite: Low-capacity individuals reached peak amplitude sooner than high-capacity individuals. This finding suggests the accumulation of distractor information over time occurs at a similar rate, irrespective of WM capacity.

Considering these conflicting but equally plausible accounts, there is a need to further investigate the nature of the information reflected in the CDA during search. Although it is theoretically interesting to determine whether it is better to rely more or less on WM during search, existing research provides conflicting accounts of the role of WM in visual search. Here, we sought to directly test whether WM is involved in maintaining rejected distractors by manipulating visual set size. This approach takes advantage of the CDA’s sensitivity to the number of objects maintained in WM (Vogel & Machizawa, 2004). Specifically, if the CDA reflects the maintenance of rejected distractors in WM during search:

  1. 1)

    The amplitude of the CDA should be larger for target absent than target-present trials because, on average, more distractors will be evaluated prior to search termination on target-absent trials.

  2. 2)

    For target-absent trials, the amplitude of the CDA during search should increase with visual set size up to WM capacity (~4 items) because more distractors would need to be maintained at larger visual set sizes.

  3. 3)

    Greater WM capacity and greater WM reliance during search should predict faster task performance.

In Experiment 1, we failed to find evidence in support of these predictions in a search task that used real-world stimuli. In Experiment 2, we used a search task with a higher degree of feature overlap between targets and distractors and found partial evidence in favor of distractor maintenance. These results suggest that rejected distractors do not need to be encoded into WM when targets and distractors can be distinguished relatively easily based on their visual features. However, WM use increases with the number of distractors in visual search tasks that have high target/distractor feature overlap, which may reflect the encoding of rejected distractors into WM. In both experiments, there was little evidence that WM was related to overall task performance. Together, these findings suggest that distractor maintenance may play a relatively minor role in visual search, particularly in more realistic search tasks.

Method

Participants

Participants were recruited from the University of Utah and received course credit or monetary compensation ($15/hr.) in exchange for their participation. There were 51 participants in Experiment 1 and 18 participants in Experiment 2. Eleven participants were excluded in Experiment 1 (five did not complete the study, two had too many EEG artifacts, and four were missing behavioral data due to technical difficulties), leaving 40 participants (27 females, average age = 21.9 years). Three participants were excluded in Experiment 2 (one had too many EEG artifacts and two did not complete the study), leaving 15 participants (three females, average age = 24.93 years). Note that the sample size was larger in Experiment 1 because this research question was a secondary aim of study examining individual differences in visual search performance, whereas Experiment 2 solely aimed to replicate the effects observed in Experiment 1. The procedure was approved by the University of Utah Institutional Review Board, and participants provided informed consent.

Procedure

Participants were seated approximately 36-in. from a 24-in. monitor with a 60-Hz refresh rate, and the tasks were presented using MATLAB with Psychtoolbox (Brainard, 1997; Kleiner, Brainard, Pelli, & Ingling, 2007; Pelli, 1997).

WM task

First, participants completed a change detection WM task (Luck & Vogel, 1997), with six practice and 120 experimental trials. Four or eight colored squares were presented for 100 ms while participants fixated on a central cross. Following a 900-ms retention interval, one of the squares returned to its original location. In 50% of trials, the square changed color from its original state, and participants were asked to indicate if the square was the same or a different color. WM capacity was calculated using Cowan’s k formula: WM capacity = (hit rate − false alarm rate) × set size (Cowan, 2001).

Visual search task

Participants completed a visual search task with 12 practice and 1,296 experimental trials in Experiment 1 and 1,200 experimental trials in Experiment 2 (see Fig. 1). Each trial began with a preparation period for 600 ms in Experiment 1 and 500 ms in Experiment 2. During this period, only the central fixation cross and placeholders for the search array were visible. Next, a target (~3.35° of visual angle) was presented at the center of the screen for 500 ms in Experiment 1 and 1,000 ms in Experiment 2. The target presentation time was increased after piloting in Experiment 2 in order to increase the number of correct trials entered into our analyses. Following target presentation, an arrow cue pointing to the left or right side of the screen (direction chosen at random) was presented for 800–1,000 ms. A lateralized search array with two, four, or six objects on each side (~3.14° of visual angle) was presented until the participant made a target-present (50%) or target-absent (50%) response using J or F key presses. Participants were instructed to search only the cued side of the array. Finally, a blink interval was presented for 1,000 ms in Experiment 1 and 600 ms in Experiment 2. In Experiment 1, the stimuli were grayscale real-world objects chosen at random from a database of >2,000 objects (Brady, Konkle, Alvarez, & Oliva, 2008). In Experiment 2, the stimuli were Landolt Cs with six possible orientations.

Fig. 1
figure 1

Visual search task for Experiments 1 and 2

Full size image

EEG procedure

Electrophysiological (EEG) activity was recorded at 500 Hz using Brain Products’ ActiCap and ActiCHamp systems. There were 32 Ag/Cl active electrodes from the International 10–20 system, and the data were referenced online to the average of the left and right mastoids. Impedance at each electrode was reduced to 15 kiloohms or less using light scalp abrasion and conductive gel.

EEG analysis

The data were processed off-line using EEGLAB and ERPLAB (Delorme & Makeig, 2004; Lopez-Calderon & Luck, 2014). First, the data was down-sampled to 250 Hz. Next, the continuous data was band-pass filtered from .01 to 100 Hz. For eye-movement detection, we placed two electrodes ~1 cm from the external canthi of each eye. An HEOG channel was created as the difference between the left and right eye. To detect eye movements, a step-function was performed on the HEOG channel (threshold 40 μV). For blink detection, we used the frontal electrodes (Fp1/2) above each eye. Blinks were corrected using independent component analysis (ICA; Delorme & Makeig, 2004). A moving window function was used to detect other large artifacts (threshold 140μV). As appropriate, individual thresholds were adjusted slightly to increase signal and reduce noise For the search-locked analysis, in Experiment 1, 10.08% of trials were rejected, and in Experiment 2, 10.4% of trials were rejected. For the response-locked analysis, in Experiment 1, 6.3% of trials were rejected, and in Experiment 2, 14.8% of trials were rejected. For plotting purposes, we used a low-pass Butterworth filter with a half-amplitude cutoff of 30 Hz. All statistical analyses were performed on the waveforms prior to the application of the low-pass filter. Only correct trials were included in the ERP analyses.

Search-locked CDA

The lateralized waveforms were calculated as the contralateral-ipsilateral activity averaged across electrodes PO7/O8 and P7/8 relative to the cued side of the search array. We used these electrodes because the CDA is maximal at these sites across all conditions. The ERP waveforms were time locked to the search array and extended for 1,000 ms. The waveforms were baselined to the 200 ms prior to search onset, and the average CDA amplitude was measured from 300 to 1,000 ms.

Response-locked CDA

In addition to the search-locked analysis, we performed a response-locked analysis to account for differences in response time between conditions and experiments. For the search-locked analysis, a participant might have already identified the target during the CDA window at smaller set sizes but still be evaluating the distractors during this time at larger set sizes. This could result in a smaller mean CDA for smaller set sizes over a given time window due to differences in response time rather than differences in distractor maintenance. In contrast, by time locking to the response, we can account for these differences in response time by evaluating WM activity during the moments preceding the response, which should index the maximum number of distractors maintained in WM for all conditions.

The ERP waveforms were time locked to the participant’s response and baseline corrected to the 200 ms prior to search onset. The waveforms extended from −1,000 to 0 ms in Experiment 1 and −3,000 to 0 ms in Experiment 2. The CDA starts ~300 ms from search onset (Vogel & Machizawa, 2004), so we chose a time window for the response-locked CDA that was approximately 300 ms shorter than the smallest mean response time in each experiment: −400 to 0 ms in Experiment 1, and −800 to 0 ms in Experiment 2. Data from two participants exceeded our threshold for exclusion (>30% artifacts) in the response-locked analysis due to excessive eye movements, leaving a total of 39 participants in Experiment 1 and 14 participants in Experiment 2.

Experiment 1

Results

Prior to the analysis of the behavioral data, responses less than 200 ms or greater than 5,000 ms were removed. Only correct trials were included in the RT and CDA analyses. In the event of an omnibus effect, Tukey’s multiple comparison test was used to determine which means differed from the rest. In addition to frequentist statistics, Bayes factors were calculated to aid in the interpretation of null results. A BF10 greater than 3 is considered at least moderate evidence in favor of the alternative hypothesis, and a BF10 less than .33 is considered at least moderate evidence in favor of the null hypothesis (Jeffreys, 1961).

Response time

RT was calculated for correct trials only. RT was significantly longer for target absent (M = 981ms, SD = 267ms) than target-present trials (M = 838 ms, SD = 163 ms), t(39) = 6.55, p < .001. For target-absent trials, RT significantly increased with visual set size (SS), F(2, 78) = 75.54, p < .001 (see Fig. 2a). SS2 RT (M = 816ms, SD = 175 ms) was significantly faster than SS4 (M = 986 ms, SD = 265 ms), p < .001, and SS6 (M = 1,145 ms, SD = 377 ms), p < .001. Similarly, RT was significantly shorter for SS4 than for SS6 (p < .001). For target-present trials, RT significantly increased with visual set size, F(2, 78) = 178.4, p < .001 (see Fig. 2b). SS2 (M = 755 ms, SD = 146 ms) had a significantly lower RT than SS4 (M = 853ms, SD = 170 ms), p < .001, and SS6 (M = 919 ms, SD = 182 ms), p < .001. Similarly, RT was significantly lower for SS4 than SS6 (p < .001).

Fig. 2
figure 2

Behavioral data for Experiment 1. a Response time for target-absent trials. b Response time for target-present trials. c Error rate for target-absent trials. d Error rate for target-present trials. Error bars represent standard error of the mean

Full size image

Error rate

Error rate was significantly lower for target-absent (M = 4.77%, SD = 3.34%) than for target-present trials (M = 14.25%, SD = 5.91%), t(39) = 9.71, p < .001. For target-absent trials, error rate significantly increased with visual set size, F(2, 78) = 17.92, p < .001 (see Fig. 2c). SS2 (M = 3.24%, SD = 2.46%) had a significantly lower error rate than SS4 (M = 5%, SD = 3.6%), p = .001, and SS6 (M = 6.06%, SD = 4.85%), p < .001. Error rate did not significantly differ between SS4 than SS6 (p = .07). For target-present trials, error rate significantly increased with visual set size, F(2, 78) = 160.3, p < .001 (see Fig. 2d). SS2 (M = 7.36%, SD = 4.05%) had a significantly lower error rate than SS4 (M = 13.65%, SD = 6.44%), p < .001, and SS6 (M = 21.74%, SD = 8.54%), p < .001. Similarly, SS4 had a significantly lower error rate than SS6 (p < .001).

Search-locked CDA

If distractors are maintained in WM during search, we would expect to see a larger CDA amplitude (i.e., more negative) for target-present than for target-absent trials because, on average, only half of the distractors should be evaluated before identifying the target on target-present trials. Contrary to our prediction, the amplitude of the CDA was significantly larger for target-present than for target-absent trials, F(1, 39) = 40.99, p < .001, BF10 = 143,559.22 (see Fig. 3a–b; Table 1). We also expected to see an increase in CDA amplitude with visual set size. This pattern of results should be particularly evident for target-absent trials because, on average, more distractors should be evaluated before the observer can definitively say the target is not there. However, CDA amplitude significantly decreased with visual set size, F(2, 78) = 16.69, p < .001, BF10 = 15,182.24, suggesting greater WM involvement at smaller set sizes (see Fig. 3c–f; Table 1). The Set Size × Target Present/Absent interaction was also statistically significant, F(2, 78) = 5.05, p = .009, BF10 = 4.29. Contrary to our prediction, the set size effects were primarily driven by the target-present trials. For target-present trials, SS2 was significantly larger than SS4 (p = .02), and SS6 (p < .001). SS4 was significantly larger than SS6 (p = .002). There were no statistically significant set size effects for target-absent trials (all p values > .09). Overall, this pattern of results is the opposite of what would be expected if the CDA reflects information about maintained distractors during search.

Fig. 3
figure 3

Search-locked analysis for Experiment 1. a Lateralized difference waveforms for target-present and target-absent trials. b Mean CDA amplitude for target-present and target-absent trials. c Lateralized difference waveforms by visual set size for target-absent trials. d Mean CDA amplitude by visual set size for target-absent trials. e Lateralized difference waveforms by visual set size for target-present trials. f Mean CDA amplitude by visual set size for target-present trials. Shaded regions represent standard error. Error bars represent standard error of the mean

Full size image
Table 1 Mean CDA amplitude for each condition in Experiment 1
Full size table

Response-locked CDA

In addition to the search-locked analysis, CDA amplitude was also calculated relative to the moment a response was made to ensure that differences in response time were not driving amplitude differences between visual set sizes. Overall, these results follow the same pattern as the search-locked analysis. The response-locked CDA was significantly larger (i.e., more negative) in amplitude for target-present than for target-absent trials, F(1, 38) = 29.34, p < .001, BF10 = 24,73.63 (see Fig. 4a–b; Table 1). In addition, the CDA decreased in amplitude with visual set size, F(2, 76) = 10.72, p < .001, BF10 = 402.00 (see Fig. 4c–f; Table 1). The Set Size × Target Present/Absent interaction was also statistically significant, F(2, 76) = 4.85, p = .01, BF10 = 4.04. For target-present trials, SS6 was significantly smaller than SS2 (p < .001), and SS4 (p = .001). SS2 and SS4 did not significantly differ (p = .31). There were no statistically significant set size effects for target-absent trials (all p values > .31).

Fig. 4
figure 4

Response-locked analysis for Experiment 1. a Lateralized difference waveforms for target-present and target-absent trials. b Mean CDA amplitude for target-present and target-absent trials. c Lateralized difference waveforms by visual set size for target-absent trials. d Mean CDA amplitude by visual set size for target-absent trials. e Lateralized difference waveforms by visual set size for target-present trials. f Mean CDA amplitude by visual set size for target-present trials. Shaded regions represent standard error. Error bars represent standard error of the mean

Full size image

Relationship between WM and performance

Based on previous research, greater WM capacity is thought to either (a) lead to more WM reliance during search and better performance because more distractors can be maintained in WM (Emrich et al., 2009) or (b) lead to a smaller reliance on WM during search and better performance because those with a greater capacity are better able to filter out irrelevant information (Luria & Vogel, 2011). Overall, WM capacity (M = 1.51, SD = 1.46) did not predict better search performance in terms of RT (M = 911.8, SD = 210.5), r(36) = −.16, p = .33, BF10 = .32, or error rate (M = 9.51%, SD = 3.67%), r(36) = −.033, p = .842, BF10 = .21, in this task. In addition, WM capacity did not predict the degree of WM reliance (i.e., CDA amplitude) for target present, r(36) = −.15, p = .37, BF10 = .20, or target-absent trials, r(36) = −.10, p = .55, BF10 = .21. Finally, if distractors are maintained in WM during search, we would expect individuals with greater WM capacity to be better able to store those distractors, resulting in larger set size effects. To test this proposal, we divided the data into high-WM-capacity and low-WM-capacity participants using a median split of their WM capacity. Next, we performed a 2 (WM capacity) × 3 (set size) mixed analysis of variance (ANOVA). The overall effect of WM capacity was not statistically significant, F(1, 36) = .56, p = .46, BF10 = .43, and there was not a significant Set Size × WM Capacity interaction, F(2, 72) = 1.37, p = .26, BF10 = .37 (see Fig. 5a–d). Together, these results suggest that WM capacity is not a limiting factor during visual search, which is consistent with previous findings (Williams & Drew, 2018).

Fig. 5
figure 5

ERP waveforms broken down by median split of WM capacity for Experiment 1. a Lateralized difference waveforms by visual set size for target-absent trials for high-WM-capacity participants. b Lateralized difference waveforms by visual set size for target-absent trials for low-WM-capacity participants. c Lateralized difference waveforms by visual set size for target-present trials for high-WM-capacity participants. d Lateralized difference waveforms by visual set size for target-present trials for low-WM-capacity participants. Shaded regions represent standard error

Full size image

If the CDA indexes information about distractors during search, we would expect the amplitude of the CDA to predict search performance. However, the amplitude of the search-locked CDA did not predict RT for target-present, r(38) = .20, p = .21, BF10 = .42 (see Fig. 6a), or target-absent trials, r(38) = .10, p = .55, BF10 = .23 (see Fig. 6b). Similarly, the amplitude of the CDA did not predict error rate for target-present, r(38) = −.10, p = .55, BF10 = .23, or target-absent trials, r(38) = −.06, p = .70, BF10 = .21. Finally, CDA amplitude predicted search efficiency for target-present trials, r(38) = .36, p = .02, BF10 = 2.34, but not target-absent trials, r(38) = .15, p = .35, BF10 = .30. However, the Bayes factor for the relationship between search efficiency and CDA amplitude for target-present trials is inconclusive.

Fig. 6
figure 6

Relationship between WM use and task performance in Experiment 1. a Correlation between stimulus-locked CDA and response time for target-absent trials. b Correlation between stimulus-locked CDA and response time for target-present trials. c Correlation between response-locked CDA and response time for target-absent trials. d Correlation between response-locked CDA and response time for target-present trials. Dashed lines represent the 95% confidence interval

Full size image

Overall, the results of the response-locked CDA show a similar pattern. The amplitude of the response-locked CDA did not predict RT for target-present, r(37) = −.02, p = .90, BF10 = .20 (see Fig. 6c), or target-absent trials, r(37) = −.08, p = .63, BF10 = .22 (see Fig. 6d). The amplitude of the CDA did not predict error rate for target-present, r(37) = −.03, p = .86, BF10 = .20, or target-absent trials, r(37) = −.06, p = .71, BF10 = .21. Finally, CDA amplitude for the response-locked waveforms did not predict search efficiency for target present, r(37) = .19, p = .25, BF10 = .38, or target-absent trials, r(37) = .02, p = .91, BF10 = .20.

Discussion

The results of Experiment 1 do not support any of the predictions for a distractor maintenance interpretation of the CDA during visual search. Although the number of rejected distractors should be approximately twice as large, on average, in target-absent trials, CDA amplitude was larger for target-present trials. In addition, WM activity decreased with visual set size, which is difficult to reconcile with the proposal that distractors are stored in WM during search (see the General Discussion for additional thoughts on this finding). The same pattern of results was found for both search array and response-locked waveforms, suggesting that differences in response time between the conditions cannot explain these results. Finally, there was not a relationship between CDA amplitude or WM capacity and search performance. This is inconsistent with the findings of Luria and Vogel (2011). However, the search array was only presented for a limited time (200 ms) in their studies, which might have increased the need for WM involvement. Overall, these results suggest that visual search does not depend on maintaining rejected distractors in WM in this task. However, one key difference between our study and previous research (Emrich et al., 2009; Luria & Vogel, 2011) is the choice of stimuli (real-world objects vs. letters). Real-world objects may decrease the need for distractors to be tagged during search because it is relatively easy to distinguish targets from distractors. In Experiment 2, we used stimuli with a high degree of feature overlap (Landolt Cs) to account for this possibility.

Experiment 2

Experiment 2 used Landolt C stimuli with six possible orientations rather than real-world objects to increase the degree of feature overlap between targets and distractors. Otherwise, all methods were identical to Experiment 1, except where noted above (see Fig. 1).

Results

Response time

RT was calculated for correct trials only. RT was significantly faster for target-present (M = 1480 ms, SD = 343 ms) than for target-absent trials (M = 1,881 ms, SD = 482 ms), t(14) = 7.63, p < .001. For target-absent trials, RT significantly increased with visual set size, F(2, 28) = 93.65, p < .001 (see Fig. 7a). SS2 RT (M = 1,216 ms, SD = 271 ms) was significantly lower than SS4 (M = 1,914 ms, SD = 476 ms), p < .001, and SS6 (M = 2,551 ms, SD = 734 ms), p < .001. Similarly, RT was significantly lower for SS4 than SS6 (p < .001). For target-present trials, RT significantly increased with visual set size, F(2, 28) = 85.69, p < .001 (see Fig. 7b). SS2 (M = 1,103 ms, SD = 252 ms) was significantly faster than SS4 (M = 1,541 ms, SD = 382 ms), p < .001, and SS6 (M = 1,865 ms, SD = 443 ms), p < .001. Similarly, RT was significantly lower for SS4 than SS6 (p < .001).

Fig. 7
figure 7

Behavioral data for Experiment 2. a Response time for target-absent trials. b Response time for target-present trials. c Error rate for target-absent trials. d Error rate for target-present trials. Error bars represent standard error of the mean

Full size image

Error rate

Error rate was significantly lower for target-absent (M = 6.34%, SD = 5.88%) than for target-present trials (M = 15.52%, SD = 5.68%), t(14) = 5.83, p < .001. For target-absent trials, error rate significantly increased with visual set size, F(2, 28) = 6.59, p = .005 (see Fig. 7c). SS2 error rate (M = 3.2%, SD = 2.56%) was significantly lower than SS6 (M = 9.17%, SD = 10.1%), p = .003. There were no significant differences between SS2 and SS4 (M = 6.67%, SD = 6.01%), p = .11, or between SS4 and SS6 (p = .30). For target-present trials, error rate significantly increased with visual set size, F(2, 78) = 37.93, p < .001 (see Fig. 7d). SS2 error rate (M = 8.03%, SD = 3.48%) was significantly lower than SS4 (M = 14.13%, SD = 5.53%), p = .009, and SS6 (M = 24.4%, SD = 10.4%), p < .001. Similarly, SS4 error rate was significantly lower than SS6 (p < .001).

Search-locked CDA

If WM is used to store distractors during search, the CDA should be larger in amplitude for target-present than target-absent trials. Contrary to this prediction, the amplitude of the CDA did not significantly differ between target-present and target-absent trials, F(1, 14) = .12, p = .74, BF10 = .23 (see Fig. 8a–b; Table 2). In addition, CDA amplitude should increase with the number of distractors in the search array, up to WM capacity. However, the amplitude of the CDA did not significantly differ between visual set sizes, F(2, 28) = 2.18, p = .13, BF10 = 1.21 (see Fig. 8c–f; Table 2). However, the Bayes factor for this comparison is inconclusive. The Set Size × Target Present/Absent interaction was also not statistically significant, F(2, 28) = .96, p = .40, BF10 = .28.

Fig. 8
figure 8

Search-locked analysis for Experiment 2. a Lateralized difference waveforms for target-present and target-absent trials. b Mean CDA amplitude for target-present and target-absent trials. c Lateralized difference waveforms by visual set size for target-absent trials. d Mean CDA amplitude by visual set size for target-absent trials. e Lateralized difference waveforms by visual set size for target-absent trials. f Mean CDA amplitude by visual set size for target-present trials. Shaded regions represent standard error. Error bars represent standard error of the mean

Full size image
Table 2 Mean CDA amplitude for each condition in Experiment 2
Full size table

Response-locked CDA

We repeated the above analyses on the response-locked waveforms to account for RT differences between conditions. Note that because of these large differences in RT, the response-locked ERP waveforms are equal to zero (i.e., baseline) at different time points in the waveform for each condition (see Fig. 9). As with the search-locked CDA, the amplitude of the response-locked CDA did not significantly differ between target-present and target-absent trials, F(1, 13) = .11, p = .75, BF10 = .24 (see Fig. 9a–b; Table 2). However, contrary to Experiment 1, the amplitude of the CDA significantly increased with visual set size, F(2, 26) = 8.28, p = .002, BF10 = 340.28 (see Fig. 9c–f; Table 2). SS2 was significantly smaller than SS4 (p = .01), and SS6 (p = .002). There were no significant differences between SS4 and SS6 (p = .99). The Set Size × Target Present/Absent interaction was not statistically significant, F(2, 26) = .28, p = .76, BF10 = .21. Together, these results provide partial support for the hypothesis that distractors are maintained in WM during search.

Fig. 9
figure 9

Response-locked analysis for Experiment 2. a Lateralized difference waveforms for target-present and target-absent trials. b Mean CDA amplitude for target-present and target-absent trials. c Lateralized difference waveforms by visual set size for target-absent trials. d Mean CDA amplitude by visual set size for target-absent trials. e Lateralized difference waveforms by visual set size for target-present trials. f Mean CDA amplitude by visual set size for target-present trials. Shaded regions represent standard error. Error bars represent standard error of the mean

Full size image

Relationship between WM capacity and performance

In Experiment 1, we found little evidence of a relationship between WM capacity and task performance. Here, we replicated those results. WM capacity (M = 2.34, SD = .73) did not predict overall RT for correct trials (M = 1,689 ms, SD = 408 ms), r(13) = −.34, p = .22, BF10 = .64. Similarly, WM capacity did not predict error rate (M = 10.93%, SD = 4.91%), r(13) = .20, p = .48, BF10 = .40. Finally, WM capacity did not predict the CDA amplitude for target-present trials, r(13) = −.24, p = .38, BF10 = .48, or target-absent trials, r(13) = −.09, p = .76, BF10 = .34. to determine whether those with a greater WM capacity were able to maintain more distractors in WM, we ran a 2 (WM capacity) × 3 (set size) mixed ANOVA. There was no significant main effect of WM capacity, F(1, 13)=.18, p = .68, BF10 = .71, and there was no significant Set Size × WM Capacity interaction, F(2, 26) = .94, p = .40, BF10 = .47 (see Fig. 10a–d).

Fig. 10
figure 10

ERP waveforms broken down by median split of WM capacity for Experiment 2. a Lateralized difference waveforms by visual set size for target-absent trials for high-WM-capacity participants. b Lateralized difference waveforms by visual set size for target-absent trials for low-WM-capacity participants. c Lateralized difference waveforms by visual set size for target-present trials for high-WM-capacity participants. d Lateralized difference waveforms by visual set size for target-present trials for low-WM-capacity participants. Shaded regions represent standard error

Full size image

In Experiment 1, the amplitude of the CDA did not closely relate to task performance. In contrast, the results of Experiment 2 show that CDA amplitude predicts RT for target-present trials, r(13) = .63, p = .01, BF10 = 5.84, such that greater WM activity is associated with faster RTs (see Fig. 11a). There was not a significant relationship between CDA amplitude and RT for target-absent trials, r(13) = .34, p = .22, BF10 = .65 (see Fig. 11b). The amplitude of the CDA did not predict error rate for target-present, r(13) = −.10, p = .72, BF10 = .34, or target-absent trials, r(13) = .04, p = .89, BF10 = .32. Finally, CDA amplitude did not predict search efficiency for target-present, r(13) = .48, p = .07, BF10 = 1.48, or target-absent trials, r(13) = .20, p = .47, BF10 = .41.

Fig. 11
figure 11

Relationship between WM use and task performance in Experiment 2. a Correlation between stimulus-locked CDA and response time for target-absent trials. b Correlation between stimulus-locked CDA and response time for target-present trials. c Correlation between response-locked CDA and response time for target-absent trials. d Correlation between response-locked CDA and response time for target-present trials. Dashed lines represent the 95% confidence interval

Full size image

Although the search-locked CDA was correlated with RT, response-locked CDA amplitude did not significantly predict RT for target-present trials, r(12) = −.06, p = .84, BF10 = .34 (see Fig. 11c), or target-absent trials, r(12) = −.04, p = .89, BF10 = .33 (see Fig. 11d). This discrepancy may be due to RT differences between participants. Participants with slower response times might reach peak amplitude toward the end of the CDA time window, resulting in a lower overall mean amplitude. However, relative to the response, slow and fast participants show a similar CDA amplitude. The amplitude of the CDA did not predict error rate for target-present, r(12) = −.04, p = .90, BF10 = .33, or target-absent trials, r(12) = −.14, p = .64, BF10 = .36. Finally, CDA amplitude did not predict search efficiency for target-present, r(12) = −.02, p = .96, BF10 = .33, or target-absent trials, r(12) = −.14, p = .63, BF10 = .37.

Discussion

In Experiment 2, we used a search task with a higher degree of feature overlap between targets and distractors and found partial support for the proposal that distractors are maintained in WM during search. There were no significant differences in CDA amplitude between target-present and target-absent trials, and the Bayes factors for this analysis indicated sufficient evidence for the null relative to the alternative hypothesis. In addition, the search-locked CDA did not differ in amplitude with visual set size. However, the response-locked CDA increased in amplitude with visual set size, suggesting the search-locked effects may have been washed out due to differences in RT between conditions. Bayes factors indicated sufficient evidence for the alternative hypothesis. Contrary to our prediction, the set size effects did not significantly differ in magnitude between target-present and target-absent trials. In addition, contrary to our predictions and previous research, we did not find evidence for a relationship between WM capacity and task performance (Emrich et al., 2009; Luria & Vogel, 2011). Overall, these results indicate that WM activity increases with visual set size when targets and distractors have a high degree of feature overlap, which may reflect the maintenance of rejected distractors in WM. However, in conjunction with Experiment 1, our findings suggest that distractor maintenance appears to be limited to search arrays with high target/distractor similarity, which is a relatively rare occurrence in real-world search tasks.

General discussion

Our studies were designed to directly test the proposal that distractors are actively maintained in WM during visual search. By manipulating set size, we were able to observe changes in WM activity as a function of the number of distractors in the search array. If distractor objects are maintained in WM during search, we reasoned the amplitude of the CDA should increase with visual set size until WM capacity is reached. For real-world stimuli in Experiment 1, we failed to find evidence in favor of distractor maintenance. The CDA did not increase in amplitude with the number of distractors. Furthermore, although approximately twice as many distractors should be evaluated prior to search termination on target-absent trials, CDA amplitude was larger for target-present trials. Finally, neither WM activity nor WM capacity predicted visual search performance. In Experiment 2, we used Landolt C stimuli instead of real-world stimuli to increase the degree of feature overlap between targets and distractors. Here, we found partial evidence in favor of distractor maintenance. The CDA increased in amplitude with visual set size relative to the moment a response was made and plateaued at ~4 items. However, there were no significant differences in CDA amplitude between target-present and target-absent trials, and there was little evidence of a relationship between WM and task performance. Together, these findings suggest that role of working memory in distractor maintenance during search appears to depend on the nature of the visual search task in question. If targets and distractors can be easily differentiated, strong search guidance to target-relevant features appears to make it unnecessary to actively maintain distractors in WM during search. In contrast, if targets and distractors are more similar to each other, there appears to be a role for distractor maintenance during visual search.

Although we found partial support for distractor maintenance in Experiment 2, previous research suggests this process may play a relatively limited role in visual search. Evidence for distractor maintenance in visual search has been attributed to the usage of systematic search strategies, such as searching from “left to right” (Dickinson & Zelinsky, 2007; Godwin, Benson, & Drieghe, 2013; Peterson, Beck, & Vomela, 2007). Notably, memory for previous fixations exceeds WM capacity in orderly search arrays, which is more consistent with a coarse representation of a search path than the use of WM to store rejected distractors (Dickinson & Zelinsky, 2007; Peterson et al., 2007). Alternatively, search might be guided to novel locations through an oculomotor mechanism that discourages refixations for a short amount of time (Wolfe, 2003). However, eye movements were restricted in our studies to avoid EEG artifacts, thus providing a measure of the covert mechanisms that guide search behavior and minimizing the influence of oculomotor mechanisms. Under these conditions, it is clear that memory for rejected distractors is much more limited than suggested by early models of visual search (e.g., Treisman & Gelade, 1980). At best, behavioral evidence points to a memory system that is very limited in capacity (McCarley et al., 2003), is easily disrupted (Klein & MacInnes, 1999; Takeda & Yagi, 2000), and does not appear to have a large impact on search performance (Horowitz & Wolfe, 1998). Our results add to this growing body of evidence that distractor maintenance likely plays a relatively minor role during visual search by demonstrating that it may be limited to situations with a high degree of feature overlap between targets and distractors, which is often not the case in more realistic search environments. In addition, the results of Experiment 2 provide converging evidence that distractor maintenance is restricted to only a few items during search, and WM capacity does not appear to be a limiting factor on search performance.

In previous analyses of the CDA during search, the proposed role of working memory during search has been inconsistent (Emrich et al., 2009; Luria & Vogel, 2011). Although both groups of researchers found that the CDA indexes distractors in WM, they differed in whether they interpreted this as a beneficial strategy during search. According to Emrich et al. (2009), greater WM capacity should be associated with better performance and more WM reliance during search. These individuals should be better able to maintain rejected distractors, resulting in fewer distractor reevaluations and more efficient search. In contrast, Luria and Vogel (2011) suggested that those with a greater WM capacity should rely less on WM during search because they are better able to filter out the distractors. In the current study, we found little evidence of a relationship between WM and visual search performance, and individuals with high and low WM capacity had very similar set size effects during search. Overall, our results suggest that although WM plays a role in visual search, it is likely not a limiting factor (see also Williams & Drew, 2018). Because of inconsistencies between these three studies, it is worth considering how methodological differences might account for these discrepancies. Our studies, as well as Luria and Vogel (2011), used circular search arrays, whereas the objects were randomly dispersed in Emrich et al. (2009). Orderly search arrays may further limit the utility of actively maintaining distractors in WM, thus diminishing the relationship between WM and task performance (Dickinson & Zelinsky, 2007; Peterson et al., 2007; Godwin et al., 2013). In addition, both Emrich et al. (2009) and Luria and Vogel (2011) used set sizes above typical WM capacity, and search processes may be fundamentally different at lower visual set sizes (Wolfe, 2007). It is currently unclear if distractor maintenance plays a role at smaller visual set sizes, but this possibility could account for our relatively limited evidence in favor of distractor maintenance in comparison to Emrich et al. (2009).

Although the results from Experiment 1 are clearly inconsistent with the idea of distractor maintenance for real-world stimuli, the finding that the CDA was largest in amplitude for the smallest set size merits further discussion. These results were unexpected, but a larger CDA amplitude for smaller set sizes is not without precedent. In a visual working memory task, Quirk, Adam, and Vogel (2020) found a larger CDA amplitude at Set Size 1 than Set Sizes 3 and 5 for complex, real-world objects but not for colored squares (see also, Vogel & Machizawa, 2004). These findings were attributed to WM reaching capacity at smaller set sizes for more complex, real-world objects, resulting in overloading effects at larger set sizes. Another possible explanation for this unexpected result is that search at set sizes below WM capacity might rely on a different strategy than search at higher set sizes. For example, multiple candidate items might be compared to the target template simultaneously when visual set size is below WM capacity. Finally, it is possible the CDA reflects additional processes occurring in WM during search beyond distractor maintenance. Existing models of visual search propose that target templates are maintained in WM for novel targets and that WM is the site of comparison between target templates and candidate search items (Bundesen, 1990; Desimone & Duncan, 1995; Duncan & Humphreys, 1989). In Experiment 1, the amplitude of the CDA was larger for target present than target-absent trials, which is consistent with the proposal that the CDA indexes the target verification process. It is possible that long-term memory processes (e.g., familiarity) can be recruited for real-world stimuli when the search task becomes sufficiently difficult, thus reducing the amount of WM resources needed for comparing candidate items with the target template (Drew, Williams, Jones, & Luria, 2018). For current purposes, it is clear this result is the opposite of what would be expected if the CDA indexed the number of rejected distractors in WM during search for real-world items. However, additional research will be necessary to investigate the underlying reasons for the reverse set size effect observed in Experiment 1.

Our studies corroborate prior evidence that the CDA, an index of WM activity, is present during visual search tasks (Emrich et al., 2009; Luria & Vogel, 2011). In previous research, the CDA was attributed to the maintenance of rejected distractors in WM. In our studies, we sought to test this idea more directly by manipulating the number of distractors in the search array. Although WM activity increased with the number of distractors during search for artificial stimuli, there was no evidence of distractor maintenance during search for real-world stimuli. Furthermore, the increase in CDA with the number of distractors in Experiment 2 plateaued at ~3–4 items, providing additional evidence that distractor maintenance is limited-capacity process. These findings provide additional insight on the circumstances that lead to the active maintenance of distractors in WM. In addition, this research contributes to a growing body of work highlighting differences in how cognitive resources are used between real world and artificial stimuli (e.g., Brady, Störmer, & Alvarez, 2016; Drew et al., 2018). More generally, we believe these results highlight the substantial potential of the CDA as a method for better understanding the role of WM during visual search.

Conclusion

Using the CDA as an online index of WM activity during a visual search task, we did not find evidence that rejected distractors are maintained in WM during visual search for real-world stimuli. WM activity was greater for target-present than for target-absent trials, and the CDA decreased in amplitude with the number of distractors present during the visual search task. Furthermore, neither WM capacity nor WM activity during search were predictive of task performance. In a follow-up study, we found partial support for distractor maintenance during search for artificial stimuli with greater feature overlap between targets and distractors. WM use increased with the number of distractors present in the search array until ~4 items. However, WM activity did not significantly differ between target-present and target-absent trials, and we did not see strong evidence in favor of a relationship between WM and task performance. Overall, these findings suggest that, despite the potential benefit for such a mechanism, distractor maintenance appears to play a relatively limited role in visual search.