Two fundamental issues in the study of visual attention are the control and the effects of attention. Attentional control refers to those mechanisms that are used to direct attention to items in the visual field. Theories of visual attention have proposed two control parameters: bottom-up (stimulus-driven) parameters, in which attention is captured by an environmental event, such as an abruptly appearing object or a uniquely colored item, and top-down (goal-driven) parameters, in which attention is directed in a voluntary manner, such as searching for a specific object in a cluttered scene (Theeuwes, 2010). In contrast, the effects of attention refer to how an attended stimulus is processed differently from an unattended stimulus. For example, the representation of an attended item could be enhanced relative to the representations of unattended items, or attending an item could integrate together the visual attributes of that stimulus, as in Treisman’s feature integration theory (Treisman, 1988; Treisman & Gelade, 1980).

In the present study, we focused on the confluence of attentional control and effects by investigating how the contents of visual working memory (VWM) affect information processing during visual search. Following the predictions of “biased competition” accounts of attention (e.g., Desimone & Duncan, 1995), a number of recent studies have demonstrated that objects held in VWM can influence attention, biasing it toward items that match those features. For example, in one of the earliest studies on memory-guided attention, Downing (2000) presented participants with a to-be-remembered face. After a delay, two faces appeared, one that was stored in memory and another, nonremembered face. A small target appeared on one of the two faces, and participants reported the identity of this target. Participants identified the target faster when it appeared on the face stored in memory than when it appeared on the other face, suggesting that memory guided the allocation of attention. Although Downing’s results might have been produced by strategies used by the participants (see Woodman & Luck, 2007), it has been demonstrated that objects in VWM influence attentional deployment in various ways (e.g., Emrich, Al-Aidroos, Pratt, & Ferber, 2010; Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005; Woodman & Luck, 2007).

Although the contents of VWM can control the allocation of attention, less is known about the effects of VWM on the processing of attended items. There are at least two possibilities suggested in the attention literature. A “perceptual enhancement” view predicts that targets sharing features with objects held in VWM are perceived more effectively than targets that do not (e.g., Carrasco, Ling, & Read, 2004; Luck, Hillyard, Mouloua, & Hawkins, 1996; Yeshurun & Carrasco, 1999). In contrast, an “uncertainty reduction” account predicts that the contents of VWM operate postperceptually to reduce the uncertainty of a target’s location or identity when these attributes are hard to discern or are highly variable, prioritizing attention toward items whose features match those held in memory (e.g., Eriksen & Spencer, 1969; Palmer, 1995; Shiu & Pashler, 1994).

Recently, Soto and colleagues (Soto, Wriglesworth, Bahrami-Balani, & Humphreys, 2010) advocated a perceptual enhancement effect of VWM on attentional processing. In this account, objects in visual memory increase the perceptibility of items that share features with the remembered objects. Using an adapted version of a typical memory-cuing task (e.g., Downing, 2000; Soto et al., 2005), participants remembered the color of a single box while searching displays containing two to four colored boxes, one of which contained a target line. To differentiate perceptual enhancement and uncertainty reduction accounts, Soto et al. (2010) used briefly presented, masked displays and measured performance using a signal detection measure of sensitivity (A'). Participants performed better on the target discrimination task (as evidenced by higher A') when the color of the box containing the target matched that held in working memory (valid trials) than when it did not (invalid trials). The authors interpreted these results as supporting a perceptual enhancement account of memory-driven cuing effects, because differences were observed in A', a measure of sensitivity, despite a number of manipulations designed to rule out nonperceptual effects.

Although there is convincing evidence for perceptual enhancement in spatial cuing tasks (e.g., Carrasco et al., 2004; Yeshurun & Carrasco, 1999), these studies have used rigorous methods to rule out nonenhancement effects. Unfortunately, the results from search tasks cannot provide unequivocal support for perceptual enhancement of features matching those stored in VWM, because search tasks include uncertainty regarding the target’s location (Palmer, 1994, 1995; Shiu & Pashler, 1994). For example, even though Soto et al. (2010) attempted to eliminate “competition for selection” (p. 448) as a factor in their results by reducing display size to two items, the mere presence of a distractor box in the display introduced uncertainty about the target’s location (see Soto et al., 2010, Exp. 5). Despite the fact that they observed no difference in the magnitudes of the cuing effect in the two- versus four-item displays, given the brief duration of the search displays and the use of valid versus invalid cues, one would not expect such a difference unless uncertainty were reduced entirely.

Thus, it is quite possible that the contents of memory acted to reduce the uncertainty of the target’s position by prioritizing the memory-matching item in visual search. In short, attention might be directed first toward items with features that match those held in memory, and only later to other items. Such attentional prioritization need not be an explicit strategy, but instead could operate automatically whenever the target’s location is unknown. This alternative account readily explains Soto et al.’s (2010) results: Targets would be attended earlier in valid trials than in invalid trials, reducing the deleterious effects of the mask more on the former than on the latter and producing both A' and RT effects. No mention of perceptual enhancement is needed to explain these results. Instead, valid cues may simply reduce the chances that a shift of attention is made to a distractor rather than to a target. Furthermore, using A' as a dependent measure does not ensure that any A' differences arise from perceptual processes. A' is a measure that can be applied to any (relevant) data, irrespective of the cognitive process(es) generating those data.

The uncertainty reduction account we propose has been used in the spatial cuing literature to account for attentional cuing effects. Shiu and Pashler (1994) asked participants to discriminate targets that were either validly, neutrally, or invalidly cued; the target was followed by either a single pattern mask at the target’s location or multiple pattern masks that appeared at all possible target locations. The single-mask condition features little uncertainty, because the target’s location is easy to determine; the multiple-mask condition has higher levels of uncertainty, because the multiple masks obscured the target’s location.

Shiu and Pashler (1994) found effects of spatial cuing on accuracy in the multiple-mask condition, but not in the single-mask condition, consistent with the notion that spatial cues operate to reduce uncertainty about the potential target location. Under high-uncertainty (multiple-mask) conditions, participants used the spatial cue to restrict processing to the cued location; when the target appeared at that location, target identification was more accurate than when the target appeared elsewhere. In short, valid cues reduced the probability that a nontarget location would be mistaken as the target location.

As discussed above, because previous studies never completely eliminated uncertainty regarding the target’s location, it is possible that during search the contents of VWM drive uncertainty reduction processes rather than perceptual enhancement. This leaves open the question of which effect, perceptual enhancement or uncertainty reduction, best characterizes attentional guidance from VWM representations. To investigate the effects of attentional guidance from VWM while circumventing the criticisms of the previous work noted above, we used a task identical to that used in Soto et al. (2010), but added a masking manipulation similar to that used by Shiu and Pashler (1994). Participants remembered the color of a box presented at fixation, and then searched for a tilted line contained in a box that either matched the item held in memory (valid trials) or did not (neutral trials). Neutral trials were included in order to assess evidence of perceptual enhancement; if information held in VWM enhances the perceptual representation of targets matching that information, then validly cued targets should show higher A' values than neutrally cued targets (Shiu & Pashler, 1994). Critically, the search displays contained either a single, masked item (always the target) or a target item presented along with three distractors, all of which were masked (a condition nearly identical to that used in the main experiment of Soto et al., 2010).

These conditions allow perceptual enhancement and uncertainty reduction effects to make different predictions: If information held in working memory drives perceptual enhancement, participants should show higher A' values on valid than on neutral trials regardless of set size. In contrast, uncertainty reduction predicts that VWM will have a minimal effect in the low-uncertainty single-item, single-mask condition. Instead, the contents of working memory should only affect A' in the high-uncertainty multiple-item, multiple-mask trials. Uncertainty reduction predicts higher A' values for valid trials over neutral trials only for the multiple-item, multiple-mask condition; no differences would be expected for the low-uncertainty, single-item, single-mask condition.

Experiment 1

Method

Participants

The participants were 18 University of Iowa undergraduates who participated for course credit.

Stimuli and procedure

The stimuli were presented on a Mac Mini computer using MATLAB and the Psychophysics Toolbox (Brainard, 1997) on a 17-in. CRT screen. Participants sat 60 cm from the screen and performed a working memory task interspersed with a visual search task. For the time course of a typical trial, see Fig. 1. A trial began with the presentation of a fixation point for 500 ms, followed by the presentation of a single, colored 1.7° × 1.7° box (the “cue”) for 500 ms. Participants were asked to remember the color of this box because they would be tested on it at the end of the trial. After a 200-ms blank interval, the search array was presented for 70 ms and consisted of either a single 1.7° × 1.7° box that always contained a 0.6° target line tilted 12° right or left (single-item, single-mask condition) or four identically sized boxes, one of which contained the target line while the others contained vertical distractor lines (multiple-item, multiple-mask condition). On half of the trials, the cue color matched the color of the target (valid trials), and on the other half, the color did not match any of the items in the array (neutral trials). Directly following the presentation of the search array, each location containing a box was masked with a random dot-pattern mask, with masks remaining on the screen until participants responded. Participants pressed the “z” key for left-tilted targets and the “x” key for right-tilted targets. In all conditions, boxes could appear at the corner(s) of an imaginary square centered on fixation, with boxes appearing at an eccentricity of 6° of visual angle. All stimuli were presented on a black background, and the colors used for the cue/search boxes were red, green, blue, yellow, magenta, and white; the target and distractor lines were gray.

Fig. 1
figure 1

Task diagram showing the time course of events for a typical trial

Full size image

Following the response to the search task, participants were shown a colored box that either did or did not match the one they were asked to remember at the beginning of the trial, and they used the “.” key to respond same color and the “/” key to respond different color, with accuracy emphasized over speed. Response hands for the search and memory tasks were counterbalanced across subjects. Cue validity (valid vs. neutral) varied across trials, and mask condition was blocked and counterbalanced; half of the participants performed the single-item, single-mask condition first, and the other half performed the multiple-item, multiple-mask condition first. The remaining blocks were then alternated until the experiment was completed (e.g., single, multiple, single, multiple). Participants performed a 24-trial practice block for each search condition, followed by two blocks of 48 trials in each of the search conditions, for a total of 192 experimental trials (96 in each condition).

Results

We computed A' values for each condition (Green & Swets, 1966; Zhang & Mueller, 2005), with left responses to left targets treated as “signal present” and left responses to right targets treated as “false alarms” (see Table 1). Trials in which participants responded incorrectly to the memory test were discarded, and memory accuracy was high (mean = 94% correct), with no significant difference in memory accuracy between the mask conditions, t(17) = 1.6, p > .13. A' values for each condition (Fig. 2) were submitted to a two-factor ANOVA with Mask Condition (single vs. multiple masks) and Validity (valid vs. neutral) as factors. We found no significant main effects (Fs < 1.1, ps > .30). However, we did find a significant Mask Condition x Validity interaction, indicating that the magnitude of the validity effect varied between the two mask conditions, F(1, 17) = 5.5, p = .03. In order to elaborate on this interaction, we performed planned comparisons on A' values in valid and neutral conditions for the two mask conditions separately. We found a significant difference between valid and neutral trials in the multiple-item, multiple-mask condition, t(17) = 3.2, p < .001. Importantly, there was no difference between valid and neutral trials in the single-item, single-mask condition, t(17) < 1, n.s., consistent with the predictions of an uncertainty reduction account.

Table 1 Accuracy and probability of hits and false alarms across conditions
Full size table
Fig. 2
figure 2

Mean A' values for each condition in Experiment 1. Error bars represent 95% confidence intervals (Cousineau, 2005; Loftus & Masson, 1994)

Full size image

Reaction time (RT) data were submitted to an identical two-factor ANOVA. The RT data paralleled the A' data: The main effect of neither mask condition, F(1, 17) < 1, n.s., nor validity, F(1, 17) = 3.5, p = .08, reached significance, but a significant Mask Condition x Validity interaction was observed, F(1, 17) = 4.3, p = .05. Planned comparisons revealed a significant validity effect (valid RT = 952 ms, neutral RT = 1,024 ms) in the multiple-mask condition, t(17) = 2.7, p = .02, but not in the single-mask condition (valid RT = 994 ms, neutral RT = 1,009 ms), t(17) < 1, n.s.

These data support an uncertainty reduction account of memory-based cuing effects, with the contents of VWM only affecting target detection when there are multiple, competing objects in the search array. However, it is possible that blocking the mask conditions led observers to adopt block-wide strategies, obscuring any effect of the contents of VWM on target detection in the single-mask condition. To this end, in Experiment 2 we used a design identical to that in the present experiment, but intermixed single- and multiple-mask trials in order to decrease the likelihood that participants would adopt such block-wide strategies.

Experiment 2

Method

Participants

The participants were 15 University of Iowa undergraduates, all of whom participated for course credit.

Stimuli and procedure

The present experiment was identical to Experiment 1, except that mask types were intermixed within blocks. As in Experiment 1, participants completed 48 practice trials total, followed by four blocks of 48 trials each, for a total of 192 experimental trials (96 in each condition).

Results

The data were analyzed in a manner identical to that for Experiment 1. As in that earlier experiment, memory accuracy was high (mean = 92% correct), with no significant difference in memory accuracy between the mask conditions, t(14) < 1. A' values for each condition (Fig. 3) were submitted to a two-factor ANOVA with Mask Condition (single vs. multiple masks) and Validity (valid vs. neutral) as factors. We found significant main effects of both mask condition, F(1, 14) = 17.7, p < .001, and validity, F(1, 14) = 5.4, p = .04. Importantly, we observed a significant Mask Condition x Validity interaction, indicating that the magnitude of the validity effect varied between the two mask conditions, F(1, 14) = 8.2, p = .01. As in Experiment 1, planned comparisons showed a significant difference between valid and neutral trials in the multiple-item, multiple-mask condition, t(14) = 2.7, p = .01, but no such difference in the single-item, single-mask condition, t(14) < 1, n.s.; these results were again consistent with the predictions of an uncertainty reduction account.

Fig. 3
figure 3

Mean A' values for each condition in Experiment 2. Error bars represent 95% confidence intervals (Cousineau, 2005; Loftus & Masson, 1994)

Full size image

We analyzed RTs with a two-factor ANOVA identical to the one above. These data paralleled the A' data: We observed main effects of mask condition, F(1, 14) = 23.0, p < .0001, and validity, F(1, 14) = 10.7, p < .01, and a significant Mask Condition x Validity interaction was observed, F(1, 14) = 8.8, p < .01. As in the A' data, planned comparisons revealed a significant validity effect (valid RT = 866 ms, neutral RT = 942 ms) in the multiple-mask condition, t(14) = 4.6, p < .001, but not in the single-mask condition (valid RT = 807 ms, neutral RT = 815 ms), t(14) < 1, n.s.

Ceiling effect analysis

Although the evidence above supports an uncertainty reduction account, it is possible that the lack of a validity effect in the single-mask condition was due to a ceiling effect in the single-mask condition. To rule out this possibility, we performed supplementary analyses on Experiments 1 and 2. First, we analyzed detection performance from Experiments 1 and 2 with a mixed ANOVA, with experiment as a between-subjects factor and mask condition (single vs. multiple masks) and validity (valid vs. neutral) as within-subjects factors. This analysis showed no main effect of experiment, F(1, 31) < 1, and no three-way interaction, F(1, 31) < 1, indicating that neither overall performance nor the mask condition x validity interaction differed across experiments.

With these results in hand, we combined the data from Experiments 1 and 2 and performed a median split on the basis of overall accuracy in the single-mask condition, allowing us to examine the interaction between mask condition and validity separately in participants with accuracy either near or well below ceiling. Our median split resulted in two groups, with overall accuracy of 95% in the high-accuracy group (\( \overline {\hbox{x}} \) single mask = 97%, \( \overline {\hbox{x}} \) multiple masks = 92%) and 80% in the low-accuracy group (\( \overline {\hbox{x}} \) single mask = 82%, \( \overline {\hbox{x}} \) multiple masks = 79%). On the basis of this median split, we calculated A' scores for each group and performed separate two-factor within-subjects ANOVAs with Mask Condition (single vs. multiple masks) and Validity (valid vs. neutral) as factors. Importantly, the data from the low-accuracy group (Fig. 4) showed a significant Mask Condition x Validity interaction, F(1, 15) = 5.8, p = .03. Additionally, even the high-accuracy group, with overall accuracy near ceiling (95%), showed a trend toward such an interaction, F(1, 16) = 3.5, p = .08. Taken together, these results argue against ceiling effects in the single-mask condition as an alternative explanation of our results.

Fig. 4
figure 4

Mean A' values for each condition in participants who showed the lowest accuracy in the single-mask condition of Experiments 1 and 2 (see the Ceiling Effect Analysis section). Error bars represent 95% confidence intervals (Cousineau, 2005; Loftus & Masson, 1994)

Full size image

General discussion

The present results suggest that features held in VWM serve to reduce uncertainty in visual search by prioritizing the memory-matching item, rather than enhancing its perceptual representation. In cases where the target location is known with certainty, the contents of VWM do not affect sensitivity to the target as measured by A'. These results parallel those in the spatial attention literature, arguing that VWM representations can affect search performance through postperceptual mechanisms that serve to reduce uncertainty regarding the location of task-relevant information in the environment (Dosher & Lu, 2000; Palmer, 1994, 1995; Shiu & Pashler, 1994).

Furthermore, an important point related to our data is that signal detection measures alone are not sufficient to differentiate enhancement and uncertainty reduction accounts of attentional processing. Instead, these measures must be coupled with an experimental design that eliminates target uncertainty in order to unequivocally provide information regarding the locus of attentional effects. That said, we neither wish to argue that the contents of VWM can’t alter the perceptual processing of visual information, nor do we feel that uncertainty reduction and enhancement accounts are mutually exclusive. For example, when there are multiple items in a search array, these items compete for processing resources, and top-down biasing mechanisms determine the strength with which attended and unattended information is processed (e.g., Desimone & Duncan, 1995; Kastner, de Weerd, Desimone, & Ungerleider, 1998; Reynolds, Chelazzi, & Desimone, 1999). These top-down mechanisms may, for example, enhance the perceptual processing of relevant items, suppress the processing of irrelevant items, or reduce uncertainty regarding the possible target location or identity. However, in a case where only a single item is present, these mechanisms may not be obligatorily recruited, and as a result one may expect little effect of top-down factors on perceptual analysis; bottom-up information from the single target item may be sufficient to preclude the operation of top-down control.

Thus, we argue that to date there is little evidence that representations held in VWM obligatorily affect the perceptual processing of the search target, with our evidence favoring an uncertainty reduction account. Such an interpretation is in line with those of previous studies examining both feature-based (Moore & Egeth, 1998; Theeuwes & Van der Burg, 2007) and spatial (Dosher & Lu, 2000; Shiu & Pashler, 1994) attention, and we argue that information held in VWM can reduce uncertainty regarding a target’s location during visual search without altering its perceptibility.