In search of the focus of attention in working memory: 13 years of the retro-cue effect
- 4.6k Downloads
The concept of attention has a prominent place in cognitive psychology. Attention can be directed not only to perceptual information, but also to information in working memory (WM). Evidence for an internal focus of attention has come from the retro-cue effect: Performance in tests of visual WM is improved when attention is guided to the test-relevant contents of WM ahead of testing them. The retro-cue paradigm has served as a test bed to empirically investigate the functions and limits of the focus of attention in WM. In this article, we review the growing body of (behavioral) studies on the retro-cue effect. We evaluate the degrees of experimental support for six hypotheses about what causes the retro-cue effect: (1) Attention protects representations from decay, (2) attention prioritizes the selected WM contents for comparison with a probe display, (3) attended representations are strengthened in WM, (4) not-attended representations are removed from WM, (5) a retro-cue to the retrieval target provides a head start for its retrieval before decision making, and (6) attention protects the selected representation from perceptual interference. The extant evidence provides support for the last four of these hypotheses.
KeywordsAttention Retro-cue Working memory
An information-processing system faces two big challenges: the creation of processing priorities among relevant pieces of information, and the selection of relevant input amidst irrelevant ones. Attention is assumed to serve both of these functions for our cognitive system (Carrasco, 2011; Yantis, 2008). Attention can be directed both to information coming through the senses, and to information generated or maintained internally in the absence of corresponding perceptual input (Chun, Golomb, & Turk-Browne, 2011). Notwithstanding the theoretical relevance of attention in both domains, attentional effects are far better understood in the perceptual domain (see the reviews by Carrasco, 2011; Egeth & Yantis, 1997; Serences & Yantis, 2006) than in the realm of internal information.
In the present review, we aim to bring to focus the research on how attention affects internal information, and in particular, information in working memory (WM). WM is the system enabling online maintenance of representations for ongoing cognition. Accordingly, performance in WM tasks is associated with several measures of complex cognition, such as reading comprehension, reasoning, intelligence, and educational outcomes (e.g., Bull, Espy, & Wiebe, 2008; Conway, Kane, & Engle, 2003; Daneman & Carpenter, 1980; Süß, Oberauer, Wittmann, Wilhelm, & Schulze, 2002). Despite its central role for higher cognition, WM storage capacity is severely limited: Only a few representations at a time can be maintained accessible (Cowan, 2010). Moreover, representations in WM are not all in the same state of accessibility, and not all are equally relevant for the current task goals (Oberauer & Hein, 2012; Olivers, Peters, Houtkamp, & Roelfsema, 2011). Investigating how an attentional set can be applied to the representations actively maintained in mind can help shed light onto the control mechanisms of WM and how its contents can be flexibly updated, strengthened, and/or discarded when information about their relevance becomes available (Tamber-Rosenau, Esterman, Chiu, & Yantis, 2011).
This special issue honors Steve Yantis and his many contributions to the understanding of the mechanisms of the control of attention. In one of his more recent publications, Yantis and collaborators (Tamber-Rosenau et al., 2011) timely warned us that we are still far from fully understanding how the selection of representations in WM takes place and how it relates to the selection of perceptual representations (see also Yantis, 2008). This is not a hopeless state; this gap is being gradually narrowed down, and several experimental paradigms are available for probing the mechanisms of attentional selection in WM (for a review, see Oberauer & Hein, 2012). Here we focus on one of the approaches that has been particularly fruitful in recent years: the retro-cue paradigm.
This review of the retro-cue effect is structured in four sections. In the first section, we describe the retro-cue effect and expand on why this effect has attracted considerable interest in the past 13 years. We then move on to describe the many procedures used to measure retro-cue effects, providing a quick overview of the implementations of the retro-cue paradigm in different visual WM tasks. In the second section, we review five sets of findings characterizing how internal attention works in the retro-cue paradigm. In the third section, we march through six hypotheses that have been proposed to explain the retro-cue effect and evaluate them in light of the available evidence. Finally, we conclude this review with a discussion of potential future directions to inspire developments in the field.
The retro-cue effect
The retro-cue effect was first described by two independent groups of researchers: Landman, Spekreijse, and Lamme (2003) and Griffin and Nobre (2003). The retro-cue paradigm is in many regards similar to the classical Posner cueing paradigm (Posner, 1980) used to investigate the orienting of attention in the perceptual domain. In the Posner cueing task, a spatial cue guides attention to the location where a relevant stimulus will occur, thereby enhancing detection and processing of the stimulus, as revealed by faster and more accurate responses to the stimulus at the cued location. In the retro-cue paradigm, a spatial cue (called a retro-cue, due to its retroactive or retrospective effect) is shown during the retention interval of a visual WM task. The retro-cue guides attention to one of the representations in WM, thereby tagging it as the most relevant one for an upcoming memory test.1 The retro-cue correctly indicates the to-be-tested item in an above-chance proportion of trials, referred to as the cue reliability. In line with the cueing advantages observed in perceptual tasks, retro-cues improve performance in tests of the validly cued item, whereas costs are observed for tests of one of the noncued items—henceforth, invalid retro-cue trials (Griffin & Nobre, 2003; Landman et al., 2003). These effects are not due to modulation of the response to a preferred perceptual stimulus over others (since no perceptual input is present), and no eye movement artifacts (e.g., Matsukura, Luck, & Vecera, 2007) or eccentricity effects (Tanoue & Berryhill, 2012) have been found to explain it.
Since its first description in 2003, there has been much research interest in the retro-cue effect. Retro-cues in visual WM paradigms are a versatile tool for making the concept of a focus of attention in WM (Oberauer & Hein, 2012) empirically tractable. Guiding internal attention through a retro-cue enables researchers to investigate how the information in WM is selected for processing, which information can be selected, how it remains selected, and what consequences attentional selection has for the maintenance of information in WM. Finally, the retro-cue effect indicates that more information can be extracted from WM than what is measured with standard test procedures, hence showing that previously assumed fixed capacity limitations may be due less to the rigid structure of WM and more to how the information in WM is accessed. In a nutshell, the study of the retro-cue effect provides important clues in understanding the structure and functions of WM, and on how capacity limitations can be mitigated by efficiently managing information to match the current task goals.
In the present review, our goal is to synthesize the growing body of research on the retro-cue. Our main focus is on behavioral studies, but we will refer to brain data inasmuch as they help to adjudicate between competing cognitive theories of the retro-cue effect. For the reader interested in the brain profiles associated with directing attention to internal representations in comparison to orienting to perceptual ones, we refer to the recent review by Gazzaley and Nobre (2012).
Measuring the benefits and costs of retro-cues
The retro-cue has remained, for the most part, an effect investigated in visual WM. Most studies have used simple visual stimuli such as colors (e.g., Griffin & Nobre, 2003) or oriented bars (Landman et al., 2003), following the footsteps of the authors first demonstrating this effect. However, there are reports of retro-cue effects with more complex visual stimuli, such as alphanumeric or horoscope symbols (Sligte, Scholte, & Lamme, 2008), shape–color conjunctions (Delvenne, Cleeremans, & Laloyaux, 2010; Gilchrist, Duarte, & Verhaeghen, 2016; Li & Saiki, 2015), colored line-drawing of objects (Astle, Nobre, & Scerif, 2012; Sligte, Vandenbroucke, Scholte, & Lamme, 2010), object drawings with 3-D details (Beck & van Lamsweerde, 2011), real-world pictures of objects (LaRocque et al., 2015), visual illusions (Vandenbroucke, Sligte, Fahrenfort, Ambroziak, & Lamme, 2012), and pictures of faces or scenes (Lepsien & Nobre, 2007; Lepsien, Thornton, & Nobre, 2011). There is evidence that retro-cues modulate access to auditory (Backer & Alain, 2012; Backer, Binns, & Alain, 2015), tactile (Katus, Andersen, & Müller, 2012a, b, 2014), and verbal stimuli in WM (Oberauer, 2001, 2002, 2003, 2005). However, given the more prominent role of retro-cues in the visual domain, we will limit this review to studies using visual stimuli.
All of the above-mentioned tasks offer a binary measure of memory performance: The participant responds either correctly (i.e., a hit or correct rejection) or incorrectly (i.e., a miss or a false alarm). Accordingly, accuracy in these tasks has been analyzed using measures such as proportions of correct responses, d′, or an estimate of capacity (K) derived from hit and false-alarm rates (Rouder et al., 2011). Some studies have also assessed the speed of access to WM’s contents, as measured by the reaction times (RTs) on correct trials. For both accuracy (no matter the specific measure) and RTs, retro-cue benefits and costs have been consistently reported (see Gressmann & Janczyk, 2016, for an overview of benefits and costs).
More recently, tasks have been developed that allow measurement of the fidelity with which an item is remembered in WM (Prinzmetal, Amiri, Allen, & Edwards, 1998; Wilken & Ma, 2004; Zhang & Luck, 2008). In the continuous delayed-estimation task, participants are asked to reconstruct the feature of a target item to match the feature stored in memory (see Fig. 1A) using a continuous scale (e.g., a color wheel, or a response dial to rotate an item). The dependent measure in this type of task is response error, measured as the distance between the target item’s true feature and the one reported by the participant. The closer the mean response error is to zero, the higher the fidelity of the representation in memory. The distribution of response errors can also be submitted to modeling to extract parameters related to the probability of having recalled the target item (as opposed to guessing or recalling one of the nontarget items) and the variability (precision) with which features from memory are recalled (Bays, Catalao, & Husain, 2009; Zhang & Luck, 2008).
One final task variant that also offers a measure of WM fidelity is the so-called discrimination-of-change task (Bays & Husain, 2008). In this task, a test display is shown containing a single probe that is always changed in relation to the corresponding memory item (e.g., a probe arrow displaced 20° to the left; see Fig. 1A). Participants have to judge in which direction (e.g., left or right) the probe stimulus has changed. The degree of displacement is varied, allowing researchers to measure precision as the slope of the psychometric function relating change identification accuracy to the size of a change, and the probability of reporting information from memory (as opposed to guessing) as the asymptote of the psychometric function at large sizes of change (Bays & Husain, 2008; Murray, Nobre, Clark, Cravo, & Stokes, 2013).
The latter two tasks—delayed estimation and change discrimination—have the advantage of allowing researchers to assess which memory parameter, as estimated by a measurement model, is affected by the presentation of a retro-cue (i.e., probability of recall vs. precision). So far, all studies assessing retro-cue effects in continuous tasks have consistently reported benefits in the probability of retrieving the retro-cued item (Gunseli, van Moorselaar, Meeter, & Olivers, 2015; Makovski & Pertzov, 2015; Murray et al., 2013; Souza, manuscript submitted for publication; Souza, Rerko, Lin, & Oberauer, 2014; Souza, Rerko, & Oberauer, 2016; van Moorselaar, Gunseli, Theeuwes, & Olivers, 2015; Wallis, Stokes, Cousijn, Woolrich, & Nobre, 2015; Williams, Hong, Kang, Carlisle, & Woodman, 2013). Regarding the precision of recall as measured by the mixture model, the findings are mixed. Only about half of the above-mentioned studies observed benefits in this parameter (Gunseli et al., 2015; Makovski & Pertzov, 2015; Souza, manuscript submitted for publication; van Moorselaar, Gunseli, et al., 2015; Wallis et al., 2015; Williams et al., 2013). At present, it is not clear why some studies find effects on precision, whereas others don’t. One pattern that seems to emerge from these studies is that effects on precision tend to appear when memory load is low (i.e., less than four items).2
The typical implementation of a retro-cue trial in a WM task comprises presentation during the retention interval of a central arrow cue or a peripheral spatial cue (see the examples in Fig. 1A). Central and peripheral cues are equally effective in producing retro-cue benefits (Matsukura, Cosman, Zachary, Vatterott, & Vecera, 2014). Despite the ubiquitous use of spatial retro-cues in the literature, there is more to the retro-cue effect than spatial selection. Some studies have used color or shape retro-cues (Heuer & Schubö, 2016; Li & Saiki, 2015; Pertzov, Bays, Joseph, & Husain, 2013), hence engaging feature-based attention to the cued item(s). There is also evidence of effective selection using semantic cues (Backer et al., 2015), or even verbal retro-cues (Gilchrist et al., 2016; Hollingworth & Maxcey-Richard, 2013).
Typically, retro-cues are presented after a retention interval of at least 0.5 s, to guarantee that the cue is operating outside of the time range of iconic memory (Irwin & Thomas, 2008; Sperling, 1960). Retro-cue benefits have been consistently reported when the cue appears between 1 and 5.5 s (Sligte et al., 2008) and up to 9.6 s (Astle, Nobre, & Scerif, 2012) after memory array offset, demonstrating that they cannot be reduced to selective readout from iconic memory.
Retro-cue trials have been compared to different baselines, as is shown in Fig. 1B.3 The first possibility is to use no-cue trials. No-cue trials can be implemented in two ways: Either the total retention interval is matched to the one used in the retro-cue trial (no-cue long), or the retention interval is reduced to match the time until presentation of the retro-cue in retro-cue trials (no-cue short). The latter condition has the advantage of controlling for time-based forgetting up until the presentation of the retro-cue.
Another popular choice is to use neutral-cue trials. Neutral cues are presented at the same point in time as retro-cues, but unlike the latter, these cues are uninformative, hence not guiding the allocation of attention (see the top of Fig. 1A). This baseline is usually preferred in studies measuring neural markers of attentional control (e.g., Griffin & Nobre, 2003; Kuo, Stokes, & Nobre, 2012; Lepsien, Griffin, Devlin, & Nobre, 2005; Lepsien et al., 2011; Myers, Walther, Wallis, Stokes, & Nobre, 2015; Nobre, Griffin, & Rao, 2008), because it equates the retro-cue and baseline trials in terms of the responses to perceptual stimulation. Studies that have included both no-cue and neutral-cue trials (Makovski, Sussman, & Jiang, 2008; Murray et al., 2013) have observed that performance does not differ between these conditions, provided that they are matched regarding overall retention interval (i.e., no-cue long vs. neutral cue).
Retro-cue trials can also be compared to simultaneous-cue trials (simu-cues, for short). Simu-cues are usually used to restrict the decision processes to one candidate probe stimulus in a change detection test array (e.g., Luck & Vogel, 1997; Wheeler & Treisman, 2002), or even in local-recognition tasks (Makovski et al., 2008). Simu-cues have sometimes been found to yield performance similar to that in no-cue trials (Luck & Vogel, 1997; Wheeler & Treisman, 2002). However, there is some evidence that simu-cues improve performance relative to no-cue trials by reducing the number of comparisons in memory when the stimuli are complex (e.g., Beck & van Lamsweerde, 2011; Hollingworth, 2003; Makovski et al., 2008). Finally, retro-cue trials have occasionally been compared to postcues—that is, cues presented after the presentation of the test display. Similarly to simu-cues, postcues could potentially be used to narrow down the comparison process to the cued item in change detection, and hence may yield some improvement in comparison to no-cue trials (Beck & van Lamsweerde, 2011; Landman, Spekreijse, & Lamme, 2004).
The empirical characteristics of retro-cue effects
The retro-cue effect is thought to arise due to the allocation of attention to the cued item. In this section, we briefly review five lines of investigation speaking to how attention is guided by a retro-cue.
How much time does one need to make effective use of a retro-cue? A few studies have addressed this question by varying the temporal separation between the retro-cue and onset of the test display (hereafter, the cue–test delay). Tanoue and Berryhill (2012) varied the cue–test delay from 100 to 700 ms in a local-recognition task. Significant retro-cue benefits emerged after 300 ms, and they did not increase in size with longer cue–test delays. Souza, Rerko, and Oberauer (2014) also varied the cue–test delay (100, 400, 1,000, or 2,000 ms) in local recognition (see also Gressmann & Janczyk, 2016). Retro-cue benefits for accuracy were stable after a delay of 400 ms, but benefits for RTs continued to increase up to 1,000 ms. Studies varying cue–test delay in delayed-estimation tasks (Pertzov et al., 2013; Souza et al., 2016) have also shown retro-cue effects for delays of 300 ms or more.4 Some studies have also assessed how long it takes to achieve a stable retro-cue benefit by varying the stimulus onset asynchrony (SOA) between the offset of the retro-cue and presentation of a visual mask (Pinto, Sligte, Shapiro, & Lamme, 2013; van Moorselaar, Gunseli, et al., 2015). The visual mask diminished the retro-cue benefit with SOAs up to 300 ms, further supporting this interval as the time needed to observe the full retro-cue benefit.
In sum, people need at least 300–500 ms to make full use of the retro-cue to improve the accuracy of their responses, with performance remaining stable thereafter. The speed of processing, however, can continue to improve up to 1,000 ms after the retro-cue.
A second question concerns the voluntary control of the allocation of attention to the retro-cue. Some studies have investigated whether the retro-cue triggers an automatic shift of attention to the cued location or whether this shift is voluntary and dependent on the informational value of the cue—a distinction much like that between stimulus-driven versus goal-directed perceptual attention (Yantis, 2000). The degree of voluntary control has been measured through manipulations of cue reliability—that is, the proportion of trials on which the cue validly indicates the test item.5
Gunseli, van Moorselaar, Meeter, and Olivers (2015) compared retro-cue benefits and costs in an orientation delayed-estimation task with cue reliabilities of 50% and 80% (random selection of the target of recall would yield a 25% reliability). In both conditions, retro-cue benefits were observed, but these benefits were larger under the higher-reliability condition. Invalid retro-cue costs, conversely, showed up only in the high-reliability condition. These results seem to indicate that people use the retro-cue differently on the basis of how predictive it is of the to-be-tested item. Berryhill, Richmond, Shay, and Olson (2012) compared 100%-reliable to completely uninformative retro-cues. Presenting a retro-cue improved testing of the retro-cued item even when this cue was uninformative, but the retro-cue effect was smaller in the nonpredictive condition.6 Shimi, Nobre, Astle, and Scerif (2014) compared the performances of different groups of participants exposed to retro-cues with 100% or 50% reliability. Retro-cue benefits were reduced under the low-reliability condition, being only significant when compared against invalid-cue trials. In sum, retro-cue effects (benefits and costs) depend on the reliability of the retro-cue, indicating that people use it strategically to modulate WM maintenance. When retro-cues are mostly valid, noncued items become irrelevant and may be worth discarding to reduce memory load, even though that incurs a performance cost on the rare invalid-cue trials. When retro-cues are informative but have low reliability, removing noncued items is not a good strategy, but attention can still be used to strengthen the retro-cued item, thereby generating a benefit for valid-cue trials with little cost for invalid-cue trials (Gunseli et al., 2015).
Resistance to distraction
A third line of investigation has assessed whether attention needs to remain focused on the retro-cued item until test to obtain a retro-cue benefit. Hollingworth and Maxcey-Richard (2013) tested whether the retro-cue requires sustained spatial visual attention to the cued item by introducing, in some trials, a visual search task after presentation of either a neutral cue or a retro-cue. The visual search task was meant to prevent spatial attention from being persistently focused on the retro-cued item. Although the visual search task impaired memory, it did not reduce the retro-cue benefit. Rerko, Souza, and Oberauer (2014) applied a similar logic to test the contribution of feature-based attention. The interruption task in this study required participants to make a perceptual judgment (warm–cold color decision) on a centrally displayed stimulus during the retention interval (presented after the retro-cue in retro-cue trials). Again, the interruption task impaired overall memory performance but did not reduce the retro-cue benefit. In a second step, Rerko et al. also tested whether the retro-cue benefit is robust against distraction from the focus of attention within WM: After retro-cueing an item A, participants were first tested on another item B before being tested on the cued item A. Because retrieving item B required selecting B at the exclusion of all other items in the memory set, the focus of attention could not remain on A continuously until it was tested. Yet the retro-cue benefit for item A was undiminished by the intervening retrieval of another item from WM.
The maintenance of retro-cue benefits after distraction from the focus of attention has also been observed in the delayed-estimation task. Makovski and Pertzov (2015) inserted a parity decision on visually or auditorily presented digits after the presentation of a cue (retro-cue or neutral cue). The parity task (in both visual and auditory forms) impaired performance in both cue conditions, but less so in the retro-cue condition, consistent with the idea that the retro-cueing benefit survives distraction of central attention.
This conclusion is further supported by a study measuring the neural substrate of attention after presentation of a retro-cue. Wallis et al. (2015) tracked alpha-power lateralization, which is thought to index the deployment of attention, during the time following a retro-cue. They observed that alpha lateralization occurs only transiently (for about 500 ms) after the retro-cue, suggesting that attention does not persistently linger on the retro-cued item, even when its focus is not distracted by a secondary task. These findings also converge with the time course of obtaining a retro-cue benefit, which seems to require no more than 500 ms.
In a related vein, Zokaei, Ning, Manohar, Feredoes, and Husain (2014) investigated whether the maintenance of a retro-cued item could be disrupted by the application of transcranial magnetic stimulation (TMS) to areas involved in the sensory perception of the stimulus that is currently maintained in WM. Zokaei et al. asked participants to maintain the motion direction of two dot patterns. After presenting a retro-cue, TMS was applied to motion-sensitive area MT+. The TMS impaired performance overall, but the retro-cue benefit was not reduced by it. This finding contrasts with the observation that stimulation of MT+ has been found to disrupt other effects that have been thought to reflect attention in WM, such as the recency effect and an incidental cueing benefit (Zokaei, Manohar, Husain, & Feredoes, 2014).
Taken together, the results above establish that the retro-cue benefit is robust against distraction of spatial or feature-based attention, shifts of attention within WM, and disruption of maintenance through TMS over stimulus-relevant sensory areas. These findings converge with other studies that have observed that one item can be retained in a privileged state (of heightened accessibility) even when other items are cued between the initial cueing and testing (Rerko & Oberauer, 2013; Souza, Rerko, & Oberauer, 2015) or are subsequently encoded into WM (Gorgoraptis, Catalao, Bays, & Husain, 2011; Maxcey-Richard & Hollingworth, 2013; Souza, Rerko, & Oberauer, 2014). In sum, focused attention can be used to prioritize one WM item in a manner that is robust against distractions from the external or internal focus of attention.
Central attention demands
Some recent evidence has been provided to bear on the question of the attentional demands of using the retro-cue. Janczyk and Berryhill (2014) assessed whether there is a critical period in which central attention to the retro-cued item is required, by measuring dual-task interference using the psychological refractory period (PRP) paradigm. In the absence of perceptual and motor overlap between the two tasks, dual-task costs in the PRP paradigm reflect competition for central attention, conceptualized as a general processing bottleneck (Pashler, 1994) or resource allocated (Tombu & Jolicœur, 2003) for processes such as response selection and retrieval from long-term memory. Janczyk and Berryhill inserted a tone discrimination task during the retention interval of a local-recognition task and varied the temporal separation (SOA) between presentation of the tone and subsequent presentation of a cue (retro-cue or neutral-cue). If central attention is required temporarily for using the retro-cue, then short SOAs should create a situation of competition for central attention, leading to smaller retro-cue benefits than would be observed at longer SOAs. This is exactly what they observed when comparing SOAs of 150 and 850 ms, showing therefore that central attention is required over a critical period to make use of the retro-cue.
Splitting attention between multiple WM items
Retro-cues can be used to guide attention to one single item or to multiple items in WM. To date, there is still dispute whether simultaneously retro-cueing multiple items (hence, requiring concurrent selection of several representations in WM) leads to an advantage for memory performance relative to a no-cue baseline. Makovski and Jiang (2007) compared the efficacy of one-cue, two-cue, and three-cue displays in yielding a retro-cue benefit. Only single retro-cues improved performance in their study. Delvenne and Holt (2012) showed evidence of two-cue benefits, provided that the cues pointed to different items across left–right hemifields.7 Other studies have used cues pointing to entire memory sets (set cues), such as cues indicating all of the items on the left or the right side of the screen as being relevant (Matsukura et al., 2007; Williams & Woodman, 2012), which were successful in improving performance. Recently, Matsukura and Vecera (2015) tested the boundary conditions for observing benefits yielded by simultaneously retro-cueing multiple items. These authors compared single cues, muti-item cues (cueing a random subset of three out of six items), and set cues (cueing groups of three contiguous items). Critically, in this study, the testing procedure was varied, testing a single item from the retro-cued set (i.e., a local-recognition test) or testing the entire set of retro-cued items (i.e., a change detection test on the cued subset of the array). Multi-item cues and set cues only benefited performance when the entire set was tested, suggesting that the test display has to match the attentional set created by the retro-cue. Yet another recent study conducted by Heuer and Schubö (2016) has opened up more possible alternative explanations for these inconsistencies. In their study, they cued two items from a memory array simultaneously using either spatial cues (arrows), symbolic cues (numbers mapped to locations), or feature-based cues (color or shape). Moreover, they also varied the spatial distance between the cued items, such that the items could be from neighboring or nonneighboring positions. Spatial and symbolic cues only improved performance relative to a neutral-cue condition when neighboring locations were cued. In contrast, feature-based cues yielded benefits, irrespective of the spatial distance between cued items. These findings suggest that the selection of multiple items simultaneously may be contingent on how different forms of attention (spatial or feature-based) are engaged by the retro-cue.
Whereas the use of simultaneous multi-item cues has yielded a mixed pattern, sequential presentation of multiple retro-cues has shown that participants can shift attention between items in WM. When a second retro-cue follows the first, and the last-cued item is the one relevant to the test, a retro-cue benefit is observed for the second-cued item (Landman et al., 2003; Li & Saiki, 2014; Maxcey, Fukuda, Song, & Woodman, 2015; Rerko & Oberauer, 2013). When all of the sequentially cued items are relevant, improvements have been reported for all cued items (Li & Saiki, 2014; Souza et al., 2015), and this improvement is larger, the more frequently the same item is cued during the retention interval (Souza et al., 2015). There is only one exception to this pattern, revealed in a study by van Moorselaar, Olivers, Theeuwes, Lamme, and Sligte (2015). In this study, a second retro-cue did not bring performance above baseline, but it did counteract the costs otherwise incurred by an invalid first retro-cue. One factor that may explain these discrepant results is the frequency of two-cue trials, which were much less frequent than single-cue trials in van Moorselaar, Olivers, et al. These results therefore suggest that the strategic use of retro-cues is not only sensitive to cue reliability, but also to the frequency with which a first cue is followed by a second one.
In a nutshell, the studies reviewed above show that the retro-cue induces a voluntary shift of (central) attention to the cued item for about 300–500 ms. After this period, the retro-cued item reaches a robust, privileged state in WM. This state involves more than remaining selected within the focus of attention, because distracting attention from the cued object by requiring the selection of a response to a perceptual stimulus, or selection of another item within WM, does not diminish the retro-cue benefit.
The emergent picture is that a state of robustness can be achieved for multiple items in WM, provided that they are focused on sequentially. Focusing of multiple items simultaneously may also provide a benefit, but the boundary conditions for that effect are not yet firmly established.
Explanations of the retro-cue effect
The retro-cue effect is puzzling to explain from a pure informational standpoint. Consider a task in which a single item is tested (e.g., in local recognition or change discrimination). In the retro-cue condition, a retro-cue at time t 1 only provides one piece of information: It indicates which item will be tested. All further information is delayed until the test display is presented at a later time t 2. In contrast, in a no-cue short condition (see Fig. 1), the memory test instead occurs at time t 1. The test display contains more information than the retro-cue: It identifies which item is tested, provides the probe to compare that item to, and allows decision making to start. Yet, robust retro-cue benefits have been found in comparison to a no-cue short condition (Makovski et al., 2008; Murray et al., 2013; Rerko et al., 2014; Souza, Rerko, Lin, et al., 2014; Souza et al., 2015; Souza, Rerko, & Oberauer, 2014). How can less information be helpful? Intriguingly, people can use the retro-cue to access and modulate WM contents more effectively than when they are given a probe that includes more information than the retro-cue.
Explanations of the retro-cue benefit
Assumptions and Predictions
H1. Protection from time-based decay
Item in visual WM suffer from time-based decay. Decay affects all items in memory unless some active mechanism is used to counteract forgetting.
A retro-cued item is protected from time-based decay, whereas noncued items are left unprotected from time-based decay.
H2. Prioritization for comparison
At test, items are serially searched in memory to be compared to the probe stimulus. Each comparison step is time-consuming and error-prone. The more comparisons that are made to reach a decision, the slower and more error-prone the response is.
The retro-cue reduces the number of comparisons made in memory, yielding faster and more accurate responses.
Memory items interfere with each other (interitem competition), as reflected in the set-size effect.
The retro-cue indicates which information is no longer relevant and can therefore be removed (discarded) from WM. Removal reduces memory load, freeing capacity to process the relevant (cued) information and to add new input to WM.
H4. Attentional strengthening or refreshing
Attention is used to augment the accessibility of representations in WM (refresh its trace). Arguably, in no-cue trials refreshing is distributed among all items in memory.
The retro-cue indicates which item can be selectively attended to, and thereby differentially strengthened in memory.
H5. Retrieval head start
Items have to be retrieved (selected) from working memory for making a decision. Retrieval can be understood as the gradual accumulation of evidence for an item in memory.
The retro-cue allows evidence to accumulate for the target item ahead of decision making (i.e., response selection), increasing the probability that the correct information will be selected for decision making.
H6. Protection from perceptual interference
Visual input (e.g., a visual mask, the probe stimulus, or a color wheel) following encoding of the memory array can replace or distort memory representations.
The retro-cued item is insulated from interference by subsequent visual input.
H1. Protection from time-based decay
Protection from decay could arguably explain the retro-cue benefit in comparison to a baseline condition that was matched in terms of overall retention interval (i.e., the no-cue long or neutral-cue conditions in Fig. 1). However, protection from decay cannot explain retro-cue benefits in comparison to no-cue short trials, because in that comparison there is as much time for decay before the retro-cue as in the control condition. In Fig. 2A, this is illustrated by the common decay curve of all conditions up to the onset of the cue, which coincides with the onset of the test probe in the no-cue short condition. Makovski et al. (2008) were the first to demonstrate that retro-cue benefits are observed relative to a no-cue short baseline. This finding has since been replicated in several other reports (Murray et al., 2013; Rerko et al., 2014; Souza, Rerko, Lin, & Oberauer, 2014; Souza et al., 2014, 2015). These studies show that there is more to the retro-cue benefit than protection from time-based decay.
Whether time-based decay contributes to forgetting from visual WM is in itself still a matter of debate. In visual WM tests, performance has been found to be inversely related to the duration of the retention interval (Morey & Bieler, 2013; Pertzov et al., 2013; Ricker & Cowan, 2010, 2014; Ricker, Spiegel, & Cowan, 2014; Shipstead & Engle, 2013; Souza & Oberauer, 2015; Zhang & Luck, 2009). This is line with a decay explanation of forgetting, but also with an explanation by temporal distinctiveness: As the memory stimuli of the current trial recede into the past, they become less distinguishable in time from preceding trials. An effect of temporal distinctiveness is predicted by interference theories assuming that temporal context is a retrieval cue, such that events close in psychological time compete with each other for retrieval, leading to interference (Brown, Neath, & Chater, 2007; Brown, Sala, Foster, & Vousden, 2007). There is experimental support for an interference explanation of time-based forgetting in visual WM (Mercer, 2014; Shipstead & Engle, 2013; Souza & Oberauer, 2015; but see Ricker et al., 2014, for conflicting results).
In sum, protection from decay is a noncompelling explanation of retro-cue benefits for three reasons: (1) Retro-cue benefits are observed even in conditions that control for the contribution of time-based decay (no-cue short); (2) there is no consistent evidence for progressive forgetting of noncued items, as would be predicted if these items were unprotected from decay; and (3) the power of time-based decay to explain forgetting in visual WM has been questioned by several recent studies demonstrating the viability of an alternative explanation in terms of time-based interference.
H2. Prioritization for comparison
This account starts from the assumption that multiple items in visual WM must be searched through sequentially at test. When an item is retro-cued, it receives priority in the search sequence, as is shown in Fig. 2B (Astle, Summerfield, Griffin, & Nobre, 2012; Makovski et al., 2008; Matsukura et al., 2007; Pertzov et al., 2013). Under the reasonable assumption that each comparison step in memory is a time-consuming and error-prone process, this hypothesis predicts benefits in RTs and accuracy when the number of memory comparisons is reduced. According to this hypothesis, the retro-cue benefit is a function of how much participants rely on a serial search through memory to arrive at a response. It is conceivable that participants use serial search in the classical change detection task, in which every stimulus in the test display has to be compared to its counterpart in the remembered memory array, or in a global-recognition task, in which a single, central probe stimulus must be compared to all memory items. However, prioritization is a less plausible explanation when the task clearly limits the comparison process to a single item, as in a local-recognition task, in which the probe has to be compared only to the item presented in the same location. A prioritization account is even less plausible in delayed-estimation tasks, in which a single item is reconstructed from memory. Unlike the predictions shown in Fig. 2B, retro-cue benefits have been shown with all five types of WM tests shown in Fig. 1, rendering the prioritization account nonappealing.
Williams, Hong, Kang, Carlisle, and Woodman (2013) provided strong evidence for the first prediction. Participants encoded two items for a delayed-estimation test, and in half of the trials, one item was retro-cued as irrelevant. Performance on tests of the remaining, relevant item improved in retro-cue trials relative to no-cue trials, and a surprise test of the cued to-be-forgotten item showed virtually no memory for that item. Furthermore, several studies have shown invalid retro-cueing costs, which are consistent with a role of removal as an explanation of the retro-cue benefit (Astle, Summerfield, et al., 2012; Gözenman, Tanoue, Metoyer, & Berryhill, 2014; Gressmann & Janczyk, 2016; Griffin & Nobre, 2003; Gunseli et al., 2015; Pertzov et al., 2013; van Moorselaar, Olivers, et al., 2015).
Souza et al. (2014) tested Predictions 2 and 3 in a local-recognition task. They presented no-cue trials or 100%-valid retro-cues, and varied memory load (two, four, or six items). The performance in no-cue trials decreased with memory load. The retro-cue led to a larger improvement of performance at larger memory loads, though it did not entirely eliminate the effect of memory load on accuracy. Several other studies have also revealed larger retro-cue benefits with increasing numbers of noncued items (Gilchrist et al., 2016; Gressmann & Janczyk, 2016; Kuo et al., 2012; Nobre et al., 2008; Sligte et al., 2008; Souza, Rerko, Lin, & Oberauer, 2014; Vandenbroucke, Sligte, de Vries, Cohen, & Lamme, 2015; van Moorselaar, Olivers, et al., 2015), but it is worth noting three studies that have not observed such an interaction of retro-cueing and memory load (Makovski et al., 2008; Matsukura et al., 2007; Trapp & Lepsien, 2012).
To test whether this interaction indeed reflected freeing of WM capacity after a retro-cue, Souza et al. (2014) asked participants to encode two sequentially presented memory arrays separated by a 2-s interset interval. During the interset interval, an item from Array 1 was retro-cued in half of the trials. The usual retro-cue benefit was obtained in tests of Array 1. Critically, testing memory for Array 2 also showed improved performance after a retro-cue to an item in Array 1, showing that the retro-cue freed capacity to encode Array 2 into WM.
A contribution of removal to the retro-cue effect has also been supported by neural evidence. Kuo et al. (2012) assessed how the contralateral delay activity (CDA; a lateralized electroencephalography [EEG] marker of WM load) is modulated by neutral versus retro-cues. In trials without retro-cues, the CDA was larger when participants maintained four items instead of two items in WM. After retro-cueing, however, the CDA was reduced and converged to similar levels for both set sizes, suggesting that only the cued item was maintained in WM (see also Duarte et al., 2013; Schneider, Mertes, & Wascher, 2015). Another set of studies assessed the neural markers of WM maintenance by using pattern classifiers trained to identify the class of stimuli (words, orientations, etc.) that a person was holding in WM from their brain activity as measured by means of fMRI or EEG. When people hold stimuli from two different classes in mind and one class of stimuli is subsequently retro-cued, the neural activity pattern reflecting the noncued stimulus class drops to baseline, whereas the activity related to the cued class remains measurable (LaRocque, Lewis-Peacock, Drysdale, Oberauer, & Postle, 2013; Lewis-Peacock, Drysdale, Oberauer, & Postle, 2012).
These studies show that noncued items are removed from the set of representations maintained through persistent neural activity. At the same time, these items are not irreversibly lost—when initially noncued items are subsequently cued as being relevant, the neural activity pattern reflecting their stimulus class reemerges, and participants can make accurate judgments on these items (LaRocque et al., 2013; Lewis-Peacock et al., 2012). Information about these items is apparently maintained in an “activity-silent” state (Stokes, 2015). It is plausible that outsourcing representations to an activity-silent form of maintenance reduces the degree to which they interfere with access to the cued items. When tested directly, the information maintained in activity-silent form might be less accessible than information maintained through persistent neural activity, resulting in an invalid-cue cost. Yet, activity-silent information can apparently be partially (van Moorselaar, Olivers, et al., 2015) or fully (Landman et al., 2003; Rerko & Oberauer, 2013) recovered when the initially noncued items are cued by a second retro-cue before being tested.
To conclude, there is compelling evidence that the removal of noncued information from WM contributes to the retro-cue benefit. It is unlikely, however, that removal alone suffices for a complete explanation of the retro-cue benefit, because such benefits have been observed also at low levels of cue reliability, such that the removal of noncued items is not advantageous, and experiments have shown no cost of accessing an invalidly cued item under these conditions (Gunseli et al., 2015).
H4. Attentional strengthening or refreshing
Bringing an item to the focus of attention is assumed to strengthen its trace in WM, improving its accessibility for later use, a process often referred to as refreshing (Chun & Johnson, 2011; Johnson, 1992; Rerko & Oberauer, 2013; Ricker & Cowan, 2010; Souza et al., 2015). This hypothesis states that focused attention makes the representation of the retro-cued item, and possibly the binding to its context (e.g., its spatial location), stronger than it originally was after encoding, as is illustrated in Fig. 3B (Kuo, Yeh, Chen, & D’Esposito, 2011; Lepsien et al., 2011; Nobre et al., 2008; Rerko & Oberauer, 2013; Rerko et al., 2014; Souza et al., 2014, 2015).
The strengthening of items and of item–context bindings can be distinguished in the local-recognition paradigm (see Fig. 1), by using intrusion probes. These probes test an item B with a probe matching a different item A from the current memory set. If a retro-cue to item A strengthens the representation of that item, then A should be a very strong lure, leading to increased false-alarm rates as compared to an intrusion probe matching a noncued item C. In contrast, if a retro-cue to A strengthens the bindings between A and its location, people should find it easier to correctly reject an intrusion probe matching A than an intrusion probe matching C. The latter pattern was observed by Rerko and Oberauer (2013), implying that a retro-cue results in stronger item–context bindings, not stronger item representations independent of their context.
A distinguishing feature of the strengthening or refreshing explanation is that noncued items are assumed to be maintained in WM unchanged, whereas the focused item increases in accessibility (Rerko & Oberauer, 2013). Strengthening does not mean to shift memory strength away from other items and toward the cued item; rather, attending to an item in WM leads to a stronger binding of that item to its context, without affecting the strength of other item–context bindings.
One prediction of this hypothesis is that a retro-cue benefit for validly cued items may occur without a cost for invalidly cued items. Several studies have demonstrated costs of invalid retro-cues (e.g., Astle, Summerfield, et al., 2012; Griffin & Nobre, 2003; Pertzov et al., 2013). Retro-cue costs are observed even when participants are aware that one of the noncued items will also be tested (Rerko et al., 2014). However, these studies compared invalid retro-cue trials against a no-cue long or a neutral-cue baseline. In the baseline condition, participants probably distributed their attention among several items, refreshing each of them to a small extent, whereas in the retro-cue condition they concentrated attention more exclusively on the cued item. Therefore, the observed costs may not reflect a weakening of the noncued items, but rather a reduction of refreshing opportunities for the noncued items in retro-cue trials as compared to the baseline.
Alternatively, the invalid retro-cue costs may reflect the operation of other processes that, in addition to refreshing, take place in retro-cue trials (e.g., the removal of noncued items, as we explained above). It follows that to observe a pure effect of refreshing (if there is one), one needs to reduce the incentive to remove noncued items from WM. Li and Saiki (2014) have provided a first hint in this direction. In addition to having a retro-cue condition, they presented in some trials a “withdraw cue” after the retro-cue. These cues were used to instruct participants to withdraw attention from the retro-cued item and to reset to a state of maintaining all items equally. Despite presenting this withdraw cue, the probability of testing the retro-cued item or one of the noncued items was maintained the same as in a regular retro-cue trial. In the standard retro-cue trial, testing of validly and invalidly retro-cued items yielded benefits and costs, respectively. In the withdraw condition, valid retro-cue benefits were maintained, but invalid retro-cue costs disappeared.8 These results are in line with the idea that participants can strengthen an item without costs for noncued items. Similar findings have been reported from studies that have varied cue reliability: Under low retro-cue reliability conditions, valid retro-cue benefits are still observed, but invalid retro-cue costs disappear (Berryhill et al., 2012; Gunseli et al., 2015; Shimi et al., 2014; Souza et al., 2015).
Another prediction from the strengthening hypothesis is that several items can be focused in succession, leading to an incremental benefit for all of them. Rerko and Oberauer (2013) have provided initial evidence that multiple successive retro-cues do cumulatively strengthen items in WM: When participants see up to three successive retro-cues (with the last-cued item being the one relevant for the test), memory is better for items cued twice (sequence of cueing A–B–A) than for items cued once (sequence C–B–A). These findings provide a hint that focusing attention on an item in WM strengthens its representation in a cumulative fashion.
Souza et al. (2015) have provided a stronger test of the strengthening explanation. In this study, a sequence of four central arrow cues was embedded in the retention interval of a color delayed-estimation task. Participants were instructed to “think of” (i.e., refresh) the cued items. They were instructed that the cues were nonpredictive of the test item, to reduce any incentive to remove items from WM. Across the sequence of four cues, individual memory items were cued zero, one, or two times. Response errors decreased as the number of refreshing steps directed at the target increased, showing that refreshing benefited memory in a cumulative fashion. Moreover, the zero-cued items in the refreshing condition were recalled as well as items in a no-cue short condition, demonstrating that focused attention not merely protects memory representations from forgetting, but improves memory over time, as is predicted by the strengthening hypothesis (see Fig. 3B). Hence, this study showed that participants can strengthen the cued item without necessarily changing the accessibility of the noncued items.
Finally, there is evidence that retro-cues counteract the detrimental effect of reducing attention to the memory array during encoding (Vandenbroucke, Sligte, & Lamme, 2011). Vandenbroucke et al. showed that WM performance suffers when, at encoding, attention is diverted from the memory array by, for example, creating uncertainty about which display is memory-relevant, or by asking people to perform another task concurrently with encoding of the memory array. In these conditions, presenting a retro-cue during the retention interval of the memory task reduces the deleterious effect of distraction during encoding. This finding suggests that reduced attention during encoding results in the formation of relatively weak memory traces, which can be strengthened by focusing attention on the relevant item prior to test.
To conclude, the strengthening of WM representations by focused attention is a likely cause of the retro-cue benefit.
H5. Retrieval head start
We started this section by pointing out a conundrum: In many retro-cue experiments, a probe stimulus is presented in the no-cue condition at the same time as the retro-cue is presented in the retro-cue condition, and the probe has more information to offer than the retro-cue, yet it yields worse performance. Perhaps, then, the retro-cue benefit is less about the information the retro-cue provides, and more about what it temporarily withholds. The last two hypotheses about the causes of the retro-cue benefit that we will consider state that access to visual WM is improved by delaying part of the information provided at test. The retro-cue delays the (potentially interfering) visual input of the test (e.g., the memory probe; or the continuous response scale). Moreover, it also delays all of the cognitive operations related to decision making (response selection and response execution), but leaves the opportunity to retrieve the cued item intact.
The retrieval head start hypothesis rests on the assumption that decision making in no-cue trials commences before retrieval has extracted all information from WM, so that decision making is based on incomplete accumulated information. Consequently, the decision might be unduly influenced by other sources of information than the target, including nontarget representations in WM as well as the visual stimuli present at test (e.g., probe or distracting stimuli). In contrast, a retro-cue gives retrieval a head start, ensuring more advanced accumulation of evidence from memory before decision making can begin.
One may wonder why enforcing a longer delay helps performance in nonspeeded tests. There are at least two possibilities. First, in nonspeeded tasks, participants need to set a threshold of evidence accumulation before responding, thereby controlling the speed–accuracy trade-off. This threshold can vary across trials and individuals, and nothing guarantees that it will optimize accuracy. Hence, enforcing a delay may help to increase the probability of optimal evidence accumulation. Second, it might not be optimal to immediately use all retrieved information as evidence toward one or another response, because in the initial stages of retrieval, information from irrelevant sources of information (e.g., nontarget items in WM) is likely to be retrieved alongside relevant information about the target feature. Temporally separating retrieval and decision making, as a retro-cue does, could give the retrieval process time to narrow down its output to the relevant information before feeding it into the decision process, thereby reducing the influence of misleading information on response selection.
The idea that retro-cues give retrieval stages a head start is in line with the event-related potential (ERP) topographies elicited by the probe stimulus in the neutral versus retro-cue trials reported by Nobre et al. (2008). They observed a load-sensitive ERP component (which they termed the N3 RS ) related to the search through items in WM in neutral-cue trials. By contrast, this component was not elicited by the probe in retro-cue trials. This suggests that by the time the probe was displayed in retro-cue trials, participants might already have finished retrieving the target item, providing tentative evidence in support of the retrieval head start hypothesis. This evidence is only tentative, though, because in this study they did not investigate whether the N3RS was elicited by the retro-cue (instead of the probe) and whether it was somehow modulated in retro-cue trials, showing the facilitation of retrieval that would be expected from the retrieval head start hypothesis.
H6. Protection from perceptual interference
The retro-cue not only delays response selection, it also delays the onset of interference from the visual input at test relative to the onset of the retrieval cue. Accordingly, one prominent hypothesis of the retro-cue benefit is that focused attention shields WM representations from perceptual interference (Makovski & Jiang, 2007; Matsukura et al., 2007; Sligte et al., 2008). As is shown in Fig. 4B, such a hypothesis predicts that retro-cueing an item would make it more robust to interference from visual stimuli. The underlying assumption is that the WM representations in no-cue trials, as well as the noncued representations in retro-cue trials, are easily distorted by subsequent visual input. This leads to the prediction that after using the retro-cue to focus attention on one item, the cued item is protected from visual interference, whereas noncued representations remain unprotected, hence being impaired by interference.
Two lines of research speak to this hypothesis. First, some studies have assessed the impact of presenting irrelevant interfering information during the retention interval. Second, other studies have assessed how much interference is introduced by the visual stimuli in the test display, and how retro-cues can protect WM representations from it.
The first of these approaches tests the prediction that irrelevant visual input impairs memory for the target item less after a retro-cue to that item than in the absence of a retro-cue (Fig. 4B). Two studies have shown that presenting an irrelevant visual mask during the retention interval (after the retro-cue in retro-cue trials) yields results matching these predictions (Makovski & Jiang, 2007; van Moorselaar, Gunseli, et al., 2015). In three other studies, interference was manipulated by presenting a secondary task during the retention interval—essentially creating a dual-task condition (Hollingworth & Maxcey-Richard, 2013; Makovski & Pertzov, 2015; Rerko et al., 2014). Makovski and Pertzov introduced an even–odd digit classification task during the retention interval of a delayed-estimation task, and observed that retro-cue benefits were significantly larger under the dual-task condition, as is predicted by the protection-from-interference hypothesis. Hollingworth and Maxcey-Richard as well as Rerko et al. also observed a numerical trend in this direction in local recognition; however, the Cueing × Dual-Task interaction was not significant, hence not providing evidence that could be used to either support or reject this hypothesis. All in all, studies testing for protection from irrelevant visual interference suggest that retro-cues can protect WM representation from interference by irrelevant visual input.
The second approach to testing the protection-from-interference hypothesis is concerned with perceptual interference arising from the process of testing memory. Attending to the test display is required in order to complete the memory task, but at the same time, this introduces visual input that can harm memory. Retro-cues can serve to protect memory from this deleterious effect. Evidence in this direction has been provided by Makovski, Watson, Koutstaal, and Jiang (2010). These authors compared performance in a local-recognition task (single-probe display) with performance in a two-alternative forced choice task (two-probe display) with and without a retro-cue. Performance was overall lower with the two-probe display test, but retro-cues reduced this disadvantage.
Souza et al. (2016) investigated the role of protection from visual interference in local-recognition and delayed-estimation tasks. In recognition, visual interference arises from the memory probe. The retro-cue delays onset of the probe, giving focused attention time to make the cued item robust enough to withstand distortion. The retro-cue also postpones other processes involved in decision making, as we discussed above. To test separately the contributions of delaying decision and of delaying probe interference, Souza et al. (2016) created a delayed-selection condition that was in-between the classical no-cue and retro-cue conditions (see Fig. 5). In this condition, a probe was shown at the same point in time as the retro-cue, but participants were forced to delay decision making for the same amount of time as in the retro-cue condition. Performance in the retro-cue condition was better than in this delay condition, indicating unambiguously that delaying probe interference contributes to the retro-cue benefit in this task, over and above the benefit from delaying the decision process.
Interference from the test display can also impair memory in the delayed-estimation paradigm. In a color delayed-estimation task, a color wheel is shown at test. Souza et al. (2016) showed that the color wheel interferes with memory: When attending to the location of the target of recall, participants attend as well to the portion of the color wheel that is spatially close to it. Attending to this location attracts responses to spatially close colors in the color wheel, leading to a memory distortion effect that Souza et al. (2016) termed wheel attraction. The wheel attraction effect disappeared when the onset of the color wheel was delayed relative to the onset of the cue, and when the color wheel was replaced by a gray wheel. These results show that color information presented at test interferes with memory for colors, and that attention can be used to protect memory from this interference.
Taken together, there is strong evidence that part of the retro-cue benefit is due to protection of the focused WM representation from perceptual interference. For the purpose of this review, we treated the protection-from-interference hypothesis as a separate hypothesis, but we note that protection from perceptual interference could be a by-product of strengthening a focused representation in WM, or of a head start for retrieving that representation before the onset of interfering information. The two possibilities could be distinguished by a sequential-cueing paradigm in which first item A, then item B is retro-cued, with the instruction that the last-cued item will be the most likely target. Strengthening of an item A in WM persists after attention focuses on another item B. Therefore, if strengthening underlies protection from perceptual interference, a visual mask following the second retro-cue should affect item A (and item B) less than a noncued item C. In contrast, after the second retro-cue, only item B is given a head start for retrieval, whereas there is no reason to retrieve item A at all at this point in time. Therefore, if protection from perceptual interference is a by-product of having retrieved the item before the onset of interference, the effect of a visual mask should be reduced only for item B, not for A.
Fragile/robust WM stores
A discussion of explanations of the retro-cue effect would be incomplete without considering the hypothesis of multiple visual memory stores, advanced by Sligte, Lamme, and colleagues (Sligte et al., 2008; Sligte et al., 2010): Items selectively attended at encoding yield robust representations (robust WM store). These representations are protected from decay and from interference, and underlie performance in traditional tests of visual WM (e.g., in postcue or no-cue trials). Items coarsely attended at encoding yield fragile traces (fragile WM store). These fragile representations are highly susceptible to interference from visual input presented at the same location and in the same feature dimension as the memoranda (Pinto et al., 2013; Sligte et al., 2008). Hence, a probe presented in the same location as the target—as in local-recognition or change discrimination tasks—erases any fragile representation of the target, so that performance can only rely on the contents of the robust WM store. When a fragile representation is attended during the retention interval—for instance, in response to a retro-cue—it is turned into a robust representation in WM, and thereby can contribute to performance. Finally, for nonattended items, only an iconic trace persists for ~500 ms (Sligte et al., 2008; Sligte et al., 2010; Sperling, 1960). This iconic store has a high capacity, and quickly orienting attention to their fleeting representations allows their transfer to WM, hence dramatically increasing capacity estimates (Schmidt, Vogel, Woodman, & Luck, 2002; Sligte et al., 2008; Sligte et al., 2010).
The theory of multiple visual stores makes assumptions and predictions that go substantially beyond an explanation of the retro-cue effect (Sligte, Scholte, & Lamme, 2009; Vandenbroucke et al., 2011, 2015). A full evaluation of its merits also goes beyond the scope of this review. Here we focus on how the multiple-store theory explains the retro-cue effect. The explanation is to a large part a version of H6: Retro-cued items are encoded in robust WM, and are thereby protected against interference from subsequent visual input. Unlike our general formulation of H6, which assumes that perceptual interference can arise from a broad range of sources, the multiple-store theory is committed to the assumption that only perceptual input in the same location as the memory items disrupts the fragile representations of these items (Pinto et al., 2013; Sligte et al., 2008). Therefore, the multiple-store theory in its current form cannot easily explain the retro-cue benefit in the delayed-estimation paradigm, in which no probe is presented at the location of the target. This problem could be overcome by assuming that fragile visual short-term memory can also be disrupted by visual input in other locations than the target, such as the onset of a color wheel in the delayed-estimation task (Souza et al., 2016).
One unique prediction of the multiple-store theory is that retro-cues will only be beneficial when presented prior to the onset of visual interference, which is assumed to entirely erase fragile representations so that a subsequent retro-cue cannot resuscitate them. The evidence for this strong prediction is currently mixed. Sligte et al. (2008) and Pinto et al. (2013) observed that presenting a visual mask consisting of similar items in the same location as the memoranda prior to onset of the retro-cue abolished the retro-cue benefit. In contrast, Makovski (2012) observed retro-cue benefits even when a visual mask (multicolored dots presented at the same location as the color memoranda) was displayed prior to the retro-cue.
In sum, the multiple-store theory provides a mechanistic basis for the hypothesis that retro-cues protect memory representations from perceptual interference, and as such, the empirical support for this hypothesis (H6) also supports the multiple-store theory. At the same time, our discussion of H6 has shown that it does not offer a complete explanation of the retro-cue effect on its own, and the same is true for the multiple-store theory. Moreover, H6 does not require the assumption of multiple stores: A theory of visual WM could assume a single WM store in which representations can vary on a continuum of robustness, and attending to a representation increases its robustness, thereby protecting it from perceptual interference.
Protection from decay might contribute to the retro-cue effect in experiments comparing the retro-cue condition to a no-cue long baseline, but the hypothesis is contingent on the assumption that decay contributes to forgetting in visual WM, for which evidence is mixed at best.
Prioritization for comparison could contribute to the retro-cue benefit only in tasks that require search through multiple memory items. Most retro-cue tasks constrain the comparison to a single item in both retro-cue and no-cue trials, making this explanation nonappealing.
Highly reliable retro-cues indicate the relevant item for the test, and render noncued items irrelevant. Removing irrelevant information frees WM capacity. The evidence supporting removal of noncued items from WM converges with the finding that visual WM’s contents can be flexibly discarded to encode new, more up-to-date information on an item basis (Kessler et al., 2015), and with evidence for the removal of outdated information as a component of updating of verbal WM (Ecker, Lewandowsky, & Oberauer, 2014; Ecker, Oberauer, & Lewandowsky, 2014).
According to the strengthening/refreshing hypothesis, attention can be used to cumulatively strengthen representations in WM, thereby augmenting, in an incremental fashion, their accessibility to levels higher than the one obtained just after encoding. There is compelling evidence that strengthening contributes to the retro-cue benefit. The relative contributions of refreshing and removal as mechanisms of the retro-cue benefit seems to depend critically on the reliability of the retro-cue. The latter finding underscores the voluntary, controlled use of the information provided by the retro-cue to modulate WM maintenance.
The retro-cue benefit is in part due to a head start for retrieval. Evidence for this hypothesis comes from the observation that merely delaying decision making improves performance on visual WM tests, even in the presence of perceptual interference. This finding shows that WM performance is constrained not only by limitations during maintenance but also at decision stages, due to nonoptimal selection of the information in WM.
Retro-cues make the representation of the cued item more robust to subsequent visual interference. The studies investigating protection from interference have shown that capacity limits in visual WM arise in part from interference from subsequent visual input (either irrelevant or relevant to the memory test). Retro-cues help stabilize representations to survive interference from that input.
In a nutshell, these results show how one can get the most out of visual WM, indicating possible avenues to explore when trying to bypass the severe capacity limitation that has been a hallmark of visual WM.
Outlook and conclusion
Research on retro-cue effects has provided important insights into the mechanisms of visual WM and of attention to its contents: The representations in visual WM are not as robust as has previously been thought, and much of visual WM’s capacity limitations may arise from interference (interitem interference and perceptual interference) and incomplete retrieval. Simply put, the retro-cue effect has shown that traditional tests substantially underestimate the information available in memory. In the WM literature, it has usually been assumed that performance on WM tests is more or less a direct function of the limitations of encoding and storage. Hence, the precision and number of items stored in memory are assumed to have the values estimated in traditional memory test conditions. The retro-cue effect shows that this is not the case. Representational quality could be higher, the number of items accessible from memory could be higher, or both, because many of the limitations we observe in traditional tests arise at retrieval and decision stages. For example, an increase in the estimated representational quality (precision) following a retro-cue does not mean that the retro-cue yielded an information gain from nowhere, but that some precision is lost through testing memory. The retro-cue prevents this loss from happening, surprisingly showing higher memory precision when attention is focused on the target item prior to testing.
Questions for future research
What is selected by the retro-cue (e.g., item, context, or item–context bindings)?
What is the mechanism supporting the state of heightened accessibility and robustness achieved for the retro-cued item?
What is the relation between WM capacity measures in standard conditions (no-cue baseline) and performance in retro-cue trials? So far we do not know whether people with high capacity benefit more or less from the retro-cue than do low-capacity individuals. This information might be helpful in relating the retro-cue effect to models of capacity limitations in WM.
How does the retro-cue effect relate to the control of attention in other domains? We are not aware of studies that have tried to relate the retro-cue effect to other measures of the efficiency of attention control in the perceptual domain (e.g., cueing advantages in perceptual tasks), or even to other effects ascribed to the focus of attention in WM, such as object-switch costs (Garavan, 1998; Oberauer, 2003) or the last-item recency effect (McElree, 2001; Nee & Jonides, 2011).
Does focusing attention on the retro-cued item change its status in long-term memory (LTM)? Relatedly, do LTM representations mediate the retro-cue effect? A first hint that retro-cueing an item does not change its status in LTM was provided by the recent study by LaRocque et al. (2015). However, more studies will be needed to settle this issue.
Do the different processes contributing to the retro-cue benefit (e.g., strengthening vs. removal) follow different developmental trajectories?
As is the case in many maturing fields of research in cognitive psychology, the richness of the hypotheses about the nature of retro-cue effects and of the empirical findings speaking to them is beginning to weigh down on our ability to navigate the space of theoretical and empirical possibilities. The complexity of theoretical assumptions taxes the WM capacity of researchers trying to derive predictions from them. In this review, we handled this complexity by selectively focusing attention on hypotheses individually, but we acknowledge that this can only be an intermediate step. Most of the hypotheses we discussed are not mutually exclusive, and the evidence we reviewed strongly suggests that at least four of them contribute to retro-cue effects. Future theorizing about the role of attention in visual WM therefore will have to consider how these processes, and potentially others, work together.
At this juncture, computational modeling can help as a cognitive prosthesis (Farrell & Lewandowsky, 2010) for building and evaluating more complex theories about the mechanisms and processes underlying attention to the contents of visual WM. Several computational models already exist for visual WM (Bays, 2014; Matthey, Bays, & Dayan, 2015; Swan & Wyble, 2014; van den Berg, Shin, Chou, George, & Ma, 2012). We recently proposed a computational model of attention to items in verbal WM (Oberauer, 2013; Oberauer, Souza, Druey, & Gade, 2013) that explains the effects of switching attention between items in WM and of swapping entire memory sets into and out of WM. Integrating the principles and mechanisms of these two lines of modeling might be a fruitful way toward a computational model of attention to the contents of (visual) WM.
One reason for this may be due to unreliable estimation of the memory precision parameter in the mixture model under the low levels of performance (high guessing rate, low precision, or both) that are usually observed in high-memory-load conditions (Lawrence, 2010).
In this review, we are only considering studies that have assessed retro-cue effects against a baseline.
In Pertzov et al. (2013), retro-cue benefits in recall error continued to increase with longer cue–test delays. It may be worth noting, however, that the overall retention interval in the baseline condition was also lengthened to match the total duration of the retro-cue trial (no-cue long condition). Hence, this effect may have been due to increased time-based forgetting (due to decay or interference) in the baseline.
The proportion of valid cues is often referred to as the cue validity—here we use the term validity to characterize whether an individual cue points to the target (i.e., a valid cue) or not (i.e., an invalid cue), and use reliability for the overall proportion of valid cues.
These authors did not present an analysis of retro-cue costs for invalid-cue trials.
These results are only tentatively informative, because those authors did not provide any means of assessing the amount of support for the null hypothesis.
This project was supported by a grant from the Forschungskredit (Project No. FK-13-083) to A.S.S. and by a grant from the Swiss National Science Foundation (Project No. 126766/1) to K.O.
- Bays, P. M., Catalao, R. F. G., & Husain, M. (2009). The precision of visual working memory is set by allocation of a shared resource. Journal of Vision, 9(10), 7:1–11. doi: 10.1167/9.10.7
- Heuer, A., & Schubö, A. (2016). Feature-based and spatial attentional selection in visual working memory. Memory & Cognition. Advance online publication. doi: 10.3758/s13421-015-0584-5
- Makovski, T., Watson, L. M., Koutstaal, W., & Jiang, Y. V. (2010). Method matters: Systematic effects of testing procedure on visual working memory sensitivity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 1466–1479. doi: 10.1037/a0020851 PubMedPubMedCentralGoogle Scholar
- Myers, N. E., Walther, L., Wallis, G., Stokes, M. G., & Nobre, A. C. (2015). Temporal dynamics of attention during encoding versus maintenance of working memory: Complementary views from event-related potentials and alpha-band oscillations. Journal of Cognitive Neuroscience, 27, 492–508. doi: 10.1162/jocn_a_00727 PubMedCrossRefGoogle Scholar
- Souza, A. S., Rerko, L., & Oberauer, K. (2016). Getting more from visual working memory: Retro-cues enhance retrieval and protect from visual interference. Journal of Experimental Psychology: Human Perception and Performance. Advance online publication. doi: 10.1037/a0036331
- van Moorselaar, D., Olivers, C. N. L., Theeuwes, J., Lamme, V. A. F., & Sligte, I. G. (2015). Forgotten but not gone: Retro-cue costs and benefits in a double-cueing paradigm suggest multiple states in visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41, 1755–1763.PubMedGoogle Scholar
- Wilken, P., & Ma, W. J. (2004). A detection theory account of change detection. Journal of Vision, 4(12), 11:1120–1135. doi: 10.1167/4.12.11
- Yantis, S. (2000). Goal-directed and stimulus-driven determinants of attentional control. In S. Monsell & J. Driver (Eds.), Control of cognitive processes: Attention and performance XVIII (pp. 73–103). Cambridge, MA: MIT Press.Google Scholar