Learning changes the attentional status of prospective memories
Objects in visual working memory (VWM) that are only prospectively relevant can nevertheless affect the guidance of attention in an ongoing visual search task. Here we investigated whether learning changes the attentional status of such prospective memories. Observers performed a visual search while holding an item in memory for a later memory test. This prospective memory was then repeated for several trials. When the memory was new, it interfered with the ongoing search task. However, with repetition, memory performance increased but memory-based interference rapidly diminished, suggesting that observers learned to shield the prospective memory from the ongoing task. This contrasts with earlier findings showing stronger attentional biases from learned memories when these are immediately task-relevant. Interestingly, interference resurfaced again in anticipation of a new memory, suggesting a reactivation of VWM. These effects were sensitive to task context, indicating that the attentional status of prospective memories is flexible.
KeywordsVisual working memory Visual attention Visual search
Visual working memory (VWM) is assumed to play an important role in guiding selective attention toward task-relevant visual objects—for example, in visual search (Bundesen, Habekost, & Kyllingsbæk, 2005; Desimone & Duncan, 1995; Wolfe, 1994; Woodman & Chun, 2006). However, VWM is not the only memory system guiding attention, since long-term memory (LTM) can also bias selection during visual search (Hutchinson & Turk-Browne, 2012; Olivers, 2011).
Learning through repetition induces a transition from VWM to LTM (Shiffrin & Schneider, 1977). Recent electroencephalography studies have indicated that this transition is remarkably rapid, since the contralateral delay activity (CDA, a marker of VWM) drops to asymptote within a handful of trials (Carlisle, Arita, Pardo, & Woodman, 2011; Gunseli, Meeter, & Olivers, 2014; Gunseli, Olivers, & Meeter, 2014; Reinhart & Woodman, 2014). Moreover, this reduction in VWM-related activity appears to be accompanied by a reduction in the P170, which has been interpreted as a marker of a more implicit longer-term memory (Woodman, Carlisle, & Reinhart, 2013; though see Gunseli, Olivers, & Meeter, 2014). In line with this, target repetitions have led to robust, automatic, and largely implicit selection benefits in visual search (Kruijne & Meeter, 2015; Maljkovic & Nakayama, 1994; Müller, Krummenacher, & Heller, 2004; Theeuwes, 2013).
So far, studies of attentional learning have investigated conditions in which the memory representation is directly relevant for the search—that is, the target is what is learned. However, for the cognitive system to be adaptive, it must be able to distinguish currently relevant learned information from currently irrelevant learned information. Here we investigated the influence on attention from memories that are being learned for a prospective task, but that are not relevant for the current task. Specifically, how does learning an item for such a prospective task affect attentional priorities?
To answer this question, we made use of an established paradigm in which observers search for a specific target, while holding an accessory item in memory for a later test. Thus, the accessory memory item is only relevant in a prospective sense. Nevertheless, attention during search has been shown to be inadvertently biased toward distractors that match the accessory memory item, resulting in interference (Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005; but see Downing & Dodds, 2004; Woodman & Luck, 2007). The evidence suggests that the prospective relevance of the accessory memory causes it to be kept active in VWM, from where it guides attention. However, it is unknown how learning changes the attentional status of such prospective memories. There seem to be two possibilities: (1) Learning strengthens the prospective memory but is blind to task relevance, and thus results in increased interference stemming from prospective memories. (2) Learning strengthens the prospective memory, but at the same time observers also learn to shield it from currently ongoing tasks. In this case, as the prospective memory is deactivated within VWM, it no longer drives visual attention, and interference should decrease.
Experiments 1A and 1B: reduced interference from learned memories
A planned number of 15 volunteers (14 females, one male; age 18–26 years, M = 21 years) participated in Experiment 1A, and another 15 volunteers (13 females, two males; age 18–24 years, M = 21 years) participated in Experiment 1B, in exchange for course credit or €8 per hour. The participants reported normal or corrected-to-normal acuity. Procedures were approved by the Scientific and Ethical Review Committee (Faculty of Behavioral and Movement Sciences, VU University).
Apparatus, stimuli, procedure, and design
A Windows 7 PC running OpenSesame v0.28 (Mathôt, Schreij, & Theeuwes, 2012) generated the stimuli on a Samsung SyncMaster 2233 120-Hz screen, at 70 cm viewing distance. Participants sat in a dimly lit cubicle. The screen background was gray (29 cd/m2).
A trial started with a 500-ms black fixation dot with a white (99 cd/m2) rim (0.11° × 0.11°). Then a memory item was presented for 1,000 ms at the center. After a 1,000-ms delay, a search display was shown, until response. The fixation dot then reappeared for 250 ms, or, in the case of an error, the word “incorrect” (in red). Finally, a memory test was shown until response.
The memory item was a colored disk (radius 1.16°) and remained identical for nine trials (Repetition factor). At the start of a sequence, a color was selected at random from five color categories (red, green, yellow, blue, and purple), with the restriction that the selected category differed from the preceding one. Within each category, the specific hue and chroma varied randomly between any of nine different combinations, chosen on the basis of the Munsell color system (Munsell Color, 1929), such that the brightness of each color was kept constant at around 26 cd/m2, except for yellow (66 cd/m2).
The search display consisted of seven distractor disks (radius 1.16°) and one diamond-shaped target (2.91° × 2.91°), all white-rimmed and all placed on an imaginary circle (radius 4.65°) centered on fixation (Theeuwes, 1992). The target was placed at one of four possible locations (i.e., top left, bottom left, top right, and bottom right). Participants were instructed to localize the diamond as quickly and accurately as possible. The target locations were coupled to keys on the QWERTY keyboard (Q for top left, A for bottom left, P for bottom right, and L for bottom left). One distractor disk was colored. The position of the colored distractor was random but at least two positions away from the target. Crucially, there were two distractor type conditions. In the related condition, the distractor color was the to-be-memorized one. In the unrelated condition, the color was chosen from a different category.
To limit the number of cells, only the first, third, fifth, seventh, and ninth trials in the repetition sequence could contain a memory-matching distractor. In Experiment 1A, the unrelated color was selected randomly on each trial. In Experiment 1B, the unrelated color was fixed for the odd-numbered trials, to make sure that both distractor colors (i.e., related and unrelated) were presented equally often, and thus that the relevant effects in our paradigm were not due to the fact that one type of distractor simply occurred more often (Vatterott & Vecera, 2012).
Each trial ended with a forced choice recognition task in which participants had to select the memory-matching color from three colored disks. Participants could select the memory-matching exemplar by moving an outline by pressing either A (to the left) or L (to the right; random starting point), and could submit their response with either Q or P. Participants thus could keep their fingers on the relevant buttons throughout the trial.
In both experiments, participants completed 18 practice trials and ten experimental blocks of 54 trials each. Each experimental block contained six memory displays: three related and three unrelated distractor trials per repetition (randomly mixed). This resulted in 30 trials per repetition for each distractor condition. After each block, feedback was given on reaction times (RTs; search) and accuracy (search and memory). Participants were encouraged to take a break between blocks.
Results and discussion
Only odd-numbered trials in the sequence of nine trials were analyzed, since only those could contain related distractors. Incorrect trials were removed (Exp. 1A = 1.5 %, Exp. 1B = 1.3 %), and then a two-step trimming procedure was applied. First, trials with RTs shorter than 200 ms and longer than 5,000 ms were excluded. Next, the data were trimmed on the basis of a cutoff value of 2.5 standard deviations (SDs) from the mean RT per participant per condition (Exp. 1A = 1.3 %, Exp. 1B = 2.9 %), resulting in an overall loss of 3.5 % of trials.
The remaining RTs were entered into a repeated measures analysis of variance (ANOVA) with the factors Distractor Type (related, unrelated) and Repetition (1, 3, 5, 7, 9), and with α = .05. A Greenhouse–Geisser correction was applied in the case of sphericity violations. Experiment (1A, 1B) was added as a between-subjects factor to assess whether any effects were due to repetition of the distractor in the search display, rather than in memory. We observed no main effect of experiment (F = 1.44, p = .24), nor did it interact with any of the other factors (all Fs < 1.67, all ps > .16), indicating that there was no effect of distractor repetition per se.
We did find a main effect of distractor type [F(1, 28) = 69.39, p < .001, ηp2 = .71], reflecting overall slower RTs in the related relative to the unrelated distractor condition. The main effect of repetition was also significant [F(4, 112) = 27.95, p < .001, ηp2 = .50], as RTs were modulated by repetition of the accessory memory item. Importantly, the distractor type by repetition interaction was highly significant [F(4, 112) = 12.30, p < .001, ηp2 = .31]. This pattern also held when the experiments were analyzed separately (all ps < .01; see the supplementary material).
As can be seen from Fig. 2b, memory-based interference with search decreased with repetition. It thus appears that transferring an accessory memory from VWM to LTM makes it easier to ignore memory-matching distractors. Also, consistent with previous studies (e.g., Carlisle et al., 2011), this transition was rather rapid, occurring within one or two trials.
However, Fig. 2b also suggests that as learning progressed, interference did not continue to decrease, but reemerged somewhat toward the end of the sequence. An ANOVA in which we only entered Repetitions 5, 7, and 9 indeed suggested an interaction [F(2, 56) = 6.10, p < .01], but we emphasize that this analysis was post hoc. This resurgence was unexpected, and we will return to it after Experiment 2. However, here it could have driven the distractor type by repetition interaction, so to make sure that the initial repetition-driven decline in interference was reliable, we repeated the ANOVA, but only for Repetitions 1 to 5. This again revealed a significant distractor type by repetition interaction [F(2, 56) = 18.06, p < .001].
Experiment 2: interference from learned prospective memories is under cognitive control
Experiment 1 showed that, with repetition, a prospective memory ceases to interfere with the current task. In Experiment 2, we tested the extent to which learning-based decrement in interference is under cognitive control. With a relatively long series of repetitions, as in Experiment 1, offloading the accessory memory to LTM as soon as possible may be a viable strategy. However, we hypothesized that when memory content is more changeable, observers may choose to keep VWM online, to anticipate updating with the new content and prevent spurious learning of soon-to-be-irrelevant representations (Braver & Cohen, 2000; O’Reilly & Frank, 2006). An adaptive system should thus be able to choose between VWM- and LTM-based processing, depending on the task context. To test this, in Experiment 2 we compared a blocked condition in which the accessory memory item was repeated for nine trials (as in Exp. 1) to a blocked condition in which the memory changed every three trials. On the basis of Experiment 1, we predicted a repetition-related decrease in interference in the nine-repetitions block. We were also curious whether we could replicate the resurgence of interference near the end of the sequence. However, for the three-repetitions condition, in which items changed frequently, we hypothesized that observers are inclined to keep VWM involved, and thus memory-based interference was predicted to be sustained across repetitions.
The method was the same as in Experiment 1A, except for the following changes: Eighteen new volunteers (15 females, three males; age 18–26 years, M = 21 years) participated. The participants completed separate blocks, in which the memory color was repeated either three or nine times. Note that the three- and the nine-repetitions blocks only differed in the number of repetitions. All participants completed 16 experimental blocks (eight blocks for each condition), in counterbalanced order. Each block contained six memory displays, such that a three-repetitions block contained 18 trials and a nine-repetitions block contained 54 trials. This resulted in 24 trials per repetition for each distractor condition.
Results and discussion
These results show that the repetition-based decrement in interference from prospective memories depends on the anticipated length of the repetition sequence. Consistent with our hypothesis, when the memory task changed frequently, memory-based interference did not diminish, whereas it did for more stable sequences of nine representations. This suggests that, with learning, a prospective memory is not automatically shielded from the current task, but that this transfer is under cognitive control. When a new color memory is expected soon, VWM for color is kept active, leading to interference from the current color memory.
Experiment 3: reactivating VWM is under cognitive control
The finding that the initial offloading from VWM to LTM is under cognitive control may also provide an explanation for the resurgence of memory-based interference. Just as observers decide to keep VWM online when they anticipate an imminent change in color (as in the three-repetitions blocks of Exp. 2), they may decide to bring VWM back online at the end of a nine-repetition sequence, to facilitate the encoding of a new, upcoming memory. A study by Reinhart and Woodman (2014) provided evidence that VWM can be strategically reinstated, even though the object in memory no longer requires VWM. They used the CDA to track VWM’s involvement during repeated search, and found that with repetition of the search target the CDA amplitude decreased, consistent with an offloading of the target representation to LTM. Importantly, when a large reward was then promised, the CDA amplitude returned to starting levels, suggesting that VWM was strategically recruited again in anticipation of the rewarding trial. Interestingly, other experiments using the CDA method have also provided hints of reinstatement of VWM toward the end of repetition sequences, without any promise of reward (Exp. 3 in Carlisle et al., 2011; Exp. 2 in Gunseli, Meeter, & Olivers, 2014).
Although our observers had no obvious incentive to reinstate VWM, in all experiments the length of the sequence was kept constant, and at least a subset of observers might have developed an estimate of when a new color would appear, and resumed VWM-based control in its anticipation. Here we tested whether the strategic reinstatement of VWM indeed accounts for the observed reemergence of memory-based interference. We used the same procedure as in Experiment 1, except that at the end of the sequence, participants first had to perform a different task that did not require them to remember a new color. Only after completing this intermediate task did they return to the color memory task. We reasoned that there should now be no reason to reinstate VWM for color toward the end of the repetition sequence, and thus we should not observe a return of memory-based interference. Alternatively, it is possible that no active anticipation was involved, and that the increased interference resulted from, for example, automatic long-term priming mechanisms taking over the guidance of attention as learning accumulated (Kruijne & Meeter, 2015). In that case, we should still observe increased interference at the end of the repetition sequence.
The method was the same as in Experiment 1A, except for the following changes. Fifteen new volunteers (seven females, eight males; age 19–30 years, M = 24 years) participated. Here, after every nine trials, following a 2,000-ms interval, participants switched to an intermediate task. To minimize memory requirements, this task was very similar to the search task; however, no color was involved (i.e., none of the distractor disks was colored). To further ensure that this task was easily dissociated from the regular search task, the stimuli were not presented simultaneously, but appeared sequentially. Every 750 to 1,250 ms, a randomly selected item was added to the display, until all items were visible. Participants had to respond immediately when the diamond appeared by pressing the button that corresponded to the matching search location.
Results and discussion
The lack of any resurgence of memory-based interference suggests that it can be suppressed, and is thus under cognitive control. Whereas the previous experiments showed resurgence of memory-based interference prior to the storage of a new color in VWM, the present experiment showed no such increased interference prior to a task for which little working memory was required, and certainly no working memory for color. Taken together, the reemergence of memory-based interference appears to have been driven by the anticipated role that VWM would play in the updating of task-relevant memories.
We investigated how learning representations that are only prospectively relevant (i.e., for a later task) affect visual attention on the current task. Earlier evidence had indicated that such accessory memory items, when represented in VWM, interfere with current selective attention tasks (Olivers et al., 2006; Soto et al., 2005). Here, we assessed whether repeated accessory memories continue to guide attention when they are transferred from VWM to LTM. All experiments demonstrated that the first occasion an accessory item was encountered, and thus presumably held in VWM, memory-based interference occurred during search. However, within three repetitions of the accessory memory, interference reached a minimum, indicating that with learning, its effect on attentional selection rapidly diminishes.
Previous research indicated that memory-related capture should be attributed to active maintenance of the memory representation, and not to passive priming effects (Olivers et al., 2006; Soto et al., 2005). The present results support this finding by demonstrating that priming contributes little, even when the accessory memory is presented repeatedly. Whereas memory improved across repetitions, memory-related attentional capture actually decreased. Furthermore, the speed with which active maintenance of the color appeared to be reduced matches similar reductions found in studies using the CDA as a measure of activity (Carlisle et al., 2011; Gunseli, Meeter, & Olivers, 2014; Gunseli, Olivers, & Meeter, 2014).
The finding that memory improves while attentional guidance diminishes is in line with the idea that not the quality, but the status of a memory representation determines whether or not it will guide attention (Hollingworth & Hwang, 2013; Olivers, Peters, Houtkamp, & Roelfsema, 2011; van Moorselaar, Theeuwes, & Olivers, 2014), and extends this idea to LTM. Where others have assumed increased attentional guidance from LTM as search performance improved with target repetition (Carlisle et al., 2011; Gunseli, Meeter, & Olivers, 2014; Gunseli, Olivers, & Meeter, 2014; Reinhart & Woodman, 2014), here we found decreased attentional guidance from repeated accessory memories. Taken together, this points to an important functional dissociation within LTM: Observers can learn either to use a currently relevant memory for attention or to shield a prospectively relevant memory from attention.
Also, the results demonstrate that the learning-based attentional status of memory contents is context-dependent, since both the initial reduction and the subsequent reappearance were sensitive to overall task expectations. We believe that these effects should be attributed to the level of VWM involvement during maintenance. As was shown in Experiment 2, even though three repetitions were sufficient to reduce memory-based interference, this reduction was only observed when participants knew the memory would be repeated on subsequent trials. In contrast, when a memory update was expected soon, interference did not diminish, suggesting that the memory was kept active in VWM, and thus guided attention. Similarly, Experiment 3 showed that the increased interference toward the end of the repetition sequence was no longer present when observers switched to an intermediate task that required no color memory whatsoever before starting a new memory sequence. This argues against long-term priming mechanisms driving the increase in memory-based interference observed in Experiments 1 and 2. Instead, we propose that VWM was reinstated in expectation of a new color memory, even though the current memory no longer required VWM. A similar mechanism of reinstating VWM was shown by Reinhart and Woodman (2014). In that study, observers had a clear incentive to reinstate VWM, because it served to supplement the cognitive control already afforded by LTM. In our study, VWM reinstatement, if anything, interfered with search, yet it was not suppressed. This raises the question of whether VWM was strategically reinstated to regain task control, or whether VWM is automatically recruited when anticipating a changing task environment. Future research will be necessary to dissociate these options.
Finally, it is worth pointing out that, even though there was a rapid reduction with repetition, interference from the accessory memory item never completely disappeared (except in Exp. 1A), a finding consistent with earlier studies that have shown some remaining markers of VWM involvement after search target repetition (Carlisle et al., 2011; Gunseli, Meeter, & Olivers, 2014; Gunseli, Olivers, & Meeter, 2014; Reinhart & Woodman, 2014). One could argue that VWM needs to stay involved in order to maintain at least some level of control and flexibility, given the complexity of the task, as well as the fact that the specific memory representations are only relevant within the context of the experiment. Alternatively, it is possible that the handoff from VWM to LTM is actually complete, and that attentional guidance by accessory memories from LTM is simply less strong. Future research will need to dissociate these options.
We conclude that learning prospective memories affects their attentional status, but at the same time increases cognitive flexibility, since different memory systems can be recruited depending on the task context.
- Braver, T. S., & Cohen, J. D. (2000). On the control of control: The role of dopamine in regulating prefrontal function and working memory. In S. Monsell & J. Driver (Eds.), Control of cognitive processes: Attention and performance XVIII (pp. 713–737). Cambridge, MA: MIT Press.Google Scholar
- Color, M. (1929). Book of color. Baltimore, MD: Munsell Color Co.Google Scholar
- Gunseli, E., Meeter, M., & Olivers, C. N. L. (2014a). Is a search template an ordinary working memory? Comparing electrophysiological markers of working memory maintenance for visual search and recognition. Neuropsychologia, 60, 29–38. doi:10.1016/j.neuropsychologia.2014.05.012 CrossRefPubMedGoogle Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.