Advertisement

Attention, Perception, & Psychophysics

, Volume 79, Issue 6, pp 1643–1651 | Cite as

Looking sharp: Becoming a search template boosts precision and stability in visual working memory

  • Jason Rajsic
  • Natasha E. Ouslis
  • Daryl E. Wilson
  • Jay Pratt
Article

Abstract

Visual working memory (VWM) plays a central role in visual cognition, and current work suggests that there is a special state in VWM for items that are the goal of visual searches. However, whether the quality of memory for target templates differs from memory for other items in VWM is currently unknown. In this study, we measured the precision and stability of memory for search templates and accessory items to determine whether search templates receive representational priority in VWM. Memory for search templates exhibited increased precision and probability of recall, whereas accessory items were remembered less often. Additionally, while memory for Templates showed benefits when instances of the Template appeared in search, this benefit was not consistently observed for Accessory items when they appeared in search. Our results show that becoming a search template can substantially affect the quality of a representation in VWM.

Keywords

Visual search Template Visual attention Working memory 

The source of voluntary visual attention—our ability to control what information we will and will not process—has long been debated in psychology (Awh, Belopolsky, & Theeuwes, 2012). One of the main tools to investigate voluntary attention has been visual search, where one attempts to determine whether a particular object is present among an array of objects. To do so, one must maintain a “template,” that is, a mental representation of the object that one is looking for, to know whether the desired object is one of the many objects visible. Visual working memory (VWM), a limited capacity store that maintains visual information in the service of ongoing cognitive operations (see Luck, 2008 for a review), has been proposed to be the cognitive basis of templates used in visual search (Desimone & Duncan, 1995). This has typically been tested by measuring whether items merely stored in memory lead to attentional capture towards memory-matching objects that appear in the context of a search (Soto, Hodsoll, Rotshtein, & Humphreys, 2008; Olivers, Meijer, & Theeuwes, 2006; Olivers, 2009). Whereas many studies have found such memory-driven capture, despite the fact that memory-matching objects are task-irrelevant (but see Woodman & Luck, 2007), this type of attentional capture seems to occur only when templates used for the search task have been practiced for several trials or more (Woodman, Luck, & Schall, 2007; Olivers, 2009; Carlisle, Arita, Pardo, & Woodman, 2011). Indeed, memory-driven capture also tends not to occur when multiple objects are held in VWM (van Moorselaar, Theeuwes, & Olivers, 2014). These findings are consistent with the proposal that novel search targets occupy a special state in VWM (Olivers, Peters, Houtkamp, & Roelfsema, 2011), such that one item can serve as a template, which can interact with ongoing perceptual processing, but other items held as “accessory” items that cannot interact.

Whereas there is converging evidence that search templates have a special state in VWM (Houtkamp & Roelfsema, 2009; Carlisle et al., 2011; Greene, Kennedy, & Soto, 2015; van Moorselaar et al., 2014), relatively little is known about the properties of these representations. For example, Hollingworth and Hwang (2013) showed that items that do not guide search need not be lower precision but simply deprioritized (van Moorselaar et al., 2014), but this leaves open the question of whether search templates have higher memory precision than accessory items. Evidence from neural data show that search templates, compared with accessory items, are associated with a sustained increase in the activity of relevant visual areas and a selective, transient increase in activation of fronto-parietal and visual areas when the search template, but not when accessory items, appear (Peters, Roelfsema, & Goebel, 2012). Although such neural differences suggest that the representation of search templates in VWM may be qualitatively different from accessory items, a direct measurement of the quality of memory for templates and accessory items is lacking.

What is not lacking is repeated demonstrations that observers are able to prioritize particular representations in VWM (see Souza & Oberauer, 2016, for a recent review). Such studies have relied on the retro-cuing technique, wherein a set of objects are encoded into VWM, and only afterwards is one designated to be the object that will be tested more often than not. When retro-cued, objects can be remembered more often (Murray, Nobre, Clark, Cravo, & Stokes, 2013) and sometimes more precisely (Gunseli, van Moorselaar, Meeter, & Olivers, 2015) than other items. This line of research shows that substantial differences can exist between items held in VWM, supporting the possibility that search templates may be remembered better than accessory items.

It is important to note that with retro-cuing there is an obvious benefit to shifting memory to the cued item, as memory for the cued items is tested more often than memory for uncued items. Retro-cue benefits are larger when retro-cues more often predict the tested item (Gunseli et al., 2015), consistent with the notion that participants will increasingly bias internal attention to cued items as the payoff increases, assuming participants intend to minimize their performance errors. This is not to say that shifts of attention are completely strategic; Berryhill, Richmond, Shay, and Olson (2012) have shown that retro-cuing effects persist when retro-cues do not predict the tested item, albeit after participants had gained experience with retro-cues that were completely valid. Similarly, Li and Saiki (2014) showed that a cue that loses its predictive validity later in a trial nonetheless produces a retro-cue effect, albeit when mixed with trials in which this cue is helpful on half of the overall trials. Our experiments differ in that the cues used to indicate which item should be searched for were always at chance in terms of predicting the tested item, therefore any differences between cued and uncued items (or, in the present terminology, template and accessory items) cannot be due to participants’ intention to minimize error in memory reports, but presumably instead to the need to represent templates with greater fidelity.

The question addressed in the current study is straightforward: does assigning template status to a memory, holding constant the testing probability of template and accessory items, nonetheless affect the quality of memory of the template item akin to that observed in retro-cuing? Indeed, if our results do show an enhancement of memory quality for templates compared to accessory items analogous to studies using retro-cues (Gunseli et al., 2015; Murray et al., 2013), then this would provide converging evidence for the notion that search templates require internal attention (Olivers et al., 2011). Such a finding would be consistent with Souza, Rerko, and Oberauer (2015), who found that memory error is lower when specific items are cued to be “refreshed” during the retention interval, lending support to the notion that attention can be shifted in memory even when it produces no clear performance gains. Relatedly, van Moorselaar et al. (2014) showed in their final experiment that memory-driven capture selectively occurred for the one item (out of two) in memory that participants expected to be tested on first, even when both were ultimately tested. Indeed, memory performance was better for the first item tested than the second. Both studies suggest that memory resources can be unevenly distributed in VWM even when these altered distributions might not be expected to reduce memory error. The goal of the present study was to assess directly the quality of memory for search templates compared to accessory items, with the hypothesis that assigning “search template” status to one item would shift resources in VWM towards the template, improving the quality of its memory. To do so, we conducted two experiments in which one of two items encoded into VWM was designated a search template, and subsequently measured the quality of memory for this search template, or an accessory item in VWM.

Experiment 1

In Experiment 1, we compared the memory for templates and accessory objects after a visual search to memory for identical objects when no search occurred to determine their relative memory quality. If objects serving as search templates indeed enter a special representational state in VWM, they should show superior memory to accessory objects.

Method

Participants

Eighteen undergraduate students (5 males) participated in this experiment as partial fulfillment of course credit for a first-year psychology course. All participants reported normal vision. Three participants were excluded due to excessive incorrect search responses, leading to fewer than 50 trials in one or more cells of the factorial design. Given that we intended to model memory performance using the Bays’ three-component model (Bays, Catalao, & Husain, 2009), we excluded these participants to preclude the possibility of poor model-fitting from small number of trials. Our goal was to collect approximately 15 participants whose data could be included as it is in the typical range of the number of participants collected in experiments on memory-driven capture (Olivers, 2009).

Materials and Procedure

Stimuli were created and presented using Matlab and the Psychophysics toolbox 3.0.8 (Kleiner et al., 2007). Each participant completed 756 trials (for a sample trial, see Fig. 1), over two, 1-hour sessions, broken up into blocks of 54 trials (14 blocks in total). Each trial began with 1,000-ms display of a fixation cross, followed by a 500-ms presentation of the memory stimuli—distinct shapes (created by modulating the radius of a circle using sine, square, or saw waves with power at 1, 2, or 3 cycles within the circumference, approximately 7° in diameter; 9 different shapes in total) centered approximately 8° left and right of fixation on the horizontal meridian. The colors of the shapes were randomly selected, from a range of eight evenly spaced angular values, with a randomly applied rotation; that is, the available color values changed, but their relative differences did not. The angular values defined colors on an imaginary circle in L*A*B color space centred on [50, 20, 35] with fixed a radius of 50.
Fig. 1

Sample trial for the experimental task. Responses for the visual search and memory tests were made using the mouse. On memory tests, the tested shape was drawn initially in gray. After the mouse was moved to a color on the color wheel, the shape was filled in with this color

After the offset of the memory items, a 500-ms blank display preceded the search instructions display. The instruction display, which lasted 500 ms, consisted of a left-facing arrow, a right-facing arrow, or an X drawn at fixation. Participants were instructed that if an arrow appeared, then they were to search for the object that had appeared on that side of fixation just previously in the upcoming display. This allowed us to designate one object in memory as the search template, and one object as the accessory item. If an X appeared, participants were told that they did not have to respond to the upcoming display and that it would offset on its own. This allowed us to measure the baseline memory for two items in VWM when no search occurred but with identical stimulus conditions during the retention interval. Each cue type (left arrow, right arrow, or an X) was equiprobable.

Next, the search display, which consisted of six peripheral shapes evenly spaced along an imaginary circle around fixation, appeared. The shapes were drawn approximately 6° from fixation and were the same size as the shapes presented in the memory array (approximately 7° in diameter). These shapes were colored using the six nonsampled color values from the array of values used to select the memory colors, except on trials where one shape matched the memory shapes, in which case five nonsampled colors were used for the nonmemory-matching shapes. Participants were instructed to use the computer mouse to report whether the search target was present or absent on search trials. For nonsearch trials, the search display offset after a random amount of time (drawn from a log-normal distribution with mean 0 and SD 0.5, with the constraint that samples could not exceed 4,000 ms, producing a mean time of 1,120 ms and standard deviation [SD] of 560 ms). Actual search RTs (when excluding only trials with >4,000 ms RTs for direct comparison) had a mean of 1,040 ms and an average SD of 557 ms.

To assess how visual repetition affects memory for search templates and accessory items, for two thirds of trials one of the shapes in the search display matched one of the objects in VWM (Fig. 1). For half of these trials, this shape was the search template (i.e., the search was target present), and for the other half, the shape was the accessory item (i.e., the search was target absent). As such, stimulus repetition effects could be measured independently of memory status (either template or accessory).

After the offset of the search array, and a 500-ms delay, the cued recall memory test occurred. Memory error was measured by presenting a gray shape at fixation accompanied by a peripheral color wheel. The shape matched one of the two shapes presented at the beginning of the trial, and participants were instructed to report the associated color for the probed shape by clicking a color on the color wheel. Participants again used the computer mouse to select the color that they believed belonged with the presented shape. The next trial began following this response.

Results and Discussion

For all analyses, trials were excluded when search RT fell below 100 ms or two standard deviations above a participant’s search RT. Mean correct search response time (RT) was affected by target presence, F(2, 28) = 4.26, p = 0.024, η2 p = 0.23, with search response times being shorter when the target was present (M = 919 ms, SE = 56 ms) but not differing between target absent trials when the accessory object appeared (M = 991 ms, SE = 61 ms) and when neither memory object appeared (M = 993 ms, SE = 61 ms), demonstrating that participants indeed searched for the instructed template. Accuracy was high, M = 94.8%, SE = 1.0%, but a slight speed-accuracy trade-off occurred, F(2, 28) = 7.91, p = 0.002, η2 p = 0.36, such that 3% more errors occurred on target present trials than target absent trials, which we attribute to the low prevalence of targets (33%; Wolfe et al., 2007).

Memory performance was first evaluated in terms of raw error, defined as the precision (1/sample SD in degrees) of report errors (Fig. 2a). Memory type (baseline, accessory, and template; F(2, 28) = 78.38, p < 0.001, η2 p = 0.85) and repeated exposure (neither present [NP], nontested present [NTP], and tested present [TP]; F(2, 28) = 50.13, p < 0.001, η2 p = 0.78) both affected precision (Fig. 2a). The two factors also interacted, F(4, 56) = 51.07, p < 0.001, η2 p = 0.78), such that memory improved when either of the two remembered items appeared in the search array, even though participants were not required to attend to these items on baseline trials, ts > 4.38, ps < 0.001. While it is intuitive that seeing a tested item during the retention interval improves memory performance, it is somewhat surprising that seeing the non-tested item also improves memory. One possible reason for this is that in both of these trial types fewer non-remembered colors are presented, potentially reducing visual interference. Alternatively, seeing either item again might reduce ambiguity regarding the specific color-shape bindings being remembered, which could improve cued-recall by reducing swap errors.
Fig. 2

Effects of memory type (squares: baseline, triangles: accessory Items, circles: search templates) and repeated exposure (x-axis: NP; neither present; NTP; nontested present; TP: tested present) on memory performance. A Raw memory precision (1/σ in degrees). B Estimated memory SD. C Estimated p(Correct). D Estimated p(Swap). Mean values are depicted with “+” markers, with “−” markers depicting the mean ±1 SE; square, triangle, and circle markers depict individual participants’ mean values, with random x jitter added to reduce occlusion

On the trials where one item was a search template, precision of the template was better following searches where a template-matching item appeared in search compared to when neither the template nor the accessory item appeared in search (i.e., tested-present [TP] vs. neither present [NP], for search templates in Fig. 2), t(14) = 8.21, p < 0.001, but was not affected when the accessory item appeared (i.e., nontested present [NTP[vs. NP), t(14) = 1.61, p = 0.13. Accessory precision, on the other hand, benefitted from both the appearance of the accessory item in search (i.e., TP vs. NP for accessory item memory), t(14) = 2.52, p = 0.024, and, arguably also, the appearance of the template (i.e., NTP vs. NP), t(14) = 1.89, p = 0.08.

One potential reason for differences in memory between repeated exposure conditions is differences in overall search time, given that target present searches were faster than target absent searches. To test this possibility, we performed a median split on search display times for no search trials, comparing subsequent memory precision. Despite a large difference in display times, M short = 689 ms, SE = 3.7 ms, M long = 1349 ms, SE = 7.2 ms, t(14) = 92.73, p < 0.001, memory did not differ, M short = 0.022, SE = 0.0025, M long = 0.026, SE = 0.0029, t(14) = 0.93, p = 0.37. As such, differences in retention duration are unlikely to account for the present findings.

To better understand the nature of the changes in memory quality caused by searching, we applied the three-component model (Bays et al., 2009) to our data, which expresses memory performance as a mixture of three types of responses: correct responses (i.e., responses drawn from a distribution centered around the probed object’s color, with an estimated SD), swap responses (i.e., responses drawn from a distribution centred around the nonprobed object’s color, with the same SD as correct responses), and guess responses (i.e., responses drawn from a uniform distribution, where every color-response is equally likely). Data in each cell of our design, for each participant, were fitted with the model, and the resulting parameter estimates were analysed. The SD of correct responses for baseline VWM in the three repeated exposure conditions (neither present, nontested present, tested present) was 28° (SE = 2.5°), 29° (SE = 2.1°), and 18° (SE = 1.6°), respectively, and estimated p(Correct) for baseline VWM in the three repeated exposure conditions was 0.59 (SE = 0.06), 0.68 (SE = 0.06), and 0.67 (SE = 0.06), respectively. SD of correct responses was determined by memory type, F(2, 28) = 6.19, p = .006, η2 p = 0.31, repeated exposure, F(2, 28) = 4.42, p = 0.022, η2 p = 0.24, and their interaction, F(4, 56) = 6.39, p < 0.001, η2 p = 0.31 (Fig. 2b). However, the probability of a correct response was only affected by memory type, F(2, 28) = 31.37, p < 0.001, η2 p = 0.69, with no interaction with repeated exposure, F(4, 56) = 0.98, p = 0.43, η2 p = 0.07, but a marginal main effect of repeated exposure, F(2, 28) = 3.02, p = 0.065, η2 p = 0.18 (Fig. 2c). Critically, the SD of correct responses was lower for templates even when no memory-matching object appeared in search, t(14) = 2.09, p = 0.055. Repeated-exposure had opposite effects for accessory items and search templates; templates had lower SD on target present trials compared with none-present, target-absent trials, t(14) = 6.35, p < 0.001, and accessory-item present, target-absent trials, t(14) = 9.00, p < 0.001. Accessory items, however, showed no SD reduction when the accessory item appeared in search (i.e., TP vs. baseline for Accessory items), t(14) = 0.01, p = 0.99, η2 p = 0.001. Instead, their SD was lower when the template appeared in search (i.e., NTP vs. baseline), t(14) = 2.68, p = 0.02.

An analysis of the probability of swap errors (Fig. 2d) revealed a main effect of memory type, F(1, 14) = 7.13, p = 0.003, η2 p = 0.34, with a large difference in the probability of a swap errors between template-tested trials and accessory-tested trials, F(1, 14) = 8.86, p = 0.01, η2 p = 0.39. This shows that true “swaps” were not occurring, and participants were likely reporting the only color they knew (the template color) when the accessory item was tested. Because this would imply that the accessory color was unavailable, instead of truly swapped with the template color, then these excess swap responses on accessory-tested trials are better considered as guesses—trials in which the accessory color was unknown. Thus, we conclude that, in our task, both swap and guess responses reflected a loss of information about the tested item. The estimated p(Swap) for baseline VWM in each repeated exposure condition was 0.19 (SE = 0.04), 0.15 (SE = 0.03), 0.09 (SE = 0.01), respectively.

Finally, we analysed search performance as a function of subsequent memory quality. To equate the number of trials, we performed a median split on mean squared memory error for both template and accessory memories, thus comparing search when memory was “good” to when it was “bad.” Search was faster overall when memories were recalled with less error, F(1, 14) = 14.83, p = 0.002, η2 p = 0.51, M good = 938 ms, SE good = 54 ms, M bad = 998 ms, SE bad = 60 ms. In addition, searches were faster when the template was tested, which reflects a larger contribution of the memory quality of templates to search speed; trials with good template memory showed faster search than trials with good accessory memory, t(14) = 3.66, p = 0.003, but no such difference occurred when memory was bad, t(14) = 0.38, p = 0.71. No interactions were found between memory quality and search conditions (template present, accessory present, neither present), Fs < 0.98, ps > 0.38, corroborating the conclusion that, when a template is held in VWM, accessory items do not interfere with search (Woodman, Carlisle, & Reinhart, 2013; Hollingworth & Hwang, 2013).

Experiment 2

Experiment 1 showed that objects represented in VWM that become search templates are remembered both more often and with greater precision. However, it is not clear whether the change in memory states occurs in anticipation of search or during the search itself. To resolve this ambiguity, we ran a second experiment where participants were again told which of two remembered items needed to be search for but included trials where no search occurred. If changes in memory states occur when a representation is selected for use as a search template, then we should observe differences between templates and accessory items even when no search is performed.

Methods

Participants

Twenty-four undergraduate students, enrolled in a first-year Psychology course at the University of Toronto, were recruited as participants in Experiment 2. All provided informed consent before participating, and none had participated in Experiment 1.

Materials and Procedure

Stimuli and Procedure were identical to Experiment 1 with the following exceptions. First, participants completed only a single, 1-hour session consisting of 288 trials. This reduction in trial numbers was motivated by exploratory analysis of data from Experiment 1, which showed that the model-fitted memory data did not appreciably differ when only the first of the two sessions for each participant was analysed. Second, the trials with “no search” cues (Xs) from Experiment 1 were removed. Instead, four possible trial types followed a search cue (an arrow pointing left or right). No search trials occurred when, 500 ms after the offset of the search cue, the memory probe display was presented. These trials occurred in one quarter of all trials. For the remaining three fourths of trials, the search display was presented with neither of the memory items present, with the accessory item present, or with the template item present. A schematic of the possible events in a given trial for Experiment 2 is depicted in Fig. 3.
Fig. 3

Schematic of events in Experiment 2. Not depicted are the 1,000-ms fixation period at the beginning of each trial, the 500-ms interstimulus interval between the offset of the template cue and the search display, and the 500-ms interstimulus interval between the offset of the search array and the memory probe, all of which consisted of a fixation mark on a blank screen

Results and Discussion

Five of the 24 participants were excluded from data analysis for having poor search accuracy (<80% correct). Once again, trials with overly fast (<100 ms) or overly slow (>2 SD of overall search RT) were excluded in all analyses. For the remaining participants, correct mean search time surprisingly did not differ between no target, accessory present, and template present trials, F(2, 36) = 0.49, p = 0.62, η2 p = 0.03, M template present = 737 ms, SE template present = 23 ms; M accessory present = 749 ms, SE accessory present = 22 ms; M neither present = 753 ms, SE neither present = 25 ms. Search accuracy, however, did differ, F(1, 36) = 16.90, p < 0.001, η2 p = 0.48, such that accuracy was lower on template present trials, M = 83%, SE = 1.9%, than on accessory present trials, M = 93%, SE = 1.4%, and neither present trials, 94%, SE = 1.4%. While unusual, this reduction in accuracy for target present trials may have occurred because of the low prevalence of targets in our experiment, as in Experiment 1.

Inverse root mean squared memory error (Precision) was lower for template than accessory items, F(1, 18) = 39.03, p < 0.001, η2 p = 0.68, and also was affected by search condition (i.e., the no-search [NS], neither present [NP], nontested present [NTP], and tested-present [TP]), F(3, 54) = 12.03, p < 0.001, η2 p = 0.40 (Fig. 4). For both template and accessory items, searching incurred a memory cost. Furthermore, template memory improved when the template appeared in search compared with when the accessory item appeared in search, t(18) = 2.84, p = 0.01, but the reverse was not true; seeing an accessory item in search did not improve accessory item memory, t(18) = 0.39, p = 0.70.
Fig. 4

Memory results for Experiment 2. The x-axes depict memory for the four trial conditions (NS: no search; NP; neither present; NTP; nontested present; TP: tested present) for accessory items (red triangles) and templates (green circles). A Raw memory precision (1/σ of memory report errors). B Estimated SD of correct memory responses. C Estimated probability of a correct response. D Estimated probability of a swap response

To determine the nature of these memory errors, we again analysed memory parameter estimates given by the three-component model (Bays et al., 2009). Memory SD was again better for templates than accessory items, F(1, 18) = 5.89, p = 0.03, η2 p = 0.25, and better after no-search (NS) trials and TP (target present) trials for both template and accessory items, F(3, 54) = 3.47, p = 0.022, η2 p = 0.16. However, the benefit in memory SD of seeing the tested object in search was greater for the template than for the accessory item (i.e., template vs. accessory, for TP trials), t(18) = 2.33, p = 0.032. Critically, even when no search occurred, template memory SD was lower than accessory memory SD (i.e., template vs. accessory for NP trials), t(18) = 3.70, p = 0.002, showing that template precision is increased in anticipation of search.

Estimated p(Correct) also was better for templates than accessory items, F(1, 18) = 18.82, p < 0.001, η2 p = 0.51, and was affected by search, F(3, 54) = 5.95, p = 0.001, η2 p = 0.25. As shown in Fig. 4, performing a visual search was more deleterious to accessory items than templates. Whether this is due to the increased retention intervals associated with search trials or the visual and cognitive interference that they likely produced is not clear. However, even when no search occurred, templates were more often remembered than accessory items, t(18) = 2.68, p = 0.015. Overall, these results show that changes in the representational status of objects in VWM occur in anticipation of, and not only as a consequence of, visual search.

We again analysed correct search RT as a function of memory quality, as in Experiment 1. As in Experiment 1, searches were faster when memory quality was higher, F(1, 18) = 8.82, p = 0.008, η2 p = 0.33, M good = 736 ms, SE good = 22 ms, M bad = 757 ms, SE bad = 21 ms. No other differences were observed, Fs < 1.65, ps < 0.21.

General Discussion

In the present experiments, we measured the quality of memory for objects in VWM that were (templates) and were not (accessory items) used to guide search. Overall, we found that Templates were recalled with greater precision and were less likely to be forgotten. Our inclusion of baseline conditions showed that search templates and accessory items compete for limited memory resources; when one object in VWM became a template, it caused the other item to be forgotten more often. Seeing an object held in memory during the context of visual search also improved its precision, although this did not always occur for accessory items. Experiment 2, however, showed that actually searching is not necessary for such a change to occur. Templates were remembered more often and with more precision even when no search occurred. These results paint a picture of how VWM representations are modulated by visual search and show that search templates are not just prioritized, but better represented (Olivers et al., 2011). However, they also go beyond this proposal in showing a representational cost for accessory items: Search templates’ colors were nearly always recalled at test, whereas accessory items’ colors were correctly recalled on approximately half of the memory tests; approximately 15% less often than our baseline VWM condition.

The present data allow us to distinguish between states in VWM based on memory quality; accessory items are fragile and have lower precision, whereas items being used for concurrent tasks (e.g., search templates) are robust and have relatively higher precision representations. Although it is too early to tell whether these findings primarily reflect task demands (e.g., the requirement to maintain color-shape bindings, the amount of color-precision required to distinguish targets from nontargets in search) or more fundamental differences in representation between states in VWM, we nonetheless provide initial evidence that the need to use a representation for search can affect its memory representation.

The present data fill a gap in previous investigations of the relationship between memory precision and search guidance. Whereas Hollingworth and Hwang (2013) showed that memories that do not guide attention are not necessarily less precise than memories being actively maintained, we show that memories being actively used for search are more precise and stable than accessory items. Additionally, Dowd, Kiyonaga, Beck, and Egner (2015) showed that the primary difference between instances in which memory-driven capture occurs and does not occur seems to lie in the probability that a memory is maintained, rather than the precision with which it is held. However, this result speaks to dynamics of whether a single item, which can vary in its task-relevance, affects search. In our task, two items in memory likely competed for representation, and thus the differences in precision may have resulted in a shift in representational resources towards the Template and away from the Accessory item due to its momentary task-relevance. An intriguing finding from Experiment 1 was that memory precision for the accessory item was, counter intuitively, impaired when that very item appeared in search. This finding may reflect the operation of distractor-suppression mechanisms that occur during search (Lamy, Tsal, & Egeth, 2003; Emrich, Al-Aidroos, Pratt, & Ferber, 2010); however, we are hesitant to draw strong conclusions from this finding given that it did not re-emerge in Experiment 2.

Our results support and extend the proposal of Olivers et al. (2011), who suggest that visual memories used to guide attention exist in a different state than other visual memories (see also Carlisle & Woodman, 2011). We have provided empirical evidence that the representational quality of Templates is superior to Accessory items. Our findings are likely related to the shifts in memory quality found using retro-cues (Murray et al., 2013; Gunseli et al., 2015) to the extent that guided visual search requires a form of internally focused attention within visual working memory. Indeed, the present design is similar to the “retro-cuing” paradigm used to study voluntary shifts of attention within VWM (Griffin & Nobre, 2003; Murray et al., 2013) but without the change in testing probability (see Zokaei, Manohar, Husain, & Feredoes, 2014; Zokaei, Ning, Manohar, Feredoes, & Husain, 2014; and van Moorselaar et al., 2014 for nonsearch alternatives). Despite this difference, the present effect and the retro-cue effect may reflect common mechanisms. In this case, the suggestion that templates occupy a special state in visual working memory can be seen as an application of Oberauer’s concept of the Focus of Attention (Oberauer, 2002); a single-item capacity state in working memory that maintains the representation currently being used for a mental task. In any case, critical future research needs to address underlying mechanisms behind these state differences, which could be due to differences in the temporal dynamics of task-relevant memories (Kiyonaga & Egner, 2014), differences in representational resources (e.g., active neural representation for Templates; Lewis-Peacock, Drysdale, Oberauer, & Postle, 2012), or even both.

In summary, by measuring memory for VWM representations that guide search and those that do not, we have provided direct evidence for a privileged state in VWM for search templates over nonsearch items. This is not to say that voluntary attention necessarily requires VWM (Carlisle et al., 2011), but when it does, search templates enjoy a representational benefit in VWM. Becoming a selection template thus appears to shift VWM resources for the upcoming search task, demonstrating the role of internal attention in visual selection (Kiyonaga & Egner, 2013).

Notes

Author note

Funding provided by an NSERC Discovery grant (194537) to Jay Pratt, as well as an NSERC Doctoral Scholarship (PGS-D) to Jason Rajsic. The authors thank Dorothy Yu, Harendri Perera, and Kai Zhou for their assistance with this project.

References

  1. Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012). Top-down versus bottom-up attentional control: A failed theoretical dichotomy. Trends in Cognitive Sciences, 16(8), 437–443.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Bays, P. M., Catalao, R. F., & Husain, M. (2009). The precision of visual working memory is set by allocation of a shared resource. Journal of Vision, 9(10).Google Scholar
  3. Berryhill, M. E., Richmond, L. L., Shay, C. S., & Olson, I. R. (2012). Shifting attention among working memory representations: Testing cue type, awareness, and strategic control. The Quarterly Journal of Experimental Psychology, 65(3), 426–438.CrossRefPubMedGoogle Scholar
  4. Carlisle, N. B., Arita, J. T., Pardo, D., & Woodman, G. F. (2011). Attentional templates in visual working memory. The Journal of Neuroscience, 31(25), 9315–9322.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Carlisle, N. B., & Woodman, G. F. (2011). When memory is not enough: Electrophysiological evidence for goal-dependent use of working memory representations in guiding visual attention. Journal of Cognitive Neuroscience, 23(10), 2650–2664.CrossRefPubMedPubMedCentralGoogle Scholar
  6. Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18(1), 193–222.CrossRefPubMedGoogle Scholar
  7. Dowd, E. W., Kiyonaga, A., Beck, J. M., & Egner, T. (2015). Quality and accessability of visual working memory during cognitive control of attentional guidance: A Bayesian model comparison approach. Visual Cognition, 23(3), 337–356.CrossRefGoogle Scholar
  8. Emrich, S. M., Al-Aidroos, N., Pratt, J., & Ferber, S. (2010). Finding memory in search: The effect of visual working memory load on visual search. The Quarterly Journal of Experimental Psychology, 6(8), 1457–1466.CrossRefGoogle Scholar
  9. Greene, C. M., Kennedy, K., & Soto, D. (2015). Dynamic states in working memory modulate guidance of visual attention. Evidence from an n-back paradigm. Visual Cognition, 235, 546–560.CrossRefGoogle Scholar
  10. Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive Neuroscience, 15(8), 1176–1194.CrossRefPubMedGoogle Scholar
  11. Gunseli, E., van Moorselaar, D., Meeter, M., & Olivers, C. N. L. (2015). The reliability of retro-cues determines the fate of noncued visual working memory representations. Psychonomic Bulletin & Review, 22, 1334–1341.CrossRefGoogle Scholar
  12. Hollingworth, A., & Hwang, S. (2013). The relation between visual working memory and attention: Retention of precise color information in the absence of effects on perceptual selection. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 368, 20130061.CrossRefPubMedPubMedCentralGoogle Scholar
  13. Houtkamp, R., & Roelfsema, P. R. (2009). Matching of visual input to only one item at any one time. Psychological Research, 73(3), 317–326.CrossRefPubMedGoogle Scholar
  14. Kiyonaga, A., & Egner, T. (2013). Working memory as internal attention: Toward an integrative account of internal and external selection processes. Psychonomic Bulletin & Review, 20(2), 228–242.CrossRefGoogle Scholar
  15. Kiyonaga, A., & Egner, T. (2014). Resource-sharing between internal maintenance and external selection modulates attentional capture by working memory content. Frontiers in Human Neuroscience, 8. doi: 10.3389/fnhum.2014.00670
  16. Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What’s new in Psychtoolbox-3. Perception, 36(14), 1.Google Scholar
  17. Lamy, D., Tsal, Y., & Egeth, H. E. (2003). Does a salient distractor capture attention early in processing? Psychonomic Bulletin & Review, 10(3), 621–629.CrossRefGoogle Scholar
  18. Lewis-Peacock, J. A., Drysdale, A. T., Oberauer, K., & Postle, B. R. (2012). Neural evidence for a distinction between short-term memory and the focus of attention. Journal of Cognitive Neuroscience, 23(1), 61–79.CrossRefGoogle Scholar
  19. Li, Q., & Saiki, J. (2014). The effects of sequential attention shifts within visual working memory. Frontiers in psychology, 5, 1–10.Google Scholar
  20. Luck, S. J. (2008). Visual short-term memory. In S. J. Luck & A. Hollingworth (Eds.), Visual memory (pp. 43–108). New York: Oxford University Press.CrossRefGoogle Scholar
  21. Murray, A. M., Nobre, A. C., Clark, I. A., Cravo, A. M., & Stokes, M. G. (2013). Attention restores discrete items to visual short-term memory. Psychological Science, 24(4), 550–556.CrossRefPubMedPubMedCentralGoogle Scholar
  22. Oberauer, K. (2002). Access to information in working memory: Exploring the focus of attention. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(3), 411.PubMedGoogle Scholar
  23. Olivers, C. N. (2009). What drives memory-driven attentional capture? The effects of memory type, display type, and search type. Journal of Experimental Psychology: Human Perception and Performance, 35(5), 1275.PubMedGoogle Scholar
  24. Olivers, C. N., Meijer, F., & Theeuwes, J. (2006). Feature-based memory-driven attentional capture: Visual working memory content affects visual attention. Journal of Experimental Psychology: Human Perception and Performance, 32(5), 1243.PubMedGoogle Scholar
  25. Olivers, C. N., Peters, J., Houtkamp, R., & Roelfsema, P. R. (2011). Different states in visual working memory: When it guides attention and when it does not. Trends in Cognitive Sciences, 15(7), 327–334.PubMedGoogle Scholar
  26. Peters, J. C., Roelfsema, P. R., & Goebel, R. (2012). Task-relevant and accessory items in working memory have opposite effects on activity in extrastriate cortex. The Journal of Neuroscience, 32(47), 17003–17011.CrossRefPubMedGoogle Scholar
  27. Soto, D., Hodsoll, J., Rotshtein, P., & Humphreys, G. W. (2008). Automatic guidance of attention from working memory. Trends in Cognitive Sciences, 12(9), 342–348.CrossRefPubMedGoogle Scholar
  28. Souza, A. S., & Oberauer, K. (2016). In search of the focus of attention in working memory: 13 years of the retro-cue effect. Attention, Perception, & Psychophysics, 78(7), 1839–1860.CrossRefGoogle Scholar
  29. Souza, A. S., Rerko, L., & Oberauer, K. (2015). Refreshing memory traces: Thinking of an item improves retrieval from visual working memory. Annals of the New York Academy of Sciences, 1339, 20–31.CrossRefPubMedGoogle Scholar
  30. van Moorselaar, D., Theeuwes, J., & Olivers, C. N. L. (2014). In competition for the attentional template: Can multiple items within visual working memory guide attention? Journal of Experimental Psychology: Human Perception and Performance, 40(4), 1450–1464.PubMedGoogle Scholar
  31. Wolfe, J. M., Horowitz, T. S., Van Wert, M. J., Kenner, N. M., Place, S. S., & Kibbi, N. (2007). Low target prevalence is a stubborn source of errors in visual search tasks. Journal of Experimental Psychology: General, 136(4), 623–638.CrossRefGoogle Scholar
  32. Woodman, G. F., Carlisle, N. B., & Reinhart, R. M. G. (2013). Where do we store the memory representations that guide attention? Journal of Vision, 13(3), 1–17.CrossRefPubMedPubMedCentralGoogle Scholar
  33. Woodman, G. F., & Luck, S. J. (2007). Do the contents of visual working memory automatically influence attentional selection during visual search? Journal of Experimental Psychology: Human Perception and Performance, 33(2), 363–377.PubMedPubMedCentralGoogle Scholar
  34. Woodman, G. F., Luck, S. J., & Schall, J. D. (2007). The role of working memory representations in the control of attention. Cerebral Cortex, 17, i118–i124.CrossRefPubMedPubMedCentralGoogle Scholar
  35. Zokaei, N., Manohar, S., Husain, M., & Feredoes, E. (2014). Causal evidence for a privileged working memory state in early visual cortex. The Journal of Neuroscience, 34(1), 158–162.CrossRefPubMedPubMedCentralGoogle Scholar
  36. Zokaei, N., Ning, S., Manohar, S., Feredoes, E., & Husain, M. (2014). Flexibility in representational states in working memory. Frontiers in Human Neuroscience, 8, 1–12.CrossRefGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Jason Rajsic
    • 1
  • Natasha E. Ouslis
    • 1
  • Daryl E. Wilson
    • 2
  • Jay Pratt
    • 1
  1. 1.Department of PsychologyUniversity of TorontoTorontoCanada
  2. 2.Queen’s UniversityKingstonCanada

Personalised recommendations