Advertisement

Attention, Perception, & Psychophysics

, Volume 79, Issue 3, pp 782–795 | Cite as

Pupillary correlates of covert shifts of attention during working memory maintenance

  • Nash UnsworthEmail author
  • Matthew K. Robison
Article

Abstract

The pupillary light reflex (PLR) was used to track covert shifts of attention to items maintained in visual working memory (VWM). In three experiments, participants performed a change detection task in which rectangles appeared on either side of fixation and at test participants indicated if the cued rectangle changed its orientation. Prior to presentation or during the delay, participants were cued to the light or dark side of the screen. When cued to the light side, the pupil constricted, and when cued to the dark side, the pupil dilated, suggesting that the PLR tracked covert shifts of attention. Similar covert shifts of attention were seen when the target stimuli remained onscreen and during a blank delay period, suggesting similar effects for attention to perceptual stimuli and attention to stimuli maintained in VWM. Furthermore, similar effects were demonstrated when participants were pre-cued or retro-cued to the prioritized location, suggesting that shifts of covert attention can occur both before and after target presentation. These results are consistent with prior research, suggesting an important role of covert shifts of attention during VWM maintenance and that the PLR can be used to track these covert shifts of attention.

Keywords

Attention Space-based Visual working memory 

Visual working memory (VWM) is a system responsible for active maintenance and online manipulation of visual information over short intervals. Importantly, VWM is thought to be capacity limited such that only four or so items can be actively maintained (Cowan, 2001; Luck & Vogel, 2013). Given this sharp capacity limit, it is critically important to encode and actively maintain only task-relevant information to ensure fast and accurate response. That is, because only a few items can be maintained in VWM at any given time, it is important to ensure that important items for task performance are encoded adequately into VWM and are maintained actively in VWM during a delay. A number of studies have suggested that spatial attention is important to maintain information in VWM (Awh & Jonides, 2001; Smyth & Scholey, 1994). In particular, research suggests that attention-based rehearsal processes are needed to maintain information actively in VWM via covert (or overt) shifts of attention to prioritized locations (Awh & Jonides, 2001; Awh, Vogel, & Oh, 2006). Evidence consistent with attention-based rehearsal processes comes from behavioral, neuroimaging, and ERP studies, which have demonstrated that covert shifts of attention during retention can impair performance. There is enhanced visual processing at locations that are currently being held in VWM, similar frontal-parietal regions are activated for spatial attention and VWM, and there are similar ERP effects for spatial attention and VWM (Awh et al., 1998, 1999, 2000; Jha, 2002; Smyth & Scholey, 1994). More recent research has suggested that overt shifts of attention via eye movements also are implicated in attention-based rehearsal processes (Godijn & Theeuwes, 2012; Theeuwes et al., 2009; Tremblay et al., 2006; Williams et al., 2013). For example, Williams et al. (2013) recently demonstrated that during the retention interval of a visual change detection task, participants spontaneously shifted their eyes to locations of items currently being held in VWM even though the screen was blank. Thus, similar to the notion that during the retention interval participants covertly shift their attention to locations of items in VWM, when possible participants also will shift their attention overtly to those same locations. Similarly, evidence suggests that irrelevant eye movements can impair VWM performance (Lawrence et al., 2004; Postle et al., 2006; Smyth, 1996). Collectively, prior research suggests covert (and overt) shifts of attention are important for the active maintenance of items in VWM.

In addition to the role of spatial attention to attention-based rehearsal during maintenance, recent research has suggested that attentional processes can be used to select items in VWM during maintenance (Griffin & Nobre, 2003; Landman et al., 2003; Sperling, 1960). In these studies, a retro-cue paradigm is used in which during the retention interval a spatial cue is presented, indicating which item currently maintained in VWM is most relevant. These trials are contrasted with pre-cue trials in which the spatial cue is presented before the presentation of the array and neutral trials in which no spatial cue is presented. A typical finding is that retro-cues enhance performance relative to neutral cues (see Stokes & Nobre, 2011 for a review). Similar to the attention-based rehearsal hypothesis, it is assumed that with both spatial pre- and retro-cues attention covertly shifts to the prioritized location leading to enhanced processing (possibly via a number of different mechanisms; see Souza & Oberauer, 2016 for a review).

In the present study, we suggest that pupil diameter can be used to track covert attentional shifts while performing VWM tasks. Much prior research has shown that the pupil dilates in response to the cognitive demands of a task (Beatty & Lucero-Wagoner, 2000). Furthermore, research suggests that pupil dilation is sensitive to the number of items being maintained in working memory (Kahneman & Beatty, 1966; Peavler, 1974) and this is true for VWM tasks (Unsworth & Robison, 2015). Additionally, recent research has examined the extent to which the pupillary light reflex (PLR) can be used to track covert shifts of attention (Binda et al., 2013, 2014; Mathôt et al., 2013). The PLR is a pupillary response that occurs when the pupils constrict in brightness and dilate in darkness. Typically thought to be a low-level reflex, more recent research suggests that the PLR is implicated in more high-level cognitive processes (Binda & Murray, 2015; Mathôt & Van der Stigchel, 2015). For example, in Binda et al., (2013) participants were shown a grey screen with two disks (one bright and one dark) on each side of the screen. Participants were cued to attend to either the bright or dark disk while maintaining central fixation. Binda et al. found that when cued to the bright disk the pupil constricted and when cued to the dark disk the pupil dilated, thus suggesting that the pupil accurately tracked covert shifts of attention. Similarly, in Mathôt et al. (2013) participants were presented with a screen in which the one side was white, one side was black, and the center was grey. Participants were instructed to maintain their gaze on center while performing a variant of the Posner (1980) cuing task in which the orientation of a target presented on the right or left had to be reported. Mathôt et al. found that the pupil was larger when participants were cued to attend to the dark side of the screen compared with when they were cued to attend to the light side of the screen. Thus, in both cases covert shifts of attention modulated the PLR (see Binda & Murray, 2015; Mathôt & Van der Stigchel, 2015 for reviews of similar effects). These results suggest a promising means of using the PLR as a tool to track covert shifts of attention.

We used the PLR to track covert shift of attention to locations currently maintained in VWM. That is, while both Binda et al. (2013) and Mathôt et al. (2013) suggested the PLR could be used as a continuous measure of covert shifts of attention, in their studies the target stimuli were always present onscreen indicating covert shifts of perceptual attention. To examine whether covert shifts of spatial attention occur during VWM maintenance as suggested by studies of attention-based rehearsal and retro-cues, it is necessary to examine the PLR during blank retention intervals where target stimuli are absent. One recent study by Blom et al. (2016) has examined the PLR in a VWM task. In this study, participants performed a change detection task in which they were instructed to remember shapes presented in a cued color. The cued shapes were either all bright or all dark. Blom et al. found that when told to remember bright shapes the pupil constricted and when told to remember dark shapes the pupil dilated during encoding but not during the retention interval. Thus, the PLR indicated the contents of VWM during encoding but not during maintenance. Although this study indicates that PLR is not modulated by the contents (bright or dark stimuli) of working memory during maintenance, it does not tell us whether the PLR can be used to determine covert shifts of attention during VWM maintenance. Thus, to examine whether the PLR tracks covert shifts of attention during VWM maintenance, we performed three experiments based on the task used by Mathôt et al. (2013). Participants were presented with rectangles on the right and left side of the screen and were instructed to remember the orientation of the rectangles. Before presentation (Experiments 1 and 2) or during the delay (Experiments 2 and 3), participants were cued to the right or left side of the screen. Similar to Mathôt et al. (2013), one side of the screen was white while the other was black with a grey center. If covert shifts of attention occur during maintenance either for attention-based rehearsal or selection, then when cued to the bright side of the screen the pupil should constrict, but when cued to the dark side of the screen the pupil should dilate during the blank retention interval. These results would provide convincing evidence that during VWM maintenance attention covertly shifts to prioritized locations even in the absence of any stimuli on screen.

Experiment 1

We examined whether covert shifts of attention during VWM maintenance were similar to covert shifts of attention to perceptual stimuli. Participants performed a change detection task during which encoding two rectangles were presented (one to the right and one to the left of fixation). Before presentation, participants were cued to attend to the right or left side of the screen. The participant’s job was to indicate at test whether the cued rectangle changed its orientation. For the attention condition, the stimuli remained on-screen. For the memory condition, the stimuli were presented for 500 ms followed by a 2,000-ms delay. For both conditions, participants were instructed to maintain their eyes on central fixation at all times. Finally, for the gaze condition, participants were instructed to move their eyes to the cued condition (see also Binda et al., 2013). This condition was added as a control condition to demonstrate that covert shifts of attention to the cued location are similar (although much smaller) to overt shifts of attention to the cued locations.

Method

Participants

Participants were 32 undergraduate students recruited from the subject pool at the University of Oregon. Participants were between aged 18 to 35 years and received course credit for their participation. Data from three participants were excluded from analyses because of data collection problems with the Eyetracker, leaving a final sample of 29 participants.

Procedure

Participants were tested individually in a dark room. Pupil diameter was continuously recorded binocularly at 120 Hz using a Tobii T120 eyetracker. Participants were seated 60 cm from the monitor. After providing informed consent and after calibrating the eyetracker, participants performed a change detection task (Fig. 1). Participants were first presented with a dark grey fixation cross (1 cm × 1 cm) in the middle of the screen on a light grey background (7.5-cm wide) in the center flanked by a white screen (13-cm wide) and a black screen (13-cm wide) for 2,000 ms. Next, participants were presented with a cue (the words “Right” or “Left”) onscreen below the fixation cross for 500 ms. Cue validity was 100%. Participants were then presented with the target stimuli consisting of a rectangle to the right and a rectangle to the left of fixation. The rectangles (1.5 cm × 5 cm) were presented in one of four different orientations (vertical, horizontal, diagonal right, diagonal left). In the Attention and Gaze conditions, the rectangles remained onscreen for 2,000 ms. In the Memory condition, the rectangles remained onscreen for 500 ms. The presentation of the target stimuli was followed by a delay period. In the Attention and Gaze conditions, the delay was 500 ms. In the Memory condition, the delay was 2,000 ms. At test, a rectangle was presented in the cued location and participants responded as to whether or not the rectangle had changed orientation by pressing one of two keys. Participants performed five practice trials to get acquainted with the task. In each condition, there were 64 trials. Half of the trials cued the bright side, and the other half cued the dark side. Half of the trials were change trials. The order of the Attention and Memory conditions were counterbalanced between participants. The Gaze condition always followed the other two conditions. This was done to minimize eye movements in the other two primary conditions.
Fig. 1

Schematic example of trials for Experiment 1

Pupil data analysis

Data from each participant’s left eye was used (left and right eye pupil diameter were highly correlated r = 0.95). Missing data points due to blinks, off-screen fixations, and/or eyetracker malfunction were removed. The relevant period of missing data was not included in the averaging. The Tobii T120 uses a normalized gaze coordinate system [0, 1]. Specifically, the center of the screen has a value of [0.5, 0.5]. The leftmost horizontal position is 0, and the rightmost horizontal position is 1.0. Similar to Mathôt et al. (2013), trials where a participant made an eye movement off of the center grey background were removed (1.5%). For the remaining trials, the average horizontal gaze position was 0.50 (SD = 0.02) with similar positions for right (M = 0.49, SD = 0.02) and left (M = 0.50, SD = 0.02) cues, indicating that participants were staring at the fixation cross. Task related pupil responses were baseline corrected by subtracting out the baseline pupil diameter during the fixation screen on a trial-by-trial basis for each participant. During the presentation of the target stimuli and delay period, the pupil data were averaged into a series of 100-ms time windows for each trial and each 100-ms window was baseline corrected.

Results and discussion

Behavioral data

Examining overall accuracy, there were no differences among the conditions, F(2, 56) = 1.68, MSE = 0.001, p = 0.20, partial η2 = 0.05 (Memory M = 0.89, SD = 0.04; Attention M = 0.89, SD = 0.03; Gaze M = 0.90, SD = 0.04). Examining correct reaction times suggested an effect of condition, F(2, 56) = 7.54, MSE = 8214, p = 0.001, partial η2 = 0.21 (Memory M = 831 ms, SD = 202; Attention M = 875 ms, SD = 168; Gaze M = 783 ms, SD = 211). The Gaze condition was faster than the Attention condition, t(28) = 4.07, p < 0.001, and the difference between the Memory and Gaze conditions was not quite significant, t(28) = 1.86, p = 0.07. Similarly, the difference between the Memory and Attention conditions was not quite significant, t(28) = 1.96, p = 0.06.

Pupil data

Next for the primary analyses of interest, we compared pupillary responses for the Memory and Attention conditions when participants were cued to attend to the Light or the Dark side of the screen. The pupil data were submitted to a 2 Condition (Memory vs. Attention) × 2 Cue (Dark vs. Light) × 25 Bin repeated measures ANOVA. There was a main effect of condition, F(1, 28) = 12.08, MSE = 0.167, p = 0.002, partial η2 = 0.30, in which there was more dilation in the Memory condition than the Attention condition. There was a main effect of Cue, F(1, 28) = 5.01, MSE = 0.167, p = 0.033, partial η2 = 0.15, with more dilation for Dark cues than Light Cues. There also was a main effect of Bin, F(24, 672) = 2.19, MSE = 0.003, p = 0.001, partial η2 = 0.07, with the pupil dilating more as time progressed. The Condition × Cue interaction was not significant, F(1, 28) = 2.56, MSE = 0.031, p = 0.121, partial η2 = 0.08. There was Condition × Bin interaction, F(24, 672) = 13.50, MSE = 0.009, p < 0.001, partial η2 = 0.33, in which the difference between the Memory and Attention conditions increased as time increased during the delay (Fig. 2). There was a Cue × Bin interaction, F(24, 672) = 3.70, MSE = 0.002, p < 0.001, partial η2 = 0.12, in which the difference between the Dark and Light cues increased with time during the delay. Finally, there was a Condition × Cue × Bin interaction, F(24, 672) = 2.21, MSE = 0.001, p = 0.001, partial η2 = 0.07, which qualified these lower level effects. Specifically, there was a difference in the pupil waveforms when cued for Dark vs. Light side of the screen for both the Memory and Attention conditions (Fig. 2). However, the effect was larger in the Memory condition than in the Attention condition (Fig. 2c). This is likely due to the fact that participants found the Memory condition to be more effortful than the Attention condition. As shown in Fig. 2, when participants were cued to attend to the dark side of the screen their pupils tended to dilate, whereas when they were cued to attend to the light side of the screen their pupils dilated less. These effects were seen for both the Memory condition (Fig. 2a) and the Attention condition (Fig. 2b). Although it should be noted that in the Memory condition it is possible that the effects are due to a PLR that occurs while the items are still onscreen, but because overall pupillary responses are quite slow, this effect bleeds over into the delay period. This will be further examined in Experiments 2 and 3. Collectively, these results suggest that even though participants’ eyes remained fixated on center, in both the Memory and Attention conditions participants covertly shifted their attention to the cued locations.
Fig. 2

(a) Change in pupil diameter as a function of cue and time point during the delay for the Memory condition. (b) Change in pupil diameter as a function of cue and time point during the delay for the Attention condition. (c) Average difference between light and dark cued conditions for the Memory and Attention conditions during the delay

Next, we examined the Gaze condition to examine if overt shifts of attention to the cued location would be similar (although much larger) to the effects observed with covert shifts of attention. The pupil data for the Gaze condition were submitted to a 2 Cue (Dark vs. Light) × 25 Bin ANOVA. There was a main effect of Cue, F(1, 28) = 95.66, MSE = 0.457, p < 0.001, partial η2 = 0.77, a main effect of Bin, F(24, 672) = 28.31, MSE = 0.004, p < 0.001, partial η2 = 0.50, and a Cue × Bin interaction, F(24, 672) = 88.61, MSE = 0.003, p < 0.001, partial η2 = 0.76. When participants shifted their gaze to the dark side of the screen the pupil dilated, whereas when participants shifted their gaze to the light side of the screen the pupil constricted (Fig. 3). Furthermore, the difference between the two cues increased throughout the delay period. These results are similar (although much larger) to the covert orienting conditions, suggesting that similar effects arise when covertly and overtly orienting attention to light or dark locations.
Fig. 3

(a) Change in pupil diameter as a function of cue and time point during the delay for the Gaze condition. (b) Average difference between light and dark cued conditions for the Gaze condition during the delay.

Overall, the results from the Attention condition replicate prior research demonstrating that the PLR indexes covert shifts of attention to locations when the objects are still onscreen (Binda et al., 2013; Mathôt et al., 2013). Furthermore, the current experiment extends these results to situations in which the target items are no longer present onscreen, but rather are being maintained in VWM (the Memory condition). These results were similar to overt shifts of attention (although much smaller), suggesting that PLR tracks shifts of both overt and covert attention. Collectively, these results are consistent with the attention-based rehearsal hypothesis, suggesting that participants covertly shift their attention to prioritized locations to help actively maintain items in VWM (Awh & Jonides, 2001).

Experiment 2

Experiment 1 demonstrated that covert shifts of attention to locations of items maintained in VWM were similar to covert shifts of attention to items still present onscreen in accord with the attention-based rehearsal hypothesis. In Experiment 2, we examined whether similar covert shifts of attention occur when selecting items within VWM. Specifically, we examined whether the PLR would index similar shifts in covert attention for pre-cues and retro-cues. Participants performed the same task as Experiment 1 and on half of the trials were pre-cued to the target location (same as the Memory condition) and on the other half of trials participants received a retro-cue during the delay. If retro-cues represent a covert shift of attention to items in VWM, then we expect pre- and retro-cues to demonstrate similar PLR effects. For pre-cue trials, there should be differences between light and dark sides of the screen early in the delay period, but for the retro-cue trials, differences should only appear after the retro-cue. These results would suggest that not only do participants covertly shift their attention to the location of items being maintained in VWM, but also these covert shifts of attention can occur retrospectively during the delay.

Method

Participants

Participants were 43 new undergraduate students recruited from the subject pool at the University of Oregon. Participants were aged 18 to 35 years and received course credit for their participation. Data from four participants were excluded from analyses due excessive eye movements on all trials, and one participant was excluded for having poor accuracy (13% in each condition, likely due to mismapping the keys) leaving a final sample of 38 participants.

Procedure

Participants were tested individually in a dark room. Pupil diameter was continuously recorded binocularly at 120 Hz using a Tobii T120 eyetracker. Participants were seated 60 cm from the monitor. After providing informed consent and after calibrating the eyetracker, participants performed the same change detection task as Memory condition in Experiment 1 with the following exceptions. First, the delay for both the pre-cue and retro-cue conditions was 4,500 ms. Second, in the retro-cue condition the cue was presented after 2,000 ms into the delay and remained onscreen for 500 ms. The pre-cue condition was the same as the Memory condition in Experiment 1. The only difference between the pre- and retro-cue conditions was when the cue was presented. Participants performed 10 practice trials to get acquainted with the task. In each condition there were 64 trials. Half of the trials cued the bright side, and the other half cued the dark side. Half of the trials were change trials. The order of the pre-cue and retro-cue conditions was counterbalanced between participants.

Pupil data analysis

Same as Experiment 1. Trials where a participant made an eye movement off of the center grey background were removed (2.6%). For the remaining trials the average horizontal gaze position was 0.50 (SD = 0.02) with similar positions for right (M = 0.49, SD = 0.02) and left (M = 0.50, SD = 0.02) cues, indicating that participants were staring at the fixation cross.

Results and discussion

Behavioral data

Examining overall accuracy, there was no difference between the pre- (M = 0.88, SD = 0.05) and retro-cue conditions (M = 0.88, SD = 0.05), t(37) = 0.122, p = 0.904. Examining correct reaction times similarly suggested no differences between pre- (M = 1075, SD = 358) and retro-cue conditions (M = 1051, SD = 374), t(37) = 0.885, p = 0.382. The fact that there were no differences is likely due to the fact that only two items were presented rather than the more typical four items.

Pupil data

Next for the primary analyses of interest we compared pupillary responses for the pre- and retro-cue conditions when participants were cued to attend to the light or the dark side of the screen. The pupil data were submitted to a 2 Condition (Pre-cue vs. Retro-cue) × 2 Cue (Dark vs. Light) × 45 Bin ANOVA. The main effect of condition was not significant, F(1, 37) = 2.40, MSE = 0.054, p = 0.130, partial η2 = 0.06. There was a main effect of Cue, F(1, 37) = 4.89, MSE = 0.078, p = 0.033, partial η2 = 0.12, with more dilation for Dark cues than Light Cues. There was also a main effect of Bin, F(44, 1628) = 6.92, MSE = 0.004, p < 0.001, partial η2 = 0.16, with the pupil dilating more as time progressed. The Condition × Cue interaction was not significant, F(1, 37) = 1.12, MSE = 0.085, p = 0.298, partial η2 = 0.03. There was Condition × Bin interaction, F(44, 1628) = 15.32, MSE = 0.002, p < 0.001, partial η2 = 0.29, which is likely driven by the additional phasic response in the Retro-cue condition when the cue is presented during the delay (Fig. 4). There was a Cue × Bin interaction, F(44, 1628) = 4.30, MSE = 0.001, p < 0.001, partial η2 = 0.10, in which the difference between the Dark and Light cues increased with time during the delay. The Condition × Cue × Bin interaction was not significant, F(44, 1628) = 0.59, MSE = 0.001, p = 0.984, partial η2 = 0.02.
Fig. 4

(a) Change in pupil diameter as a function of cue and time point during the delay for the Pre-cue condition. (b) Change in pupil diameter as a function of cue and time point during the delay for retro-cue condition. (c) Average difference between light and dark cued conditions for the pre- and retro-cue conditions during the delay

As shown in Fig. 4 and similar to Experiment 1, when cued to attend to the dark side the pupil dilated, but when cued to attend to the light side the pupil dilated less. Thus, the pre-cue condition replicates the Memory condition from Experiment 1. Furthermore, in the pre-cue condition, differences between light and dark side cues occurred early in the delay period. Specifically, in the pre-cue condition the difference between Light and Dark side cues becomes significant around 2,000 ms, t(37) = 2.13, p = 0.040 (Fig. 4c). However, in the retro-cue condition there were no differences between Light and Dark side cues until 1,300 ms after the presentation of the retro-cue or 3,800 ms into the delay period, t(37) = 2.73, p = 0.010 (Fig. 4c). Thus, in both the pre- and retro-cue conditions, participants covertly shifted their attention to the prioritized locations, but this only occurred after the presentation of the cue. These results are consistent with the notion that following a retro-cue, participants covertly shift their attention to the cued location of the item being selected in VWM.

Experiment 3

Experiment 2 suggested that the PLR tracked covert shifts of attention to locations of items maintained in VWM that were cued both before item presentation (pre-cues) and after item presentation (retro-cues). However, one potential issue with these results is that it is not clear that the shifts of attention reflect shifts to external spatial locations or shifts to internal representations. That is, because the items are encoded against a light or dark background, it is possible that lightness and darkness are integrated into the memory representations during encoding, and thus any observed pupillary changes reflect shifts in attention to bright vs. dark memory representations or the retrieval of bright vs. dark representations.1 As noted previously, Blom et al. (2016) demonstrated pupillary changes to light or dark stimuli during encoding but not during maintenance, suggesting that the PLR does not track brightness/darkness associated with VWM representations during maintenance. However, to examine this alternative possibility, we ran an third experiment in which items were either encoded with light and dark backgrounds similar to the prior experiments or encoded against a grey background (Fig. 5). Following a retro-cue, the screen was either all grey or partitioned into light and dark sides similar to Experiments 1 and 2. If lightness vs. darkness is being integrated with the memory representations during encoding then when switched to all an grey background (Condition 1 shown in Fig. 5a), we should see similar results as found in Experiment 2 in which items encoded against a dark background dilate more than items encoded against a light background. If, however, the PLR is tracking covert shifts of attention to locations during maintenance (regardless of encoding), then when switching from an all grey background to a background with light and dark partitions (Condition 2 shown in Fig. 5b) differences should only emerge in this condition.
Fig. 5

Schematic example of trials for Condition 1 (a) and Condition 2 (b) in Experiment 3

Method

Participants

Participants were 40 new undergraduate students recruited from the subject pool at the University of Oregon. Participants were aged 18 to 35 years and received course credit for their participation. Data from six participants were excluded from analyses because of data collection problems with the eyetracker, data from one participant was excluded due to excessive eye movements on all trials, and three participants were excluded for having poor accuracy (roughly 50% in each condition) leaving a final sample of 30 participants.

Procedure

Participants were tested individually in a dark room. Pupil diameter was continuously recorded binocularly at 120 Hz using a Tobii T120 eyetracker. Participants were seated 60 cm from the monitor. After providing informed consent and after calibrating the eyetracker, participants performed the same change detection task as retro-cue condition in Experiment 2 with the following exceptions. As shown in Fig. 5a in Condition 1, the background stimuli consisted of white, grey, and black partitions up to and including the retro-cue. Following the retro-cue, the screen was screen was grey. As shown in Fig. 5b in Condition 2, the background was grey up to and including the retro-cue. Following the retro-cue, the screen consisted of white, grey, and black partitions. Participants performed four practice trials to get acquainted with the task. In each condition there were 64 trials. Half of the trials cued the bright side, and the other half cued the dark side. Half of the trials were change trials. The order of the conditions was counterbalanced between participants.

Pupil data analysis

Same as Experiment 1 and 2. Trials where a participant made an eye movement off of the center grey background were removed (2.8%). For the remaining trials, the average horizontal gaze position was 0.50 (SD = 0.01) with similar positions for right (M = 0.50, SD = 0.01) and left (M = 0.49, SD = 0.01) cues, indicating that participants were staring at the fixation cross.

Results and discussion

Behavioral data

Examining overall accuracy, there was no difference between Condition 1 (M = 0.89, SD = 0.05) and Condition 2 (M = 0.88, SD = 0.05), t(29) = 1.06, p = 0.298. Examining correct reaction times similarly suggested no differences between Condition 1 (M = 955 ms, SD = 316) and Condition 2 (M = 974 ms, SD = 335), t(29) = −0.318, p = 0.753.

Pupil data

Next for the primary analyses of interest, we compared pupillary responses for the two conditions when participants were retro-cued to attend to the Light or the Dark side of the screen. The pupil data were submitted to a 2 Condition (1 vs. 2) × 2 Cue (Dark vs. Light) × 45 Bin ANOVA. There was a main effect of Condition, F(1, 29) = 19.92, MSE = 0.394, p < 0.001, partial η2 = 0.41, with more dilation in Condition 1 than Condition 2 (Fig. 6a). The main effect of Cue was not significant, F(1, 29) = 0.131, MSE = 0.043, p = 0.720, partial η2 = 0.004. There was also a main effect of Bin, F(44, 1276) = 42.31, MSE = 0.006, p < 0.001, partial η2 = 0.59. The Condition × Cue interaction was not significant, F(1, 29) = 1.13, MSE = 0.047, p = 0.297, partial η2 = 0.04. There was Condition × Bin interaction, F(44, 1276) = 151.32, MSE = 0.007, p < 0.001, partial η2 = 0.84, which was driven by the large change in luminance following the retro-cue in each condition (Fig. 6). There was a Cue × Bin interaction, F(44, 1276) = 2.60, MSE = 0.001, p < 0.001, partial η2 = 0.08, in which the difference between the Dark and Light cues increased with time during the delay. Importantly, there was a Condition × Cue × Bin interaction, F(44, 1276) = 4.45, MSE = 0.001, p < 0.001, partial η2 = 0.13. As shown in Fig. 6, in Condition 1 where following the retro-cue the screen was all grey, there was a large constriction response to the change in luminance, but importantly there was no differences in the PLR as a function of cue. Specifically, examining the last second of the delay period in Condition 1 there was no effect of Cue, F(1, 29) = 0.71, MSE = 0.017, p = 0.41, partial η2 = 0.02, nor a Cue × Bin interaction, F(44, 1276) = 0.69, MSE = 0.001, p = 0.74, partial η2 = 0.02. However, in Condition 2, where following the retro-cue the screen was split into white, grey, and black portions, there were differences in the PLR as a function of cue. Specifically, examining the last second of the delay period in Condition 2 suggested there was an effect of Cue, F(1, 29) = 8.36, MSE = 0.021, p = 0.007, partial η2 = 0.22, such that when cued to attend to the dark side the pupil dilated, but when cued to attend to the light side the pupil dilated less replicating Experiment 2. Furthermore, there was a Cue × Bin interaction, F(44, 1276) = 1.88, MSE = 0.001, p = 0.047, partial η2 = 0.06, suggesting that the effect increased during the latter part of the delay period. Collectively, these results are consistent with the notion that following a retro-cue, participants covertly shift their attention to the cued location of the item being selected in VWM, rather than retrieving contextual information (lightness vs. darkness) that has been integrated with the target items during encoding. As such, the results from Experiment 3 replicate and extend the prior experiments in suggesting that participants are covertly shifting their attention to the locations of items being maintained in VWM.
Fig. 6

(a) Change in pupil diameter as a function of cue and time point during the delay for Condition 1. (b) Change in pupil diameter as a function of cue and time point during the delay for Condition 2. (c) Average difference between light and dark cued conditions for Conditions 1 and 2 during the delay

General discussion

In three experiments, we used the PLR to examine covert (and overt) shifts of attention to items either still onscreen or to items being maintained in VWM. Replicating prior research, we found (Experiment 1) that when cued to attend to the dark side of the screen the pupil dilated and when cued to attend to the light side of the screen the pupil dilated less, suggesting that the PLR is modulated by covert shifts of attention to items still present onscreen (Binda et al., 2013; Mathôt et al., 2013). These results were similar (although smaller) to the effects seen with overt shifts of attention to the cued locations (Experiment 1; Binda et al., 2013). Importantly, we extended these results by demonstrating that the PLR was similarly modulated by covert shifts of attention to the locations of items being maintained in VWM even though no targets were currently presented onscreen (Experiments 1-3). Furthermore, in Experiments 2 and 3, we demonstrated that these effects occur not only with pre-cues but also with retro-cues where selection of items in VWM occurred after target presentation during the delay period.

The current results are consistent with a recent study by Blom et al. (2016), which found that when participants were instructed to remember bright shapes the pupil constricted and when told to remember dark shapes the pupil dilated during encoding the phase but not during the retention interval. These results indicated that the PLR was sensitive to the contents of VWM during encoding but not during maintenance. Similarly, in Experiment 3 we found that in the condition where light and dark sides were present during encoding, but not following the retro-cue during maintenance, there was no difference in the pupil response. However, in the condition where light and dark sides were only present following the retro-cue, a PLR effect was found, suggesting that participants shifted their attention to the cued location. Thus, both studies indicated that the PLR is not necessarily modulated by the contents (bright or dark stimuli) of working memory during maintenance, but the PLR can be used to determine covert shifts of attention during VWM maintenance.

These results are consistent with the attention-based rehearsal hypothesis, which suggests that items are maintained in VWM via covert shifts of attention to prioritized information during the delay period of working memory tasks (Awh & Jonides, 2001; Awh, Vogel, & Oh, 2006). The fact that the PLR changed as a function of cued locations suggests that participants covertly shifted their attention to the prioritized location even when there was no explicit instructions to do so. These results suggest that participants likely shifted their attention to the cued location to help maintain the prioritized information in VWM. The results also are consistent with recent research on the retro-cue paradigm suggesting that items can be retrospectively selected in VWM via covert shifts of attention to items during the delay period (Griffin & Nobre, 2003; Stokes & Nobre, 2011; Souza & Oberauer, 2016). Similar to the pre-cue condition, the PLR results from the retro-cue condition suggest that early in the delay participants maintained attention on central fixation (or attention was split to both sides of the screen), but after the presentation of the retro-cue participants covertly shifted their attention to the cued location to select and maintain the cued item in VWM. In both cases the PLR was used to measure continuously the location of the focus of attention even when no items were presented onscreen. These results along with prior research suggest that the PLR can be used to track covert shifts of attention to perceptual items and items maintained in VWM, thereby providing an important means to test theories that rely on the notion that covert shifts of attention occur during working memory maintenance.

Although these results are consistent with the attention-based rehearsal hypothesis and with the notion of covert selective attention in the retro-cue paradigm, it is important to point out that the current results are correlational in nature. That is, the current results simply tell us that during the delay period participants covertly shift their attention to the cued location. This does not tell us whether these covert shifts of attention are beneficial to working memory maintenance, nor do they tell us that these covert shifts of attention are necessary for working memory maintenance (Belopolsky & Theeuwes, 2009) or for the beneficial effect of retro cues (Rerko, Souza, & Oberauer, 2014).2 Future research is needed to better assess the extent to which these shifts of attention measured via the PLR are beneficial and necessary for working memory maintenance.

Along with prior research, the current results suggest a link between covert shifts of attention and pupillary responses. A possible neural mechanism for this relation is the superior colliculus (SC). A great deal of research suggests that the SC is important for overt and covert shifts of attention (see Krauzlis, Lovejoy, & Zenon, 2013; Corneil & Munoz, 2014 for reviews) and is a potentially important reason for the connection between working memory, attention, and eye movements (Theeuwes, Belopolsky, & Olivers, 2009). Furthermore, research suggests that the SC is important for mediating the PLR via direct connections from the SC to pupil pathways (Corneil & Munoz, 2014; Loewenfeld, 1999). Recent research also has shown that weak microstimulation of the SC resulted in transient pupil dilation in the absence of any saccades (Joshi et al., 2016; Wang et al., 2012). Wang and Munoz (2015) proposed a circuit for pupil orienting responses in which the SC was critical for modulating pupillary responses during orienting, and thus Wang and Munoz suggested “SC-mediated pupil pathways could provide the substrate required for pupil size modulation by various cognitive processes” (p. 139). That is, as suggested by Corneil and Munoz (2014), SC-mediated pupillary responses can be seen as overt markers of covert orienting.

Conclusions

Results from the current study demonstrated a PLR modulated by covert shifts of attention to both the location of perceptual information and to the location of information being maintained in VWM. These PLR indexed covert shifts of attention occurred both when cued prior to the appearance of the target stimuli (pre-cues) and when cued retrospectively (retro-cues) during the delay after the appearance of the target stimuli. These results along with prior research add to the growing body of literature suggesting a promising avenue of using the PLR as a means to track continuously covert shifts of attention during working memory maintenance.

Footnotes

  1. 1.

    Thanks to Naseem Al-Aidroos for suggesting this alternative explanation and for suggesting the experiment.

  2. 2.

    We examined PLR differences between correct and error trials in each experiment, but none of the effects were statistically significant. Given overall high accuracy rates, there were relatively few error trials per condition, resulting in high standard errors associated with the PLRs for error trials. Perhaps with a more difficult task differences may emerge. Additionally, we did not examine the correlations between accuracy or reaction time with each individual’s PLR effects given that there are too few participants for an individual differences analyses in each experiment.

Notes

Acknowledgments

Thanks to Naseem Al-Aidroos and Sebastiaan Mathôt for comments on a previous version of the article.

This research was supported by Office of Naval Research grant N00014-15-1-2790.

References

  1. Awh, E., Anllo-Vento, & Hillyard, S. A. (2000). The role of spatial selective attention in working memory for locations: Evidence from event-related potentials. Journal of Cognitive Neuroscience, 12, 840–847.CrossRefPubMedGoogle Scholar
  2. Awh, E., & Jonides, J. (2001). Overlapping mechanisms of attention and working memory. Trends in Cognitive Sciences, 5, 119–126.CrossRefPubMedGoogle Scholar
  3. Awh, E., Jonides, J., Smith, E. E., Buxton, R. B., Frank, L. R., Love, T., … Gmeindl, L. (1999). Rehearsal in spatial working memory: Evidence from neuroimaging. Psychological Science, 10, 433–437.CrossRefGoogle Scholar
  4. Awh, E., Jonides, J., & Reuter-Lorenz, P. A. (1998). Rehearsal in spatial working memory. Journal of Experimental Psychology: Human Perception and Performance, 24, 780–790.PubMedGoogle Scholar
  5. Awh, E., Vogel, E., & Oh, S.-H. (2006). Interactions between attention and working memory. Neuroscience, 139, 201–208.CrossRefPubMedGoogle Scholar
  6. Beatty, J., & Lucero-Wagoner, B. (2000). The pupillary system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (pp. 142–162). New York: Cambridge University Press.Google Scholar
  7. Belopolsky, A. V., & Theeuwes, J. (2009). No functional role of attention-based rehearsal in maintenance of spatial working memory representations. Acta Psychologica, 132, 124–135.CrossRefPubMedGoogle Scholar
  8. Binda, P., & Murray, S. O. (2015). Keeping a large-pupilled eye on high-level visual processing. Trends in Cognitive Science, 19, 1–3.CrossRefGoogle Scholar
  9. Binda, P., Pereverzeva, M., & Murray, S. O. (2013). Attention to bright surfaces enhances the pupillary light reflex. Journal of Neuroscience, 5, 2199–2204.CrossRefGoogle Scholar
  10. Binda, P., Pereverzeva, M., & Murray, S. O. (2014). Pupil size reflects the focus of feature-based attention. Journal of Neurophysiology, 12, 3046–3052.CrossRefGoogle Scholar
  11. Blom, T., Mathôt, S., Olivers, C. N. L., & Van der Stigchel, S. (2016). The pupillary light response reflects encoding, but not maintenance, in visual working memory. Journal of Experimental Psychology: Human Perception and Performance, 42, 1716–1723.PubMedGoogle Scholar
  12. Corneil, B. D., & Munoz, D. P. (2014). Overt responses during covert orienting. Neuron, 82, 1230–1243.CrossRefPubMedGoogle Scholar
  13. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 97–185.Google Scholar
  14. Godijn, R., & Theeuwes, J. (2012). Overt is no better than covert when rehearsing visuo-spatial information in working memory. Memory & Cognition, 40, 52–61.CrossRefGoogle Scholar
  15. Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive Neuroscience, 15, 1176–1194.CrossRefPubMedGoogle Scholar
  16. Jha, A. (2002). Tracking the time-course of attentional involvement in spatial working memory: An event-related potential investigation. Cognitive Brain Research, 15, 61–69.CrossRefPubMedGoogle Scholar
  17. Joshi, S., Li, Y., Kalwani, R. M., & Gold, J. I. (2016). Relationship between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron, 89, 221–234.CrossRefPubMedGoogle Scholar
  18. Kahneman, D., & Beatty, J. (1966). Pupil diameter and load on memory. Science, 154, 1583–1585.CrossRefPubMedGoogle Scholar
  19. Krauzlis, R. J., Lovejoy, L. P., & Zenon, A. (2013). Superior colliculus and visual spatial attention. Annual Review of Neuroscience, 36, 165–182.CrossRefPubMedGoogle Scholar
  20. Landman, R., Spekreijse, H., & Lamme, V. A. F. (2003). Large capacity storage of integrated objects before change blindness. Vision Research, 43, 149–164.CrossRefPubMedGoogle Scholar
  21. Lawrence, B. M., Myerson, J., & Abrams, R. A. (2004). Interference with spatial working memory: An eye movement is more than a shift of attention. Psychonomic Bulletin & Review, 11, 488–494.CrossRefGoogle Scholar
  22. Loewenfeld, I. E. (1999). The pupil: Anatomy, physiology, and clinical applications. Boston: Butterworth-Heinemann.Google Scholar
  23. Luck, S. J., & Vogel, E. K. (2013). Visual working memory capacity: From psychophysics and neurobiology to individual differences. Trends in Cognitive Science, 17, 391–400.CrossRefGoogle Scholar
  24. Mathôt, S., van der Linden, L., Grainger, J., & Vitu, F. (2013). The pupillary response to light reflects the focus of covert visual attention. PLoS ONE, 8(10), e78168.CrossRefPubMedPubMedCentralGoogle Scholar
  25. Mathôt, S., & Van der Stigchel, S. (2015). New light on the mind’s eye: The pupillary light response as active vision. Current Directions in Psychological Science, 24, 374–378.CrossRefPubMedPubMedCentralGoogle Scholar
  26. Peavler, W. S. (1974). Pupil size, information overload, and performance differences. Psychophysiology, 11, 559–566.CrossRefPubMedGoogle Scholar
  27. Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3–25.CrossRefPubMedGoogle Scholar
  28. Postle, B. R., Idzikowski, C., Della Sala, S., Logie, R. H., & Baddeley, A. D. (2006). The selective disruption of spatial working memory by eye movements. Quarterly Journal of Experimental Psychology, 59, 100–120.CrossRefGoogle Scholar
  29. Rerko, L., Souza, A. S., & Oberauer, K. (2014). Retro-cue benefits in working memory without sustained focal attention. Memory & Cognition, 42, 712–728.CrossRefGoogle Scholar
  30. Smyth, M. M. (1996). Interference with rehearsal in spatial working memory in the absence of eye movements. Quarterly Journal of Experimental Psychology, 49, 940–949.CrossRefPubMedGoogle Scholar
  31. Smyth, M. M., & Scholey, K. A. (1994). Interference in immediate spatial memory. Memory & Cognition, 22, 1–13.CrossRefGoogle Scholar
  32. Souza, A. S., & Oberauer, K. (2016). In search of the focus of attention in working memory; 13 years of the retro-cue effect. Attention, Perception, & Psychophysics, 78, 1839–1860.CrossRefGoogle Scholar
  33. Sperling, G. (1960) The information available in brief visual presentations. Psychological Monographs, 74(11, Whole No. 498).Google Scholar
  34. Stokes, M., & Nobre, A. C. (2011). Top-Down biases in visual short-term memory. In G. R. Mangun (Ed.), Neuroscience of attention (pp. 209–228). New York: Oxford University Press.Google Scholar
  35. Theeuwes, J., Belopolsky, A., & Olivers, C. N. L. (2009). Interactions between working memory, attention and eye movements. Acta Psychologica, 132, 106–114.CrossRefPubMedGoogle Scholar
  36. Tremblay, S., Saint-Aubin, J., & Jalbert, A. (2006). Rehearsal in serial memory for visual–spatial information: Evidence from eye movements. Psychonomic Bulletin & Review, 13, 452–457.CrossRefGoogle Scholar
  37. Unsworth, N., & Robison, M. K. (2015). Individual differences in the allocation of attention to items in working memory: Evidence from pupillometry. Psychonomic Bulletin & Review, 22, 757–765.CrossRefGoogle Scholar
  38. Wang, C. A., Boehnke, S. E., White, B. J., & Munoz, D. P. (2012). Microstimulation of the monkey superior colliculus induces pupil dilation without evoking saccades. Journal of Neuroscience, 32, 3629–3636.CrossRefPubMedGoogle Scholar
  39. Wang, C.-A., & Munoz, D. P. (2015). A circuit for pupil orienting responses: implications for cognitive modulation of pupil size. Current Opinion in Neurobiology, 33, 134–140.CrossRefPubMedGoogle Scholar
  40. Williams, M., Pouget, P., Boucher, L., & Woodman, G. F. (2013). Visual-spatial attention aids the maintenance of object representations in visual working memory. Memory & Cognition, 41, 698–715.CrossRefGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2017

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of OregonEugeneUSA

Personalised recommendations