RETRACTED ARTICLE: Unexpected events disrupt visuomotor working memory and increase guessing
When an unexpected event, such as a car horn honking, occurs in daily life, it often disrupts our train of thought. In the lab, this effect was recently modeled with a task in which verbal working memory (WM) was disrupted by unexpected auditory events (Wessel et al. in Nature Communications, 7, 11195, 2016). Here we tested whether this effect extends to a different type of WM—namely, visuomotor. We found that unexpected auditory events similarly decremented visuomotor WM. Moreover, this effect persisted for many more trials than had previously been shown for verbal WM, and the effect occurred for two different types of unexpected auditory events. Furthermore, we found that unexpected events decremented WM by decreasing the quantity, but not necessarily the quality, of items stored. These results showed an impact of unexpected events on visuomotor WM that was statistically robust and endured across time. They also showed that the effect was based on an increase in guessing, consistent with a neuroscience-inspired theory that unexpected events “wipe out” WM by stopping the ongoing maintenance of the trace. This new task paradigm is an excellent vehicle for further explorations of distractibility.
KeywordsInhibitory control Unexpected events Working memory
Daily life is littered with unexpected events: the jingling of a phone’s ringtone, the honking of a car horn, the screech of a passing crow. These events are distracting, often interrupting our train of thought and jolting us away from the task at hand. A recent study investigated such distraction using a laboratory task in which working memory (WM) was interrupted by unexpected auditory events (Wessel et al., 2016). On each trial, participants encoded a letter string and were then presented with a standard tone (80%) or a novel birdsong (20%). The novel stimuli led to lower WM accuracy upon probe. Although the typical view of the underlying mechanism is that it is “caused” by an attention shift, the same authors provided a deeper insight. The novel stimulus recruited the same brain network as outright stopping of an action (Wessel & Aron, 2013; Wessel et al., 2016), and the more it did so, the greater the WM disruption (Wessel et al., 2016). It was proposed that the stop system can interrupt some forms of cognition just as it can interrupt skeletomotor activity (Wessel & Aron, 2017). This account suggests intriguing links across several disparate areas of psychology—namely attention, working memory, surprise, response inhibition, and distractibility (Banich, Mackiewicz Seghete, Depue, & Burgess, 2015; Horstmann, 2006; Leiva, Parmentier, Elchlepp, & Verbruggen, 2015; Levy & Anderson, 2002; Rissman, Gazzaley, & D’Esposito, 2009). Here we set out to more thoroughly substantiate the behavioral effect, to elaborate the paradigm with several new psychometric features, and to test several behavioral implications. (Note that, throughout, we refer to the unexpected events as “novels,” in keeping with a substantial literature: Parmentier, 2008, 2014; Parmentier, Elsley, Andres, & Barcelo, 2011).
We established the basic effect that novels decremented visuomotor WM. We then tested (in Exp. 2) whether this effect was a generalizable feature of unexpected events. Along with the birdsong novels, we included trials in which the standard tone was simply increased in duration (cf. Näätänen, Pakarinen, Rinne, & Takegata, 2004).
In Experiment 3 we tested whether the “WM decrementing effect” was truly due to the novelty of the tone or could instead be attributable to the different sensory elements of the novels versus the standards. We now used a birdsong on every trial, with some of these remaining unexpected. We also probed how WM was impacted by novels. As above, our theory was that visuomotor WM would be maintained via cortico-basal-ganglia loops that were interrupted by unexpected-event-driven subthalamic activation. This might predict that WM itself could be partially “wiped out” (i.e., a reduction in “quantity”), which would lead to increased guessing at the probe, rather than to a change in the “quality” of the representation, which would lead to a reduction in precision. Accordingly, we used a model-fitting procedure (Suchow, Brady, Fougnie, & Alvarez, 2013) based on models of WM storage (Brady, Konkle, & Alvarez, 2011; Ma, Husain, & Bays, 2014; Zhang & Luck, 2008, 2011) to test for changes in guessing, precision, or both.
Sixteen students (15 female, one male; mean age 20.8 ± 2.1 years) from the University of California, San Diego (UCSD), participated for course credit. All were right-handed and had normal or corrected-to-normal visual acuity. They signed written consent according to the local ethics committee (IRB #140033).
An earlier study of the effect of unexpected events on verbal WM had shown a within-subjects novel-versus-standard effect size of Cohen’s d = 0.78 (Wessel et al., 2016). From this we estimated that 16 participants would give 90% power to detect a significant effect with alpha of .05, one-tailed.
Apparatus and stimuli
Participants sat ~22 in. in front of a 19-in. iMac personal computer (Apple) supported by a chinrest. The study was run with MATLAB 2015b (The MathWorks) and Psychophysics Toolbox 3 (Brainard, 1997). The sounds were delivered through Sennheiser HD 280 Pro headphones. Responses were recorded using an Apple mouse. The stimuli were turquoise (HEX #00EDFF) squares, measuring approximately 0.5 in. × 0.5 in. (subtended 1.3° × 1.3° of visual angle).
Procedure and design
Testing was divided into (1) determining WM span and (2) the main experiment.
The WM span procedure was designed to derive the correct span for each participant to have good sensitivity to detect WM decrements. Each trial began with a central fixation cross, around which a number of square stimuli (20 trials each of three, four, five, and six squares) were shown in succession, with each presented for 500 ms in one of 12 locations on the screen (no delay between presentation times). No locations were repeated within a trial. Then the central fixation cross was once again displayed alone for a delay of 1.4–2.9 s, jittered. This was followed by a tone, which in the case of the span test was always the standard tone—a 600-Hz sine wave presented for 200 ms—followed by a delay of 300 ms. After the delay, a number would appear in the center of screen for 500 ms, indicating which square, based on the order, the participant needed to remember. Finally, a square would reappear, in either the exact same location as the target square or a slightly different location. Participants were given 3 s to respond by indicating whether the probe square was in the same location as (“press 1”) or a different location from (“press 2”) the target square. We then calculated the number of individual squares remembered for each participant by using Cowan’s k: K = (H – FA)*N, where K is the number of items stored, H is the hit rate, FA is the false alarm rate, and N is the number of items presented (Alvarez & Cavanagh, 2004; Cowan, 2001), which was then used as set size for the main experiment.
For the main experiment, each trial proceeded as in the span test, with two key differences (Fig. 1). First, the trials were divided into two conditions: standard and novel. In the standard condition (80% of trials), the tone in the delay period was the same 600-Hz sine wave as in the span test. However, in the novel condition (20% of trials), a unique birdsong segment was presented, instead. Additionally, probe squares were no longer presented; instead the mouse cursor reappeared in the center of the screen, and participants clicked the position where they believed the target had been located. Trials were presented in a pseudorandom order, constrained so that two novel trials could never occur in a row and the first three trials of every block were standards. A total of 320 trials were presented, divided into eight blocks of 40 trials (eight of which were novel trials and 32 of which were standard trials).
Response times (RTs), defined as the times between when the mouse cursor reappeared in the center of the screen and when participants clicked, did not differ between novels and standards [t(15) = 1.74, p = .103].
As an auxiliary analysis, we tested whether this WM decrement was limited to the current trial or also extended into the following trial. We calculated response errors and RTs for standard trials (t), with trials divided as a function of the previous trial (t – 1)—that is, the current trial (t) was now analyzed as either “postnovel” or “poststandard.” Response errors were significantly higher on postnovel trials (mean distance = 1.78 ± 0.44) than on poststandard trials (mean distance = 1.54 ± 0.24) [t(15) = 2.85, p = .012]. We observed no significant difference in RTs between postnovel and poststandard trials [t(15) = 0.53, p = .602].
This study thus extended the earlier verbal WM results (Wessel et al., 2016) by showing that visuomotor WM is also decremented by auditory novels. Additionally, these effects carried over into the next trial.
We now tested whether the novel-induced decrement occurs for stimuli other than birdsongs.
Given the partial eta-squared value of .372 in Experiment 1, 16 participants would give us 99% power to detect an effect. We thus kept N equal to 16 to be consistent across studies. Sixteen students (ten female, six male; 13 right-handed; mean age 21.0 ± 4.2 years) were recruited and gave consent as in Experiment 1.
Apparatus, stimuli, and procedure
All aspects were the same as in Experiment 1, except that (1) the set size for the experimental task was now determined as Cowan’s k + 1 (we reasoned that this would make WM even more prone to disruption), and (2) novels were subdivided into two types: Half (10% of all trials) were the 200-ms birdsong segments used in Experiment 1, and the other half (10% of all trials) were instead long tones (600-Hz sine waves) lasting 500 ms.
An ANOVA was conducted with the factors condition (standard vs. novel) and block (1 to 8), with response errors and outliers being determined as in Experiment 1. Again, we found a significant main effect of condition [F(1, 15) = 24.98, p < .001, ηp2 = .625], with greater response errors for novels than for standards (see Fig. 2). There was no main effect of block [F(1, 15) = 0.77, p = .617] or block by condition interaction [F(1, 15) = 1.15, p = .337]. Furthermore, post hoc tests using the Bonferroni correction revealed that the effect was present for both types of novels (birdsong vs. standard, p = .009; long tone vs. standard, p = .001). We observed no significant effect of condition on RT [t(15) = 1.17, p = .260].
Response errors and RTs were also calculated for postnovel and poststandard trials as in Experiment 1. Response errors were significantly greater for postnovel trials (mean distance = 2.31 ± 0.59) than for poststandard trials (mean distance = 1.96 ± 0.58) [t(15) = 4.83, p < .001], although there was no significant difference in RTs [t(15) = 1.85, p = .085].
This experiment thus replicated Experiment 1, and went further by showing that a different kind of novel (a longer tone) is also capable of decrementing visuomotor WM.
To be consistent across all three studies, we again ran 16 participants (14 female, two male; 15 right-handed, mean age 21.8 ± 4.9 years), who were recruited and gave consent as in Experiment 1.
Apparatus, stimuli, and procedure
All aspects were the same as in Experiment 2, except that for the main task, only birdsongs were now used. These were now counterbalanced, with each acting as a standard or a novel, depending on the block (specifically, the main experiment consisted of nine blocks [360 trials] with nine unique birdsong segments used throughout the experiment; for each block, one of the birdsong segments served as the standard, with the other eight serving as novels, and the standard rotated across blocks). We also increased the number of trials (with now 72 instead of 64 novels) and standardized the stimulus locations to a fixed eccentricity (i.e., each box occurred at some point on an invisible circle ringing the screen).
The increased number of trials and standardized stimulus locations now allowed us to attempt to measure differences in capacity and precision. For each trial, we calculated the response error as the angular distance between the correct and reported locations. We then modeled these error distributions as a mixture of a circular normal distribution (centered around the correct location) and a uniform distribution (random guesses), with two parameters—a mixture parameter (G, indicating how many trials were “guesses”1) and an SD parameter (indicating how wide the normal distribution was—i.e., precision; Zhang & Luck, 2008). The data were fit using MemToolbox (Suchow et al., 2013), and parameter estimates were derived using Markov chain Monte Carlo simluations to find the maximum a posteriori values. The model was fit separately for each participant and task condition (novel vs. standard). We also analyzed the data using a modified model that estimated how frequently participants reported one of the other square locations as the target location (“swap errors”; Bays et al., 2009). The results converged across the two models, and parameter estimates for this alternative model are listed in the supplementary materials.
An ANOVA with the factors condition (standard vs. novel) and block (1 to 9) revealed a significant effect of condition [F(1, 15) = 17.19, p = .001, ηp2 = .534]. This showed, again, that response errors increased on novel relative to standard trials (see Fig. 2). We found no significant main effect of block [F(1, 15) = 0.84, p = .566] or interaction [F(1, 15) = 1.84, p = .076]. Interestingly, the interaction was at trend level, and this trend was in the direction of the unexpected-event-induced decrement in WM increasing over time. Indeed, the effect was significant even in the last block [t(15) = 2.79, p = .014]. Again, no significant effect of condition emerged in RTs [t(15) = 1.57, p = .137]. There was also no significant difference between postnovel and poststandard trials with regard to either response errors [t(15) = 1.51, p = .151] or RTs [t(15) = –1.15, p = .269].
Mean values for the novel and standard conditions, for response error distance and response time
As an auxiliary analysis, we examined response errors on standard trials following novel trials, as compared to standard trials following other standard trials. Response errors increased in the former trial type in two of the three experiments (Exps. 1 and 2). This makes sense, even within the context of a brief, transient stop process being recruited by the unexpected event, as evidence of a later potential post-error slowing or distraction effect. Conceptually, this is consistent with the idea that the unexpected event during the delay period on trial n recruits the stop network and impacts the maintenance of WM, thus causing the WM decrement on that trial. Then, general distraction and loss of attention may be a secondary consequence of the unexpected event and may impact encoding on trial n + 1. Because this “trial-after” effect was only present in two of the three experiments and was not the main focus of the study, we do not discuss it further.
The large, enduring effect of novel-induced distraction seen in this study is to be contrasted with the effects for the verbal WM paradigm (Wessel et al., 2016). There, although significant effects did emerge in several participant groups, the effects were not of so dramatic a magnitude and did not last aso long. We conjecture that visuomotor WM is more fragile—that is, rehearsing a motor sequence of spatial locations is more easily interruptible than verbal WM, which is presumably rehearsed via the phonological loop. We also showed that even stimuli that are well-balanced in their physical characteristics interrupt WM when they are unexpected. All of these features make the present paradigm an excellent vehicle for further explorations of distractibility—it produces a reliable effect that can be studied across many trials and is not confounded by stimulus differences.
It is notable that novels decremented WM by increasing guessing, without any apparent effect on precision. This is interesting in relation to our neural theory that visuomotor WM is maintained via thalamocortical drive. A stop-induced interruption in that activation might be expected to produce a disruption of memory for a visuomotor sequence, which may manifest as a loss of one of more item representations rather than as an overall “blurring” of precision.
An interesting question is whether there is something special, with regard to interruptibility, about visuomotor or verbal WM relative to other kinds of WM, such as for colors or faces. It is likely that both visuomotor and verbal WM are motor-based. In that sense, an unexpected-event-driven stop system might interrupt them in the same way that it “suppresses” the skeletomotor system (Wessel & Aron, 2013). More generally, this theory provides new perspectives on the connection between the basal ganglia and WM (cf. Chatham & Badre, 2015).
Notably, the impact of distraction on WM has been difficult to demonstrate empirically, and many of those studies that do show an effect use a distractor (e.g., visual) which 'shapes' the (visual) WM representation (Ester, Zilber, & Serences, 2015; Hakun & Ravizza, 2016; Rademaker, Bloem, De Weerd, & Sack, 2015; Yoon, Curtis, & D’Esposito, 2006; however, see Awh, Vogel, & Oh, 2006; Cools, Miyakawa, Sheridan, & D’Esposito, 2010; Ester et al., 2015; Gazzaley, Cooney, Rissman, & D’Esposito, 2005; McNab & Klingberg, 2008; Zanto & Gazzaley, 2009). In the real world, however, distraction is often in a different modality than the target information (e.g., a cell phone chirping while you are trying to remember the exit from the highway), and moreover, the distractor is typically unexpected. In these ways, the present approach is a more realistic paradigm for understanding distraction, and as we have shown, the distraction it produces is statistically robust and endures across trials. Further studies could test whether unexpected events interrupt visuomotor WM and induce guessing via “erasing” current traces through the stopping system, whether this extends to nonmotor forms of WM, and whether this explains clinical conditions of under- and over-distractibility.
The authors thank Fabrice Parmentier and Sirawaj Itthiipuripat for helpful advice on the task design, and Francesco Marini for programming. This research was funded by grants from the National Institutes of Health (R21 NS085543) and the James S. McDonnell Foundation (#220020375).
- Corsi, P. M. (1972). Human memory and the medial temporal region of the brain. Dissertation Abstracts International: Section B. Sciences and Engineering, 34(2), 891.Google Scholar