Introduction

Many models of working-memory have focused on understanding how attention can be used to mitigate forgetting in dual-tasking situations (Barrouillet et al., 2004; Barrouillet & Camos, 2015; Cowan et al., 2014; Oberauer et al., 2012; Oberauer & Lewandowsky, 2011; Raye et al., 2007; Vergauwe et al., 2010). Working memory, the information available for immediate use in ongoing cognitive processing, is critical to performance of higher-level cognitive tasks (Halford et al., 1998; Hitch, 1978; Süß et al., 2002; Sweller, 2016). This makes understanding successful working memory retention in the face of concurrent processing important for understanding human performance in a variety of common contexts. Models of working memory have been quite successful in predicting dual-task memory performance by focusing on the cognitive load of concurrent tasks (Barrouillet et al., 2004; Lemaire & Portrat, 2018; Oberauer et al., 2012; Oberauer & Lewandowsky, 2011; Portrat & Lemaire, 2015). These concurrent tasks generally include one or more two-alternative forced-choice decisions such as judging number parity, shape symmetry, or whether a simple math equation is correct. The cognitive load of concurrent tasks is often manipulated by either increasing the number of decisions to be made or decreasing the time available for making the decisions.

In particular, working memory models such as the Time-Based Resource-Sharing model (Barrouillet et al., 2004; Barrouillet & Camos, 2015) and the Serial Order in-a-Box Complex-Span model (Oberauer et al., 2012; Oberauer & Lewandowsky, 2014) predict a strong relationship between the cognitive load of concurrent tasks and the amount of forgetting observed in dual-task situations requiring the short-term retention of information in the face of concurrent processing. In these models, cognitive load is defined as the proportion of time a concurrent task distracts attention from memory maintenance activities during working memory retention, i.e., during the time that separates study and test (Barrouillet et al., 2004; Barrouillet & Camos, 2015). This class of models computes the cognitive load as a ratio of time occupied by the concurrent task during retention and the total duration of retention. Processing activities during retention result in forgetting, and free time available during retention is used to counteract forgetting with active maintenance. Thus, cognitive load is assumed to reflect constant mitigation of forgetting by attention-based mechanisms during working memory retention. That this ratio does well in explaining dual-task performance deficits is shown by a strong record of replication of the cognitive load effect in the existing literature (e.g., Barrouillet et al., 2007; Barrouillet et al., 2011; Hudjetz & Oberauer, 2007; Vergauwe et al., 2009; Vergauwe et al., 2010).

Whereas agreement exists as to the notion that the cognitive load effect reflects mitigation of ongoing forgetting by attention-based maintenance, an intensive debate has centered around the question of whether this ongoing forgetting is driven by passive working memory decay (e.g., Barrouillet et al., 2004; Barrouillet et al., 2012) or, conversely, by representation-based interference from concurrent processing (e.g., Oberauer et al., 2012; Oberauer & Lewandowsky, 2014). While the nature of forgetting driving the cognitive load effect is regularly debated, the existence of the cognitive load effect is rarely debated. However, some work suggests that cognitive load explanations may be limited to standard memoranda such as letters, words, or locations in matrices (Ricker & Cowan, 2010; Ricker & Vergauwe, 2020; Vergauwe, Camos, & Barrouillet, 2014a). Most studies examining cognitive load effects use these common memoranda presented one at a time, with a processing task following each memory item in the format often referred to as a complex span task. These tasks most often use verbal memory items, but sometimes use familiar visuo-spatial memory items, such as arrows or locations (Barrouillet et al., 2012; Langerock et al., 2014; Vergauwe et al., 2009, 2010; Vergauwe et al., 2012; Vergauwe et al., 2015). Ricker and Cowan (2010) diverged from the standard methodology and used a visual array task instead of a complex span approach to studying dual-task effects. Participants saw three unfamiliar visual memory items simultaneously and then had to perform a processing task during a single retention period before making a new/old recognition judgement about one of the memory items. They found that the cognitive load of the processing task explained some of the observed disruptions of memory performance, but that a significant portion of the observed loss of memory was related to the total length of the processing task and retention interval, rather than to the ratio between the durations of the processing task and the retention interval. In particular, Ricker and Cowan observed an effect of retention interval duration on memory performance, over and above any cognitive load effect. This is directly in conflict with cognitive load explanations of memory performance that state that the ratio of the duration of the processing task and the duration of the retention interval is the only temporal factor that should be relevant.

Vergauwe, Camos, et al. (2014) found similar results. They presented a visual array of three fonts, followed by a single retention interval that was either empty or that was filled by a concurrent processing task. At the end of the trial, participants had to make a new/old recognition judgement about one of the memory items. They found that the requirement to perform a processing task during retention resulted in memory loss, but also that the fonts were lost over time, even when attention was available for maintenance throughout the entire retention interval (i.e., when there was no concurrent processing to be carried out). Thus, in line with Ricker and Cowan (2010)’s observations, the absolute length of the retention interval was needed to account for forgetting. In cognitive load theories of dual-task forgetting, such as the Time-Based Resource-Sharing Model and Serial Order in-a-Box Complex-Span, time is only relevant in determining the ratio of occupied to free time. Studies showing an absolute effect of retention time are problematic at a theoretical level because there is no explanation in the theory for why attention-based maintenance mechanisms cannot fully mitigate forgetting over time.

Both Ricker and Cowan (2010) and Vergauwe, Camos, et al. (2014) proposed that the additional forgetting as a function of the absolute duration of retention interval may be related to the specific memoranda that were used, proposing passive decay of visual sensory features that cannot be maintained through attention-based mechanisms. If certain visual features cannot be maintained through attention-based mechanisms, their ongoing forgetting cannot be mitigated by attention-based maintenance mechanisms. From this, it follows that no cognitive load effect would be expected in tasks requiring retention of stimuli that are predominantly relying on visual sensory features. Recently, Ricker and Vergauwe (2020) examined this idea across four experiments in which participants had to reproduce the orientation of a simple visual memory item at test. Memory items were presented sequentially with a single retention period containing a concurrent processing task following presentation. In the first two experiments (Experiments 1a and 1b) participants retained sequentially presented memory items that could vary continuously in their orientation, promoting a representation of the visual sensory features. These items were followed by a single processing phase during which participants performed a concurrent processing task (tone discrimination or parity judgment).

If Ricker and Cowan (2010) and Vergauwe, Camos, et al. (2014) were correct, and visual sensory features cannot be actively maintained by attention-based mechanisms, then the experiments focusing on continuous orientation memory of Ricker and Vergauwe (2020) should show no cognitive load effect. This was indeed the case. However, in two additional experiments (Experiments 2a and 2b), the visual memoranda could only be in one of eight canonical orientations, promoting the retention of conceptual directions corresponding to top, bottom, left, right, and the related diagonals. We expected that the use of more categorical, conceptual representations should result in strong cognitive load effects, because previous studies have shown that these can be maintained through attention-based mechanisms. Surprisingly, no cognitive load effect was induced by the secondary task in these experiments either, casting doubt on the generality of cognitive load effects.

The cognitive load effect has been described as a Priority-A benchmark finding in working memory that all theories should be able to explain (Oberauer et al., 2018) and as a law relating storage and processing in working memory (Barrouillet et al., 2011). If the cognitive load effect is limited to a specific set of task parameters, this has important implications for working memory research and theory. Here we investigate whether cognitive load is truly the invariable law as has been previously argued or whether, instead, cognitive load has important boundary conditions. Any benchmark finding of cognition should be broadly applicable, not limited to a narrow set of task parameters.

In the following we show that cognitive load effects can be fickle and only generated under specific circumstances when using visual memoranda. The overall pattern of results producing cognitive load effects is inconsistent with any theory of cognitive load present in the existing literature. In each of four Experiments (1, 2a, 2b, and 3), we vary the cognitive load during memory retention. Across experiments, we manipulate our variables of theoretical interest to observe whether they modify the presence of absence of the cognitive load effect. Experiments 1 and 2 use a complex span (CS) task paradigm while Experiment 3 uses a Brown-Peterson (BP) task paradigm. Experiment 1 uses a long consolidation time following memory item presentation, while Experiments 2 and 3 use a short consolidation time following memory item presentation. This gives us the following three types of experiments that vary paradigm and consolidation time, CS-long consolidation (Experiment 1), CS-short consolidation (Experiments 2a and 2b), and BP-short consolidation (Experiment 3). Experiments 2a and 2b differ only in whether the memory items vary continuously in nature or take on only a few discrete values.

Experiment 1

The previous studies that provide some evidence against a cognitive load explanation of dual-task forgetting (Ricker & Cowan, 2010; Ricker & Vergauwe, 2020; Vergauwe, Camos, et al., 2014) used a Brown-Peterson task (Jarrold et al., 2011; Lucidi et al., 2016; Neath et al., 2014). In the Brown-Peterson task all memory items are presented sequentially and in a massed fashion, followed by a single retention interval containing a processing task of some sort, after which memory is tested (Brown, 1958; Peterson & Peterson, 1959). In contrast, most studies showing evidence of cognitive load as the driver of dual-task forgetting used a complex span task (e.g., Barrouillet et al., 2004; Barrouillet et al., 2007; Oberauer et al., 2012; Vergauwe et al., 2009, 2010) in which items are presented sequentially and in an interleaved fashion, with a retention interval following each memory item (Daneman & Carpenter, 1980; Turner & Engle, 1989). The processing task is performed during every retention interval. After all memory items and processing episodes are complete, memory is tested.

In Experiment 1, we used a complex span approach to investigate the cognitive load effect (see Fig. 1). The memoranda and processing tasks are the same as those used by Ricker and Vergauwe (2020), who failed to find an effect of cognitive load, except that their work used a Brown-Peterson task. Here, we present three memory items (locations of a dot on a ring), each followed by a 6-s retention interval during which a parity judgement task was to be carried out. This processing task varied in its cognitive load across trials. After all memory-processing episodes, the orientation of the memory items had to be reproduced. If the use of a Brown-Peterson task is responsible for the failure to observe a cognitive load effect in previous studies, then we should now observe a cognitive load effect in the present experiment.

Fig. 1
figure 1

An example of a single experimental trial in Experiment 1. Each memory item presentation was followed by a masking stimulus and then a full 6-s retention interval before the following memory item or recall phase. During each retention interval two, four, or six numbers appeared on screen and required a parity judgment response

Full size image

Method

Participants

Thirty-six students (22 female, ages 18–41 years) from the College of Staten Island participated in the experiment in exchange for partial course credit. We arrived at the existing sample size in this experiment, and all following experiments, by allowing participants to sign up for the experiment until we had collected data from at least 32 participants, at which point we stopped signing up new participants. The actual samples sizes vary across experiments because the number of participants that had signed up to participate in the experiments varied at the time we closed enrollment. All participants gave informed consent prior to participation in the study. The experiment was approved by the City University of New York Integrated Institutional Review Board under IRB File #2015-1156, entitled, “The Roles of Time and Interference in Working Memory.”

To ensure that any null effects of the cognitive load manipulation were due to cognitive load failing to impact memory performance we had to rule out the possibility that participants may have ignored the processing task. We filtered the data to remove trials with low performance on the processing task using a cutoff of 0.7 processing-task accuracy. See Table 1 for the results of all experiments under several different data filtering methods. After excluding these trials, we removed three participants from the analyses because they did not have data remaining in all experimental conditions. This left 33 participants.

Table 1 Performance metrics across experiments and inclusion criteria
Full size table

Design

Cognitive Load was manipulated within-participants and had three levels (low, medium, or high load). The Cognitive Load level on each trial was selected at random.

Materials and procedure

An example of an experimental trial is presented in Fig. 1. Participants initiated each trial by pressing the space bar. Each trial began with the presentation of a white fixation cross on a black background for 500 ms. Next the first item presentation occurred. Memory items were each a ring, 3.6 cm in diameter, with a dot, 0.5 cm in diameter, located somewhere along its circumference (see Fig. 1). Participants were to remember the location of the dot on the ring. Each dot appeared at one of eight locations. These locations corresponded to the four cardinals (top, bottom, right, and left) as well as the four locations halfway between the cardinals. Dot location was selected randomly with repetition allowed within a trial. Memory item color was determined based upon its presentation location. Each location was yoked to a unique color and all items presented in that location were presented in the yoked color for all trials and participants. Memory items were presented one at a time at one of eight locations 6.7 cm from the center of the screen, selected randomly without repetitions on any given trial. These locations were equally spaced from one another and arranged in a circle around the center of the screen. The eight locations were directly above the center of the screen and the seven remaining locations that resulted from 45° steps around the circle circumference. Each memory item was presented for 750 ms, followed by a mask stimulus presented for 200 ms. Masks consisted of eight circles slightly displaced from the location of the memory item they were masking, and eight dots located randomly within the rectangular area occupied by the masking circles. Each masking circle and dot were a unique color (see Fig. 1). This was followed by a blank consolidation period for 200 ms before a 6,000-ms retention period. This sequence was repeated for the second and third memory items with new stimuli for each item. During each 6,000-ms retention period the secondary task occurred.

During the secondary task a series of single-digit numbers were presented one at a time at the center of the screen in 30-pt font. The numbers were each chosen at random from the set of integers 1–8, inclusive. Processing stimuli remained on the screen until a response was made or until the next stimulus was presented. Participants were instructed to press the ‘a’ key if the number was odd or the ‘s’ key if the number was even. Two, four, or six numbers were presented during each retention interval, corresponding to the low-, medium-, and high-load conditions, respectively. The numbers were presented at a rate of one digit per 3, 1.5, or 1 s in the low-, medium-, and high-load conditions, respectively. Cognitive load condition was randomly selected on each trial. Participants were not notified of the cognitive load condition.

After all memory items and retention periods were complete, the memory probes were presented. Response probes were the memory rings presented in their original locations and colors with the dot located in the middle of the ring. Grey dots were located on the probe ring at each of the possible presentation locations. Each probe was presented alone in the order of memory item presentation and stayed on the screen until a response was entered. Participants moved the computer mouse to place the dot at the location it was originally seen on the edge of the ring, then clicked the left button to move on to the next response. The dependent measure of interest was response error in circular degrees, which could range from 0° to 180°. Feedback on memory task performance was given immediately after all three responses were made. Feedback consisted of all rings presented concurrently with the correct dot location marked in white and the participants response marked in the color of the memory stimulus. Mean response error for that trial determined which one of three tones played during feedback. Mean response error of less than 20° resulted in a rising/happy tone sequence. Mean response error between 20° and 60° resulted in a neutral tone sequence. Mean response error of greater than 60° resulted in a dropping/sad tone sequence.

All participants completed 12 practice trials followed by four blocks of 15 experimental trials. Cognitive Load condition was determined randomly for each trial. Table 2 details the key methodological factors differentiating each experiment.

Table 2 Main methodological factors in each experiment
Full size table

Analysis

We use the Bayes factor as our inferential statistic (Rouder et al., 2012), calculated for ANOVA using the “Bayesfactor” package in R (R. D. Morey & Rouder, 2013). We used a Cauchy prior with a standard deviation of √2/2 to model our effect size. The Bayes factor in the context of an ANOVA model represents the probability of the data when assuming an effect is present relative to the probability of the data when assuming no effect is present. A Bayes factor of 10 in favor of the alternative would indicate that you should update your current beliefs about the existence of the effect to be 10 times more likely than they were prior to seeing the data. A general rule of thumb is that Bayes factors larger than 3 indicate evidence in favor of a model (in the ANOVA context the models are the null or alternative hypotheses), while a Bayes factor of 10 or larger indicates strong evidence for a model. In reporting our results, we always explicitly report whether the Bayes factor favors the alternative or null hypothesis. All results reported in the text are those presented in column 3 of Table 1 (in bold font).

Results

Visual examination of Fig. 2 shows a small trend toward a Cognitive Load effect on mean response error. A repeated-measures ANOVA of mean response error as a function of Cognitive Load provides, however, strong evidence in favor of a null effect, F(2,64) = 1.29, dz = 0.28, Bayes factor = 6.30 in favor of the null (means: low = 42, medium = 43, high = 46).

Fig. 2
figure 2

Mean response error in degrees of angle by cognitive load condition observed in Experiment 1 (CS-long consolidation). Error bars represent standard error of the mean. The blue line shows the linear regression of mean response error on cognitive load

Full size image

As expected, secondary task performance was very high, with mean accuracy of .90. As often observed in cognitive load studies, secondary task performance decreased as the pace of the secondary increased (means: low = .95, medium = .89, high = .84; e.g., see Barrouillet et al., 2007; Barrouillet et al., 2011; Vergauwe et al., 2009, 2010).

Discussion

Experiment 1 failed to produce a cognitive load effect on memory performance. This result is consistent with the findings of Ricker and Vergauwe (2020) despite the use of a complex span task in the current experiment. It indicates that the use of a complex span task is not enough in itself to produce a cognitive load effect with visual memoranda. The secondary task we have used here (i.e., parity task) has been shown to produce cognitive load effects elsewhere under similar cognitive load conditions (Barrouillet et al., 2007; Barrouillet et al., 2011; Camos & Portrat, 2015; De Schrijver & Barrouillet, 2017), so the nature of the secondary task is unlikely to be the reason we fail to observe a cognitive load effect. Our primary memory task difficulty is also unlikely to be the source of the cognitive load failure. One could imagine that if it were too easy, participants could coordinate performance of both tasks with little strain on their cognitive resources. Mean error in the present experiment was 44°, far from either floor or ceiling effects.

In the next experiment we used the same approach as in Experiment 1 but tried to induce a cognitive load effect by manipulating the total time available for consolidation, i.e., free time from memory item presentation until another attention-demanding task was required. This manipulation has been shown to modify the size of the cognitive load effect in two previous experiments (De Schrijver & Barrouillet, 2017; but see Bayliss et al., 2015) and is thought to modify the memory trace’s vulnerability to forgetting under many conditions (Jolicœur & Dell'Acqua, 1998; Ricker, 2015; Ricker & Cowan, 2014), with some notable exceptions (Ricker et al., 2019).

Experiment 2

De Schrijver and Barrouillet (2017) explored the role of consolidation time in the complex span task. Working memory consolidation is the process by which a fragile sensory memory trace is stabilized into working memory and made resistant to forgetting (Chun & Potter, 1995; Jolicœur & Dell'Acqua, 1998; Ricker, 2015; Ricker & Cowan, 2014). In De Schrijver and Barrouillet’s work a variable period of free time between item presentation and secondary task onset was introduced during which attention could be used to focus on the memory item for consolidation. When this consolidation period was longer, they observed smaller cognitive load effects on memory performance. De Schrijver and Barrouillet proposed that cognitive load effects are modified by the time available for consolidation into working memory before any secondary task was presented.

If this is true, then perhaps the present failure to observe a cognitive load effect stems from allowing relatively long consolidation times for our present memory items. Although the presentation times we used in Experiment 1 were similar to the shortest consolidation periods used by De Schrijver and Barrouillet (2017), the memory stimuli they used were different from our own. De Schrijver and Barrouillet asked participants to remember consonant lists, whereas we ask participants to remember simple visual orientations. Perhaps the consolidation time needed to avoid cognitive load effects is shorter with our stimuli than with De Schrijver and Barrouillet’s consonant lists. Indeed, both Ricker and Hardman (2017) and Ricker and Sandry (2018) explored the duration of consolidation with a continuously variable version ring-dot stimuli used here and found that consolidation was complete in less than 700 ms.

In Experiment 1, the time available for consolidation of each item into working memory was 1,150 ms (i.e., 750-ms presentation, 200-ms mask, and 200-ms blank screen). In Experiment 2, we reduced this time to 500 ms by reducing the presentation durations of the memory stimulus, mask, and blank-screen period following each mask. In Experiment 2a memory items were canonical, meaning that, like in Experiment 1, the location of the dot could only occur at one of eight stereotypical locations (top, bottom, left, right, and the four diagonals between the cardinals). In Experiment 2b memory items were continuous meaning that the location of the dot could be anywhere along the edge of the circle. In all other ways, Experiments 2a and 2b are the same as Experiment 1. If these time-related changes lead to the observation of a cognitive load effect, then we can conclude that the previous experiment’s null effect was due to a sufficiently long consolidation period rendering the memory items immune to cognitive load-related effects.

Method

Participants

Ninety-seven students (66 female, ages 18–47 years) from the College of Staten Island participated in the experiment in exchange for partial course credit. All participants gave informed consent prior to participation in the study. The experiment was approved by the City University of New York Integrated Institutional Review Board under IRB File #2015-1156, entitled, “The Roles of Time and Interference in Working Memory.” One participant was excluded from the analysis because they did not complete the experiment. After excluding trials with less than 0.7 secondary task accuracy, we removed six participants from the analyses because they did not have data remaining in all experimental conditions. See Table 1 for the results of all experiments under several different data filtering methods. This left 90 participants, 45 in the analysis of Experiment 2a and 45 participants in the analysis of Experiment 2b.

Design

The design was the same as in Experiment 1.

Materials and procedure

The procedure was the same as in Experiment 1 with the following changes. In both Experiments 2a and 2b the presentation time for each memory item was 300 ms. The presentation time for the mask was 100 ms and the blank consolidation period lasted 100 ms. In Experiment 2b the dots on the edge of the memory items could appear at any one of 360 locations on the edge of the ring. Each potential location was separated by 1° on the ring. Compared to Experiment 2a, the probe stimuli in Experiment 2b lacked the grey placeholder dots on its perimeter to reflect this change. Participants again completed four blocks of 15 experimental trials. See Table 2 for the major methodological differences in procedures across experiments.

Analysis

The analysis was the same as in Experiment 1.

Results

Experiment 2a (canonical stimuli)

Visual examination of Fig. 3a clearly shows an effect of Cognitive Load condition on mean response error. A repeated-measures ANOVA of mean response error as a function of Cognitive Load confirms this statistically, F(2,88) = 7.10, dz = 0.51, Bayes factor = 16.22 in favor of an effect (means: low = 46, medium = 49, high = 55).

Fig. 3
figure 3

Mean response error in° of angle by cognitive load condition observed in Experiment 2a (panel a) and Experiment 2b (panel b) (both CS-short consolidation). Error bars represent standard error of the mean. The blue lines show the linear regression of mean response error on cognitive load

Full size image

As expected, secondary task performance was very high, with mean accuracy of .87. As often observed in cognitive load studies, secondary task performance decreased as the pace of the secondary increased (means: low = .93, medium = .88, high = .80; e.g., see Barrouillet et al., 2007; Barrouillet et al., 2011; Vergauwe et al., 2009, 2010).

Experiment 2b (continuous stimuli)

Visual examination of Fig. 3b clearly shows an effect of Cognitive Load condition on mean response error. A repeated-measures ANOVA of mean response error as a function of Cognitive Load confirms this statistically, F(2,88) = 12.37, dz = 0.68, Bayes factor = 908 in favor of an effect (means: low = 54, medium = 59, high = 66).

As expected, secondary task performance was very high, with mean accuracy of .88. As often observed, secondary task performance decreased as the pace of the secondary increased (means: low = .93, medium = .87, high = .81; e.g. see Barrouillet et al., 2007; Barrouillet et al., 2011; Vergauwe et al., 2009, 2010).

Discussion

Experiment 2 demonstrated a clear effect of cognitive load on memory performance, in contrast to both Experiment 1 and the four experiments by Ricker and Vergauwe (2020). Experiment 2 differed from these previous experiments in that it had a shorter consolidation time designed to make the memory items more vulnerable to forgetting from cognitive load effects (De Schrijver & Barrouillet, 2017; Ricker, 2015; Ricker & Cowan, 2014). The re-emergence of cognitive load effects at short consolidation times occurred regardless of whether the memory items were canonical, Experiment 2a, or continuous, Experiment 2b, in nature. In this respect, Experiment 2 refutes the theory that categorical conceptual representations should show a cognitive load effect while memory representations for specific sensory features should not show a cognitive load effect (Ricker & Cowan, 2010; Vergauwe, Camos, et al., 2014; see also Ricker & Vergauwe, 2020).

In a final experiment we explore whether consolidation time alone is enough to explain which experiments produce cognitive load effects with visual memoranda and which experiments do not.

Experiment 3

Together, Experiments 1 and 2 established that a shortened consolidation time is needed for an effect of cognitive load to manifest in a complex span task using visual memoranda. We now explored whether shortened consolidation time is enough to evoke the cognitive load effect or whether the use of a complex span task is necessary as well. This question is motivated by the results of Ricker and Vergauwe (2020), who found no effect of cognitive load when using a Brown-Peterson task with the same memory stimuli and a long consolidation time. Based on the results of the current Experiments 1 and 2, and the results of Ricker and Vergauwe (2020), it could is not clear whether (1) shorter consolidation times in any task would result in a cognitive load effect, and thus, short consolidation times in the Brown-Peterson task would result in an effect of cognitive load, or, alternatively, (2) shorter consolidation times would result in a cognitive load effect in the complex span task, but not in the Brown-Peterson paradigm. The latter would suggest that the Brown-Peterson paradigm may be another condition that eliminates the cognitive load effect, in addition to long consolidation times.

Therefore, in Experiment 3, we use a Brown-Peterson task paired with the same fast presentation and consolidation conditions as in Experiment 2 (see Table 2 for a methodological summary). If shortened consolidation times are enough in themselves to produce a cognitive load effect in our visual memory task, then we should also see an effect of cognitive load in Experiment 3. If the combination of shortened consolidation times and a complex span task are both necessary to observe cognitive load effects in our visual memory task, then we should not see a cognitive load effect in Experiment 3.

Method

Participants

Forty-eight students (36 female, ages 18–51 years) from the College of Staten Island participated in the experiment in exchange for partial course credit. All participants gave informed consent prior to participation in the study. The experiment was approved by the City University of New York Integrated Institutional Review Board under IRB File #2015-1156, entitled, “The Roles of Time and Interference in Working Memory.”

We raised the processing task cutoff for trial inclusion in Experiment 3 to perfect accuracy on the processing task. We did this for two reasons. First, there were only two to six secondary task iterations of the processing task in Experiment 3, whereas there were six to 18 iterations in Experiments 1 and 2. This means we should expect fewer trials with motor error or other errors unrelated to cognitive effort in Experiment 3. We wished to retain as many of those trials as possible in Experiments 1 and 2, leading to a more liberal cutoff threshold. Second, we wished to keep our data filtering method consistent with Ricker and Vergauwe (2020), who used a very similar Brown-Peterson paradigm to investigate cognitive load effects. Doing so allows the data to be directly comparable across studies.

See Table 1 for the results of all experiments under several different data filtering methods. After excluding trials with less than perfect processing task performance, we removed four participants from the analyses because they did not have data remaining in all experimental conditions. This left forty-four participants.

Design

The design was the same as in the previous experiments.

Materials and procedure

The procedure was the same as Experiment 2a, except that there was no retention interval or processing task after the first or second memory item presentations. Only the third memory item was followed by a retention interval filled with a processing task. As in the previous experiments, the retention interval lasted for 6,000 ms and two, four, or six numbers were presented. Participants completed four blocks of 30 experimental trials. See Table 2 for the major methodological differences in procedures across experiments.

Analysis

All analyses were the same as in the previous experiments.

Results

Visual examination of Fig. 4 indicates no effect of Cognitive Load condition on mean response error. A repeated-measures ANOVA of mean response error as a function of Cognitive Load confirms this statistically, F(2,86) = 0.88, dz = 0.17, Bayes factor = 11.19 in favor of the null (means: low = 63, medium = 64, high = 66).

Fig. 4
figure 4

Mean response error in° of angle by cognitive load condition observed in Experiment 3 (BP-short consolidation). Error bars represent standard error of the mean. The blue line shows the linear regression of mean response error on cognitive load

Full size image

Trials without perfect secondary task performance were removed, and thus, no accuracy analysis was performed.

Discussion

The change from a complex span task in Experiment 2 to a Brown-Peterson task in Experiment 3 abolished the cognitive load effect on memory performance, even though the same shortened consolidation times were used in these experiments. It appears that a single episode of secondary task execution after all memory items are presented in a massed fashion is insufficient to disrupt visual memory performance, even under conditions of impoverished consolidation. This is problematic for decay-based cognitive load approaches to forgetting where loss of memory items is assumed to happen over just a few hundred milliseconds (Barrouillet et al., 2004; Barrouillet & Camos, 2015) as well as interference-based accounts involving distractor removal during free time (Oberauer et al., 2012; Oberauer & Lewandowsky, 2014) as both predict a relationship between cognitive load and memory performance regardless of the number of retention intervals. Interference models of cognitive load, such as Serial-Order in-a-Box Complex-Span have not been applied to the Brown-Peterson task, but it is difficult to see how they would account for the changing pattern of effects across the present data and that of Ricker and Vergauwe (2020). To the best of our knowledge, no existing theory predicts an effect of cognitive load that only emerges in the presence of multiple successive retention intervals and a short consolidation time.

General discussion

Our findings across four experiments demonstrate that a combination of impoverished consolidation and repeated episodes of concurrent processing are necessary to observe cognitive load effects for our visual memoranda. Experiment 1 used a complex span task procedure with a long consolidation time (which we refer to as CS-long consolidation). Experiment 3 used a short consolidation period in a Brown-Peterson task (which we refer to as BP-short consolidation). Neither of these experiments produced a cognitive load effect. Only when the complex span task was combined with a short consolidation time in Experiments 2a and 2b (which we refer to as CS-short consolidation) did we observe a cognitive load effect.

These findings represent clear boundary conditions for cognitive load effects in visual working memory. Studies exploring cognitive load typically use verbal memoranda presented in a complex span task. Results from these studies are then assumed to reflect universal properties of working memory within models such as the Time-Based Resource-Sharing model (Barrouillet et al., 2004; Barrouillet & Camos, 2015) and the Serial Order in-a-Box Complex-Span model (Oberauer et al., 2012; Oberauer & Lewandowsky, 2014). Here we show that this generalization is not appropriate by demonstrating boundary conditions for when cognitive load effects are present in visual working memory. This does not mean, however, that cognitive load effects are never observed in visual working memory. Indeed, in some specific conditions here, we did find the cognitive load effect. Some studies that do not use sequential memory presentation, and thus for which it is more difficult to derive predictions from cognitive load theories, have indeed found a cognitive load effect for visual memory materials (e.g., Loaiza & Souza, 2019; Ricker & Cowan, 2010; Vergauwe, Langerock, & Barrouillet, 2014b).

Although we found evidence for a null effect of cognitive load in Experiments 1 and 3 of the present work, as well as in Experiments 1a, 1b, 2a, and 2b of Ricker and Vergauwe (2020), four of these experiments did show a very small trend in the direction predicted by cognitive load theories and none of them showed a trend in the opposite direction (Experiments 1a and 2a of Ricker and Vergauwe showed no change in memory performance across conditions). One could argue this null effect represents a reduced cognitive load effect, and not a missing cognitive load effect. Even if one takes this view, the present work is still inconsistent with current theories. The size of this small effect is only 4° in Experiment 1 and 3° in Experiment 3, an inconsequential difference by any practical standard. Current theories cannot explain why we should see a reasonably sized effect in Experiments 2a and 2b, but a tiny effect in Experiments 1 and 3 and in the work of Ricker and Vergauwe (2020). Whether the effect is truly absent (i.e., a true effect equal to exactly 0) or simply very tiny, the same boundary conditions for observing a meaningful cognitive load effect remain, as do their implications for theory.

In the following we address the implications of these boundary conditions for current theories of working memory and then provide a speculative account that may better describe when one will and will not observe a cognitive load effect.

Implication for current theories of cognitive load

Major theories that explain the cognitive load effect focus on a balance of forgetting and attention-based maintenance, with attention-based maintenance acting through either reactivation of memory traces, often called refreshing, or removal of distractors. This includes, for example, the original TBRS model (Barrouillet et al., 2004; Barrouillet et al., 2007; Barrouillet et al., 2011; Barrouillet & Camos, 2015; Vergauwe et al., 2009, 2010), variants of the TBRS model such as TBRS* and TBRS2 models (i.e., different computational implementations of the TBRS model; e.g., Gauvrit & Mathy, 2018; Lemaire & Portrat, 2018; Oberauer & Lewandowsky, 2011), and conceptually related interference models such as the Serial Order in-a-Box Complex-Span modelFootnote 1 (Oberauer et al., 2012; Oberauer & Kliegl, 2006; Oberauer & Lewandowsky, 2011). Although these models have significant differences from one another, they all predict that the effect of cognitive load on memory performance should be similar across different experiments if memory set size and cognitive load ratio remain constant across the experiments. In the current work, all four experiments used the same memory set size and the same cognitive load ratio and thus, should have resulted in a similar cognitive load effect if a balance between forgetting and attention-based maintenance drives the cognitive load effect. The differing results across our experiments indicate that a balance between forgetting and maintenance does not drive the cognitive load effect and it is unclear how the existing models of cognitive load could account for the pattern of data in all the present experiment a coherent way. A different explanation of the cognitive load effect is needed.

One alternative explanation would be that cognitive load effects are driven by the gradual build-up of interference over time. Indeed, not all interference theories assume forgetting and attention-based maintenance reach a balance that completely counteracts forgetting. Many interference theories posit that interference gradually builds up over time and events (Lewandowsky et al., 2004; Lewandowsky & Oberauer, 2009; Nairne, 2002). These theories predict that memory performance is determined by an interaction between the length of retention time and the level of cognitive load. Higher cognitive loads show more forgetting because they contain more interfering events. Tasks with multiple, successive retention phases filled with processing activities typically contain more interference than tasks with a single retention phase filled with the processing activities, leading to more predicted forgetting in complex span tasks than in Brown-Peterson tasks. This loss over time should be at a faster rate with a higher cognitive load due to the greater amount of interference per unit of time. At first glance, our data appear to fit this pattern. When memory traces are vulnerable to forgetting because of short consolidation time, and when there is a large amount of processing task steps providing interference in the same experiment, we see a cognitive load effect (CS-short consolidation in Experiment 2). In the absence of the combination of these conditions, we do not (CS-long consolidation in Experiment 1 and BP-short consolidation in Experiment 3).

However, there is a problem with this interference explanation. As can be seen by comparing Figs. 3 and 4, memory performance in Experiment 3 (BP-short consolidation) is considerably worse than performance in Experiment 2 (CS-short consolidation), at all cognitive loads. This is problematic for the notion that memory performance depends on the gradual build-up of interference because the amount of interference is three times lower in Experiment 3 than in Experiment 2. For example, there were 18 processing items in the High-Load condition of Experiment 2 (CS-short consolidation) and only six processing items in the High-Load condition of Experiment 3 (BP-short consolidation), but memory performance was considerably worse in Experiment 3 than in Experiment 2 (mean response error of 66 and 55, respectively). This counter-intuitive finding of improved performance in the complex span task relative to a comparable Brown-Peterson task has been observed in other studies (Jarrold et al., 2011; Tehan et al., 2001) and is directly opposed to a pure interference-based explanation that posits more interference leads to lower memory performance.

Alternative accounts of the cognitive load effect

The first alternative approach we considered for understanding the present data was a post-error processing account. In this approach, making an error on the processing task often results in distraction and internally-generated interference from the participants’ conscious reflection upon the error (Kleiter & Schwarzenbacher, 1989). In higher cognitive load conditions, more processing task errors are made, resulting in more internal distraction/interference. This, in turn, results in more dual-task forgetting in higher cognitive load conditions (see also Lewandowsky & Oberauer, 2009, for a similar argument). We conducted several post hoc analyses to test whether this post-error processing explanation could be leading to the cognitive load effects we do observe in the present study. These analyses were conducted on the data from Experiment 2b because this experiment demonstrated the largest cognitive load effect in the present study. Post-error analyses are reported in detail in the Appendix.

Our rationale in the post-error processing analyses was the following. If errors on the secondary task cause errors in the primary memory task, then participants who make more errors on the secondary task should also produce more errors on the primary memory task. To test this, we calculated mean error rates of each task (secondary processing and primary memory tasks) for each participant and computed the correlation between the two measures. We found no relationship between error rates on the secondary task and error rates on the memory task across participants. Next, a second set of analyses was conducted on within-participant effects of errors on the secondary task. If post-error processing causes the observed cognitive load effects, then trials with more errors on the secondary task should display lower performance on the primary memory task. Our analyses within each cognitive load condition show, however, that making more errors on the secondary task did not predict lower performance on the memory task. Taken together, these additional analyses of our data rule out post-error processing as a plausible explanation of the present findings. Oberauer and Lewandowsky (2013) came to a similar conclusion in testing a post-error processing account of cognitive load effects in a different data set.

Another account worth considering is the multiple-component models of working memory (e.g., Baddeley, 1986; Baddeley & Logie, 1999). This approach proposes process-based and/or domain-based fractionations of working memory resources such that only within process/domain tasks should interfere with one another. A multi-component approach is satisfying when considering any of the present experiments in isolation but cannot account for the full pattern of results. Assuming processing and storage rely on separate resources could account for the lack of an effect of cognitive load in Experiments 1 and 3 but cannot at the same time explain why the effect appeared while using the same memory processing tasks in Experiment 2.

Finally, the Embedded-Processes model of Cowan (1988) would predict interference effects between concurrent processing and storage activities in all experiments, because of a shared central resource. While the model explicitly assigns an important role to consolidation in working memory (Cowan et al., 2021), it is not clear to us how this model could account for the entire data pattern presented here which also presents marked differences between complex span tasks and Brown-Peterson tasks.

Taken together, it appears that a fundamentally different approach to understanding the cognitive load effect is needed. In this way, the present findings complement work by Joseph and Morey (2021) showing that current theories of cognitive load do not adequately explain the effects of storage on processing.

An enrichment account of the cognitive load effect?

One new approach could be what we refer to as the memory enrichment approach. The memory enrichment approach posits that the lower memory performance associated with increased cognitive load reflects the blocking of strategic memory enrichment processes, rather than a change in the balance between forgetting and maintenance mechanisms. According to this approach, memory enrichment is strategic, employed when participants feel that the memory representations held in mind are more impoverished than necessary for optimal task performance and when there is enough free time available to enrich memory representations. Impoverished memory representations could result from a lack of sufficient consolidation for visual items, a lack of semantic elaboration when using meaningful verbal items, or the absence of any other process that leads to a more useful memory representation. Stimulus and task affordances must support memory enrichment for it to occur. If insufficient time is available or if the memory representation held is sufficient for optimal performance, then participants will use very little of their free time to enrich their memories.

In the enrichment approach, there is little to no role for forgetting, or for maintenance activities that counteract forgetting, to explain the cognitive load effect. Instead, cognitive load effects would be assumed to arise from the availability of free time to strategically enrich memory representations as a way of improving task performance. As a result, cognitive load effects are only observed when the memory representations are less complete than is necessary for optimal task performance. This enrichment approach could account for several important aspects of our findings: (1) better overall memory performance in Complex-Span tasks than in Brown-Peterson tasks, when short consolidation times are used, (2) the observation that the present cognitive load effects depend on consolidation time and task situation, and (3) the observation that cognitive load effects become more apparent when including trials and participants with poor processing accuracy. We discuss these points in turn below. However, at this point, the enrichment account is speculative and further study is needed to test its adequacy.

First, within an enrichment approach, the performance boost observed when going from the Brown-Peterson version of our task to the complex span version reflects the increase in the availability of free time for memory enrichment to compensate for low consolidation time. This time is not being used to compensate for forgetting that occurs during retention, but rather to compensate for poor initial memory creation. In the present experiments, the Brown-Peterson task does not have free time after each memory item. This makes it more difficult/effortful to enrich each memory representation than in the complex span task because enrichment cannot occur online, immediately following each item presentation. Instead, participants must think back to previous items before they can be enriched. This leads to a severe reduction in the amount of enrichment in the Brown-Peterson task and thus in poorer memory performance compared to the Complex-Span task (mean error of 64 and 50, in Experiments 3 and 2, respectively), when consolidation times are short. Moreover, and consistent with the notion that enrichment is used when memory representations are less complete than is necessary for optimal task performance, the performance boost observed when going from the Brown-Peterson version of our task to the complex span version is not observed when long consolidation times are used (mean error of 41 and 44, in Experiment 2b of Ricker & Vergauwe, 2020 and current Experiment 1, respectively), presumably because more consolidation time reduces the need for memory enrichment for the type of visual memoranda we used.

Relatedly, and second, the present work demonstrates that when memory items are encoded with a sufficient representation, as in Experiment 1 (CS-long consolidation), participants do not or cannot use free time to improve memory further, at least not when using the type of visual memoranda we did. Experiment 1 (CS-long consolidation) showed no cognitive load effect and an overall performance level similar to the low load condition of Experiment 2 (CS-short consolidation). This indicates that in the low cognitive load condition of Experiment 2 (CS-short consolidation), participants were able to use their free time to improve their memory performance until it reached the level of Experiment 1 (CS-long consolidation), but not further. This finding indicates that task and stimulus affordances dictate what strategies participants will use for enrichment. As such, the task to be performed, the material involved, and the repertoire of enrichment strategies of the participants will together determine whether or not a cognitive load effect will be observed (see Macken et al., 2015, for a similar proposal).

Finally, further evidence that the cognitive load effect reflects memory enrichment and not a balance of maintenance and forgetting comes from the filtering data presented in Table 1. When participants ignore the processing task, forgetting approaches to cognitive load predict that processing task performance should be low, and the cost of the processing task on memory performance should be minimal, reducing the effect size of the cognitive load manipulation. This concern is why researchers typically filter their data in some way to exclude low-effort participants/trials when testing for cognitive load effects. Applying this type of filter here, we observed evidence for a cognitive load effect in Experiment 2, but not in Experiments 1 and 3. If, however, we do not remove any participants or trials from our analysis, clear cognitive load effects with larger effect sizes emerge in all experiments. In other words, including participants and trials with low processing-task effort increased the size of the cognitive load effect, in direct opposition to what would be expected from a forgetting account of cognitive load. Lack of filtering should shrink the cognitive load effect if the processing task load is causing forgetting, but our data show an enhancement of the cognitive load effect when no filtering is applied (also see Ricker & Vergauwe, 2020, for a similar finding).

An enrichment approach to cognitive load effects predicts a different relationship between processing accuracy and the cognitive load effect. In short, it predicts that including participants and trials with poor processing task performance should lead to a larger cognitive load effect. When memory items are encoded with lower-than-average quality representations, these trials require more enrichment to reach an acceptable representation quality. Lower-than-average quality representations can result from stimuli/task conditions, such as low consolidation, but also from participant-related characteristics such as fluctuations in attention (Adam & deBettencourt, 2019). On these lower-quality representation trials, it is likely that participants attempt to enrich memory representations for a longer period, potentially exceeding the available free time before processing must begin. This leads to increased failure to execute the processing task and, on average, poorer processing task performance.

When filtering the data by excluding secondary task errors one is not only removing trials in which participants may be ignoring both processing and memory demands of the dual-task situation. Trials with ongoing enrichment at the time of processing task onset are also being removed. Because higher cognitive loads have a more demanding processing task schedule, a higher proportion of high load trials are removed by filtering than low load trials. When we do not exclude any trials, our data set retains more low memory performance trials in which the processing task competes for time with enrichment, resulting in lower performance in both tasks. Most of these trials belong to the high load condition. This selectively decreases memory performance in the high load condition compared to the other conditions, creating a cognitive load effect in the unfiltered data. This pattern is what we see in the filtering data (see Table 1). While changes in the cognitive load effect as a function of data filtering give evidence against a maintenance and forgetting interpretation, they can be accounted for by a memory enrichment interpretation.

The exact mechanism responsible for memory enrichment is not certain, but several candidates exist, and the specific mechanism may differ across task situations and memory materials. In the present data, it is likely that the low and medium cognitive load conditions of Experiment 2 (CS-short consolidation) allowed for continued consolidation of the memory stimuli during the retention interval (Jolicœur & Dell'Acqua, 1998; Nieuwenstein & Wyble, 2014; Ricker & Hardman, 2017). In these conditions, the time pressure for completing the processing task would have been relatively light. Successful processing task performance and completion of memory item consolidation would both have been possible given the free time available (Ricker & Hardman, 2017; Ricker & Sandry, 2018; Vergauwe et al., 2009, 2010; Vergauwe et al., 2012). Mızrak and Oberauer's (2021) encoding resources mechanism could enrich memory performance in a similar fashion. In their theory, encoding resources are depleted when used and then require free time to replenish. If one assumes that replenishment cannot occur during secondary task processing, an assumption that is currently not in Mizrak and Oberauer’s theory, enrichment could be the use of free time to replenish encoding resources. More replenishment can occur in conditions with lower cognitive load, but this is only necessary when encoding times are short.

Joseph and Morey (2021) recently proposed that response preparation may occur during free time between processing episodes of the complex span task. This is consistent with both the present data and an enrichment approach to understanding cognitive load effects if one assumes that once a memory representation is encoded or consolidated the participant attempts to map that representation onto a motor response. The key component from an enrichment perspective is that free time is being used to enrich the memory representation, be it by adding a motor response component or by adding any other type of component (e.g., long-term memory). Mechanisms such as elaborative rehearsal, reflective elaboration of the stimuli whereby participants improve performance by deeper more meaningful encoding (Bartsch et al., 2018; Craik & Tulving, 1975; Rose et al., 2010), seem unlikely with our simple visual stimuli, but possible. Free time could be used to generate visual imagery or novel sentences relating the memory items to one another. The cognitive load condition would be serving as a depth of processing effect, allowing deeper processing of the memory stimuli under lower load conditions. With meaningful and semantically rich visual stimuli similar effects may occur. Brady and colleagues argue that additional working memory capacity can be recruited to support memory for meaningful stimuli (Brady et al., 2016) and visual group characteristics are extracted (Schurgin & Brady, 2019) in under a second or two. A richer mental representation of this sort could be considered a form of shallow semantic elaboration that occurs much faster than does classic reflective elaboration and would lead to better performance at test.

Conclusion

The present work investigated the boundary conditions of the cognitive load effect in visual working memory and showed that only under specific circumstances is cognitive load crucial to visual working memory performance. Our findings across four experiments demonstrate that a combination of impoverished consolidation and repeated episodes of concurrent processing are necessary to observe cognitive load effects in our visual memory task. Existing theories of dual-task forgetting cannot account for these results, which indicates that a balance between forgetting and maintenance does not drive the cognitive load effect. In addition to accounts designed to address cognitive load findings we also considered whether other alternative accounts could describe the current data. Post-error processing and multiple-component approaches to memory were not consistent with the observed patterns. Instead of these existing accounts of the cognitive load effect, we provide a speculative memory enrichment approach to understanding cognitive load effects.