Memory and metacognition for piano melodies: Illusory advantages of fixed- over random-order practice
Some learning schedules can foster an illusion of competence, whereby the learner feels that the skill will be retained better than it actually will be. Consider fixed-order practice, in which a person practices a task repeatedly before switching to the next task (e.g., task order A, A, B, B); in contrast, in random-order practice, a person randomly alternates among two or more tasks (e.g., task order C, D, D, C). In the present experiment, participants (n = 25) who had formal training in piano practiced melodies under fixed- or random-order conditions (within-subjects), and then returned for a retention test 2 days later. Initially, the participants performed faster on melodies practiced in a fixed-order. However, on a retention test 2 days later, participants were faster with melodies from the random-order condition. Despite the within-subjects design, which facilitated the comparison of practice conditions, participants’ metacognitive judgments indicated an illusion of competence, whereby they erroneously believed that fixed-order practice would result in faster retention performance. Our results suggest that even some trained musicians may use ease of acquisition as a heuristic for predicting future performance.
KeywordsContextual interference Metacognition JOL Music cognition Motor learning
The order in which someone practices a motor skill, such as playing an instrument or typing, can have a large impact on the memory retention for that skill. Past research has revealed a paradox, which is that the practice schedule that produces superior performance at the time of acquisition may yield inferior performance at retention (Simon & Bjork, 2001, 2002). Over the years, this paradox has been demonstrated in motor learning for various skills, including keyed timing (Simon & Bjork, 2001, 2002), golfing (Porter, Landin, Hebert, & Baum, 2007), knot tying (Ollis, Button, & Fairweather, 2005), and musical instrument learning (Stambaugh, 2011). Additionally, practice schedules can influence an individual’s judgments of learning by fostering an illusion of competence, whereby the learner feels that the skill will be retained better than it actually will be. How robust is this illusion? In the present report, we sought to demonstrate both (a) that the illusion can occur even in formally trained individuals—in this case, musicians—and (b) that the illusion can occur even when participants can easily compare practice schedules.
When describing the influence of practice order on the acquisition and retention of a motor activity, one can draw upon the contextual-interference principle (Battig, 1979; see Magill & Hall, 1990, for a review). This principle proposes that arranging to-be-learned materials in a way that introduces interference between those materials typically impedes performance at acquisition, but frequently leads to superior performance at the time of retention (Lee & Simon, 2004). For example, imagine that an individual is asked to practice melodies A and B in a fixed-order (e.g., melodies A, A, A, B, B, B) and is asked to practice melodies C and D in a random-order (e.g., melodies C, D, C, C, D, D). Given that practicing melodies C and D requires the learner to alternate between the melodies in no specific pattern, arranging items in random-order is one way to introduce interference during acquisition of a skill (Shea & Morgan, 1979). Because of the high interference during learning of melodies C and D, the contextual-interference principle predicts poor acquisition and superior retention performance for those particular melodies. Additionally, the contextual-interference principle suggests that judgments made while under the influence of contextual-interference may reflect misconceptions of future performance.
Research has revealed that judgments of learning (JOLs) are sometimes an unreliable heuristic for future performance (e.g., Bjork, 1999; Jacoby, Bjork, & Kelley, 1994; Koriat & Bjork, 2005). For example, JOLs are generally sensitive to manipulations that occur throughout learning, and certain manipulations may beget an illusion of competence whereby learners feel that the skill will be retained better than it actually will be (Schmidt & Bjork, 1992; Simon & Bjork, 2001, 2002). To this end, JOLs made after a delay are often far more accurate at predicting subsequent performance than are JOLs made immediately after study (Koriat & Bjork, 2005). One possible explanation of this pattern is that JOLs made immediately after study are made in the presence of the target (i.e., the target is present in working memory), and predictions made at this time often reflect overconfidence (Koriat & Bjork, 2005). Therefore, given the notion that certain study conditions may receive more support from working memory (e.g., fixed-order practice) than others do (random-order practice; Lee & Magill, 1985), it can be presumed that the type of practice condition may influence a learner’s prediction of future performance.
JOLs can be measured in various ways, including asking participants which practice schedule they would prefer to use in the future (Kornell, Castel, Eich, & Bjork, 2010) or requiring participants to judge how quickly they believe they will later be able to execute a task (Simon & Bjork 2001, 2002). When using preference JOLs as an assessment of learning, individuals significantly prefer fixed- over random-order practice, regardless of retention outcome (Kornell et al., 2010). This misconception in learning is also reflected in time-based JOLs, whereby participants exhibit an illusion of competence for tasks practiced in fixed-order (Simon & Bjork, 2001, 2002). Such JOL miscalibration may be due to the use of retrieval fluency as a basis for predicting future performance (Koriat & Ma’ayan, 2005). In other words, the initial ease of performance of fixed-order practice may lead to overconfidence, and it may do so on a variety of metacognitive judgments.
The interaction between metacognition and type of study has been closely examined for motor learning. In a particularly relevant set of experiments (Simon & Bjork, 2001, 2002), participants learned keystroke patterns on the number pad of a computer by either fixed-order (e.g., pattern A, A, A, B, B, B, C, C, C) or random-order (e.g., pattern D, E, D, E, F, D, F, E, F) practice trials. Each to-be-learned pattern had a specific goal movement time, and participants were instructed to execute each pattern within its specified goal movement time. After the acquisition phase, participants were asked to make predictions of how close they thought their performance would be to the relevant goal movement time of each pattern when given a retention test the next day. The results from the acquisition phase showed that participants who learned patterns under fixed-order practice outperformed those who learned patterns under random-order practice. However, the results from the retention phase showed the opposite pattern: Those who had learned with random-order practice performed better than those who had learned with fixed-order practice. With regard to participants’ JOLs, those who learned under fixed-order practice showed an illusion of competence, in that they believed that their future performance would be better than it actually was. Thus, the type of practice schedule used during acquisition had a sizable effect on future retention performance, and JOLs were poor predictors of later performance.
Contextual-interference effects have been observed even with real-world scenarios, such as with batting by baseball players (Hall, Domingues, & Cavazos, 1994) and knot-tying by fire-fighters (Ollis et al., 2005). In one previous study, fixed- versus random-order practice was manipulated with novice musicians (Stambaugh, 2011). Replicating the typical behavioral pattern from the contextual-interference literature, fixed-order practice was found to lead to superior acquisition but inferior retention of clarinet pitch exercises. However, the influence of contextual-interference on musicians’ metacognition has yet to be examined. Revealing that musicians may be susceptible to the illusion of competence produced by fixed-order practice could have serious implications for music pedagogy. For example, misconceptions at learning may alter a musician’s control over his or her own self-regulated practice, leading the musician to spuriously endorse an inferior practice routine (fixed-order).
Perhaps more importantly, previous research has shown an illusion of competence in between-subjects designs (Simon & Bjork, 2001, 2002), but such designs might not be the most challenging ones for illusions. Different cues are made accessible in between-subjects and in within-subjects experimental designs (Kahneman & Tversky, 1996). In particular, within-subjects designs measure joint evaluations, thereby making the differences among conditions more salient (Kahneman, 2002). For this reason, cognitive illusions sometimes disappear when between-subjects tasks are extended to within-subjects designs (Kahneman, 2002; Kahneman & Tversky, 1996; LeBoeuf & Shafir, 2003). In other words, within-subjects designs may provide more stringent tests of the robustness of an illusion.
To the best of our knowledge, the illusion of competence observed in contextual-interference experiments has yet to be extended to a within-subjects design. On the one hand, within-subjects designs should facilitate participants’ ability to directly compare fixed- and random-order practice, perhaps eliminating the illusion of competence. However, such a direct comparison may provide little help if the cues used during this comparison are systematically biased against accurate prediction of retention. If participants are using the ease of their acquisition performance as a heuristic for predicting retention, this heuristic would still suggest to the participant that fixed-order is better than random-order practice for later retention. That is, even when participants are able to directly compare the conditions, acquisition performance may continue to thwart cognizant predictions of future performance.
Finding that the illusion of competence remains under the stringent test of a within-subjects setting not only would reveal the robustness of the illusion, but also would have implications for music pedagogy. Case studies of musicians’ self-regulated practice have revealed that, when left to their own devices, musicians often experiment with different practice schedules (Chaffin & Logan, 2006; Hallam, 2001; Nielsen, 1999; Noice, Jeffrey, Noice, & Chaffin, 2008). Therefore, testing the influences of fixed- and random-order practice on metacognitive judgments in a within-subjects setting would be more analogous to the real-world practice routines often adopted by musicians.
In the present report, our participants practiced melodies under both fixed- and random-order practice conditions. Throughout this acquisition phase, they made predictions about their future performance after a 2-day delay. After the 2-day delay, participants returned and made predictions again about their expected performance, and then completed a final retention test in which they performed the previously studied melodies under both fixed- and random-order trial presentations. Additionally, at the end of each day, participants were asked to report their preference for fixed- or random-order practice on a Likert-type scale. We hypothesized that, of the two practice types, fixed-order practice would lead to quicker (i.e., better) performance during acquisition, but slower performance after the 2-day delay. Additionally, it was hypothesized that participants would incorrectly predict that fixed-order practice would lead to better (faster) retention, and likewise that they would prefer fixed-order practice.
The participants were 25 individuals (6 male, 19 female) recruited from both the campus and the surrounding community of Charleston, South Carolina. The recruitment techniques included newspaper advertisements (both print and electronic), classroom announcements, and fliers. The participants were 18–49 years of age (M = 25.8, SD = 10.1), and all had 1–17 years of formal piano training (M = 8.0, SD = 3.9). Most of the participants (80 %) also had formal training on an additional instrument or on voice (M = 2.6 years, SD = 2.7). Their total formal music training (all instruments combined) ranged from 3.5 to 18 years (M = 10.6 years, SD = 3.9). Of those who had training on one or more additional instruments, 35 % listed wind instruments, 25 % voice, 15 % guitar, 15 % viola/violin, 15 % organ, and 5 % percussion. Also, 10 % of those participants reported having training on two additional instruments. Likewise, 24 % of the participants reported being professional musicians, but the professionals did not have significantly more years of training than did the nonprofessionals, for either piano or all instruments combined, ps > .10.
All testing occurred within two sessions separated by a 2-day delay, with the first session lasting 1 h and the second session lasting 30 min. Participants were tested individually in a private room with an experimenter and were compensated for their time at a rate of $10 per session.
Materials and design
The experimental procedures were administered using a Yamaha P-200 digital piano connected via MIDI to a desktop computer. A customized software program was used to administer the experimental trials and to collect participants’ responses for later analysis. All of the participants were given headphones to wear with volume that they could adjust to a comfortable level. A total of 12 melodies were taken from an obscure source (Ottman, 1996) and edited such that each note had the same duration (i.e., quarter notes) and could be executed for ease of right-hand performance. The melodies’ key signatures had no more than two sharps or two flats. Each to-be-learned melody was 12 notes in duration, and each had a unique goal movement time that was set in such a way as to motivate fast performance, but participants were encouraged to surpass a melody’s goal movement time, if possible.
A within-subjects design was adopted, with four melodies being practiced in fixed-order (e.g., melodies A, A, A, B, B, B) and four in random-order (e.g., melodies C, D, C, D, D, C). Eight critical melodies were used throughout the acquisition phase (four under fixed-order, four under random-order), and one melody was used for the practice phase (before the acquisition phase). Additionally, two buffer melodies were presented at the beginning and one at the end of the acquisition phase (Day 1), in order to prevent or reduce primacy and recency effects. The same eight critical melodies used for the acquisition phase were used in the retention phase (48 h later). The retention phase began with the practice melody and two buffer melodies, and concluded with two buffer melodies. Presentation of the critical melodies was divided into four blocks, and each block contained two melodies presented in either fixed- or random-order. The presentation order of the melodies was fixed, and the assignment of melodies to study condition was counterbalanced across participants.
The main dependent measure was participants’ performance times on trials in which all notes were played in the right order. Trials during which an incorrect note was played were rare (M = 6.3 % of critical trials) and were excluded from the analyses. Participants’ time-based JOLs were collected after every fifth trial of each of the melodies, whereas participants’ preference- and belief-based JOLs were collected at the endings of both days.
In addition to manipulating fixed- versus random-order practice during acquisition, we also manipulated practice order during retention, with half of the melodies being assigned to fixed and half to random-order during retention. The assignment of melodies to retention conditions was also counterbalanced across participants. This retention manipulation was included so as to consider the possibility that the effect of acquisition practice order would depend on retention practice order (i.e., the possibility that practice order benefits would be transfer appropriate). However, consistent with previous research (Simon & Bjork, 2001, 2002), practice order during retention did not qualitatively alter the pattern of results (i.e., retention practice order did not significantly interact with acquisition practice order; F < 1, p = .48). All of the other analyses were collapsed across this factor.
At the start of the acquisition phase, participants were verbally instructed to play melodies as quickly as possible while maintaining 100 % accuracy. Before each trial, the goal movement time for the particular melody (e.g., 3.4 s) appeared on the computer screen above the sheet music of the to-be-learned melody. The sheet music and the goal movement time remained on the screen until the participant had completed that trial according to the relative practice schedule. Additionally, before each of the trials, participants were reminded that they should prioritize accuracy, completing the melody as accurately as possible, even if that required playing at a slower speed in the beginning. Participants were also reminded that once their accuracy was stable, they should attempt to increase their speed to get closer to, or even surpass, the goal movement time. After each trial, feedback was displayed that provided three pieces of information: whether or not the keys were pressed correctly (i.e., correct notes were displayed in green and incorrect notes were displayed in red), the goal movement time, and the time that the participant had taken. If a melody was executed with 100 % accuracy, the feedback screen instructed participants to try to increase their performance speed.
After five executions of each melody, participants were asked to make a prediction as to how fast they believed that they would be able to execute that melody 2 days later, assuming that they had no more practice in the meantime. At the end of the acquisition phase, participants were asked, “If you had to learn new melodies in the future, which practice schedule would you prefer to use: fixed- or random-order practice?” They responded to this question using a Likert-type scale (1 = strongly prefer mixed, 2 = somewhat prefer mixed, 3 = no preference, 4 = somewhat prefer unmixed, and 5 = strongly prefer unmixed). Additionally, participants were provided with examples of both practice orders for clarification. Once their answers were completed, participants were instructed to return after 2 days at the same time that they had originally arrived to complete the retention phase of the experiment, and they were advised not to practice any of the melody patterns that they had been exposed to.
Retention phase (2 days later)
“If you had to learn melodies in the future, which practice schedule would you prefer to use: fixed- or random-order practice?”
“Which melodies do you believe you performed best on today, those that you practiced on Day 1 in a random-order or those that you practiced on Day 1 in fixed-order?” and
“Why do you believe this?”
The first two questions were based on a 5-point Likert-type scale (see the Acquisition Phase section above). Question 3 was open-ended and intended for exploratory purposes. Questions 2 and 3 were added after data collection began, so results for these questions were missing for three participants.
When sphericity was significantly violated—as shown by Mauchly’s test—the Greenhouse–Geisser-corrected statistics are reported.
Retention phase (2 days later)
Melodies originally practiced in random-order were performed significantly faster than melodies originally practiced in fixed-order (see Fig. 1a), as shown by a paired-samples t test, t(24) = 3.03, p < .01. In order to compare retention phase to acquisition phase performance, participants’ retention performances were then subjected to a 2 × 2 (Practice Type × Day) within-subjects ANOVA. Melodies practiced in fixed-order were significantly faster than those practiced in random-order on acquisition day (collapsing across all blocks); however, on the retention day, melodies originally practiced in random-order outperformed the melodies practiced in fixed-order, as revealed by a significant crossover interaction effect, F(1, 24) = 17.72, p < .01, η p 2 = .43. Moreover, that interaction was still significant even when retention day performance was compared only to the final block of acquisition day, F(1, 24) = 17.75, p < .01, η p 2 = .43.
As is shown in Fig. 1, participants’ JOLs during acquisition revealed a pattern strikingly similar to their acquisition performance speeds. That is, predicted performance times for melodies practiced in fixed-order showed a pattern similar to participants’ performance times on acquisition day, in that those melodies were predicted to be performed faster after a 2-day delay, as revealed by a 2 × 6 (Practice Type × Trial Block) within-subjects ANOVA, F(1, 24) = 6.72, p < .05, η p 2 = .22. As is shown in Fig. 1b, the predictions for future performance times for melodies practiced in fixed-order decreased as participants progressed through the trial blocks (similar to their performance), which was indicated by a significant main effect of practice type, F(1.86, 44.59) = 132.93, p < .01, η p 2 = .85. The Practice Type × Trial Block interaction was not significant, F(1.89, 14.03) = 1.20, p > .10.
Before retention phase
Participants’ JOLs made at the beginning of the retention phase indicated no significant preference for practice type, as shown by a paired-samples t test, p > .50. However, this finding may have been due to participants’ inability to recall their previous performance times for each melody after the 2-day delay. A 2 × 2 (Practice Type × Day) within-subjects ANOVA revealed a significant interaction effect, F(1, 24) = 6.11, p < .05, η p 2 = .20, showing that the pattern of JOLs for the fixed versus random conditions changed from acquisition day to retention day. Moreover, that interaction remained significant even when comparing the JOLs for the final trial block of acquisition day to the beginning-of-retention-day JOLs, F(1, 24) = 5.09, p < .05, η p 2 = .18.
Preferences and beliefs
Participants significantly preferred fixed-order practice, and nonsignificant trends in the same direction also emerged regarding participants’ beliefs. Recall that participants responded on Likert scales ranging from 1 (strong preference/belief favoring random order) to 5 (strong preference/belief favoring fixed order). At the end of acquisition day, participants on average preferred fixed-order practice (M = 3.7, SD = 1.4). A one-sample t test showed that the mean rating was significantly higher than the neutral rating of 3, t(24) = 2.37, p < .05. Likewise, at the end of retention day, participants still significantly preferred fixed-order practice (M = 3.6, SD = 1.1), t(24) = 2.50, p < .05. At the end of retention day, we also found a nonsignificant trend, such that participants believed that fixed-order practice was superior (M = 3.5, SD = 1.2), t(21) = 1.74, p < .10. In order to confirm these results without assuming an interval response scale, one-sample Wilcoxon signed rank tests were also used, testing whether medians (rather than means) were significantly different from the neutral value of 3. As was shown by the signed rank tests, the medians were significantly higher than 3 for preference judgments at the ends of both the acquisition day, p < .05, and the retention day, p < .05. At the end of retention day, a nonsignificant trend also indicated that participants believed in the superiority of fixed-order practice, p < .09. The medians for all three questions were 4.0, consistent with a “slight” preference or belief in favor of fixed-order practice. Overall, participants’ preferences and beliefs were broadly consistent with their mistaken predictions made during the acquisition phase.
Effects of formal training
Not surprisingly, more training was associated with shorter performance times (cf. Figs. 2a vs. b), and likewise, JOLs that predicted shorter performance times on the retention test (Figs. 2c vs. d). In a 2 × 2 (Practice Type × Day) ANCOVA on performance time and years of formal training showed a significant effect, F(1, 23) = 9.22, p < .01, η p 2 = .29. A 2 × 2 ANCOVA on JOLs also revealed a significant effect of years of formal training, F(1, 23) = 4.85, p < .05, η p 2 = .17. Thus, more highly trained musicians played faster and predicted that they would play faster.
Interactions between training and performance time
Of more importance, though, was whether training qualitatively altered the pattern of results—that is, whether training interacted with the other variables. For acquisition performance times (Fig. 2a and b), a 2 × 6 (Practice Type × Trial Block) ANCOVA showed a significant interaction between training and trial block, suggesting that training affected the overall rate of learning during acquisition, F(1.8, 41.0) = 3.74, p < .05, η p 2 = .14. Examining performance times across the last block of acquisition day and the retention day, a 2 × 2 (Practice Type × Day) ANCOVA showed a significant interaction between training and day, consistent with more highly trained individuals forgetting less (i.e., slowing down less) between the acquisition and retention days, F(1, 23) = 4.33, p < .05, η p 2 = .16. All other interaction effects for performance times were not significant, ps > .05.
Interactions between training and predicted retention performance times
In terms of time-based JOLs (cf. Fig. 2c and d), interaction effects with training were apparent only on the acquisition day. A 2 × 6 (Practice Type × Trial Block) ANCOVA on JOLs during the acquisition day showed a significant interaction between training and trial block, F(1.5, 34.2) = 7.93, p < .01, η p 2 = .26. This significant interaction is analogous to the one observed for retention performance, and it suggests that training affected the overall rate of learning, which in turn affected JOLs. Additionally, we found a three-way interaction between training, trial block, and practice type, F(2.2, 50.5) = 6.37, p < .01, η p 2 = .22. This three-way interaction is consistent with the pattern that low-training individuals had larger JOL differences between the fixed- and random-orders for earlier than for later blocks, but that such differences were very small in all blocks for high-training individuals (see Fig. 2c and d). All other interaction effects for JOLs were not significant, ps > .05.
Relationship of training to preferences and beliefs
On Likert-scale responses about preferences and beliefs, there was some evidence that more highly trained individuals showed less preference for fixed-order practice, particularly at the end of the acquisition day. All of the Pearson correlations between training and preferences/beliefs were negative, indicating that training might sway participants away from fixed-order practice, but none of these correlations reached significance, ps > .05. Specifically, training was negatively correlated with preference ratings on acquisition day (r = –.35) and retention day (–.14), and with the belief rating on retention day (–.15). These correlations may have been underpowered, due to the large number of tied data points, as Likert responses could only take on five possible values. To address this issue, correlations were also analyzed with Kendall’s tau-b measure. These analyses showed a significant correlation between training and acquisition day preference ratings, τ b = –.41, p < .05, but not between training and retention day preference ratings, τ b = –.15, p > .10, or beliefs, τ b = –.15, p > .10. In other words, some evidence emerged that training was associated with less of a preference for fixed-order practice, but only on the acquisition day. However, even among the high-training group, with more than 9 years of formal training, the mean Likert-scale responses never fell below the 3.0 midpoint of the scale (Ms = 3.0, 3.2, and 3.1 for acquisition day preferences, retention day preferences, and retention day beliefs, respectively). Hence, training was associated with less preference for fixed-order practice, but the high-training group was, at best, agnostic about the benefits of random-order practice.
Consistent with previous research (Stambaugh, 2011), we replicated the contextual-interference principle in the learning of a musical instrument; fixed-order practice fostered faster acquisition performance, whereas random-order practice resulted in faster retention performance. This finding suggests that random-order practice is more advantageous for long-term retention of piano melodies.
More importantly, participants showed the illusion of competence engendered by fixed-order practice in a within-subjects setting. To our knowledge, this is a novel finding. Participants’ performance times during acquisition were aligned with their metacognitive judgments, suggesting that, even when they could easily compare fixed- and random-order practice conditions, participants still erroneously espoused the short-term benefits of fixed-order practice (cf. Simon & Bjork, 2001, 2002). This pattern was also visible in participants’ preference-based judgments, which showed that, on both days, the majority of participants still favored fixed-order over random-order practice for future training.
Why did the illusion of competence prevail in a within-subjects setting? After all, within-subjects designs often increase the use of controlled processes by allowing for comparisons among conditions (Kahneman, 2002). Additionally, our participants were provided with feedback, which should also have increased the use of cognitive control (Kahneman & Klein, 2009). Increased cognitive control should alleviate some illusions by reducing the need to rely on heuristics, including retrieval fluency (Jones & Jacoby, 2001; see also Jacoby, Bishara, Hessels, & Hughes, 2007; Jacoby, Hessels, & Bopp, 2001). On the other hand, even though cognitive control may allow for easier comparison across conditions, in our scenario, participants may have been comparing conditions on the basis of immediate performance speed. Facilitating this comparison would, if anything, contribute to illusions about retention, because immediate performance poorly predicted retention performance (cf. Koriat & Bjork, 2005).
In order to avoid the illusion of competence, a learner might need to compare retention performances, and particularly, retention performances as a function of acquisition type. However, this information may prove difficult for the learner to extract. The learner not only would need to notice retention performance differences, but would also need to remember how retention performance mapped to acquisition type, sometimes from an acquisition that had occurred several days earlier. To add to the conundrum, different stimuli (in this case, musical passages) often have different inherent levels of difficulty, thus making it even more challenging for learners to correctly attribute performance differences to practice orders. Indeed, the benefits of random- over fixed-order practice may be challenging to notice from casual, nonscientific experimentation with practice orders, and that may be the reason why even experts sometimes rely on fixed-order practice (Maynard, 2006).
Another challenge that learners might experience is that it might seem counterintuitive to endorse the practice order associated with difficult acquisition and to ignore the ease of acquisition associated with fixed-order practice. This would suggest that participants may have been using acquisition performance as a heuristic for predicting both retention performance and preference-based JOLs. However, if participants were using retrieval fluency at learning to formulate their metacognitions, then why did participants still prefer fixed-order practice for future training at the end of the retention day, thereby disregarding the subpar retention performance for melodies practiced in fixed-order? There are at least two possible explanations for this pattern. First, participants could have been motivated to maintain consistency in their judgments (Abelson et al., 1968), and this may have led them to persist with their preferences made at the end of the first day. Second, after the 2-day delay, participants may have been unable to recall which melodies had appeared in the fixed- or random-order practice conditions during acquisition, which would deter the participants from comparing acquisition practice schedules to their resulting retention performance speeds. The latter possibility would explain why participants’ time-based JOLs made at the beginning of the retention day did not suggest a practice type preference. However, further research will be required to disentangle these possibilities.
In the present experiment, metacognition was assessed at both the item level (e.g., predictions of retention performance for each melody) and the global level (e.g., preference for fixed- or random-order practice in general). Item- and global-level assessments do not always converge; even when item-level assessments poorly reflect actual performance, global assessments are sometimes more accurate (e.g., Dunlosky & Hertzog, 2000; Koriat, Bjork, Sheffer, & Bar, 2004; Wahlheim, Finn, & Jacoby, 2012). In contrast, in the present experiment, both item- and global-level assessments suggested poor awareness of the benefits of random-order practice. In particular, both the predicted retention performance times and the more global preference ratings favored fixed-order. The convergence among the item-level and global-level measures underscores the robustness of the illusion of competence in trained musicians.
Of course, “trained” is a relative word, and some of the musicians had more training than others. Musicians with more training showed, if anything, smaller differences between fixed- and random-order practice (see Fig. 2), but such interaction effects often failed to reach significance. The sample may have been too small to examine the impact of a between-subjects variable (years of formal piano training) with adequate power. Additionally, it is unclear whether any such reduction in the difference between fixed- and random-order practice would reflect better metacognition or simply an approach toward asymptotic performance. That is, as performance times were smaller, differences between the two conditions were also smaller, for both high- and low-training individuals. Importantly, even individuals with more than 9 years of formal training were agnostic about the value of random-order practice. Overall, the pattern of results suggests that training might have reduced the illusion of competence, but it did not produce a clear preference for random- over fixed-order practice.
One limitation of the present experiment was that the melodic stimuli were artificially simple. For example, our stimuli comprised only quarter notes, but real melodies often contain a variety of note durations and rhythms. Previous research with other motor skills has suggested that contextual-interference’s behavioral effects remain, even with more complex stimuli (Ollis et al., 2005), but to the best of our knowledge, no empirical work has investigated the interaction between contextual-interference and task complexity on metacognition.
Another limitation is the imbalance between conditions regarding elapsed times between item-level JOLs made during acquisition. Recall that the number of stimulus repetitions between JOL assessments was held constant, with JOL assessments occurring on every fifth presentation of a particular melody. However, equating melody repetitions between JOLs necessarily led to different amounts of time elapsing between JOLs for the fixed- versus random-order practice conditions. It should be noted that this confound between inter-JOL time and practice schedule is common in experiments of this nature (e.g., Simon & Bjork, 2001, 2002), perhaps because equating inter-JOL time would require confounding practice type with arguably more critical variables, such as the number of stimulus repetitions between JOLs. In any event, one possible concern with this procedure is that more time elapses between JOL assessments of the same stimulus in the random-order condition, which could cause participants to be less certain about their rate of learning across trials. While possible, this explanation of the present JOL results seems unlikely. In the fixed-order practice condition, JOL assessments occurred after five consecutive trials, which should, if anything, provide a more accurate gauge for rates of learning, yet this condition led to less accurate predictions for future performance. Additionally, participants were instructed to estimate their future performance assuming that they would have no additional practice, so the rates of learning across trials should be a less relevant heuristic than was performance on the most recent trial.
The findings from the present report have several implications. Most importantly, they reveal that the illusion of competence is robust enough to prevail in a within-subjects design. Second, they suggest that fixed-order practice can have deleterious effects on both the long-term retention and metacognition of trained musicians. Finally, our results advise caution to musicians and other experts engaging in self-regulated practice. Even when such individuals casually experiment with both fixed- and random-order practice, their metacognitive judgments may still be contaminated by the illusion of competence engendered by fixed-order practice, thus encouraging the use of inferior learning strategies.
B.A. is now at the Department of Psychology, University of North Carolina at Greensboro. This project was completed as part of the requirements for the first author’s Bachelor’s Essay at the College of Charleston. This project was supported by a College of Charleston SURF grant. We thank Major Murphy for developing the software used in this project.
- Abelson, R. P., Aronson, E., McGuire, W. J., Newcomb, T. M., Rosenberg, M. J., & Tannenbaum, P. (Eds.). (1968). Theories of cognitive consistency: A sourcebook. Chicago: Rand-McNally.Google Scholar
- Battig, W. F. (1979). The flexibility of human memory. In L. S. Cermak & F. I. M. Craik (Eds.), Levels of processing in human memory (pp. 23–44). Hillsdale, NJ: Erlbaum.Google Scholar
- Bjork, R. A. (1999). Assessing our own competence: Heuristics and illusions. In D. Gopher & A. Koriat (Eds.), Attention and performance XVII: Cognitive regulation of performance. Interaction of theory and application (pp. 435–459). Cambridge, MA: MIT Press.Google Scholar
- Jacoby, L. L., Bjork, R. A., & Kelley, C. M. (1994). Illusions of comprehension, competence, and remembering. In D. Druckman & R. A. Bjork (Eds.), Learning, remembering, believing: Enhancing human performance (pp. 57–80). Washington, DC: National Academy Press.Google Scholar
- Jacoby, L. L., Hessels, S., & Bopp, K. (2001). Proactive and retroactive effects in memory performance: Dissociating recollection and accessibility bias. In H. Roediger, J. S. Nairne, I. Neath, & A. M. Surprenant (Eds.), The nature of remembering: Essays in honor of Robert G. Crowder (pp. 35–54). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
- Kahneman, D. (2002, December 8). Maps of bounded rationality: A perspective on intuitive judgment and choice. Nobel Prize Lecture, Stockholm.Google Scholar
- Lee, T. D., & Simon, D. A. (2004). Contextual interference. In A. M. Williams & N. J. Hodges (Eds.), Skill acquisition in sport: Research, theory and practice (pp. 29–44). London, UK: Routledge.Google Scholar
- Magill, R. A., & Hall, K. G. (1990). A review of the contextual interference in motor-skill acquisition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9, 730–746.Google Scholar
- Maynard, L. (2006). The role of repetition in the practice sessions of artist-teachers and their students. Bulletin of the Council for Research in Music Education, 167, 61–72.Google Scholar
- Ottman, R. W. (1996). Music for sight singing (4th ed.). Englewood Cliffs, NJ: Prentice-Hall.Google Scholar