A comparison of serial order short-term memory effects across verbal and musical domains

In order to explore hallmark serial order STM phenomena in verbal and musical domains, we used serial order reconstruction tasks for auditorily presented sequences of verbal and musical material. Serial order reconstruction tasks allow maximizing the demands on serial order processing as item information is fully available during the reconstruction phase (see Majerus, Poncelet, Elsen, & van der Linden, 2006; Majerus, Poncelet, van der Linden, & Weekes, 2008). Importantly, serial order reconstruction tasks further allow the assessment of musical and verbal STM abilities in a way that is not biased by output difficulties. Adult participants with no advanced musical expertise will be experts in outputting verbal stimuli, but they will be much less familiar with outputting musical stimuli. But even in musical experts, ease of musical output will depend on the type of the required musical response: professional singers will produce vocal musical responses with ease and high accuracy, but this will not necessarily be the case for professional pianists or percussionists. Therefore, the use of serial order reconstruction tasks—for which items are available at recall and only their order needs to be reconstructed—allows minimizing the impact of differential expertise levels for outputting verbal and musical responses on task performance. Also, in order to ensure comparable performance levels across the verbal and musical serial order reconstruction tasks, shorter lists were used for the musical than for the verbal task.

Finally, participants with no advanced musical expertise and experienced musicians were recruited for this experiment. This allowed us to determine whether any possible differences in serial order STM phenomena between verbal and musical STM tasks reflect fundamental serial order processing differences in the two domains, or whether they are the result of differences in expertise related to processing musical information. We furthermore expected that the experienced musicians group would show superior performance in the musical STM tasks as compared to the participants with no advanced musical expertise, in line with previous studies (Pechmann & Mohr, 1992; Schulze, Dowling, & Tillmann, 2012; Williamson, Baddeley, & Hitch, 2010).

This study aimed at investigating hallmark serial order effects across verbal and musical domains of STM. We observed similar serial order phenomena in musical and verbal short-term serial order reconstruction tasks, as highlighted by similar transposition gradients, fill-in:infill errors ratios, and sequence length effects. Recency and primacy effects also characterized recall performance in both tasks, but for musical STM tasks these effects only appeared for the longest list lengths and were less marked. Finally, in addition to outperforming participants with no advanced musical expertise on the musical STM task, musicians showed a specific pattern of serial recall errors in the musical task. Indeed, relative to the group with no advanced musical practice, musicians showed an increase of Distance-2 transposition errors and a similar amount of fill-in and infill errors.

We observed similar sequence length effects in the two STM domains, with performance decreasing as a function of increasing sequence length, in line with previous studies having shown robust list length effects across different STM domains (e.g., Anderson, Bothell, Lebiere, & Matessa, 1998; Avons, 1998; Jones et al., 1995; Maybery, Parmentier, & Jones, 2002; Smyth et al., 2005; Williamson et al., 2010). As regards serial position effects, we observed in the verbal task a classical U-shaped serial curve accompanied by both primacy and recency effects, and this for the four different list lengths that were explored, in accordance with many other studies from the verbal, musical and visuospatial domains (Avons, 1998; Cowan et al., 2002; Greene & Samuel, 1986; Guérard & Tremblay, 2008; Mondor & Morin, 2004; Oberauer, 2003; Smyth et al., 2005; Tremblay, Parmentier, Guérard, Nicholls, & Jones, 2006). For the musical task, we also observed primacy and recency effects, but only for the longest list lengths, in both musicians and participants with no advanced musical expertise. These effects, when present, were less marked as compared to the verbal task.

For the smallest musical STM lists—that is, three-tone and four-tone sequences—serial response curves were flat. This result is, however, noninformative given that it is very likely to be caused by the small number of positions used (see also Smyth et al., 2005, for similar results). The absence in the musical task of a primacy effect for List Length 5, and the relatively weaker primacy effect for List Length 6 as well as the weaker recency effects for List Lengths 5 and 6, deserves some more consideration. A possible explanation for these relatively flat serial position curves is that participants may have relied on configurational melodic processing instead of strict serial processing of the memory items (e.g., contour information reflects the overall pattern of up and down tone transitions in the sequence and can be represented in a gestalt-like manner; see Dowling, 1991; Gorin et al., 2016). In the musical literature, it is known that contour information plays an important role in short-term recognition of melodic excerpts (Dowling, 1978; Dowling & Tillmann, 2014). For example, consider the tone sequence C₂–E₂–D₂–F₂, which is characterized by a melodic contour of two rise–fall alternations (resembling, in spatial terms, to a landscape with two horizontally aligned mountains in the background). If for the maintenance of this sequence a melodic contour representation is used, then errors are more likely to involve the exchange of C₂ and D₂, or E₂ and F₂, leading to a preservation of the rise–fall pattern of the melodic contour. This view is clearly supported by the presence in four-tone sequences of more transpositions for Distance 2 than for Distance 1 and Distance 3 (see Fig. 7a). At the same time, in five-tone sequences, the proportion of transpositions decreased linearly as a function of distance of displacement (see Fig. 7b), which suggests that this type of configurational representation was not used anymore for longer musical sequences.⁷

Open image in new window

This potential shift from a gestalt-like contour representation to a more serial representation is in line with other studies in the musical STM domain. Edworthy (1985) showed that participants are better at recognizing contour information in short sequences and better at discriminating more fine-grained pitch-interval information in longer sequences. In order to check for the intervention of these possible strategies, we reanalyzed our data by scoring the responses based on contour recall, by considering two adjacent tones as correct if their interval direction (up or down) conformed to the contour of the sequence. This score was then corrected for chance-level performance and compared to the strict serial order score (also after chance-level correction as reported in the Results section), by collapsing three-tone and four-tone lists, on the one hand, and five-tone and six-tone lists, on the other hand, in order to maximize the reliability of these analyses. A mixed Bayesian repeated-measures ANOVA (2 groups × 2 scores × 2 list length) showed strong evidence for a Score × Length interaction (BF_Inclusion = 13.16), suggesting that contour and serial position scoring methods led to opposing results as a function of sequence length. Furthermore, there was some evidence for a three-way interaction (BF_Inclusion = 2.29), indicating that the Score × Length interaction was further modulated by group. As shown in Fig. 8a, the musician group showed better recall performance when the serial position scoring method was applied, and this was true for all sequence lengths. For participants with no advanced musical expertise, the contour recall score captured performance to the same extent or even better than the serial recall score for the short list lengths, while the serial recall score led to higher performance for the longer list lengths, in line with a gradual shift from contour-based to serial position-based representations.

Open image in new window

Another explanation for the presence of weak primacy and recency effects in five-tone and six-tone lists could be related to the use of faster presentation rates for the musical stimuli as compared to the verbal stimuli. A fast presentation rate had been chosen for the musical stimuli as our initial aim was to ensure that they were perceived as melodies rather than as series of isolated tones and that they were not verbally recoded. At the same time, it has been shown in the verbal domain that presentation rates can influence the shape of the serial position curve (see, e.g., Posner, 1964; Tan & Ward, 2008). Fast presentation rates of verbal stimuli have been shown to lead to reduced primacy effects (Bhatarah, Ward, Smith, & Hayes, 2009). Conversely, more pronounced primacy and recency effects have been shown in musical STM tasks requiring recall of tone sequences presented at slower rates (i.e., one tone per second; Greene & Samuel, 1986). At the same time, another factor known to influence the shape of the serial position curve is list length. Indeed, previous studies showed that when using a reconstruction method for short lists of verbal and visual memoranda, recency and primacy effects can be reduced, leading to a flattened shape of the serial response curve (Smyth et al., 2005; Ward, Tan, & Grenfell-Essam, 2010). Also, studies on free recall of spatial locations argued that the tendency to serially recall items according to their initial list position could be language specific (see, e.g., Gmeindl, Walsh, & Courtney, 2011). However, other recent studies showed that serial recall strategies can be observed in free recall tasks for nonverbal memoranda such as spatial or facial locations (Cortis, Dent, Kennett, & Ward, 2015). Future studies using similar presentation rates and list lengths when comparing performance in musical and verbal STM tasks are required to determine whether the differences in serial position curves observed in this study for verbal and musical STM tasks are due to differences in task parameters or to the existence of specific serial order coding processes in the musical domain (such as the use of contour-based configurational processes).

We also observed that, as compared to participants with no advanced musical expertise, musicians produced specific patterns of errors in the musical serial order reconstruction STM task, such as specific transposition gradients and fill-in:infill ratios. This suggests that musicians may use additional domain-specific strategies for processing musical information (see also Williamson et al., 2010). One critical variable for efficiently processing musical stimuli is meter. Meter refers to the generation of a metrical hierarchy taking the form of a recurrent alternation between strong and weak beats occurring at different periodicities (Lerdahl & Jackendoff, 1983). Musicians possess elaborate metrical mental representations they will use to structure novel musical sequences. The use of metrical representations will allow the chunking of musical elements, with an effect similar to temporal grouping. This is likely to have an effect on STM performance and error types. Mathias et al. (2015) observed that pianists produced more transpositions between items having the same metrical role, particularly in musical contexts with a stronger metrical structure. In the present case, the use of an elementary, binary metrical structure (e.g., strong, weak alternation) could have influenced the pattern of errors by enhancing the amount of serial position exchanges between positions sharing the same metrical role (i.e., every n + 2 position). This could explain why musicians showed an increase of Distance 2 transposition errors and an altered fill-in:infill ratio, relative to participants with no advanced musical expertise.

At the same time, these specificities characterized mainly performance in musicians. For participants with no advanced musical expertise, serial order effects were very similar across verbal and musical modalities (except for the reduced primacy and recency effects in the musical task). Indeed, analysis of transposition gradients showed that transposition errors were governed by a locality principle (Henson, 1996) in both modalities; the same transposition gradients have also been observed for verbal and visuospatial STM tasks (Avons, 1998; Farrell & Lewandowsky, 2004; Hartley et al., 2016; Henson, 1998; Hurlstone & Hitch, 2015; Johnson, Shaw, & Miles, 2016; Parmentier, Andrés, et al., 2006; Parmentier et al., 2006; Smyth et al., 2005). Also, fill-in errors were overall 2 times more frequent than infill errors in verbal and musical modalities (Farrell et al., 2013; Henson, 1996; Surprenant et al., 2005), mirroring again results also observed in the visuospatial (Guérard & Tremblay, 2008) and tactile domains (Johnson et al., 2016).

What do these serial order effects tell us about the nature of ordering mechanisms in musical STM? The presence in participants with no advanced musical expertise of a very similar profile of serial order behaviors in musical and verbal STM strengthens the view that musical and verbal STM systems possibly rely on common sequential processes (Gorin et al., 2016; Williamson et al., 2010). These processes have been proposed to rely on domain-general serial order marking processes, such as context signals (see Hurlstone et al., 2014; Lewandowsky & Farrell, 2008). The presence in the musical task of sequence length effect, transposition gradients, and serial position effects fit with positional marking (e.g., Brown et al., 2000; Burgess & Hitch, 1999; Hartley et al., 2016) and primacy gradient (e.g., Farrell & Lewandowsky, 2002; Page & Norris, 1998) mechanisms as proposed by verbal STM models. Moreover, the presence of more fill-in than infill errors (see Henson, 1996; Surprenant et al., 2005) provides evidence for the intervention of a primacy gradient principle to represent serial order information in musical STM while arguing against a chaining account (e.g., Kieras et al., 1999; Lewandowsky & Murdock, 1989). However, even though we obtained in participants with no advanced musical expertise evidence in favor of similar ordering mechanisms in verbal and musical STM domains, we cannot rule out the possibility that distinct, modality-specific STM systems process serial order information using similar representational mechanisms (see Logie et al., 2016). At the same time, the results observed in this study can be accommodated by an approach considering STM as an emerging process resulting from the coactivation of domain-specific systems for storage of verbal and musical item information, and domain-general mechanisms involved in serial order coding and attentional control (see,e.g., Majerus, 2013; Majerus et al., 2016).

The effects observed here stress the need for musical STM models to take into consideration the problem of serial order. There is currently no model of musical STM/working memory addressing directly this aspect of musical sequence processing. In his model, Berz (1995) suggested to add to the Baddeley and Hitch (1974) multicomponent model of working memory a module specialized in the processing of musical stimuli. However, as in the original working memory model, the model of Berz (1995) lacks details concerning the representation and the processing of serial order information. More recently, Ockelford (2007) suggested the existence of a central executive component specialized in the processing of musical material that is connected to short-term and long-term musical memory stores. Moreover, Ockelford (2007) proposed that when experiencing music, the musical executive component is responsible for tagging musical elements as well as their relationships. This tagging is proposed to rely on item-by-item associative chaining mechanisms, following the proposal of Kieras et al. (1999). However, as we have seen, chaining mechanisms are not compatible with the observation of more fill-in than infill errors. Future models of musical STM will need to incorporate realistic mechanisms for explaining serial ordering phenomena in the musical domain, and, as shown by the present data, some of these mechanisms may prove to be the same as those used in the verbal STM domain, such as context signals based on primacy gradients and/or temporal/episodic signals (Brown et al., 2000; Burgess & Hitch, 1999; Hartley et al., 2016; Page & Norris, 1998). At the same time, musical STM models also need to account for the more specific processes that characterize musical serial order processing such as the possible influence of metrical knowledge and structure, as well the use of contour-based representations. Hence, in addition to domain-general position marking and primacy gradient mechanisms, models of musical STM also need to incorporate metrical and configurational levels of processing and their interaction with serial position coding.

The present study has also methodological implications. Our results indicate that the reconstruction of the serial order of tone sequences via a virtual keyboard is feasible by musically trained participants but also by participants with no advanced musical expertise. This new method thus offers the possibility to study STM for serial order in both verbal and musical modalities, more particularly in nonmusicians. Nonetheless, it is important to acknowledge that the musical reconstruction task that we used does not perfectly match the verbal reconstruction task used in this study. In the verbal task, the memoranda were printed on cards and were directly available during serial order reconstruction, and, hence, participants had permanent access to item identity. In the musical task, participants needed to reactivate each tone by a clicking response in order to hear its identity. These methodological discrepancies could partly explain the differences in the shapes of serial position curves we observed between the verbal and the musical tasks, as discussed above.

One manner to address this issue could be to use direct recall procedures for the tasks in both the verbal and musical domains. At the same time, previous studies have shown that imprecise singing is widespread in the general population, and even people considered as accurate singers do not imitate perfectly pitch target (see Pfordresher & Brown, 2007; Pfordresher, Brown, Meier, Belyk, & Liotti, 2010). Also, the accuracy of sung productions is assessed through relative measures of deviation, determining the extent to which a produced pitch or interval deviates from its target (see Berkowska & Dalla Bella, 2013); these measures are difficult to implement in tone serial recall tasks. The use of a threshold above which the deviation of a produced pitch is considered as incorrect could represent a possibility to circumvent this methodological challenge. However, another problem raised by this method concerns the difficulty of differentiating between item-based and order-based errors, as deviating tones could be both item and order errors, depending on the target item to which they are compared (i.e., a given tone could deviate by a semitone relative to the stimulus presented in the same position, leading to an item error, or by a semitone relative to a stimulus in another serial position, which would be considered as a serial order error). Another possibility could be to rely on identical probe recognition procedures for assessing verbal and musical STM. At the same time, recognition paradigms are not suited for studying serial order errors given that participants respond simply with yes–no responses. The serial order errors a participant could have made could be inferred by the presentation of nonmatching probe stimuli containing these serial order errors, but this approach would be very indirect and of suboptimal sensitivity and efficiency. Recognition STM paradigms are indeed better suited to study recognition accuracy (Henson, Hartley, Burgess, Hitch, & Flude, 2003; Jefferies, Frankish, & Lambon Ralph, 2006). The use of a reconstruction method provides a more direct approach for studying serial recall accuracy as well as ordering errors in the musical domain. Moreover, the present study indicated that this type of task is a valid tool for measuring musical STM for serial order in both musicians and participants with no advanced musical expertise, allowing for generalization of the results to the general population.

Finally, a further finding of this study is the musicians’ overall higher musical STM performance and, to some extent, their higher verbal STM performance. Concerning musical STM, our results are in line with previous studies reporting that musicians have better abilities to keep in memory pitch information (Berti, Münzer, Schröger, & Pechmann, 2006; Pechmann & Mohr, 1992) and to recall and recognize the serial order of tone sequences (Schulze et al., 2012; Williamson et al., 2010). Other studies observed that this advantage also extends to the verbal domain (Chan, Ho, & Cheung, 1998; Franklin et al., 2008; Hansen, Wallentin, & Vuust, 2013; Y. Lee, Lu, & Ko, 2007; e.g., Parbery-Clark, Skoe, Lam, & Kraus, 2009). The present study yielded some evidence for such an advantage, even though the advantage for STM performance in the verbal modality was less reliable than the advantage for STM performance in the musical modality. Moreover, it is important to note that we did not directly control for differences in attentional capacities between musicians and participants with no advanced musical expertise, although both groups were matched for nonverbal intellectual efficiency. A number of studies have shown that musicians not only have better STM abilities but also more developed auditory attentional capacities (see Besson, Chobert, & Marie, 2011; Strait, Kraus, Parbery-Clark, & Ashley, 2010). Attentional focalization is an important process involved in many different STM tasks, if not all (Barrouillet, Bernardin, & Camos, 2004; Cowan, 1995); and higher auditory attentional focalization abilities in musician participants could at least partially explain the small advantage observed for verbal STM performance.

Also, it could be argued that the distinction we made between the two groups—that is, musically trained participants versus participants with no advanced musical expertise—is somewhat simplistic. It has been shown that individuals considered as musicians can show surprisingly low levels of performance in various tests assessing musical abilities, while other individuals considered as nonmusicians can show high performance levels on the same tests (see Law & Zentner, 2012). Likewise, even though studies on musical aptitude have shown that musical test batteries can estimate a general form of musical aptitude in both musician and nonmusician participants, they also have highlighted the existence of varying patterns of musical sophistication, with different musical aspects (such as pitch-related vs. rhythm-related skills) being developed to different extents (see, e.g., Kunert, Willems, & Hagoort, 2016; Law & Zentner, 2012; Müllensiefen, Gingras, Musil, & Stewart, 2014). Therefore, it is important that future studies interested in the link between STM capacities and level of musical sophistication take into consideration not only the musician versus nonmusician distinction but also the extent to which participants are proficient in various musical skills, such as, for example, assessment with the short version of the Profile of Music Perception Skills battery (Law & Zentner, 2012). For instance, a close link between serial order STM and rhythm processing has been observed (Gorin et al., 2016; Plancher, Lévêque, Fanuel, Piquandet, & Tillmann, 2017; Saito, 2001). Given that temporal aspects such as rhythm play a key role in some models of verbal STM for serial order (see Brown et al., 2000; Burgess & Hitch, 1999; Hartley et al., 2016), it would be theoretically useful to investigate, using an individual differences approach, the relation between rhythmic aptitudes and STM for serial order.

To sum up, this study observed similar serial order phenomena for verbal and musical STM tasks in participants with no advanced musical expertise, suggesting that domain-general mechanisms support the representation of serial order information in STM. At the same time, this study also highlights domain-specific serial processes, possibly involving configurational and metrical processes, most clearly in participants with advanced musical experience. Future models of STM need to consider both domain-general and domain-specific serial order coding mechanisms.