Similarities and differences between language and music are a timely object of debate. A theoretical landmark in this research domain is the shared syntactic integration resource hypothesis (SSIRH) proposed by Patel (2003, 2008). On the basis of the observation that both language and music are composed of similar hierarchically ordered structures, the SSIRH posits that the successful processing of linguistic and musical materials relies, at least partially, on the mastery of a common syntactic processor. By contrast, the mental representations involved in language and music are different, a postulate that allows the SSIRH to be consistent with neuropsychological dissociations (e.g., Peretz et al., 1994).

In a recent article in this journal, Slevc, Rosenberg, and Patel (2009, Experiment 1) reported data supporting the SSIRH. Participants were submitted to a self-paced reading task in which they were asked to read sentences displayed on the screen. Each sentence was divided into a variable number of segments. After reading each segment, the participants had to press a key to trigger the appearance of the next segment until the end of the sentence. In addition, a musical chord was played in synchronization with the onset of each segment, hence generating a series of visual–auditory events.

Most of the visual–auditory series were correct, meaning that the written sentences were both grammatical and meaningful and the musical sequences were consistent with the rules of Western tonal music. In other words, all the events matched participants’ expectancies. However, unexpected events, which could occur within the sentences, the musical sequences, or concurrently in both linguistic and musical components, were introduced into the remaining series. Regarding first linguistic stimuli, events could be unexpected on either a semantic basis or a syntactic basis. In the example Slevc et al. (2009) gave of semantic manipulation (see Fig. 1), “The boss warned the mailman to watch for angry pigs,” “pigs” turns out to be unexpected in this context, the expected element being “dogs.” The manipulation of syntax involved garden paths. In the sentence “After the trial the attorney advised the defendant was likely to commit more crimes,” “was” is surprising because, presumably, the reader first understands that the attorney advised the defendant. To integrate “was” into a coherent structure, the reader needs to retrospectively process the sentence as if “that” was added after “advised.” Finally, the musical manipulation consisted in trading an unexpected chord (hereafter, out of key) for an expected chord (in key). Given that this unexpected chord generated a harmonic violation, the manipulation was construed as syntactic in nature. On the trials comprising both linguistic and musical unexpected events, these events occurred on the same segment, hence allowing examination of their joint influence on reading times (RTs).

Fig. 1
figure 1

Examples of the musical and linguistic stimuli used in both Slevc, Rosenberg, and Patel (2009) and the present experiment. The critical region is shaded in gray. Each segment of the precritical, critical, and postcritical regions (i.e., the segments on which reading times were analyzed) included one, two, or three words. This figure follows the general setup of Slevc et al.’s (2009) Fig. 1, except that a semantic garden path sentence has been added. Note that this sentence is only for the sake of illustration: “Bank” is not a homophone in French, and none of the homophones we actually used was also a homophone in English

Full size image

The predictions of the SSIRH are straightforward. If both language and music rely on a common syntactic processor, a disruption due to a syntactically unexpected linguistic segment should be especially severe when it is paired with a harmonic violation. Slevc et al. (2009) reported a complex interaction pattern that exactly matched this prediction. As is shown in the upper half of Fig. 2, the garden path effect (left-hand panel) was larger when measured in out-of-key conditions than when measured in in-key conditions, whereas the effect of semantic expectancy (right-hand panel) did not differ as a function of musical expectancy.

Fig. 2
figure 2

The difference in reading times (RTs) for linguistic expectancy (unexpected − expected) for both Slevc, Rosenberg, and Patel (2009; upper panels) and the present experiment (lower panels) as a function of linguistic manipulation (left panels: garden path; right panels: semantic violation) and musical expectancy (in key vs. out of key). Error bars indicate standard errors

Full size image

The support these data provide for SSIRH, however, depends on the endorsement of a premise—namely, that the differential effects of semantic violations and garden path sentences are exclusively linked to their semantic and syntactic dimensions, respectively. Now the two types of linguistic manipulations, such as those implemented in Slevc et al.’s (2009) study, differ along multiple features. For instance, a semantic violation elicits a disruption in the processing of the sentence that cannot be repaired irrespective of the listener’s effort to solve the incongruence, while a garden path sentence is open to reanalysis and reinterpretation.

The crucial point is that these features are in no way inherent in the engagement of semantic or syntactic operations. For instance, even though most of the literature on garden path sentences has dealt with syntactic garden paths, a garden path effect may stem from a semantic ambiguity as well. Let us consider the instance of semantic garden path given in Fig. 1. In the sentence “The old man went to the bank to withdraw his net,” “net” is surprising because, presumably, the reader first coded “bank” as a function of its dominant meaning of financial establishment, thereby expecting “money.” To integrate “net” into a coherent representation, the reader needs to revisit the meaning of the homophone “bank” as the side of a river. Both syntactic and semantic garden path sentences are open to reanalysis and reinterpretation and should lead ultimately to successful integration of the different components into a meaningful entity, with the crucial difference that the former involves syntactic operations, while the latter involves semantic operations.

What would happen if semantic garden paths were used instead of syntactic garden paths in Slevc et al.’s (2009) procedure? The authors themselves raised this issue in a proceedings paper reporting the same experiments as those in their subsequent 2009 article. They wrote the following: “Would harmonically unexpected chords increase processing difficulty for ‘semantic garden path’ sentences, where the sentence’s meaning (but not syntactic form) must be reconstructed once a disambiguating word is reached? The SSIRH would not predict such a finding” (Slevc, Rosenberg, & Patel, 2008, p. 604). The aim of the following experiment was to explore this issue. The procedure replicated Slevc et al.’s (2009, Experiment 1), except that syntactic garden paths were replaced with semantic garden paths. Observing an interaction despite the absence of syntactic ambiguity would mean that it was linked to the garden path configuration, irrespective of whether semantic or syntactic operations were involved.

Method

Participants

A total of 96 undergraduate students from the University of Bourgogne, France, participated in the experiment in partial fulfillment of a course requirement. All participants were native French speakers. Participants in Slevc et al.’s (2009) Experiment 1 differed from one another in their level of formal musical training. Given that the authors reported no correlation with this variable, only students without musical schooling were recruited for the present experiment, in order to enhance the homogeneity of the population.

Materials

There were 24 critical sentences, with 12 sentences implementing a semantic violation and 12 sentences implementing a semantic garden path. For each participant and each linguistic manipulation, only 6 out of the 12 sentences were seen with the unexpected segment. The specific sentences serving in expected and unexpected conditions were counterbalanced over the participants. In addition, there were 24 filler sentences, which never included unexpected events and which were identical for all participants. For each sentence, a comprehension question was developed to ensure that participants paid attention to the sentence. Sentences manipulating semantic violations and filler sentences were a translation into French of those used in Slevc et al.’s (2009) Experiment 1.Footnote 1 Sentences manipulating garden path were especially built for this experiment, as well as all the questions.

Each sentence was divided into 8–11 segments and was associated with a specific musical sequence comprising as many chords as linguistic segments. The chord sequences were synthesized with a piano timbre using Finale® software. The chord sequences associated with the syntactic garden paths in Slevc et al. (2009) were used for the semantic garden paths, thus ensuring that our semantic garden paths were associated with the same musical sequences as Slevc et al.’s (2009) syntactic garden paths.

For the 24 critical sentences, half of the chords paired with the critical linguistic segments were in key, and the other half were out of key, in such a way that there was an equal number of in-key and out-of-key chords for each condition. The linguistic and the musical manipulations were fully counterbalanced within participants. For the 24 filler sentences, only one sixth of the musical sequences contained an out-of-key chord, and they were ascribed to the same set of sentences for all participants.

Procedure

Participants were told that they would have to read sentences displayed segment by segment, with the appearance of the next segment being triggered by pressing the space bar. They were also told that a chord would be played simultaneously with each segment, but they were instructed to pay attention to the sentences and to read them carefully enough to be able to respond to a question that would be asked immediately after each sentence. All the participants had to read the 48 sentences while hearing the 48 musical sequences, although which sentences included an unexpected segment and which were paired with an unexpected chord was counterbalanced over participants, hence generating four different lists. The only departure from Slevc et al.’s (2009) procedure was that instead of using a fixed random order for all the participants, the order of trials was randomized in such a way that each set of 4 participants exposed to the four different lists saw the same order but a different order was used for different sets of 4 participants.

Data analysis

As in Slevc et al. (2009), RTs shorter than 50 ms or longer than 2,500 ms and RTs above or below 2.5 SDs from each participant’s mean RTs were removed from the analyses. This led to the exclusion of 3.01 % of the data. RTs were log transformed for the analyses, which were carried out using linear mixed-effects models as implemented in SPSS (version 19).Footnote 2

Results

Tables 1 and 2 are patterned like Tables 1 and 2 in Slevc et al. (2009). These tables show mean RTs and mean accuracies for the comprehension questions, respectively, as a function of musical expectancy (in key vs. out of key), linguistic expectancy (expected vs. unexpected), and linguistic manipulation (semantic garden path vs. semantic violation). In Table 1, data are presented separately for the segment preceding the critical region (precritical region), the critical segment where expectancy violations occurred on half of the trials, and the segment immediately following the critical region (postcritical region).

Table 1 Mean reading times (in milliseconds) by sentence region as a function of musical expectancy, linguistic expectancy, and linguistic manipulation
Full size table
Table 2 Mean accuracies (in percentages) on the comprehension questions as a function of musical expectancy, linguistic expectancy, and linguistic manipulation
Full size table

Slevc et al. (2009) also reported RTs in two figures (reproduced herein in Fig. 2, upper panels), which directly plot the effect of linguistic manipulations, assessed as the difference between unexpected and expected conditions. The corresponding data for the present experiment are displayed in Fig. 2, lower panels.

Visual comparison of Fig. 2’s upper panels (Slevc et al., 2009) and lower panels (present experiment) suggests two main conclusions. First, when the scales of the y-axes are considered, it appears that, on the mean, the effect of syntactic garden paths in Slevc et al. (2009) was larger than the effect of semantic garden paths.Footnote 3 Attributing this difference to the syntactic versus semantic manipulation is hazardous, however, for at least two reasons. First, different sentences were (necessarily) involved in each manipulation, and it cannot be excluded that using other sentences would have resulted in an opposite outcome. Second, a similar (although smaller) difference between experiments was observed for semantic violations despite the fact that, in this case, our sentences were literal translations of the original sentences. We have no particular explanation for why participants were apparently less sensitive to the effects of linguistic violations in our experiment than in that of Slevc et al. (2009). Note that we had no prediction for the relative size of the effect, and this part of the results is inconsequential for the questions of interest.

The second, more important conclusion is that except for small departures in the precritical and postcritical regions, the overall patterns were very similar. To examine whether what the eyeballing of the patterns suggests is confirmed by statistical analysis, independent analyses were performed for each region, as in Slevc et al. (2009), with musical expectancy (in key vs. out of key), linguistic expectancy (expected vs. unexpected), and linguistic manipulation (semantic garden path vs. semantic violation) as fixed effects and participants and items as random effects.

In the precritical region, for which no effect was anticipated, we observed no significant effect at all. Slevc et al. (2009) reported two main effects. RTs were longer in the syntactically manipulated sentences than in the semantically manipulated sentences and were longer in the linguistically expected condition than in the unexpected condition. It is worth noting that although both effects have no clear significance, we get similar numerical trends.

In the critical region, the same analysis revealed only two significant effects, exactly as reported by Slevc et al. (2009). The first was a main effect of linguistic expectancy, whereby RTs were longer in unexpected conditions (M = 576, SD = 209) than in expected conditions (M = 549, SD = 182), F(1, 58.604) = 4.651, p = .035. The second effect was the crucial three-way interaction, F(1, 2169.707) = 4.017, p = .045. Planned comparisons showed that, as in Slevc et al. (2009), there was a simple interaction between musical expectancy and linguistic expectancy in the semantic garden path condition, although this effect was only marginally significant in our data, F(1, 1042.385) = 3.373, p = .067. By contrast, there was no interaction between musical and linguistic expectancy in the semantic violation condition, F(1, 1033.282) = 1.363, p = .243, as in Slevc et al. (2009).

In the postcritical region, RTs were significantly longer in the linguistically unexpected condition than in the expected condition, F(1, 48.066) = 10.679, p = .002, as in Slevc et al. (2009). These authors observed that this effect was stronger for the semantic violation than for the syntactic garden path condition. The same interactive pattern was present with semantic garden path, but the interaction did not reach significance, F(1, 48.066) = 0.985, p = .326. Slevc et al. (2009) also reported a significant interaction of musical expectancy with type of linguistic manipulation, which was replicated in our data, F(1, 2169.242) = 4.04, p = .045. Finally, we observed a significant main effect of musical expectancy, whereby RTs were longer after an out-of-key chord (M = 574, SD = 171) than after an in-key chord (M = 556, SD = 159), F(1, 2169.242) = 8.911, p = .003, while the difference was not significant in Slevc et al. (2009).

Discussion

Overall, our results appear to be remarkably consistent with those of Slevc et al. (2009), even though syntactic garden paths have been replaced with semantic garden paths. The very same pattern of interactions emerged: Participants showed enhanced garden path effects when the sentences were paired with unexpected chords, whereas the musical context had no reliable effect on the processing of semantic violations. As a consequence, the effect of a semantic garden path is much closer to the effect of a syntactic garden path than to the effect of a semantic violation, which strongly suggests that the causal element is the garden path configuration, rather than the implication of alleged syntactic or semantic processors.

Our results invalidate the experimental argument put forth by Slevc et al. (2009) in support of the SSIRH, according to which the processing of linguistic and musical materials would rely on a common syntactic processor. As was mentioned above, Slevc et al. (2008), after having envisioned the possibility of the present results, explicitly acknowledged this implication for their hypothesis. However, they added that such results “would raise the question of what sort of basic mental process could account for overlap in harmonic processing and linguistic reanalysis” (Slevc et al., 2008, p. 604).

We do believe that encompassing both the Slevc et al. (2009) and the present results in a psychologically meaningful story is important. However, the question as formulated by Slevc et al. (2008) is unduly restrictive. Positing an overlap in linguistic reanalysis and harmonic processing as the only possible cause for the interactive pattern rests on the tacit premise that both types of processing occur when an unexpected word and an unexpected chord are displayed conjointly. Yet it is worth stressing that participants were instructed to “concentrate on the sentences” and that “the chords were not task relevant” (Slevc et al., 2009, p. 377). Although surprising sounds could, in principle, divert participants’ attention from linguistic processing despite the task demand, the harmonic violations implemented in Slevc et al.’s study were probably not salient enough to produce this effect. Loui and Wessel (2007) have shown that sensitivity to harmonic violations similar to those involved in the present study was dependent on the allocation of attention and could disappear if participants’ attention was distracted from the harmonic structure (at least for nonmusicians). As a consequence, it is likely that memory and attentional resources were primarily allocated to the written sentence and that the musical violations affected performances insofar as some residual resources were still available to process music.

This hypothesis leads to the prediction of an interaction between the effects of linguistic and musical manipulations insofar as linguistic manipulations require different amounts of attentional resources. To account for the observed interactive pattern in which musical violations have a more deleterious effect on the processing of garden path sentences than on the processing of semantic violations, one needs to assume that semantic violations are more attentionally demanding than garden paths. At first glance, this postulate may appear to be counterintuitive, because dealing with a garden path implies the retrospective processing of the prior segments to reach meaning, which looks to be attention consuming. However, several points need to be considered. First, participants have no hint about anticipating whether they are faced with a garden path or a semantic violation, and they may examine whether they have taken a wrong way in the coding of the preceding segments whenever they encounter an unexpected item, irrespective of whether this retrospective processing is doomed to success or failure. Second, all instances of semantic violations presumably have a detrimental effect (e.g., “angry pigs” can never be anticipated), while a garden path works only on a probabilistic basis (e.g., some participants may have correctly inserted “that” in some of Slevc et al.’s (2009) syntactic garden paths, especially after having been trapped by the omission of “that” on preceding trials). Finally, even when a garden path works, the surprise generated by the unexpected segment ends as soon as the sense of the sentence is reestablished. By contrast, participants faced with a semantic violation may remain bogged down in their unsuccessful quest for a feeling of coherence.

Still more important, the postulate that semantic violations could be more attentionally demanding than garden paths is supported by empirical data. A rough measure of the amount of attention required by the linguistic manipulations is given by their effect on RTs (unexpected conditions minus expected conditions) in the critical region for the in-key condition. Supporting our hypothesis, this effect was significantly stronger for the semantic violations than for the semantic garden paths, F(1, 1148) = 3.855, p = .049. Slevc et al. (2009) observed a similar difference in their Experiment 1 while using a syntactic garden path (see their Table 1). Moreover, the difference persisted (with some attenuation) on the postcritical region for the two experiments.

We thus come to the conclusion that an interpretation calling for nothing else than attentional mechanisms, as involved in the interpretation proposed by Poulin-Charronnat, Bigand, Madurell, and Peereman (2005) to account for the interaction they observed between linguistic and harmonic relatedness, accounts for the present data as well. To sum up, the detrimental effect of linguistic manipulations would be amplified in the presence of a harmonic violation only if the amount of attentional resources left available for music by the focal reading task is sufficient.

So far we have dealt with Slevc et al.’s (2009) Experiment 1. In a second experiment, the harmonic manipulation was replaced with a manipulation of musical timbre, and the critical three-way interaction disappeared. This outcome is consistent with the SSIRH, given that timbre changes are not syntactic in nature. However, timbre changes also probably differ from harmonic manipulations with regard to their effects on attention. Arguably, a change in timbre, by contrast with a harmonic violation, should be able to capture attention in an obligatory fashion irrespective of the focal task. To our best knowledge, this hypothesis has never been directly tested,Footnote 4 but it seems in agreement with results showing that physical alterations such as the gender of the speaker or changes in the pitch of task-irrelevant tones capture attention and may impair performance in a visual task (e.g., Parmentier, Elford, Escera, Andrés, & San Miguel, 2008). If the change in timbre captures attention even when the reading task is complex, the cause we suggest for the three-way interaction observed in Experiment 1 disappears.

There is another, nonexclusive way to account for the disappearance of the interactive pattern in Slevc et al.’s (2009) Experiment 2 that would be in agreement with our attentional framework. In this experiment, processing syntactic garden paths and semantic violations do not seem to differ in attentional resources, at least when assessed as the effect of linguistic manipulation on RTs when there is no concurrent musical violation. This result is surprising, given that this part of the experiment is an exact replication of their Experiment 1. But whatever the source of the discrepancy between the results of their two experiments, the consequences are the same: If processing garden paths and semantic violations requires a comparable amount of attentional resources, there is no reason to observe an interactive pattern. To sum up, the lack of interaction observed in Slevc et al.’s (2009) Experiment 2 can be due to a combination of two factors: Either the timbre manipulation automatically captured attention irrespective of the linguistic manipulations, or the two types of linguistic manipulation required a similar amount of attention from the participants in this experiment.

To conclude, it is important to note that the SSIRH does not rest only on the experimental evidence provided by Slevc et al. (2009), and further studies are clearly needed to examine the extent to which the conclusion of the present study can be generalized to the other supports the SSIRH has received (e.g., Koelsch, Gunter, Wittfoth, & Sammler, 2005). However, our reappraisal shows that a part of the data that once looked to be a compelling support for a shared syntactic processor for music and language can be accounted for by the implication of general attentional mechanisms as well.