Behavior Research Methods

, Volume 44, Issue 1, pp 81–94 | Cite as

Norming the odd: Creation, norming, and validation of a stimulus set for the study of incongruities across music and language

  • Cara R. Featherstone
  • Mitch G. Waterman
  • Catriona M. Morrison
Article

Abstract

Research into similarities between music and language processing is currently experiencing a strong renewed interest. Recent methodological advances have led to neuroimaging studies presenting striking similarities between neural patterns associated with the processing of music and language—notably, in the study of participants’ responses to elements that are incongruous with their musical or linguistic context. Responding to a call for greater systematicity by leading researchers in the field of music and language psychology, this article describes the creation, selection, and validation of a set of auditory stimuli in which both congruence and resolution were manipulated in equivalent ways across harmony, rhythm, semantics, and syntax. Three conditions were created by changing the contexts preceding and following musical and linguistic incongruities originally used for effect by authors and composers: Stimuli in the incongruous–resolved condition reproduced the original incongruity and resolution into the same context; stimuli in the incongruous–unresolved condition reproduced the incongruity but continued postincongruity with a new context dictated by the incongruity; and stimuli in the congruous condition presented the same element of interest, but the entire context was adapted to match it so that it was no longer incongruous. The manipulations described in this article rendered unrecognizable the original incongruities from which the stimuli were adapted, while maintaining ecological validity. The norming procedure and validation study resulted in a significant increase in perceived oddity from congruous to incongruous–resolved and from incongruous–resolved to incongruous–unresolved in all four components of music and language, making this set of stimuli a theoretically grounded and empirically validated resource for this growing area of research.

Keywords

Music Language Incongruity Resolution 

Introduction

The study of similarities between music and language processing is a growing area of research that has benefitted from strong methodological advances over the last two decades. Be it in the study of incongruity processing (Koelsch, Gunter, Schroger, & Friederici, 2003; Patel, Gibson, Ratner, Besson, & Holcomb, 1998), of aesthetic effects (Coulson & Van Petten, 2007; Steinbeis, Koelsch, & Sloboda, 2006; Thierry et al., 2008), of brain localization (Brown, Martinez, & Parsons, 2006; Ducreux, Marsot-Dupuch, Lasjaunais, Oppenheim, & Frédy, 2003; Levitin & Menon, 2005; Tervaniemi et al., 2006), or of the nature of expectancies (Abdallah & Plumbley, 2008; Hale, 2001; Huron, 2006; Levy, 2008), researchers have demonstrated striking similarities between both domains of human cognition. The renewed research interest in this area has arisen following methodological advances permitting a noninvasive exploration of neural mechanisms associated with the processing of music and language while listeners read or listen to stimuli. However, despite these advances, conflicting results in the literature prevent strong theoretical development alongside such strong methodological advances.

In the study of incongruity processing, two key neuroimaging studies demonstrated similar neurological patterns in response to syntactically incongruous words and harmonically incongruous chords (Patel et al., 1998) and in response to harmonically incongruous chords and semantically incongruous words (Koelsch et al., 2003) . Studies in music and language incongruity processing have repeatedly demonstrated two types of ERP responses, a late parietal positivity (Besson, Faita, & Requin, 1994; Besson & Macar, 1987; Miranda & Ullman, 2007; Patel, 2008) and a late negativity (Hantz, Kreilick, Kananen, & Swartz, 1997; Jentschke, Koelsch, & Friederici, 2005; Steinbeis & Koelsch, 2008), respectively, although these response patterns have received a number of different labels (late positive component, later positivity, late posterior positivity, P600; late negativity, N5, N500, N400).

The question of the specificity of music and language processing has led to studies investigating the interference of the processing of stimuli in one domain on the processing of stimuli in the other domain. Studies investigating the interference of ERP indexes of harmony on semantics and syntax have resulted in conflicting findings. While some studies have demonstrated an interaction between semantic processing and harmonic processing, using reaction time and ERP methods (Poulin-Charronnat, Bigand, Madurall, & Peereman, 2006), others have demonstrated an interaction between syntactic processing and harmonic processing using ERP and self-paced reading methods (Fedorenko, Patel, Casasanto, Winawer, & Gibson, 2009; Slevc, Rosenberg, & Patel, 2009). In the field of aesthetics, researchers have struggled to demonstrate the pleasant effect of incongruities, advocated from Meyer (1956) and Ortony (1975) to Vuust and Kringelbach (2010) and Bowdle and Gentner (2005), through a lack of consideration for aesthetics in the creation of incongruous stimuli. Increased tension has been linked to incongruity processing (Steinbeis, Koelsch, & Sloboda, 2006), but the pleasantness associated with the resolution of musically incongruous elements is yet to be formally understood (Juslin & Västfjäll, 2008; Vuust & Kringelbach, 2010). Concerning brain localization, significant overlaps have been demonstrated between the neural underpinnings of music and language processing (Ducreux et al., 2003; Levitin & Menon, 2005), but progress in the discussion of hemispheric asymmetry has been plagued by differences in the complexity of the stimuli used by different research teams, from ecologically valid harmonized music and sentences (Brown et al., 2006; Levitin & Menon, 2005) to phonemes, pseudowords, and random tones (Schmithorst, 2005; Tervaniemi et al., 2006; Williamson, Baddeley, & Hitch, 2010).

A closer analysis of stimulus sets used by different researchers suggests barriers to theoretical advances. First, the stimuli used in key music ERP research have often violated the rules of classical Western harmony in the introduction of incongruities (the stimuli used in the foundational paper by Patel and his colleagues presented consecutive fifths and octaves; those used by Koelsch and his colleagues have typically featured unorthodox placements of chromatic chords). Linguistic analogies for these violations would be the introduction of a spelling mistake in a target semantically incongruous word or a subject–verb disagreement in a target syntactically difficult embedded clause. These therefore confound the study of experimentally controlled incongruities (elements that are statistically improbable within their context; see, e.g., Huron, 2006; Levy, 2008) with other aspects of musical grammar (the erroneous transition to and from that improbable element and the lack of harmonic closure, respectively). Second, although the incongruity-based aesthetic models studied in this field of research rely on the notion of the resolution of incongruous elements (Bowdle & Gentner, 2005; Vuust & Kringelbach, 2010), stimuli have typically consisted of sentences and musical sequences in which a specific element is changed across conditions to make it normal, improbable, or erroneous, with no regard for the resolution of the incongruity. This lack of manipulation of resolution has resulted in a lack of progress in the investigation of whether incongruity per se or harmonic closure leads to the late negativities observed in harmonically open sequences (e.g., Hantz et al., 1997). Third, progress in understanding the role of incongruities in aesthetic and affective responses to music and language has been hampered by the use of stimuli designed with no consideration for aesthetics, introducing incongruities into congruous stimuli rather than manipulating incongruities used by authors and composers for effect (although see Steinbeis et al., 2006, for an example of stimuli derived from J. S. Bach’s aesthetically purposed harmonic incongruities). Fourth, concerns over artifact control and difficulties in the development of auditory language stimuli have led researchers to compromise both ecological validity and cross-modal equivalence by presenting stimuli consisting of short chord sequences without melodic development (Jentschke et al., 2005; Koelsch et al., 2003) and visual rather than auditory language stimuli (Koelsch et al., 2004; Patel, 2003) in studies comparing music and language processing. Finally, where psycholinguists would use norming studies and corpus evidence to judge the degree of incongruity of their stimuli (Van Berkum, Brown, Zwitserlood, Koojman, & Hagoort, 2005), authors investigating music incongruities have tended to rely solely on music theory to determine the degree of a target element’s departure from the harmonic or rhythmic norm, leaving unaddressed the question of whether theoretically odd musical elements are perceived as odd.

Responding to a call for more systematic research in this area of growing interest (Juslin & Västfjäll, 2008; Steinbeis et al., 2006; Thierry et al., 2008; Vuust & Kringelbach, 2010), this article describes the creation, norming, and validation of a new set of stimuli presenting equivalent manipulations of incongruity and resolution across the four fundamental aspects of music and language: harmony and rhythm in music and semantics and syntax in language. The development of this stimulus set addresses the concerns expressed above, thereby providing a theoretically grounded and empirically validated resource for the investigation of the similarities between music and language processing.

Creating the stimulus set

Since the aim was to create stimuli that would allow the parallel investigation of incongruity processing across the four components of music and language, the same experimental manipulation was applied to incongruities used by authors and composers for effect across harmony, rhythm, semantics, and syntax. Throughout this article, harmony and rhythm are used to describe incongruities in music having to do with the chord progression and with the rhythmic pattern, respectively, and semantics and syntax are reserved for the description of linguistic semantic and syntactic incongruities. The stimuli created in this process varied along two factors: the type of stimulus (harmony, rhythm, semantics, or syntax) and its level of congruence (congruous, incongruous–resolved, or incongruous–unresolved).

All stimuli followed the same design, illustrated in Fig. 1. The red rectangle represents the target element, which remained identical across three levels of congruence. Two quavers or two syllables either side of the target element were also kept identical across the three conditions. The arrows coming toward and leaving the rectangle represent the musical or linguistic context within which the target element was embedded. It was the manipulation of this context that enabled the creation of the three different congruence conditions: The incongruous–resolved condition presents the element of interest as it was originally used for effect; the congruous condition presents the element of interest in a context adapted to match it; and the incongruous–unresolved condition changes the frame of reference of the stimulus after the element of interest, making the stimulus nonsensical as a whole. For greater clarity, the experimental manipulation is described below separately for harmony, rhythm, semantics, and syntax. The stronger nature of the semantic and syntactic constraints in language, as compared with the rules governing musical good form (Bod, 2002), led us to expect a stronger subjective reaction to linguistic incongruities than to musical incongruities, despite the fact that this manipulation resulted in an equivalent theoretical processing disruption across music and language. The claim of equivalence here lies not between the specific elements of music and language (e.g., equating a word with a chord), but in the extent to which the element of interest is at odds with its context (congruous or incongruous) and the way in which the stimulus continues after the element of interest (resolved or unresolved).
Fig. 1

Experimental manipulation applied to all stimuli in this set to create the three congruence conditions: the incongruous–resolved condition presents the element of interest as it was originally used for effect, the congruous condition presents the element of interest in a context adapted to match it, and the incongruous–unresolved condition changes the frame of reference of the stimulus after the element of interest, making the stimulus nonsensical as a whole

All music audio files were created using the standard piano sound from Sibelius 5’s inbuilt KontactPlayer2 and contained no variations in dynamics. The same sound extract was used to create the element of interest plus two quavers either side of it in all three conditions within each music triplet. All language audio files, spoken by the same female speaker in a neutral tone, were recorded and edited using ProTools7. Once again, exactly the same sound section was used for the region of interest (incongruity + two syllables either side) across the three conditions of each sentence triplet. Cross-splicing was carried out, using the Crossfade function in ProTools, to ensure that any artifacts due to sound editing were removed. Thus, although the sentences had been edited to use the same section of recording for the element of interest plus two syllables either side, all sentences sounded as if they had been spoken as a whole and flowed at the pace of natural speech. To listen to example stimuli, please visit www.carafeatherstone.co.uk/research/stimuli/, hereafter referred to as the stimulus Web page. Details on using these stimuli are provided in the Discussion section.

Music stimuli

Extracts from mainstream popular music that contained aesthetically purposed rhythmic or harmonic incongruities were systematically manipulated to form the three levels of congruence. Each stimulus was created from a different extract. In order to control for any effects of key, some stimuli extracts were transposed to ensure that all 12 possible keys were equally represented in the stimulus set. Three tempi (70, 100, and 130 beats per minute) were also equally represented across the music stimuli. In order to ensure greater control over potential confounds, the dimension that was not the one in which the incongruity occurred (i.e., rhythm for the harmony stimuli and harmony for the rhythm stimuli) was varied in a systematic fashion across the stimuli, using one of four possible patterns. For instance, if the musical extract presented an unusual chord, the chord progression in which that strange chord occurred was isolated from its original rhythmic pattern and rewritten with one of four possible rhythmic patterns adapted from rhythmically congruous mainstream music. If, on the other hand, the incongruity was rhythmic, the rhythmic sequence that contained the incongruity was rewritten with one of four possible harmonic progressions adapted from harmonically congruous mainstream music. Thus, for each extract, the melody and either the rhythmic structure or the harmonic progression differed from the original, making the original musical extracts from which these stimuli were adapted unrecognizable. This procedure allowed a good degree of ecological validity to be retained in the stimulus set, while ensuring that the variability in the element that was not of interest was systematically controlled. Considering the concerns expressed over musical stimuli that violated the rules of tonal Western classical music, extra care was taken to abide by these in this set of stimuli (i.e., no consecutive fifths or octaves, no unfinished cadences).

Harmony

Thirty-six mainstream music extracts that contained harmonic incongruities were manipulated. These consisted of three equally represented different types of incongruities: chromatic chords (e.g., a Neapolitan 6th), chords borrowed from a nearby key (e.g., a chord of D, borrowed from G, in a sequence in C), and chords borrowed from a distant key (e.g., a chord of F borrowed from the key of C in a piece in D). These three types were equally represented in the set. A link to scores providing a typical example of each of these incongruity types can be found on the stimulus Web page mentioned above. None of these incongruities per se prevented the harmonic closure of the piece. The chord sequences were stripped of their original rhythmic pattern and recreated using one of four rhythmic patterns containing no unusual rhythms. The conditions, illustrated in Fig. 2, can be described as follows:
Fig. 2

Example harmony stimulus in all three congruence conditions. Two quavers are kept identical either side of the element of interest (a chord of F7, borrowed from the key of B-flat). In the incongruous–resolved condition, the piece returns to the original key of F Major after the chord of interest; in the congruous condition, the entire piece is in the key of the chord of interest (B-flat Major); in the incongrous-unresolved condition, the piece starts in the key of F Major and continues in the key of B-flat Major after the element of interest

  • Incongruous–resolved. The incongruous chord is followed by a resolution in the key of the stimulus (e.g., a sequence in F major with a chord borrowed from B-flat major continues in F major after the incongruity).

  • Congruous. The key of the stimulus is changed to match the key of the chord of interest so that it no longer sounds odd (e.g., the whole stimulus is in B-flat major).

  • Incongruous–unresolved. The extract is the same as in the first condition up to the incongruous chord, at which point the rest of the stimulus continues in the key determined by the incongruity (e.g., a sequence in F major with a chord borrowed from B-flat major continues in B-flat major after the incongruity).

Chords leading to and from the incongruity in the congruous condition were chosen with reference to the table of usual root progressions provided in Piston’s (1978) Harmony. In order to make the chord of interest incongruous, it was necessary to introduce new pitches at this point in the stimulus, leading to concerns over confounding effects of novelty with those of harmonic processing. To avoid confounding incongruity processing with change detection, care was taken to ensure that the part of the stimulus that was kept identical across congruence conditions was not preceded by an identical section in the congruous condition. This combination of pitches was therefore no more novel in the incongruous conditions than in the congruous conditions. Furthermore, by keeping two crotchets identical on either side of the incongruous chord, this control ensured that the sensory dissonance of the incongruous chord, in comparison with the preceding chord, was identical across congruence conditions, thereby ruling out an effect of echoic memory on perceived oddity. These considerations allowed us to control for psychoacoustic factors that, otherwise, could have confounded effects of harmonic processing.

Rhythm

Thirty-six mainstream music extracts that contained rhythmic incongruities were manipulated. These also consisted of three equally represented different types of incongruities: a change in beat emphasis (e.g., a bar sounding like 3/4 in the middle of a sequence in 6/8), a change in the subdivision of the beat division (e.g., the introduction of triplets in a sequence in 4/4), or a change in the number of beats in a bar (e.g., a bar of 5/4 in a sequence in 4/4). Once more, the stimulus Web page provides a score for each of the rhythmic incongruity types. They were stripped of their harmonic sequence and recreated using one of four congruous chord progressions, systematically transposed to represent all 12 keys of Western tonal harmony evenly.
  • Incongruous–resolved. The incongruous rhythm resolves into the rhythmic pattern of the beginning of the stimulus (e.g., a sequence in 4/4 contains a bar in which the beat emphasis sounds like 9/8 and returns to a standard 4/4 pattern).

  • Congruous. The beat is adapted to fit with the element of interest, which goes unnoticed (e.g., since the original incongruous element sounded like 9/8, the whole sequence is changed to a 9/8 beat).

  • Incongruous–unresolved. The stimulus starts the same as in the first condition but continues postincongruity with the beat dictated by the incongruous pattern (e.g., a sequence in 4/4 continues in 9/8 after the incongruity).

As with the harmony stimuli, care was taken to ensure that the rhythmic element of interest did not occur before the part of the stimulus that was identical across all three congruence conditions in the congruous stimuli. This avoided the confounds of memory effects in the study of participants’ processing of incongruous rhythms.

Language stimuli

The types of language incongruities used this study were chosen with the following criteria in mind. The incongruities needed to (1) show a good fit with the integration paradigm adopted for the music stimuli and throughout the literature investigating parallels between music and language cognition, (2) possess a previously established experimental validity in event-related potential (ERP) incongruity studies, and (3) be used for effect in naturally occurring language to allow the investigation of the role of incongruities in eliciting affective and aesthetic responses.

A survey of the literature reporting an N400 or a P600 associated with semantic or syntactic violations was carried out. Metaphors, which Ortony (1975) described as using incongruities to heighten the evocative power of words, were consistently associated with an N400 of amplitude higher than that for literal sentences and lower than that for nonsensical sentences (Bonnaud, Gil, & Ingrand, 2002; Coulson & Van Petten, 2002, 2007; Kazmerski, Blasko, & Dessalegn, 2003; Pynte, Besson, Robichon, & Poli, 1996; Tartter, Gomes, Dubrovsky, Molholm, & Stewart, 2002). Metaphors therefore seemed the obvious choice for sentences containing semantic violations. Considering that previous studies have compared the processing of notes or chords (or a single beat) with the processing of single words (Besson & Macar, 1987; Patel et al., 1998; Steinbeis & Koelsch, 2008), the target element in these language stimuli was restricted to one word in length. Syntactic incongruities used for effect were more difficult to find in the ERP literature. However, a study by Thierry et al. (2008) showed that the functional shift (the use of a noun in place of a verb), which has been used for effect since the days of Shakespeare (Clark & Clark, 1979), is systematically associated with a left anterior negativity (LAN) and both early and late positivities, and not with an N400 effect, as could be expected for a linguistic device that emphasizes meaning (Lee & Federmeier, 2006; Thierry et al., 2008). Considering the fit of the functional shift with the sought ERP and aesthetic effects, this type of incongruity was an obvious choice for the syntactic stimuli.

Semantics

Thirty-six sentences in the incongruous–resolved condition were adapted from metaphors found in song lyrics, novels, creative writing tutorials, and poetry. The continuation of the sentence (after the metaphorical word) rendered the metaphor meaningful in the incongruous–resolved stimuli and nonsensical in the incongruous–unresolved stimuli. The nonsensical sentences were created by continuing the sentence with something relating literally to the metaphorical word (e.g., a sentence about someone’s mind clanking was ended with a reference to old car engines). In view of potential overlapping of effects of syntactic and semantic incongruities, the semantic sentences presented a simple syntactic structure. Anaphora were avoided as much as possible, and care was taken to ensure that no conflict between anaphora and referents affected the processing of the sentences. The sentences were manipulated as follows:
  • Incongruous–resolved. The sentence contained a metaphorical element (incongruity) that was given greater meaning postincongruity (e.g., The pianist’s left hand had become rusty because he had ceased practicing).

  • Congruous. The sentence contained the same element, but its use was literal (e.g., Paul’s expensive bike had become rusty because he had left it out in the rain).

  • Incongruous–unresolved. The sentence was made up of the beginning of the incongruous–resolved sentence and the end of the congruous sentence, rendering it nonsensical (e.g., The pianist’s left hand had become rusty because he had left it out in the rain).

Syntax

Thirty-six syntactic-shift verbs were chosen from the review of nouns used as verbs carried out by Clark and Clark (1979) and from a study by Thierry et al. (2008). The functional-shifted nouns retained for this project were those whose use as verbs had not passed into natural language (e.g. to butter one’s bread was rejected, but to tomato a singer was retained). Echoing the control imposed on semantic stimuli, extra care was taken to ensure a simple semantic content in syntactically incongruous sentences, and the sentences presented no other syntactically difficult elements, such as difficult anaphoric resolutions, subjacency violations, or embedded relative clauses, all shown to elicit specific ERP effects (Kluender & Kutas, 1993; Koornneef & Van Berkum, 2006; Patel et al., 1998). The sentences were manipulated as illustrated in Fig. 3.
Fig. 3

Example syntax stimulus in all three congruence conditions. Two syllables are kept identical either side of the element of interest (the word trousers). In the incongruous–resolved condition, the sentence presents a meaningful functional shift (noun used as a verb); in the congruous condition, the noun is used as a noun; in the incongruous–unresolved condition, the sentence continues in the syntactic structure dictated by the functional shift

  • Incongruous–resolved. The sentence used a functional-shift verb and resolved to a normal syntactic structure (e.g., Because her two-year-old never stops kicking, Mrs Matthews trousers her son with great difficulty every morning).

  • Congruous. The sentence used the same word as a noun and had a simple syntactic structure (e.g., Because Mary had ironed a hole in Matthew’s trousers her son had to wear shorts to school).

  • Incongruous–unresolved. The sentence was made up of the beginning of the incongruous–resolved sentence and the end of the congruous sentence, creating a sentence with an erroneous syntactic structure (e.g., Because her two-year-old never stops kicking, Mrs. Matthews trousers her son had to wear shorts to school).

Note that the fact that the region of interest had a different spelling in the congruous condition was of no consequence, since the stimuli were auditorily presented, using the same sound extract for the element of interest plus two syllables either side across all three congruence conditions.

Creation summary

This process for creating stimuli addresses the first four concerns expressed over stimulus sets previously used in this field of research. First, by starting from aesthetically purposed incongruities used in mainstream music and language, this set provides a means of addressing the cognitive and neurobiological underpinnings of incongruity-based aesthetic experiences. Second, by establishing cross-domain equivalence in the degree of theoretical incongruity of the target element, in comparison with its context, this set provides a means of comparing processes resulting from music and language incongruities in a meaningful way. Third, by adhering to the rules of Western tonal music, these stimuli are devoid of the confounding errors found in previous stimulus sets. Fourth, by focusing on the notion of resolution and not just on the notion of incongruity, this set provides the means of assessing the role of resolution, emphasized in the original theories of incongruity-based aesthetics. The degree of control described in the creation of stimuli allowed greater certainty of the experimental validity of the manipulation, while maintaining the ecological validity of the stimulus set. The fifth concern, having to do with the reliance on assumptions to determine what listeners perceive as odd, was addressed via a norming procedure, designed to reduce the initial 36 stimulus triplets in harmony, rhythm, semantics, and syntax to the best 24, according to the congruence condition fitness criteria described below.

Norming the stimulus set

Gathering data for the norming procedure

An initial rating study was carried out to gather the data necessary for norming the stimulus set. This norming procedure ensured that stimuli manipulated to sound normal, slightly odd, and nonsensical were indeed perceived as such by participants.

Thirty-six native British English speakers (14 of them male, mean age = 27.50 years, average number of years of musical training = 3.86) took part in this four-part rating experiment. Participants were tested individually using the experimental software E-Prime, and the stimuli were presented via noise-canceling headphones. Each participant listened to a total of 144 stimuli, containing 36 stimuli from harmony, rhythm, semantics, and syntax made up of 12 stimuli in each congruence condition (congruous, incongruous–resolved, incongruous–unresolved). Each participant heard only 1 of the 3 stimuli derived from each target element (hereafter, three stimuli derived from the same element of interest are described as a triplet), eliminating any effects of memory on participants’ judgments (e.g., participants expecting a different ending in an incongruous–resolved stimulus after hearing the incongruous–unresolved stimulus derived from the same original music or language extract). The experiment followed a Latin-square repeated measures design. The four types of stimuli (harmony, rhythm, semantics, and syntax) were presented in four separate parts, counterbalanced across participants. The stimuli were presented in a different order for each participant within each part. The instructions, identical across all stimulus types, were as follows: “While listening to the stimulus, press the space bar if you hear anything unusual. After each stimulus, rate how odd it sounded using the scale which appears on the screen.” This scale was a visual analogue scale ranging from completely normal to very odd.

Mean scores1 were analyzed in a by-items design, to investigate the congruence condition fitness of each stimulus triplet. This congruence condition fitness consisted in the three stimuli within each triplet being reliably perceived as normal, slightly odd, and very odd. This was assessed by analyzing oddity detection, indicated by space bar presses, to determine which point of the stimulus participants judged to be odd, and oddity ratings using the visual analogue scale. The aim was to arrive at a set in which the manipulated incongruity alone caused participants to press the space bar and in which oddity ratings showed the following trend for the three levels of congruence in each triplet: congruous <incongruous–resolved <incongruous–unresolved.

Prior to the exclusion of stimuli that failed the norming procedure, a significant effect of congruence was already seen across all types of stimuli, F(2, 35) = 476.46, p < .001. Simple effects analyses revealed a significant main effect of congruence within harmony, F(2, 68) = 14.27, p < .001, η2 = .30, semantics, F(2, 68) = 347.23, p < .001, η2 = .91, and syntax, F(2, 68) = 167.66, p < .001, η2 = .83. No significant effect of congruence was found within rhythm, F(2, 68) = 1.71, p = .19, η2 = .05.

Data trends were also in the expected direction (congruous <incongruous–resolved <incongruous–unresolved) in all types of stimuli bar rhythm, in which no increase from incongruous–resolved to incongruous–unresolved was observed. There are several possible explanations for the weaker differences between congruence conditions in the rhythm stimuli. First, participants may have heard both the incongruity and its resolution as odd, since both involve a change of rhythm (when the music transitions into the incongruous rhythmic pattern and when it transitions back into the previously established rhythm). Second, since participants were instructed to listen out for incongruous elements, this instruction, combined with the salience of rhythmic changes, could have led to participants’ rating stimulus oddity as a function of change detection, rather than engaging in a more holistic processing of the stimuli. A further explanation for the weaker effect of congruence in the rhythm stimuli could be that a congruous stimulus derived from an incongruity, such as a bar in 5/4 would have consisted of a piece entirely in 5/4. Since this time signature is unusual in Western tonal music, the congruous stimulus would have received a high oddity rating. The data from this experiment were primarily collected to enable a norming procedure; a postnorming validation study must, therefore, take place before the effect of incongruities on oddity perception in rhythm can be discussed more fully.

Selecting the final set of stimuli

A scoring system was applied identically across harmony, rhythm, semantics, and syntax. Its aim was to assess congruence condition fitness on the basis of participants’ responses in the initial rating study, using a by-items analysis. Seven separate evaluations resulted in either a negative or a positive score for each stimulus. Negative scores were given for stimuli in which (1) the space bar analysis showed that 2 or more participants had indicated an incongruity within the same 1,000-ms time window in the congruous condition (this condition was designed to contain no incongruities); (2) more than 50% of participants had pressed the space bar at any given time in the congruous condition; (3) the congruous stimulus was rated more odd than the incongruous–resolved stimulus; (4) the congruous stimulus was rated more odd than the incongruous–unresolved stimulus; (5) fewer than 50% of space bar presses occurred within a time window delimited by the incongruity and its resolution in the incongruous–resolved condition; (6) fewer than 50% of space bar presses occurred after the incongruity in the incongruous–unresolved condition; and (7) the incongruous–unresolved condition was not rated more odd than the other two congruence conditions. Scores for each stimulus triplet are presented in Tables 1 and 2 for the music and language stimuli, respectively.
Table 1

Norming scores for each stimulus triplet (music)

Harmony Triplet

Evaluation

Total

Selected

Rhythm Triplet

Evaluation

Total

Selected

1

2

3

4

5

6

7

1

2

3

4

5

6

7

hlal

1

2

−1

2

2

−1

2

7

yes

r1al

−1

−1

−1

2

1

−1

−1

−2

no

hla2

−1

−1

−1

2

2

−1

−1

−1

no

rla2

−1

1

−1

2

2

−1

−1

1

yes

hla3

1

1

1

−1

2

−1

1

4

yes

rla3

−1

1

1

1

1

−1

1

3

yes

hlbl

−1

1

−1

1

−1

−1

−1

−3

no

rlbl

2

2

1

1

1

−1

1

7

yes

hlb2

1

2

2

1

2

1

2

11

yes

rlb2

−1

−1

1

2

1

−1

1

2

yes

hlb3

1

2

−1

−1

−1

−1

−1

−2

no

rlb3

2

2

1

−1

−1

−1

1

3

yes

h1c1

1

2

2

−1

1

2

2

9

yes

r1c1

−1

2

−1

−1

−1

1

2

1

yes

hlc2

2

2

2

1

2

1

2

12

yes

rlc2

1

1

−1

1

2

−1

2

5

yes

hlc3

−1

−1

1

2

2

1

1

5

yes

rlc3

−1

−1

−1

−1

−1

−1

2

−4

no

h2al

1

1

1

1

1

−1

1

5

yes

r2al

1

1

1

1

–1

1

1

5

yes

h2a2

1

1

2

1

2

2

2

11

yes

r2a2

−1

1

1

1

–1

−1

1

1

no

h2a3

−1

1

–1

2

2

–1

2

4

yes

r2a3

1

1

1

–1

2

–1

1

4

yes

h2bl

1

2

2

−1

−1

1

2

6

yes

r2bl

2

2

1

1

1

1

1

9

yes

h2b2

−1

−1

−1

−1

1

–1

1

−3

no

r2b2

1

2

2

1

–1

1

2

8

yes

h2b3

1

1

–1

1

2

–1

−1

2

yes

r2b3

2

2

1

–1

−1

−1

1

3

yes

h2cl

−1

1

–1

2

1

–1

2

3

yes

r2cl

−1

1

–1

1

–1

−1

−1

−3

no

h2c2

−1

1

–1

−1

1

–1

−1

−3

no

r2c2

1

1

1

1

–1

1

1

5

yes

h2c3

−1

1

2

1

–1

2

2

6

yes

r2c3

1

2

–1

−1

2

–1

−1

1

no

h3a1

1

2

2

2

1

2

2

12

yes

r3al

−1

1

2

–1

−1

2

2

4

yes

h3a2

−1

1

–1

−1

−1

1

1

−1

no

r3a2

1

2

–1

2

1

1

2

8

yes

h3a3

2

2

1

–1

−1

−1

1

3

yes

r3a3

−1

1

1

1

1

–1

1

3

yes

h3bl

1

2

–1

−1

−1

1

–1

0

no

r3bl

1

2

–1

−1

1

–1

−1

0

no

h3b2

1

1

–1

−1

1

1

–1

1

yes

r3b2

1

1

1

2

2

–1

1

7

yes

h3b3

−1

1

1

2

−1

−1

1

2

yes

r3b3

−1

1

1

2

2

1

1

7

yes

h3cl

1

1

–1

−1

2

–1

−1

0

no

r3cl

1

–1

−1

2

–1

−1

1

0

no

h3c2

2

2

2

1

–1

1

2

9

yes

r3c2

−1

−1

−1

1

–1

−1

−1

−5

no

h3c3

−1

1

–1

−1

−1

−1

2

−2

no

r3c3

1

2

–1

1

1

1

–1

4

yes

h4al

−1

−1

−1

2

2

–1

−1

−1

no

r4al

1

2

–1

−1

1

1

–1

2

yes

h4a2

−1

1

1

2

1

–1

1

4

yes

r4a2

−1

1

1

1

2

–1

1

4

yes

h4a3

1

2

2

2

–1

2

2

10

yes

r4a3

−1

−1

−1

1

2

–1

−1

−2

no

h4bl

1

2

−1

−1

1

1

–1

2

yes

r4bl

−1

1

1

2

–1

1

1

4

yes

h4b2

2

2

2

–1

−1

2

2

8

yes

r4b2

1

1

2

1

2

2

2

11

yes

h4b3

−1

−1

−1

1

–1

1

–1

−3

no

r4b3

—1

−1

−1

−1

1

1

−1

−3

no

h4c1

1

2

2

1

–1

2

2

9

yes

r4ci

−1

1

−1

−1

−1

−1

−1

−5

no

h4c2

−1

1

–1

1

–1

−1

−1

−3

no

r4c2

−1

1

1

–1

−1

1

1

1

yes

h4c3

1

1

–1

1

2

1

–1

4

yes

r4c3

1

1

–1

1

–1

−1

−1

−1

no

Table 2

Norming scores for each stimulus triplet (language)

Semantics Triplet

Evaluation

Total

Selected

SyntaxTriplet

Evaluation

Total

Selected

1

2

3

4

5

6

7

1

2

3

4

5

6

7

seal

2

2

2

1

1

2

2

12

yes

syal

−1

−1

2

−1

2

2

2

5

no

seal0

−1

1

−1

−1

1

1

2

2

no

syal0

−1

1

−1

1

2

−1

2

3

no

seall

1

2

2

1

2

2

2

12

yes

syall

1

2

2

1

2

2

2

12

yes

seal2

2

2

2

2

−1

2

2

11

yes

syal2

2

2

2

1

2

2

2

13

yes

sea2

−1

1

2

2

1

2

2

9

no

sya2

−1

1

−1

1

2

−1

2

3

no

sea3

2

2

2

1

2

2

2

13

yes

sya3

1

2

2

1

2

2

2

12

yes

sea4

2

2

2

−1

1

2

2

10

yes

sya4

2

2

2

1

2

2

2

13

yes

sea5

−1

1

2

2

−1

2

2

7

no

sya5

1

2

2

1

1

2

2

11

yes

sea6

2

2

2

−1

1

1

2

9

no

sya6

1

2

2

1

1

2

2

11

yes

sea7

2

2

2

1

2

2

2

13

yes

sya7

1

2

2

2

−1

2

2

10

yes

sea8

2

2

2

−1

2

2

2

11

yes

sya8

−1

1

−1

2

2

−1

2

4

no

sea9

2

2

2

1

1

2

2

12

yes

sya9

−1

1

2

1

2

2

2

9

yes

seb1

1

2

2

1

−1

2

2

9

no

syb1

1

2

2

2

2

2

2

13

yes

sebl0

2

2

2

−1

1

2

2

10

yes

sybl0

2

2

2

1

2

2

2

13

yes

sebl1

1

2

1

1

−1

1

2

7

no

syb1l

−1

−1

−1

2

−1

−1

2

−1

no

sebl2

2

2

−1

1

2

1

2

9

yes

sybl2

1

2

2

2

2

2

2

13

yes

seb2

1

2

2

1

−1

2

2

9

yes

syb2

2

2

2

1

2

2

2

13

yes

seb3

−1

1

−1

1

1

−1

2

2

no

syb3

2

2

2

2

2

2

2

14

yes

seb4

1

2

2

1

2

2

2

12

yes

syb4

1

1

2

−1

2

1

2

8

no

seb5

2

2

2

1

1

2

2

12

yes

syb5

1

2

2

1

2

2

2

12

yes

seb6

1

2

−1

−1

1

1

2

5

no

syb6

−1

1

2

1

1

2

2

8

no

seb7

1

2

2

1

2

1

2

11

yes

syb7

1

2

2

2

2

2

2

13

yes

seb8

2

2

2

2

2

2

2

14

yes

syb8

1

1

2

2

1

2

2

11

yes

seb9

1

2

2

−1

2

1

2

9

no

syb9

2

2

2

2

2

2

2

14

yes

sec1

2

2

2

−1

2

2

2

11

yes

sycl

1

1

2

2

1

2

2

11

yes

secl0

1

2

2

−1

2

2

2

10

yes

sycl0

−1

1

−1

1

2

−1

2

3

no

secll

−1

1

2

1

2

2

2

9

no

syc1l

1

2

2

2

2

2

2

13

yes

secl2

2

2

2

−1

1

2

2

10

yes

syc12

−1

1

1

1

1

1

2

6

no

sec2

−1

1

2

2

2

2

2

10

yes

syc2

2

2

2

2

2

2

2

14

yes

sec3

1

2

−1

−1

−1

1

2

3

no

syc3

−1

1

2

2

1

2

2

9

yes

sec4

2

2

1

1

2

−1

1

8

no

syc4

2

2

2

1

−1

2

2

10

yes

sec5

2

2

2

−1

2

2

2

11

yes

syc5

−1

1

2

2

1

2

2

9

no

sec6

2

2

2

1

2

2

2

13

yes

syc6

−1

1

−1

2

2

−1

2

4

no

sec7

2

2

2

−1

2

2

2

11

yes

syc7

1

2

2

2

2

2

2

13

yes

sec8

2

2

2

1

1

2

2

12

yes

syc8

1

2

2

1

1

2

2

11

yes

sec9

2

2

1

1

1

1

2

10

yes

syc9

−1

−1

−1

2

2

−1

2

2

no

A number of observations can be made at this point. First, the total scores derived from these seven evaluations were generally higher in the language stimuli than in the music stimuli. This can be attributed to a greater degree of expertise in noticing incongruities in language, as compared with music, considering that all participants were native English speakers but that most participants had less than 4 years of musical training. Second, no single evaluation carried the weight of the rejection or acceptance decision. Consequently, the evaluations of the stimuli were all given equal weighting in the selection process, since no evaluation stood out as most representative of good congruence condition fitness.

Stimuli were selected in an incremental way, with the aim of obtaining a final stimulus set containing only triplets for which all three stimuli showed good congruence condition fitness. The following systematic procedure was adopted to ensure the selection of the best 24 stimulus triplets in each stimulus type (harmony, rhythm, semantics, and syntax).
  1. 1.

    Reject stimulus triplets with an overall fail score.

     
  2. 2.

    From the remaining triplets, reject those that fail in one of the congruence conditions.

     
  3. (a)

    If this brings the number of triplets below 24, keep the stimuli with the highest overall score, preferring those with similar subtotal scores to those with a strong fail in one congruence condition.

     
  4. (b)

    If the total number of triplets is still above 24, proceed to the next step.

     
  5. 3.

    From the remaining triplets, keep the stimuli with the highest overall scores, preferring those with the higher total score on the evaluations associated with the congruous condition in the case of a tie.

     

Norming summary

The systematicity of this approach ensured that stimuli were selected neither on the basis of the researcher’s intuitive appraisal of the stimuli, nor on the basis of the theoretical oddity of the target element. Using total scores as a measure of congruence condition fitness, this procedure allowed the selection of the fittest 24 stimulus triplets within harmony, rhythm, semantics, and syntax separately. Although results from the first rating study did show a significant effect for congruence in the expected direction, the strengthening of effects shown in the validation study supports this approach to refining the stimulus set and confirms the concerns relating to the selection of a priori odd stimuli in music and language incongruity research.

Validating the stimulus set

This second experiment replicated the initial rating study, using only the stimuli selected in the norming procedure. Following the observations from the first rating study, and now that confounding stimuli had been removed, it was predicted that the oddity ratings would show a significant increase from congruous to incongruous–resolved and from incongruous–resolved to incongruous–unresolved within harmony, rhythm, semantics, and syntax.

Method

A new opportunity sample of 55 participants containing musicians and nonmusicians was recruited from the University of Leeds community. The sample consisted of 36 females and 19 males, with an age range of 18–61 years (mean age, 23.52 years, SD = 9.69) and a range of 0–20 years of musical training (mean number of years of musical training, 5.50, SD = 6.31). Participants listened to 24 harmony, 24 rhythm, 24 semantics, and 24 syntax stimuli, consisting of 8 stimuli at each level of congruence (congruous, incongruous–resolved, incongruous–unresolved). The same scale as in the initial rating study was used to rate the oddity of each stimulus.

Results

The oddity ratings, z-transformed as in the first rating study, were analyzed in a 3 (congruence: congruous, incongruous–resolved, incongruous–unresolved) × 4 (stimulus type: harmony, rhythm, semantics, syntax) ANOVA in a by-subjects analysis, in keeping with the by-subjects analyses typically carried out in this field of research. Mean oddity scores for each level of these two factors are presented in Fig. 4.
Fig. 4

Mean oddity ratings by stimulus type (harmony, rhythm, semantics, and syntax) and by congruence (congruous, incongruous–resolved, incongruous–unresolved). These ratings were collected using a visual analogue scale where a higher rating indicated more odd and a lower rating indicated less odd. Within each stimulus type, a significant effect of congruence was found, as well as a significant increase in mean oddity rating from congruous to incongruous–resolved and from incongruous–resolved to incongruous–unresolved (all ps < .05)

The overall ANOVA revealed a significant main effect for congruence, F(2, 102) = 586.56, p < .001, η2 = .91, a significant stimulus type × congruence interaction effect, F(6, 306) = 76.07, p < .001, η2 = .60, but no significant main effect of stimulus type, F(6, 153) = 1.32, p = .27, η2 = .03. In line with the one-tailed predictions, planned contrasts showed a significant increase in oddity ratings from congruous to incongruous–resolved, F(1, 51) = 152.27, p < .001, η2 = .75, as well as a significant increase in oddity ratings from incongruous–resolved to incongruous–unresolved, F(1, 51) = 439.39, p < .001, η2 = .90. These are, however, difficult to interpret, in view of the strong interaction observed, calling for separate simple effects analyses within harmony, rhythm, semantics, and syntax.

Within harmony, a significant main effect for congruence was found, F(2, 106) = 45.26, p < .001, η2 = .46. The same simple contrasts were applied, revealing a significant increase in oddity rating at each level of congruence: congruous to incongruous–resolved, F(1, 53) = 13.02, p < .001, η2 = .20, and incongruous–resolved to incongruous–unresolved, F(1, 53) = 32.21, p < .001, η2 = .38.

The simple effects analysis within rhythm also revealed a significant main effect for congruence, F(2, 108) = 10.91, p < .001, η2 = .17. A significant increase in mean oddity rating was also found between each level of congruence: congruous to incongruous–resolved, F(1, 53) = 10.82, p = .002, η2 = .17, and incongruous–resolved to incongruous–unresolved, F(1, 53) = 2.75, p = .05, η2 = .05.

Semantics data showed the strongest effect for congruence, F(2, 106) = 913.05, p < .001, η2 = .95, with a significant increase in oddity ratings at each level of congruence: congruous to incongruous–resolved, F(1, 53) = 259.67, p < .001, η2  = .83, and incongruous–resolved to incongruous–unresolved, F(1, 53) = 750.84, p < .001, η2 = .93.

Finally, the same analysis applied to syntax revealed yet again a strongly significant main effect for congruence, F(2, 106) = 489.89, p < .001, η2 = .90. Once more, simple contrasts showed a significant increase at each step: congruous to incongruous–resolved, F(1, 53) = 140.66, p < .001, η2 = .73, and incongruous–resolved to incongruous–unresolved, F(1, 53) = 275, p < .001, η2 = .84.

In summary, although the trends were of differing strength, harmony, rhythm, semantics, and syntax all showed a significant increase in oddity ratings from congruous to incongruous–resolved and from incongruous–resolved to incongruous–unresolved.

Discussion

The results from this second experiment demonstrate the validity of the experimental manipulation and the reliability of the resulting refined stimulus set. Although the music and language stimuli were equivalent in their construction, the incongruous–unresolved stimuli being nonsensical as a whole across all types of stimuli, participants’ ratings indicated stronger trends in language than in music. This was expected and can be explained by several factors. All participants were native speakers of British English, but not all were highly trained musicians, making the greater expertise in language, as compared with music, a first explanation for this difference. A second explanation lies in the fact that syntactic and semantic constraints are much stronger in language than harmonic and rhythmic constraints are in music (Bod, 2002), making an incongruous sentence “wrong” but an incongruous musical extract “unusual,” rather than wrong. A third explanation lies in the subjective appraisal of music, resulting from this property, with some people finding musical incongruities interesting rather than odd, depending upon exposure and taste. This was demonstrated by a greater standard error of the mean ratings in harmony (0.04) and rhythm (0.04), as compared with semantics (0.02) and syntax (0.03), in the validation study. Nonetheless, since harmony, rhythm, semantics, and syntax all displayed significantly different ratings in each congruence condition, in line with the intention of the experimental manipulation, and strong effect sizes in the analyses of the effects of congruence, this process has achieved its goal of establishing a set of stimuli fit for use in future studies surrounding the effect of incongruities and their resolutions in music and language.

General discussion

The procedures described above were carried out with the aim of establishing a set of stimuli that showed a statistically significant increase of oddity ratings as a function of equivalent manipulations of congruence in all four components of music and language (harmony, rhythm, semantics, and syntax). The data from the validation study confirmed the reliability and congruence condition fitness of the stimuli, displaying a significant increase in oddity ratings from congruous to incongruous–resolved and from incongruous–resolved to incongruous–unresolved in all four types of stimuli in a group of participants of mixed musical ability. Although this increase was stronger in the language stimuli than in the music stimuli, the large effect sizes for congruence in the music stimuli revealed that musical training was not necessary for theoretically incongruous musical elements to be perceived as odd.

Overall, this stimulus development process combined methods used in music psychology and in psycholinguistics. In the former, stimuli have been considered more or less incongruous according to how closely they adhere to the theory and probabilities of Western tonal harmony (Patel et al., 1998). In the latter, stimuli have often been piloted in the form of sentence completion tasks, which ascertain participants’ preferred stimulus continuations, or grammaticality judgment tasks, which ascertain which stimuli are perceived as odd (Van Berkum et al., 2005). Since completion tasks are notoriously difficult to carry out in music, and in view of evidence supporting the equivalence between both types of task (Schmuckler, 1989), the procedure described in this article combined the best of both research worlds to generate a set of reliably odd stimuli.

Future directions

This set of stimuli has formed the basis of a series of studies investigating ERP effects associated with incongruity and resolution processing in music and language, in musicians and in nonmusicians (Featherstone, 2010). It is hoped that this response to the call for a unified approach (Vuust & Kringelbach, 2010) and a more systematic investigation of incongruity processing in music and language (Juslin & Västfjäll, 2008; Steinbeis et al., 2006; Thierry et al., 2008) will lead to significant progress in music and language psychology. To download the full set of validated stimuli as well as the timings necessary for setting up ERP triggers and an instructions file, please visit www.carafeatherstone.co.uk/download/. To request extensions to the stimulus set, please contact the lead author. These stimuli are freely available, but this article should be cited wherever they are referred to and used in research.

Furthermore, by focusing on structural congruence and cognitive processing difficulty, rather than on surface level incongruities such as unexpected changes in prosody, timbre, tempo, and volume, this stimulus set allows researchers to extract similarities across domains in the processing of statistically improbable elements in structured stimuli—a property far from restricted to music and language. Kutas’s (1998) review of language structures concluded by stating that the study of language cognition was useful in understanding structures underlying general cognitive principles of memory access and knowledge organization. Music and language both involve structured elements that unfold in time and are both processed according to systematicities that are, for some, universal (e.g., the existence of phonemes and notes) and, for others, culture specific (e.g., the specific sets of phonemes and notes and their combination rules; McMullen & Saffran, 2004). The same statement therefore applies to the study of music cognition and, to a greater extent, to the parallel study of music and language cognition.

Footnotes

  1. 1.

    Ratings, obtained by calculating the distance in pixels between the beginning of the scale and the location of the participant’s click on the screen, were transformed into z-scores for each participant within each type of stimulus (harmony, rhythm, semantics, and syntax). This allowed the researchers to normalize each participant’s use of the visual analogue scale and eliminate any effects of stimulus type for an unconfounded investigation of congruence × type effects

Notes

Acknowledgements

This research was funded by a +3 Open Competition ESRC Scholarship; special thanks to Naomi Tomkins for being the voice in these stimuli.

References

  1. Abdallah, S. A., & Plumbley, M. D. (2008). Information dynamics and the perception of temporal structure in music. In J. Mayor, N. Ruh, & K. Plunkett (Eds.), Connectionist models of behaviour and cognition II: Proceedings of the 11th Neural Computation and Psychology Workshop (pp. 179–190). Hackensack: World Scientific.Google Scholar
  2. Besson, M., Faita, F., & Requin, J. (1994). Brain waves associated with musical incongruities differ for musicians and non-musicians. Neuroscience Letters, 168, 101–105.CrossRefPubMedGoogle Scholar
  3. Besson, M., & Macar, F. (1987). An event-related potential analysis of incongruity in music and other non-linguistic contexts. Psychophysiology, 24, 14–25.CrossRefPubMedGoogle Scholar
  4. Bod, R. (2002). A unified model of structural organization in language and music. Journal of Artificial Intelligence Research, 17, 289–308.Google Scholar
  5. Bonnaud, V., Gil, R., & Ingrand, P. (2002). Metaphorical and non-metaphorical links: A behavioral and ERP study in young and elderly adults. Neurophysiologie Clinique, 32, 258–268.CrossRefPubMedGoogle Scholar
  6. Bowdle, B. F., & Gentner, D. (2005). The career of metaphor. Psychological Review, 112, 193–216.CrossRefPubMedGoogle Scholar
  7. Brown, S., Martinez, M. J., & Parsons, L. M. (2006). Music and language side by side in the brain: A PET study of the generation of melodies and sentences. European Journal of Neuroscience, 23, 2791–2803.CrossRefPubMedGoogle Scholar
  8. Clark, E., & Clark, H. (1979). When nouns surface as verbs. Language, 55, 767–811.CrossRefGoogle Scholar
  9. Coulson, S., & Van Petten, C. (2002). Conceptual integration and metaphor: An event-related potential study. Memory & Cognition, 30, 958–968.CrossRefGoogle Scholar
  10. Coulson, S., & Van Petten, C. (2007). A special role for the right hemisphere in metaphor comprehension? ERP evidence from hemifield presentation. Brain Research, 1146, 128–145.CrossRefPubMedGoogle Scholar
  11. Ducreux, D., Marsot-Dupuch, K., Lasjaunias, P., Oppenheim, C., & Frédy, D. (2003). Lyrical and musical auditive mental imagery in functional MRI. Journal of Neuroradiology, 30, 18–24.Google Scholar
  12. Featherstone, C. R. (2010). Sounding sparks: Incongruities and aesthetics in music and language. Ph.D. thesis, Unversity of Leeds.Google Scholar
  13. Fedorenko, E., Patel, A. D., Casasanto, D., Winawer, J., & Gibson, E. (2009). Structural integration in language and music: Evidence for a shared system. Memory & Cognition, 37, 1–9.CrossRefGoogle Scholar
  14. Hale, J. (2001). A probabilistic early parser as a psycholinguistic model. In –Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies (pp. 1–8). Pittsburgh, Pennsylvania Association for Computational Linguistics.Google Scholar
  15. Hantz, E. C., Kreilick, K. G., Kananen, W., & Swartz, K. P. (1997). Neural responses to melodic and harmonic closure: An event-related-potential study. Music Perception, 15, 69–98.CrossRefGoogle Scholar
  16. Huron, D. (2006). Sweet anticipation. Cambridge: MIT Press.Google Scholar
  17. Jentschke, S., Koelsch, S., & Friederici, A. D. (2005). Investigating the relationship of music and language in children: Influences of musical training and language impairment. In G. Avanzini, L. Lopez, S. Koelsch, & M. Manjno (Eds.), The neurosciences and music II: From perception to performance (pp. 231–242). New York: New York Academy of Sciences.Google Scholar
  18. Juslin, P. N., & Västfjäll, D. (2008). Emotional responses to music: The need to consider underlying mechanisms. Behavioural and Brain Sciences, 31, 559–621.Google Scholar
  19. Kazmerski, V., Blasko, D., & Dessalegn, B. (2003). ERP and behavioral evidence of individual differences in metaphor comprehension. Memory & Cognition, 31, 673–689.CrossRefGoogle Scholar
  20. Kluender, R., & Kutas, M. (1993). Subjacency as a processing phenomenon. Language and Cognitive Processes, 84, 573–633.CrossRefGoogle Scholar
  21. Koelsch, S., Gunter, T., Schroger, E., & Friederici, A. D. (2003). Processing tonal modulations: An ERP study. Journal of Cognitive Neuroscience, 15, 1149–1159.CrossRefPubMedGoogle Scholar
  22. Koelsch, S., Kasper, E., Sammler, D., Schulze, K., Gunter, T., & Friederici, A. D. (2004). Music, language and meaning: Brain signatures of semantic processing. Nature Neuroscience, 7, 302–307.CrossRefPubMedGoogle Scholar
  23. Koornneef, A. W., & Van Berkum, J. J. A. (2006). On the use of verb-based implicit causality in sentence comprehension: Evidence from self-paced reading and eye tracking. Journal of Memory and Language, 54, 445–465.CrossRefGoogle Scholar
  24. Kutas, M. (1998). Current thinking on language structures. Cahiers de Psychology Cognitive/Current Psychology of Cognition, 17(45), 951–969.Google Scholar
  25. Lee, C. L., & Federmeier, K. D. (2006). To mind the mind: An event-related potential study of word class and semantic ambiguity. Brain Research, 108, 191–202.CrossRefGoogle Scholar
  26. Levitin, D. J., & Menon, V. (2005). The neural locus of temporal structure and expectancies in music: Evidence from functional neuroimaging at 3 tesla. Music Perception, 22, 563–575.CrossRefGoogle Scholar
  27. Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106, 1126–1177.CrossRefPubMedGoogle Scholar
  28. McMullen, E., & Saffran, J. R. (2004). Music and language: A developmental comparison. Music Perception, 21, 289–311.CrossRefGoogle Scholar
  29. Meyer, L. B. (1956). Emotion and meaning in music. Chicago: University of Chicago Press.Google Scholar
  30. Miranda, R. A., & Ullman, M. T. (2007). Double dissociation between rules and memory in music: An event-related potential study. NeuroImage, 38, 331–345.CrossRefPubMedGoogle Scholar
  31. Ortony, A. (1975). Why metaphors are necessary and not just nice. Educational Theory, 25, 45–53.CrossRefGoogle Scholar
  32. Patel, A. D. (2003). Language, music, syntax and the brain. Nature Neuroscience, 6, 674–681.CrossRefPubMedGoogle Scholar
  33. Patel, A. D. (2008). Music, language and the brain. Oxford: Oxford University Press.Google Scholar
  34. Patel, A. D., Gibson, E., Ratner, J., Besson, M., & Holcomb, P. J. (1998). Processing syntactic relations in language and music: An event-related potential study. Journal of Cognitive Neuroscience, 10, 717–733.CrossRefPubMedGoogle Scholar
  35. Piston, W. (1978). Harmony. London: Gollancz.Google Scholar
  36. Poulin-Charronnat, B., Bigand, E., Madurall, F., & Peereman, R. (2006). Musical structure modulates semantic priming in vocal music. Cognition, 94, B67–B78.CrossRefGoogle Scholar
  37. Pynte, J., Besson, M., Robichon, F., & Poli, J. (1996). The time course of metaphor comprehension: An event-related potential study. Brain and Language, 55, 293–316.CrossRefPubMedGoogle Scholar
  38. Schmithorst, V. J. (2005). Separate cortical networks involved in music perception: Preliminary functional MRI evidence for modularity of music processing. NeuroImage, 25, 444–451.CrossRefPubMedGoogle Scholar
  39. Schmuckler, M. A. (1989). Expectation in music: Investigation of melodic and harmonic processes. Music Perception, 7, 109–150.CrossRefGoogle Scholar
  40. Slevc, L. R., Rosenberg, J. C., & Patel, A. D. (2009). Making psycholinguistic musical: Self-paced reading time evidence for shared processing of linguistic and musical syntax. Psychonomic Bulletin & Review, 16, 374–381.CrossRefGoogle Scholar
  41. Steinbeis, N., & Koelsch, S. (2008). Shared neural resources between music and language indicate semantic processing of musical tension-resolution patterns. Cerebral Cortex, 18, 1169–1178.CrossRefPubMedGoogle Scholar
  42. Steinbeis, N., Koelsch, S., & Sloboda, J. A. (2006). The role of harmonic expectancy violations in musical emotions: Evidence from subjective, physiological, and neural responses. Journal of Cognitive Neuroscience, 18, 1380–1393.CrossRefPubMedGoogle Scholar
  43. Tartter, V. C., Gomes, H., Dubrovsky, B., Molholm, S., & Stewart, R. V. (2002). Novel metaphors appear anomalous at least momentarily: Evidence from N400. Brain and Language, 80, 488–509.CrossRefPubMedGoogle Scholar
  44. Tervaniemi, M., Szameitat, A. J., Kruck, S., Schroger, E., Alter, K., De Baene, W., et al. (2006). From air oscillations to music and speech: Functional magnetic resonance imaging evidence for fine-tuned neural networks in audition. Journal of Neuroscience, 26, 8647–8652.CrossRefPubMedGoogle Scholar
  45. Thierry, G., Martin, C. D., Gonzalez-Diaz, V., Rezaie, R., Roberts, N., & Davis, P. M. (2008). Event-related potential characterisation of the Shakespearean functional shift in narrative sentence structure. NeuroImage, 40, 923–931.CrossRefPubMedGoogle Scholar
  46. Van Berkum, J. J. A., Brown, C., Zwitserlood, P., Koojman, V., & Hagoort, P. (2005). Anticipating upcoming words in discource: Evidence from ERPs and reading times. Journal of Experimental Psychology. Learning, Memory, and Cognition, 31, 443–467.CrossRefPubMedGoogle Scholar
  47. Vuust, P., & Kringelbach, M. L. (2010). The pleasure of making sense of music. Interdisciplinary Science Reviews, 35, 166–182.CrossRefGoogle Scholar
  48. Williamson, V. J., Baddeley, A. D., & Hitch, G. J. (2010). Musicians’ and nonmusicians’ short-term memory for verbal and musical sequences: Comparing phonological similarity and pitch proximity. Memory & Cognition, 38, 163–175.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2011

Authors and Affiliations

  • Cara R. Featherstone
    • 1
  • Mitch G. Waterman
    • 1
  • Catriona M. Morrison
    • 1
  1. 1.Institute of Psychological SciencesUniversity of LeedsLeedsUK

Personalised recommendations