Skip to main content
SpringerLink
Log in

Musicians have enhanced audiovisual multisensory binding: experience-dependent effects in the double-flash illusion

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

Musical training is associated with behavioral and neurophysiological enhancements in auditory processing for both musical and nonmusical sounds (e.g., speech). Yet, whether the benefits of musicianship extend beyond enhancements to auditory-specific skills and impact multisensory (e.g., audiovisual) processing has yet to be fully validated. Here, we investigated multisensory integration of auditory and visual information in musicians and nonmusicians using a double-flash illusion, whereby the presentation of multiple auditory stimuli (beeps) concurrent with a single visual object (flash) induces an illusory perception of multiple flashes. We parametrically varied the onset asynchrony between auditory and visual events (leads and lags of ±300 ms) to quantify participants’ “temporal window” of integration, i.e., stimuli in which auditory and visual cues were fused into a single percept. Results show that musically trained individuals were both faster and more accurate at processing concurrent audiovisual cues than their nonmusician peers; nonmusicians had a higher susceptibility for responding to audiovisual illusions and perceived double flashes over an extended range of onset asynchronies compared to trained musicians. Moreover, temporal window estimates indicated that musicians’ windows (<100 ms) were ~2–3× shorter than nonmusicians’ (~200 ms), suggesting more refined multisensory integration and audiovisual binding. Collectively, findings indicate a more refined binding of auditory and visual cues in musically trained individuals. We conclude that experience-dependent plasticity of intensive musical experience extends beyond simple listening skills, improving multimodal processing and the integration of multiple sensory systems in a domain-general manner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Anvari SH, Trainor LJ, Woodside J, Levy BA (2002) Relations among musical skills, phonological processing and early reading ability in preschool children. J Exp Child Psychol 83:111–130

    Article  PubMed  Google Scholar 

  • Besson M, Chobert J, Marie C (2011) Transfer of training between music and speech: common processing, attention, and memory. Front Psychol 2:94

    Article  PubMed  PubMed Central  Google Scholar 

  • Bidelman GM (2013) The role of the auditory brainstem in processing musically-relevant pitch. Front Psychol 4:1–13

    Article  Google Scholar 

  • Bidelman GM, Alain C (2015) Musical training orchestrates coordinated neuroplasticity in auditory brainstem and cortex to counteract age-related declines in categorical vowel perception. J Neurosci 35:1240–1249

    Article  CAS  PubMed  Google Scholar 

  • Bidelman GM, Krishnan A (2010) Effects of reverberation on brainstem representation of speech in musicians and non-musicians. Brain Res 1355:112–125

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Bidelman GM, Gandour JT, Krishnan A (2011a) Cross-domain effects of music and language experience on the representation of pitch in the human auditory brainstem. J Cogn Neurosci 23:425–434

    Article  PubMed  Google Scholar 

  • Bidelman GM, Krishnan A, Gandour JT (2011b) Enhanced brainstem encoding predicts musicians’ perceptual advantages with pitch. Eur J Neurosci 33:530–538

    Article  PubMed  Google Scholar 

  • Bidelman GM, Hutka S, Moreno S (2013) Tone language speakers and musicians share enhanced perceptual and cognitive abilities for musical pitch: evidence for bidirectionality between the domains of language and music. PLoS ONE 8:e60676

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Bidelman GM, Schug JM, Jennings SG, Bhagat SP (2014a) Psychophysical auditory filter estimates reveal sharper cochlear tuning in musicians. J Acoust Soc Am 136:EL33–EL39

    Article  PubMed  Google Scholar 

  • Bidelman GM, Weiss MW, Moreno S, Alain C (2014b) Coordinated plasticity in brainstem and auditory cortex contributes to enhanced categorical speech perception in musicians. Eur J Neurosci 40:2662–2673

    Article  PubMed  Google Scholar 

  • Brandler S, Rammsayer TH (2003) Differences in mental abilities between musicians and nonmusicians. Psychol Music 31:123–138

    Article  Google Scholar 

  • Chartrand JP, Belin P (2006) Superior voice timbre processing in musicians. Neurosci Lett 405:164–167

    Article  CAS  PubMed  Google Scholar 

  • Cooper A, Wang Y (2012) The influence of linguistic and musical experience on Cantonese word learning. J Acoust Soc Am 131:4756–4769

    Article  PubMed  Google Scholar 

  • DeLoss DJ, Pierce RS, Andersen GJ (2013) Multisensory integration, aging, and the sound-induced flash illusion. Psychol Aging 28:802–812

    Article  PubMed  PubMed Central  Google Scholar 

  • Diederich A, Colonius H, Schomburg A (2008) Assessing age-related multisensory enhancement with the time-window-of-integration model. Neuropsychologia 46:2556–2562

    Article  PubMed  Google Scholar 

  • Elbert T, Pantev C, Wienbruch C, Rockstroh B, Taub E (1996) Increased cortical representation of the fingers of the left hand in string players. Science 270:305–307

    Article  Google Scholar 

  • Erber NP (1975) Auditory-visual perception of speech. J Speech Hear Disord 40:481–492

    Article  CAS  PubMed  Google Scholar 

  • Foss AH, Altschuler EL, James KH (2007) Neural correlates of the Pythagorean ratio rules. NeuroReport 18:1521–1525

    Article  PubMed  Google Scholar 

  • Foss-Feig JH, Kwakye LD, Cascio CJ, Burnette CP, Kadivar H, Stone WL, Wallace MT (2010) An extended multisensory temporal binding window in autism spectrum disorders. Exp Brain Res 203:381–389

    Article  PubMed  PubMed Central  Google Scholar 

  • Foster NE, Zatorre RJ (2010) A role for the intraparietal sulcus in transforming musical pitch information. Cereb Cortex 20:1350–1359

    Article  PubMed  Google Scholar 

  • Fritz J, Shamma S, Elhilali M, Klein D (2003) Rapid task-related plasticity of spectrotemporal receptive fields in primary auditory cortex. Nat Neurosci 6:1216–1223

    Article  CAS  PubMed  Google Scholar 

  • George EM, Coch D (2011) Music training and working memory: an ERP study. Neuropsychologia 49:1083–1094

    Article  PubMed  Google Scholar 

  • Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490

    Article  CAS  PubMed  Google Scholar 

  • Grahn JA, Rowe JB (2009) Feeling the beat: premotor and striatal interactions in musicians and nonmusicians during beat perception. J Neurosci 29:7540–7548

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Herholz SC, Zatorre RJ (2012) Musical training as a framework for brain plasticity: behavior, function, and structure. Neuron 76:486–502

    Article  CAS  PubMed  Google Scholar 

  • Ho Y, Cheung M, Chan A (2003) Music training improves verbal but not visual memory: cross sectional and longitudinal explorations in children. Neuropsychology 17:439–450

    Article  PubMed  Google Scholar 

  • Kaganovich N, Schumaker J, Leonard LB, Gustafson D, Macias D (2014) Children with a history of SLI show reduced sensitivity to audiovisual temporal asynchrony: an ERP study. J Speech Lang Hear Res 57:1480–1502

    Article  PubMed  PubMed Central  Google Scholar 

  • Kaposvari P, Csete G, Bognar A, Csibri P, Toth E, Szabo N, Vecsei L, Sary G, Kincses ZT (2015) Audio-visual integration through the parallel visual pathways. Brain Res 1624:71–77

    Article  CAS  PubMed  Google Scholar 

  • Kraus N, Chandrasekaran B (2010) Music training for the development of auditory skills. Nat Rev Neurosci 11:599–605

    Article  CAS  PubMed  Google Scholar 

  • Kraus N, Slater J, Thompson EC, Hornickel J, Strait DL, Nicol T, White-Schwoch T (2014) Music enrichment programs improve the neural encoding of speech in at-risk children. J Neurosci 34:11913–11918

    Article  CAS  PubMed  Google Scholar 

  • Lappe C, Herholz SC, Trainor LJ, Pantev C (2008) Cortical plasticity induced by short-term unimodal and multimodal musical training. J Neurosci 28:9632–9639

    Article  CAS  PubMed  Google Scholar 

  • Laurienti PJ, Burdette JH, Maldjian JAW, Wallace MT (2006) Enhanced multisensory integration in older adults. Neurobiol Aging 27:1155–1163

    Article  PubMed  Google Scholar 

  • Lee H, Noppeney U (2011) Long-term music training tunes how the brain temporally binds signals from multiple senses. Proc Natl Acad Sci USA 108:E1441–E1450

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Lee HL, Noppeney U (2014) Music expertise shapes audiovisual temporal integration windows for speech, sinewave speech and music. Front Psychol 5:1–9

    Google Scholar 

  • Lu Y, Paraskevopoulos E, Herholz SC, Kuchenbuch A, Pantev C (2014) Temporal processing of audiovisual stimuli is enhanced in musicians: evidence from magnetoencephalography (MEG). PLoS ONE 9:e90686

    Article  PubMed  PubMed Central  Google Scholar 

  • McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748

    Article  CAS  PubMed  Google Scholar 

  • Mishra J, Gazzaley A (2012) Attention distributed across sensory modalities enhances perceptual performance. J Neurosci 32:12294–12302

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Mishra J, Martinez A, Sejnowski TJ, Hillyard SA (2007) Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. J Neurosci 27:4120–4131

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Mishra J, Martinez A, Hillyard SA (2008) Cortical processes underlying sound-induced flash fusion. Brain Res 1242:102–115

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Moreno S, Bidelman GM (2014) Understanding neural plasticity and cognitive benefit through the unique lens of musical training. Hear Res 308:84–97

    Article  PubMed  Google Scholar 

  • Musacchia G, Sams M, Skoe E, Kraus N (2007) Musicians have enhanced subcortical auditory and audiovisual processing of speech and music. Proc Natl Acad Sci USA 104:15894–15898

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Myers EB, Swan K (2012) Effects of category learning on neural sensitivity to non-native phonetic categories. J Cogn Neurosci 24:1695–1708

    Article  PubMed  PubMed Central  Google Scholar 

  • Neufeld J, Sinke C, Zedler M, Emrich HM, Szycik GR (2012) Reduced audio–visual integration in synaesthetes indicated by the double-flash illusion. Brain Res 1473:78–86

    Article  CAS  PubMed  Google Scholar 

  • Novembre G, Keller PE (2014) A conceptual review on action-perception coupling in the musicians’ brain: what is it good for? Front Human Neurosci 8:603

    Article  Google Scholar 

  • Pallesen KJ, Brattico E, Bailey CJ, Korvenoja A, Koivisto J, Gjedde A, Carlson S (2010) Cognitive control in auditory working memory is enhanced in musicians. PLoS ONE 5:e11120

    Article  PubMed  PubMed Central  Google Scholar 

  • Pantev C, Roberts LE, Schulz M, Engelien A, Ross B (2001) Timbre-specific enhancement of auditory cortical representations in musicians. NeuroReport 12:169–174

    Article  CAS  PubMed  Google Scholar 

  • Paraskevopoulos E, Kuchenbuch A, Herholz SC, Pantev C (2012) Evidence for training-induced plasticity in multisensory brain structures: an MEG study. PLoS ONE 7:e36534

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Paraskevopoulos E, Kraneburg A, Herholz SC, Bamidis PD, Pantev C (2015) Musical expertise is related to altered functional connectivity during audiovisual integration. Proc Natl Acad Sci USA 112:12522–12527

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Parbery-Clark A, Skoe E, Kraus N (2009a) Musical experience limits the degradative effects of background noise on the neural processing of sound. J Neurosci 29:14100–14107

    Article  CAS  PubMed  Google Scholar 

  • Parbery-Clark A, Skoe E, Lam C, Kraus N (2009b) Musician enhancement for speech-in-noise. Ear Hear 30:653–661

    Article  PubMed  Google Scholar 

  • Powers AR, Hillock AR, Wallace MT (2009) Perceptual training narrows the temporal window of multisensory binding. J Neurosci 29:12265–12274

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Rammsayer T, Buttkus F, Altenmüller E (2012) Musicians do better than non-musicians in both auditory and visual timing tasks. Music Percept 30:85–96

    Article  Google Scholar 

  • Rosenthal O, Shimojo S, Shams L (2009) Sound-induced flash illusion is resistant to feedback training. Brain Topogr 21:185–192

    Article  PubMed  PubMed Central  Google Scholar 

  • Shahin A, Bosnyak DJ, Trainor LJ, Roberts LE (2003) Enhancement of neuroplastic P2 and N1c auditory evoked potentials in musicians. J Neurosci 23:5545–5552

    CAS  PubMed  Google Scholar 

  • Shams L, Kamitani Y, Shimojo S (2000) What you see is what you hear. Nature 408:788

    Article  CAS  PubMed  Google Scholar 

  • Shams L, Kamitani Y, Shimojo S (2002) Visual illusion induced by sound. Cogn Brain Res 14:147–152

    Article  Google Scholar 

  • Slevc RL, Miyake A (2006) Individual differences in second-language proficiency: does musical ability matter? Psychol Sci 17:675–681

    Article  PubMed  Google Scholar 

  • Stevenson RA, Wilson MM, Powers AR, Wallace MT (2013) The effects of visual training on multisensory temporal processing. Exp Brain Res 225:479–489

    Article  PubMed  PubMed Central  Google Scholar 

  • Stevenson RA, Siemann JK, Schneider BC, Eberly HE, Woynaroski TG, Camarata SM, Wallace MT (2014) Multisensory temporal integration in autism spectrum disorders. J Neurosci 34:691–697

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Strait DL, Kraus N (2011) Can you hear me now? Musical training shapes functional brain networks for selective auditory attention and hearing speech in noise. Front Psychol 2:113

    Article  PubMed  PubMed Central  Google Scholar 

  • Strait DL, Kraus N, Parbery-Clark A, Ashley R (2010) Musical experience shapes top-down auditory mechanisms: evidence from masking and auditory attention performance. Hear Res 261:22–29

    Article  PubMed  Google Scholar 

  • Sumby WH, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212–215

    Article  Google Scholar 

  • Tierney AT, Bergeson TR, Pisoni DB (2008) Effects of early musical experience on auditory sequence memory. Empir Musicol Rev 3:178–186

    PubMed  PubMed Central  Google Scholar 

  • van Vugt FT, Tillmann B (2014) Thresholds of auditory-motor coupling measured with a simple task in musicians and non-musicians: was the sound simultaneous to the key press? PLoS ONE 9:e87176

    Article  PubMed  PubMed Central  Google Scholar 

  • Vatikiotis-Bateson E, Eigsti I-M, Yano S, Munhall KG (1998) Eye movement of perceivers during audiovisual speech perception. Percept Psychophys 60:926–940

    Article  CAS  PubMed  Google Scholar 

  • Wallace MT, Stevenson RA (2014) The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities. Neuropsychologia 64C:105–123

    Article  Google Scholar 

  • Wong PC, Perrachione TK (2007) Learning pitch patterns in lexical identification by native English-speaking adults. Appl Psycholinguist 28:565–585

    Article  Google Scholar 

  • Wong PC, Skoe E, Russo NM, Dees T, Kraus N (2007) Musical experience shapes human brainstem encoding of linguistic pitch patterns. Nat Neurosci 10:420–422

    CAS  PubMed  PubMed Central  Google Scholar 

  • Zatorre R, McGill J (2005) Music, the food of neuroscience? Nature 434:312–315

    Article  CAS  PubMed  Google Scholar 

  • Zatorre RJ, Chen JL, Penhune VB (2007) When the brain plays music: auditory-motor interactions in music perception and production. Nat Rev Neurosci 8:547–558

    Article  CAS  PubMed  Google Scholar 

  • Zendel BR, Alain C (2009) Concurrent sound segregation is enhanced in musicians. J Cogn Neurosci 21:1488–1498

    Article  PubMed  Google Scholar 

  • Zuk J, Benjamin C, Kenyon A, Gaab N (2014) Behavioral and neural correlates of executive functioning in musicians and non-musicians. PLoS ONE 9:e99868

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

The author thanks Haley Sanders for assistance in data collection and Amy Fehrenbach for comments on earlier versions of this manuscript. This work was supported in part by a grant from the GRAMMY® Foundation awarded to G.M.B.

Author information

Authors and Affiliations

  1. Institute for Intelligent Systems, University of Memphis, Memphis, TN, USA

    Gavin M. Bidelman

  2. School of Communication Sciences and Disorders, University of Memphis, 4055 North Park Loop, Memphis, TN, 38152, USA

    Gavin M. Bidelman

Authors
  1. Gavin M. Bidelman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gavin M. Bidelman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bidelman, G.M. Musicians have enhanced audiovisual multisensory binding: experience-dependent effects in the double-flash illusion. Exp Brain Res 234, 3037–3047 (2016). https://doi.org/10.1007/s00221-016-4705-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-016-4705-6

Keywords

Search

Navigation