The Neural Mechanism of Audiovisual Integration Modulated by Attention: A DCM Study

  • Yang Xi
  • Mengchao Zhang
  • Ning Gao
  • Yue Li
  • Lin Liu
  • Qi LiEmail author
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 109)


Attention plays an important role in audiovisual integration. To further explore the neural mechanism of audiovisual integration when the participants attended and unattended the stimulus, we designed a visual and auditory discrimination task that included ‘A stimuli’, ‘V stimuli’ and ‘AV stimuli’ presented on the right or left side of screen randomly. We adopt DCM method to model audiovisual integration in attended and unattended condition to investigate the neural mechanism of audiovisual interaction in these two conditions. The results showed that the audiovisual interaction in attention-condition was stronger than that in inattention-condition, which reflected the influence of attention on multisensory integration.


Audiovisual integration Attention DCM 


  1. 1.
    Fort, A., Delpuech, C., Pernier, J., Giard, M.H.: Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Res. Cogn. Brain Res. 14, 20–30 (2002)CrossRefGoogle Scholar
  2. 2.
    Giard, M.H., Peronnet, F.: Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11, 473–490 (1999)CrossRefGoogle Scholar
  3. 3.
    Molholm, S., Ritter, W., Murray, M.M.: Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res. Cogn. Brain Res. 14, 115–128 (2002)CrossRefGoogle Scholar
  4. 4.
    Teder-Salejarvi, W.A., Di Russo, F., McDonald, J.J., Hillyard, S.A.: Effects of spatial congruity on audio-visual multimodal integration. J. Cogn. Neurosci. 17, 1396–1409 (2005)CrossRefGoogle Scholar
  5. 5.
    Teder-Salejarvi, W.A., McDonald, J.J., Di Russo, F., Hillyard, S.A.: An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res. Cogn. Brain Res. 14, 106–114 (2002)CrossRefGoogle Scholar
  6. 6.
    Vidal, J., Giard, M.H., Roux, S.: Cross-modal processing of auditory-visual stimuli in a no-task paradigm: a topographic event-related potential study. Clin. Neurophysiol. 119, 763–771 (2008)CrossRefGoogle Scholar
  7. 7.
    Friston, K., Harrison, L., Penny, W.: Dynamic causal modelling. NeuroImage 19, 273–1302 (2003)Google Scholar
  8. 8.
    Friston, K.: Causal modelling and brain connectivity in functional magnetic resonance maging. PLoS Biol. 7, e1000033 (2009)CrossRefGoogle Scholar
  9. 9.
    Werner, S., Noppeney, U.: Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J. Neurosci. Official J. Soc. Neurosci 30(7), 2662–2675 (2010)CrossRefGoogle Scholar
  10. 10.
    Ursino, M., Cuppini, C., Magosso, E.: Multisensory Bayesian inference depends on synapse maturation during training: theoretical analysis and neural modeling implementation. Neural Comput. 29(3), 735–782 (2017)CrossRefGoogle Scholar
  11. 11.
    Driver, J., Noesselt, T.: Multisensory interplay reveals cross-modal influences on “sensory-specific” brain regions, neural responses, and judgments. Neuron 57(1), 11–23 (2008)CrossRefGoogle Scholar
  12. 12.
    Falchier, A., Clavagnier, S., Barone, P., Kennedy, H.: Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22(13), 5749–5759 (2002)CrossRefGoogle Scholar
  13. 13.
    Ghazanfar, A.A., Schroeder, C.E.: Is neocortex essentially multisensory? Trends Cogn. Sci. 10(6), 278–285 (2006)CrossRefGoogle Scholar
  14. 14.
    Rockland, K.S., Ojima, H.: Multisensory convergence in calcarine visual areas in macaque monkey. Int. J. Psychophysiol. 50(1–2), 19–26 (2003)CrossRefGoogle Scholar
  15. 15.
    Schroeder, C.E., Foxe, J.: Multisensory contributions to low-level, “unisensory” processing. Curr. Opin. Neurobiol. 15(4), 454–458 (2005)CrossRefGoogle Scholar
  16. 16.
    King, A.J., Walker, K.M.: Integrating information from different senses in the auditory cortex. Biol. Cybern. 106, 617–625 (2012)CrossRefGoogle Scholar
  17. 17.
    Musacchia, G., Schroeder, C.E.: Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hear. Res. 258, 72–79 (2009)CrossRefGoogle Scholar
  18. 18.
    Ursino, M., Cuppini, C., Magosso, E.: A neural network for learning the meaning of objects and words from a featural representation. Neural Netw. (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yang Xi
    • 1
    • 2
  • Mengchao Zhang
    • 3
  • Ning Gao
    • 1
  • Yue Li
    • 1
  • Lin Liu
    • 3
  • Qi Li
    • 1
    Email author
  1. 1.School of Computer Science and TechnologyChangchun University of Science and TechnologyChangchunPeople’s Republic of China
  2. 2.College of Information EngineeringNortheast Electric Power UniversityJilinPeople’s Republic of China
  3. 3.Department of RadiologyChina-Japan Union Hospital of Jilin UniversityChangchunPeople’s Republic of China

Personalised recommendations