Experimental Brain Research

, Volume 198, Issue 2–3, pp 391–402 | Cite as

Spatially congruent visual motion modulates activity of the primary auditory cortex

  • Mikhail ZvyagintsevEmail author
  • Andrey R. Nikolaev
  • Heike Thönnessen
  • Olga Sachs
  • Jürgen Dammers
  • Klaus Mathiak
Research article


We investigated the brain responses to the transitions from the static to moving audiovisual stimuli using magnetoencephalography. The spatially congruent auditory and visual stimuli moved in the same direction whereas the incongruent stimuli moved in the opposite directions. Using dipole modeling we found that the static-to-moving transitions evoked a neural response in the primary auditory cortex bilaterally. The response started about 100 ms after the motion onset from a negative component (mvN1) and lasted during the entire interval of the stimulus motion. The mvN1 component was similar to the classical auditory N1 response to the static sound, but had smaller amplitude and later latency. The coordinates of the mvN1 and N1 dipoles in the primary auditory cortex were also similar. The amplitude of the auditory response to the moving stimuli appears to be sensitive to spatial congruency of the audiovisual motion; it was larger in the incongruent than congruent condition. This is evidence that the moving visual stimuli modulate the early sensory activity in the primary auditory cortex. Such early audiovisual integration may be specific for motion processing.


Motion Audiovisual integration Congruency MEG Dipole fitting Auditory cortex 



This research was supported by the BMBF (BICW, Clinical MEG), the DFG (IRTG 1328, JARA-BRAIN), and EU FP6 (STREP FUGA), and IZKF Biomat (VV N68). K.M. is supported by an AstraZeneca foundation. The authors thank to Frank Boers for his excellent technical assistance and Susanne Leiberg for helpful discussions.


  1. Alais D, Burr D (2004) No direction-specific bimodal facilitation for audiovisual motion detection. Brain Res Cogn Brain Res 19:185–194PubMedCrossRefGoogle Scholar
  2. Alink A, Singer W, Muckli L (2008) Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex. J Neurosci 28:2690–2697PubMedCrossRefGoogle Scholar
  3. Arlinger S, Jerlvall L (1981) Early auditory electric responses to fast amplitude and frequency tone glides. Electroencephalogr Clin Neurophysiol 51:624–631PubMedCrossRefGoogle Scholar
  4. Baumann O, Greenlee MW (2007) Neural correlates of coherent audiovisual motion perception. Cereb Cortex 17:1433–1443PubMedCrossRefGoogle Scholar
  5. Bell A-J, Sejnowski T-J (1995) An information-maximization approach to blind separation and blind deconvolution. Neural Comput 7:1129–1159PubMedCrossRefGoogle Scholar
  6. Besle J, Fort A, Delpuech C, Giard MH (2004) Bimodal speech: early suppressive visual effects in human auditory cortex. Eur J NeuroSci 20:2225–2234PubMedCrossRefGoogle Scholar
  7. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436PubMedCrossRefGoogle Scholar
  8. Bushara OK, Hanakawa T, Immisch I, Toma K, Kansaku K, Hallett M (2002) Neural correlates of cross-modal binding. Nat Neurosci 6:190–195CrossRefGoogle Scholar
  9. Calvert GA, Thesen T (2004) Multisensory integration: methodological approaches and emerging principles in the human brain. J Physiol Paris 98:191–205PubMedCrossRefGoogle Scholar
  10. Dammers J, Schiek M, Pilz K, Boers F, Zvyagintsev M, Zilles K, Mathiak K (2006) Automatic artifact rejection from independent components of magnetoencephalographic data. Biomag 2006. Vancouver, BC, CanadaGoogle Scholar
  11. Djelani T, Pörschmann C, Sahrhage J, Blauert J (2000) An interactive virtual-environment generator for psychoacoustic research II: collection of head related impulse responses and evaluation of auditory localization. ACUSTICA/acta acustica 86:1046–1053Google Scholar
  12. Doniger GM, Foxe JJ, Murray MM, Higgins BA, Snodgrass JG, Schroeder CE, Javitt DC (2000) Activation timecourse of ventral visual stream object-recognition areas: high density electrical mapping of perceptual closure process. J Cogn Neurosci 124:615–621CrossRefGoogle Scholar
  13. Elberling C, Bak C, Kofoed B, Lebech J, Saermark K (1981) Auditory magnetic fields from the human cortex. Influence of stimulus intensity. Scand Audiol 10:203–207PubMedGoogle Scholar
  14. Gondan M, Niederhaus B, Rösler F, Röder B (2005) Multisensory processing in the redundant-target effect: a behavioral and event-related potential study. Percept Psychophys 2005(67):713–726Google Scholar
  15. Hari R, Mäkelä JP (1988) Modification of neuromagnetic responses of the human auditory cortex by masking sounds. Exp Brain Res 71:87–92PubMedCrossRefGoogle Scholar
  16. Hillyard SA, Teder-Salejarvi WA, Munte TF (1998) Temporal dynamics of early perceptual processing. Curr Opin Neurobiol 8:202–210PubMedCrossRefGoogle Scholar
  17. Kaneoke Y (2006) Magnetoencephalography: in search of neural processes for visual motion information. Prog Neurobiol 80:219–240PubMedCrossRefGoogle Scholar
  18. Kayser C, Petkov CI, Logothetis NK (2008) Visual modulation of neurons in auditory cortex. Cereb Cortex 18:1560–1574PubMedCrossRefGoogle Scholar
  19. Kitagawa N, Ichihara S (2002) Hearing visual motion in depth. Nature 416:172–174PubMedCrossRefGoogle Scholar
  20. Klucharev V, Möttönen R, Sams M (2003) Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. Brain Res Cogn Brain Res 18:65–75PubMedCrossRefGoogle Scholar
  21. Mäkelä JP, McEvoy L (1996) Auditory evoked fields to illusory sound source movements. Exp Brain Res 110:446–454PubMedCrossRefGoogle Scholar
  22. Martuzzi R, Murray MM, Michel CM, Thiran JP, Maeder PP, Clarke S, Meuli RA (2007) Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cereb Cortex 17:1672–1679PubMedCrossRefGoogle Scholar
  23. Mathiak K, Hertrich I, Kincses WE, Riecker A, Lutzenberger W, Ackermann H (2003) The right supratemporal plane hears the distance of objects: neuromagnetic correlates of virtual reality. NeuroReport 14:307–311PubMedCrossRefGoogle Scholar
  24. Neukirch M, Hegerl U, Kötitz R, Dorn H, Gallinat J, Herrmann WM (2002) Comparison of the amplitude/intensity function of the auditory evoked N1 m and N1 components. Neuropsychobiology. 45:41–48PubMedCrossRefGoogle Scholar
  25. Orban GA (2008) Higher order visual processing in macaque extrastriate cortex. Physiol Rev 88:59–89PubMedCrossRefGoogle Scholar
  26. Pantev C, Hoke M, Lütkenhöner B, Lehnertz K, Spittka J (1986) Causes of differences in the input-output characteristics of simultaneously recorded auditory evoked magnetic fields and potentials. Audiology 25:263–276PubMedCrossRefGoogle Scholar
  27. Reite M, Zimmerman JT, Zimmerman JE (1981) Magnetic auditory evoked fields: interhemispheric asymmetry. Electroencephalogr Clin Neurophysiol 51:388–392PubMedCrossRefGoogle Scholar
  28. Sanabria D, Spence C, Soto-Faraco S (2007) Perceptual and decisional contributions to audiovisual interactions in the perception of apparent motion: a signal detection study. Cognition 102:299–310PubMedCrossRefGoogle Scholar
  29. Scherg M, Von Cramon D (1985) Two bilateral sources of the late AEP as identified by a spatio-temporal dipole model. Electroencephalogr Clin Neurophysiol 62:32–44PubMedCrossRefGoogle Scholar
  30. Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, ‘unisensory’ processing. Curr Opin Neurobiol 15:454–458PubMedCrossRefGoogle Scholar
  31. Smith KR, Saberi K, Hickok G (2007) An event-related fMRI study of auditory motion perception: no evidence for a specialized cortical system. Brain Res 1150:94–99PubMedCrossRefGoogle Scholar
  32. Soto-Faraco S, Morein-Zamir S, Kingstone A (2005) On audiovisual spatial synergy: the fragility of the phenomenon. Percept Psychophys 67:444–457PubMedGoogle Scholar
  33. Spence C (2007) Audiovisual multisensory integration. Acoust Sci Tech 28:61–70CrossRefGoogle Scholar
  34. Spoor A, Timmer F, Odenthal DW (1969) The evoked auditory response (ear) to intensity modulated and frequency modulated tones and tone bursts. Int J Audiol 8:410–415CrossRefGoogle Scholar
  35. Sreenivas TV, Raykar VC, Raman R (2000) Head related impulse response interpolation for dynamic spatialization. Texas Instruments DSPS fest-2 k, Banglore, IndiaGoogle Scholar
  36. Stekelenburg JJ, Vroomen J (2007) Neural correlates of multisensory integration of ecologically valid audiovisual events. J Cogn Neurosci 19:1964–1973PubMedCrossRefGoogle Scholar
  37. Teder-Sälejärvi WA, Di Russo F, McDonald JJ, Hillyard SA (2005) Effects of spatial congruity on audio-visual multimodal integration. J Cogn Neurosci 17:1396–1409PubMedCrossRefGoogle Scholar
  38. Thornton ARD, Harmer M, Lavoie BA (2007) Selective attention increases the temporal precision of the auditory N100 event-related potential. Hear Res 230:73–79PubMedCrossRefGoogle Scholar
  39. van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci USA 102:1181–1186PubMedCrossRefGoogle Scholar
  40. Wuerger SM, Hofbauer M, Meyer GF (2003) The integration of auditory and visual motion signals at threshold. Percept Psychophys 65:1188–1196PubMedGoogle Scholar
  41. Xiang J, Chuang S, Wilson D, Otsubo H, Pang E, Holowka S, Sharma R, Ochi A, Chitoku S (2002) Sound motion evoked magnetic fields. Clin Neurophysiol 113:1–9PubMedCrossRefGoogle Scholar
  42. Zvyagintsev M, Nikolaev AR, Mathiak KA, Menning H, Hertrich I, Mathiak K (2008) Predictability modulates motor-auditory interactions in self-triggered audio-visual apparent motion. Exp Brain Res 189:289–300PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Mikhail Zvyagintsev
    • 1
    • 2
    Email author
  • Andrey R. Nikolaev
    • 3
  • Heike Thönnessen
    • 1
  • Olga Sachs
    • 1
  • Jürgen Dammers
    • 2
  • Klaus Mathiak
    • 1
    • 4
  1. 1.Department of Psychiatry and PsychotherapyUniversity Hospital Aachen, RWTH Aachen UniversityAachenGermany
  2. 2.Institute of Neuroscience and BiophysicsResearch Center Jülich GmbHJülichGermany
  3. 3.Laboratory for Perceptual DynamicsRIKEN Brain Science InstituteWako-shiJapan
  4. 4.Institute of PsychiatryKing’s College LondonLondonUK

Personalised recommendations