Imaging Cross-Modal Influences in Auditory Cortex

  • Christoph Kayser
  • Christopher I. Petkov
  • Nikos K. Logothetis


Recent studies have made considerable progress in understanding how our brain combines the information from different sensory modalities and much evidence about the cortical regions involved has been provided by functional magnetic resonance imaging. Imaging studies have, for example, shown that cross-modal influences occur already at early stages of auditory cortex. However, given our still limited understanding of the functional organization of human auditory cortex, these results are often to interpret with respect to the exact localization of cross-modal influences. Here we discuss a localization technique, which provides a functional map of individual fields in the auditory cortex of individual subjects. Using high-resolution imaging techniques in an animal model with known organization of auditory cortex, we proved the feasibility of this functional mapping technique and demonstrated its use in localizing cross-modal influences to individual auditory fields. Our results show that cross-modal influences already occur in secondary auditory cortices and increase along the auditory processing hierarchy. While these results provide good evidence that auditory processing can be affected by non-acoustic stimuli very early on, we also discuss the interpretability of these findings with regard to the underlying neuronal activity, which is considerable hampered by the still unknown neural basis of the fMRI signal.


Cerebral Blood Flow Auditory Cortex Bold Signal Bold Response Functional Imaging Study 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the Max Planck Society and the Alexander von Humboldt Foundation.


  1. Alais D, Burr D (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14:257–262PubMedGoogle Scholar
  2. Attwell D, Iadecola C (2002) The neural basis of functional brain imaging signals. Trends Neurosci 25:621–625PubMedCrossRefGoogle Scholar
  3. Bartels A, Logothetis NK, Moutoussis K (2008) fMRI and its interpretations: an illustration on directional selectivity in area V5/MT. Trends Neurosci 31:444–453PubMedCrossRefGoogle Scholar
  4. Benevento LA, Fallon J, Davis BJ, Rezak M (1977) Auditory--visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp Neurol 57:849–872PubMedCrossRefGoogle Scholar
  5. Bernstein LE, Auer ET Jr, Moore JK, Ponton CW, Don M, Singh M (2002) Visual speech perception without primary auditory cortex activation. Neuroreport 13:311–315PubMedCrossRefGoogle Scholar
  6. Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46:369–384PubMedGoogle Scholar
  7. Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110–1123PubMedCrossRefGoogle Scholar
  8. Calvert GA, Campbell R (2003) Reading speech from still and moving faces: the neural substrates of visible speech. J Cogn Neurosci 15:57–70PubMedCrossRefGoogle Scholar
  9. Calvert GA, Brammer MJ, Bullmore ET, Campbell R, Iversen SD, David AS (1999) Response amplification in sensory-specific cortices during crossmodal binding. Neuroreport 10:2619–2623PubMedCrossRefGoogle Scholar
  10. Calvert GA, Bullmore ET, Brammer MJ, Campbell R, Williams SC, McGuire PK, Woodruff PW, Iversen SD, David AS (1997) Activation of auditory cortex during silent lipreading. Science 276:593–596PubMedCrossRefGoogle Scholar
  11. Chiry O, Tardif E, Magistretti PJ, Clarke S (2003) Patterns of calcium-binding proteins support parallel and hierarchical organization of human auditory areas. Eur J Neurosci 17:397–410PubMedCrossRefGoogle Scholar
  12. Clarke S, Rivier F (1998) Compartments within human primary auditory cortex: evidence from cytochrome oxidase and acetylcholinesterase staining. Eur J Neurosci 10:741–745PubMedCrossRefGoogle Scholar
  13. Crivello F, Schormann T, Tzourio-Mazoyer N, Roland PE, Zilles K, Mazoyer BM (2002) Comparison of spatial normalization procedures and their impact on functional maps. Hum Brain Mapp 16:228–250PubMedCrossRefGoogle Scholar
  14. Desai R, Liebenthal E, Possing ET, Waldron E, Binder JR (2005) Volumetric vs. surface-based alignment for localization of auditory cortex activation. Neuroimage 26:1019–1029PubMedCrossRefGoogle Scholar
  15. Driver J, Spence C (1998) Crossmodal attention. Curr Opin Neurobiol 8:245–253PubMedCrossRefGoogle Scholar
  16. Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57:11–23PubMedCrossRefGoogle Scholar
  17. Engel SA, Rumelhart DE, Wandell BA, Lee AT, Glover GH, Chichilnisky EJ, Shadlen MN (1994) fMRI of human visual cortex. Nature 369:525PubMedCrossRefGoogle Scholar
  18. Felleman DJ, van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:1–47PubMedCrossRefGoogle Scholar
  19. Fergus A, Lee KS (1997) GABAergic regulation of cerebral microvascular tone in the rat. J Cereb Blood Flow Metab 17:992–1003PubMedCrossRefGoogle Scholar
  20. Formisano E, Kim DS, Di Salle F, van de Moortele PF, Ugurbil K, Goebel R (2003) Mirror-symmetric tonotopic maps in human primary auditory cortex. Neuron 40:859–869PubMedCrossRefGoogle Scholar
  21. Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16:419–423PubMedCrossRefGoogle Scholar
  22. Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540–543PubMedGoogle Scholar
  23. Fullerton BC, Pandya DN (2007) Architectonic analysis of the auditory-related areas of the superior temporal region in human brain. J Comp Neurol 504:470–498PubMedCrossRefGoogle Scholar
  24. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278–285PubMedCrossRefGoogle Scholar
  25. Goense J, Logothetis N (2008) Neurophysiology of the BOLD fMRI signal in awake monkeys. Curr Biol In PressGoogle Scholar
  26. Hackett TA, Stepniewska I, Kaas JH (1998) Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. J Comp Neurol 394:475–495PubMedCrossRefGoogle Scholar
  27. Hackett TA, Preuss TM, Kaas JH (2001) Architectonic identification of the core region in auditory cortex of macaques, chimpanzees, and humans. J Comp Neurol 441:197–222PubMedCrossRefGoogle Scholar
  28. Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289–293PubMedCrossRefGoogle Scholar
  29. Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60:1615–1637PubMedGoogle Scholar
  30. Hyvarinen J, Shelepin Y (1979) Distribution of visual and somatic functions in the parietal associative area 7 of the monkey. Brain Res 169:561–564PubMedCrossRefGoogle Scholar
  31. Jones EG, Powell TP (1970) An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 93:793–820PubMedCrossRefGoogle Scholar
  32. Kayser C, Logothetis NK (2007) Do early sensory cortices integrate cross-modal information? Brain Struct Funct 212Google Scholar
  33. Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48:373–384PubMedCrossRefGoogle Scholar
  34. Kayser C, Petkov CI, Augath M, Logothetis NK (2007) Functional imaging reveals visual modulation of specific fields in auditory cortex. J Neurosci 27:1824–1835PubMedCrossRefGoogle Scholar
  35. Kikuchi Y, Rauschecker JP, Mishkin M, Augath M, Logothetis N, Petkov C (2008) Voice region connectivity in the monkey assessed with microstimulation and functional imaging. In: Program No. 850.2. 2008 Neuroscience meeting planner. Society for Neuroscience, 2008, Washington, DC. OnlineGoogle Scholar
  36. Kosaki H, Hashikawa T, He J, Jones EG (1997) Tonotopic organization of auditory cortical fields delineated by parvalbumin immunoreactivity in macaque monkeys. J Comp Neurol 386:304–316PubMedCrossRefGoogle Scholar
  37. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279–292PubMedCrossRefGoogle Scholar
  38. Lakatos P, Karmos G, Mehta AD, Ulbert I, Schroeder CE (2008) Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320:110–113PubMedCrossRefGoogle Scholar
  39. Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp Brain Res 166:298–297CrossRefGoogle Scholar
  40. Lauritzen M (2005) Reading vascular changes in brain imaging: is dendritic calcium the key? Nat Rev Neurosci 6:77–85PubMedCrossRefGoogle Scholar
  41. Lauritzen M, Gold L (2003) Brain function and neurophysiological correlates of signals used in functional neuroimaging. J Neurosci 23:3972–3980PubMedGoogle Scholar
  42. Lehmann C, Herdener M, Esposito F, Hubl D, di Salle F, Scheffler K, Bach DR, Federspiel A, Kretz R, Dierks T, Seifritz E (2006) Differential patterns of multisensory interactions in core and belt areas of human auditory cortex. Neuroimage 31:294–300PubMedCrossRefGoogle Scholar
  43. Lehmann S, Murray MM (2005) The role of multisensory memories in unisensory object discrimination. Brain Res Cogn Brain Res 24:326–334PubMedCrossRefGoogle Scholar
  44. Logothetis NK (2002) The neural basis of the blood-oxygen-level-dependent functional magnetic resonance imaging signal. Philos Trans R Soc Lond B Biol Sci 357:1003–1037PubMedCrossRefGoogle Scholar
  45. Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature 453:869–878PubMedCrossRefGoogle Scholar
  46. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A (2001) Neurophysiological investigation of the basis of the fMRI signal. Nature 412:150–157PubMedCrossRefGoogle Scholar
  47. Lutkenhoner B, Lammertmann C, Simoes C, Hari R (2002) Magnetoencephalographic correlates of audiotactile interaction. Neuroimage 15:509–522PubMedCrossRefGoogle Scholar
  48. Martuzzi R, Murray MM, Michel CM, Thiran JP, Maeder PP, Clarke S, Meuli RA (2006) Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cereb Cortex 17:1672–1679PubMedCrossRefGoogle Scholar
  49. Mathiesen C, Caesar K, Lauritzen M (2000) Temporal coupling between neuronal activity and blood flow in rat cerebellar cortex as indicated by field potential analysis. J Physiol 523 Pt 1:235–246CrossRefGoogle Scholar
  50. Mathiesen C, Caesar K, Akgoren N, Lauritzen M (1998) Modification of activity-dependent increases of cerebral blood flow by excitatory synaptic activity and spikes in rat cerebellar cortex. J Physiol 512(Pt 2):555–566PubMedCrossRefGoogle Scholar
  51. McDonald JJ, Teder-Salejarvi WA, Hillyard SA (2000) Involuntary orienting to sound improves visual perception. Nature 407:906–908PubMedCrossRefGoogle Scholar
  52. Merzenich MM, Brugge JF (1973) Representation of the cochlear partition of the superior temporal plane of the macaque monkey. Brain Res 50:275–296PubMedCrossRefGoogle Scholar
  53. Morel A, Garraghty PE, Kaas JH (1993) Tonotopic organization, architectonic fields, and connections of auditory cortex in macaque monkeys. J Comp Neurol 335:437–459PubMedCrossRefGoogle Scholar
  54. Mukamel R, Gelbard H, Arieli A, Hasson U, Fried I, Malach R (2005) Coupling between neuronal firing, field potentials, and FMRI in human auditory cortex. Science 309:951–954PubMedCrossRefGoogle Scholar
  55. Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963–974PubMedCrossRefGoogle Scholar
  56. Niessing J, Ebisch B, Schmidt KE, Niessing M, Singer W, Galuske RA (2005) Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science 309:948–951PubMedCrossRefGoogle Scholar
  57. Nir Y, Fisch L, Mukamel R, Gelbard-Sagiv H, Arieli A, Fried I, Malach R (2007) Coupling between neuronal firing rate, gamma LFP, and BOLD fMRI is related to interneuronal correlations. Curr Biol 17:1275–1285PubMedCrossRefGoogle Scholar
  58. Norup Nielsen A, Lauritzen M (2001) Coupling and uncoupling of activity-dependent increases of neuronal activity and blood flow in rat somatosensory cortex. J Physiol 533:773–785PubMedCrossRefGoogle Scholar
  59. Pekkola J, Ojanen V, Autti T, Jaaskelainen IP, Mottonen R, Sams M (2005a) Attention to visual speech gestures enhances hemodynamic activity in the left planum temporale. Hum Brain Mapp 27:471–477CrossRefGoogle Scholar
  60. Pekkola J, Ojanen V, Autti T, Jaaskelainen IP, Mottonen R, Tarkiainen A, Sams M (2005b) Primary auditory cortex activation by visual speech: an fMRI study at 3 T. Neuroreport 16:125–128PubMedCrossRefGoogle Scholar
  61. Petkov C, Kayser C, Ghazanfar AA, Patterson RD, Logothetis N (2008a) Functional imaging of sensitivity to components of the voice in monkey auditory cortex. In: Program No. 851.19. 2008 Neuroscience meeting planner. Society for Neuroscience, Washington, DC. OnlineGoogle Scholar
  62. Petkov C, Kayser C, Steudel T, Whittingstall K, Augath M, Logothetis N (2008b) A voice region in the monkey brain. Nat Neurosci 11:367–374PubMedCrossRefGoogle Scholar
  63. Petkov CI, Kayser C, Augath M, Logothetis NK (2006) Functional imaging reveals numerous fields in the monkey auditory cortex. PLOS Biol 4:e215PubMedCrossRefGoogle Scholar
  64. Rademacher J, Caviness VS, Jr., Steinmetz H, Galaburda AM (1993) Topographical variation of the human primary cortices: implications for neuroimaging, brain mapping, and neurobiology. Cereb Cortex 3:313–329PubMedCrossRefGoogle Scholar
  65. Rauschecker JP (1998) Cortical processing of complex sounds. Curr Opin Neurobiol 8:516–521PubMedCrossRefGoogle Scholar
  66. Rauschecker JP, Tian B (2004) Processing of band-passed noise in the lateral auditory belt cortex of the rhesus monkey. J Neurophysiol 91:2578–2589PubMedCrossRefGoogle Scholar
  67. Rauschecker JP, Tian B, Pons T, Mishkin M (1997) Serial and parallel processing in rhesus monkey auditory cortex. J Comp Neurol 382:89–103PubMedCrossRefGoogle Scholar
  68. Recanzone GH, Guard DC, Phan ML (2000) Frequency and intensity response properties of single neurons in the auditory cortex of the behaving macaque monkey. J Neurophysiol 83:2315–2331PubMedGoogle Scholar
  69. Rivier F, Clarke S (1997) Cytochrome oxidase, acetylcholinesterase, and NADPH-diaphorase staining in human supratemporal and insular cortex: evidence for multiple auditory areas. Neuroimage 6:288–304PubMedCrossRefGoogle Scholar
  70. Schroeder CE, Molholm S, Lakatos P, Ritter W, Foxe JJ (2004) Human–simian correspondence in the early cortical processing of multisensory cues. Cogn Process 5:140–151CrossRefGoogle Scholar
  71. Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A (2008) Neuronal oscillations and visual amplification of speech. Trends Cogn Sci 12:106–113PubMedCrossRefGoogle Scholar
  72. Schurmann M, Caetano G, Hlushchuk Y, Jousmaki V, Hari R (2006) Touch activates human auditory cortex. Neuroimage 30:1325–1331PubMedCrossRefGoogle Scholar
  73. Seitz AR, Kim R, Shams L (2006) Sound facilitates visual learning. Curr Biol 16:1422–1427PubMedCrossRefGoogle Scholar
  74. Shmuel A, Augath M, Oeltermann A, Logothetis NK (2006) Negative functional MRI response correlates with decreases in neuronal activity in monkey visual area V1. Nat Neurosci 9:569–577PubMedCrossRefGoogle Scholar
  75. Stein BE, Meredith MA (1993) Merging of the senses. MIT Press, CambridgeGoogle Scholar
  76. Sumby WH, Polack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212–215CrossRefGoogle Scholar
  77. Talavage TM, Sereno MI, Melcher JR, Ledden PJ, Rosen BR, Dale AM (2004) Tonotopic organization in human auditory cortex revealed by progressions of frequency sensitivity. J Neurophysiol 91:1282–1296PubMedCrossRefGoogle Scholar
  78. Thomsen K, Offenhauser N, Lauritzen M (2004) Principal neuron spiking: neither necessary nor sufficient for cerebral blood flow in rat cerebellum. J Physiol 560:181–189PubMedCrossRefGoogle Scholar
  79. van Atteveldt N, Formisano E, Goebel R, Blomert L (2004) Integration of letters and speech sounds in the human brain. Neuron 43:271–282PubMedCrossRefGoogle Scholar
  80. van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci U S A 102:1181–1186PubMedCrossRefGoogle Scholar
  81. Vroomen J, de Gelder B (2000) Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform 26:1583–1590PubMedCrossRefGoogle Scholar
  82. Warnking J, Dojat M, Guerin-Dugue A, Delon-Martin C, Olympieff S, Richard N, Chehikian A, Segebarth C (2002) fMRI retinotopic mapping--step by step. Neuroimage 17:1665–1683PubMedCrossRefGoogle Scholar
  83. Wessinger CM, Buonocore MH, Kussmaul CL, Mangun GR (1997) Tonotopy in human auditory cortex examined with functional magnetic resonance imaging. Neuroimage 5:18–25Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2010

Authors and Affiliations

  • Christoph Kayser
    • 1
  • Christopher I. Petkov
    • 1
    • 2
  • Nikos K. Logothetis
    • 1
    • 3
  1. 1.Max Planck Institute for Biological CyberneticsTübingenGermany
  2. 2.Institute of NeuroscienceUniversity of NewcastleNewcastle upon TyneUK
  3. 3.Division of Imaging Science and Biomedical EngineeringUniversity of ManchesterManchesterUK

Personalised recommendations