Cortical Oscillations and Multisensory Interactions in Humans

  • Jochen KaiserEmail author
  • Marcus J. Naumer


Oscillatory activity in the higher (gamma) frequency range in electro- or magnetoencephalogram (EEG or MEG, respectively) has been proposed as a correlate of cortical network synchronization. This chapter reviews the evidence for an involvement of gamma-band activity in multisensory integration. The relevant studies are grouped by the level at which oscillatory EEG or MEG data were analyzed. The first section describes reports of topographically unspecific or widespread spectral activity. The second part focuses on studies demonstrating topographically local activity both at the electrode/sensor level and at the cortical source level. Finally, reports of cortico-cortical interactions as suggested by EEG/MEG coherence or phase synchrony are summarized. Gamma-band activity increases have been reported for temporally, spatially, or semantically matching bimodal stimulation. Some studies, however, have reported enhanced oscillatory signals also to incongruent stimuli, suggesting an important role of task requirements and attentional factors.


Oscillatory Activity Multisensory Integration Audiovisual Speech Audiovisual Stimulus Bimodal Stimulus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functional imaging of human crossmodal identification and object recognition. Exp Brain Res 166:559–571PubMedCrossRefGoogle Scholar
  2. Bauer M, Oostenveld R, Fries P (2009) Tactile stimulation accelerates behavioral responses to visual stimuli through enhancement of occipital gamma-band activity. Vision Res 49:931–942PubMedCrossRefGoogle Scholar
  3. Bauer M, Oostenveld R, Peeters M, Fries P (2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduces low-frequency activity in parieto-occipital areas. J Neurosci 26:490–501PubMedCrossRefGoogle Scholar
  4. Beauchamp MS (2005) See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol 15:145–153PubMedCrossRefGoogle Scholar
  5. Beauchamp MS, Lee KE, Argall BD, Martin A (2004) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809–823PubMedCrossRefGoogle Scholar
  6. Bhattacharya J, Shams L, Shimojo S (2002) Sound-induced illusory flash perception: role of gamma band responses. Neuroreport 13:1727–1730PubMedCrossRefGoogle Scholar
  7. Busch NA, Herrmann CS, Müller MM, Lenz D, Gruber T (2006) A cross-laboratory study of event-related gamma activity in a standard object recognition paradigm. Neuroimage 33:1169–1177PubMedCrossRefGoogle Scholar
  8. Doehrmann O, Naumer MJ (2008) Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res 1242:136–150PubMedCrossRefGoogle Scholar
  9. Doehrmann O, Naumer MJ, Volz S, Kaiser J, Altmann CF (2008) Probing category selectivity for environmental sounds in the human auditory brain. Neuropsychologia 46:2776–2786PubMedCrossRefGoogle Scholar
  10. Doesburg SM, Emberson LL, Rahi A, Cameron D, Ward LM (2008) Asynchrony from synchrony: long-range gamma-band neural synchrony accompanies perception of audiovisual speech asynchrony. Exp Brain Res 185:11–20PubMedCrossRefGoogle Scholar
  11. Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57:11–23PubMedCrossRefGoogle Scholar
  12. Eckhorn R, Bauer R, Jordan W, Brosch M, Kruse W, Munk M, Reitboeck HJ (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biol Cybern 60:121–130PubMedCrossRefGoogle Scholar
  13. Engel AK, Singer W (2001) Temporal binding and the neural correlates of sensory awareness. Trends Cogn Sci 5:16–25PubMedCrossRefGoogle Scholar
  14. Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540–543PubMedGoogle Scholar
  15. Fries P (2005) A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci 9:474–480PubMedCrossRefGoogle Scholar
  16. Fries P, Nikolic D, Singer W (2007) The gamma cycle. Trends Neurosci 30:309–316PubMedCrossRefGoogle Scholar
  17. Fries P, Roelfsema PR, Engel AK, König P, Singer W (1997) Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proc Natl Acad Sci U S A 94:12699–12704PubMedCrossRefGoogle Scholar
  18. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278–285PubMedCrossRefGoogle Scholar
  19. Ghazanfar AA, Chandrasekaran C, Logothetis NK (2008) Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. J Neurosci 28:4457–4469PubMedCrossRefGoogle Scholar
  20. Gray CM, König P, Engel AK, Singer W (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338:334–337PubMedCrossRefGoogle Scholar
  21. Gruber T, Müller MM, Keil A, Elbert T (1999) Selective visual-spatial attention alters induced gamma band responses in the human EEG. Clin Neurophysiol 110:2074–2085PubMedCrossRefGoogle Scholar
  22. Gruber T, Tsivilis D, Montaldi D, Müller MM (2004) Induced gamma band responses: an early marker of memory encoding and retrieval. Neuroreport 15:1837–1841PubMedCrossRefGoogle Scholar
  23. Hein G, Doehrmann O, Muller NG, Kaiser J, Muckli L, Naumer MJ (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881–7887PubMedCrossRefGoogle Scholar
  24. Herrmann CS, Munk MH, Engel AK (2004) Cognitive functions of gamma-band activity: memory match and utilization. Trends Cogn Sci 8:347–355PubMedCrossRefGoogle Scholar
  25. Hummel F, Gerloff C (2005) Larger interregional synchrony is associated with greater behavioral success in a complex sensory integration task in humans. Cereb Cortex 15:670–678PubMedCrossRefGoogle Scholar
  26. Jensen O, Kaiser J, Lachaux JP (2007) Human gamma-frequency oscillations associated with attention and memory. Trends Neurosci 30:317–324PubMedCrossRefGoogle Scholar
  27. Jokisch D, Jensen O (2007) Modulation of gamma and alpha activity during a working memory task engaging the dorsal or ventral stream. J Neurosci 27:3244–3251PubMedCrossRefGoogle Scholar
  28. Kaiser J, Lutzenberger W, Ackermann H, Birbaumer N (2002) Dynamics of gamma-band activity induced by auditory pattern changes in humans. Cereb Cortex 12:212–221PubMedCrossRefGoogle Scholar
  29. Kaiser J, Ripper B, Birbaumer N, Lutzenberger W (2003) Dynamics of gamma-band activity in human magnetoencephalogram during auditory pattern working memory. Neuroimage 20:816–827PubMedCrossRefGoogle Scholar
  30. Kaiser J, Hertrich I, Ackermann H, Lutzenberger W (2006) Gamma-band activity over early sensory areas predicts detection of changes in audiovisual speech stimuli. Neuroimage 30:1376–1382PubMedCrossRefGoogle Scholar
  31. Kaiser J, Lutzenberger W, Preissl H, Ackermann H, Birbaumer N (2000) Right-hemisphere dominance for the processing of sound-source lateralization. J Neurosci 20:6631–6639PubMedGoogle Scholar
  32. Kaiser J, Hertrich I, Ackermann H, Mathiak K, Lutzenberger W (2005) Hearing lips: gamma-band activity during audiovisual speech perception. Cereb Cortex 15:646–653PubMedCrossRefGoogle Scholar
  33. Kaiser J, Heidegger T, Wibral M, Altmann CF, Lutzenberger W (2008) Distinct gamma-band components reflect the short-term memory maintenance of different sound lateralization angles. Cereb Cortex 18:2286–2295PubMedCrossRefGoogle Scholar
  34. Kanayama N, Ohira H (2009) Multisensory processing and neural oscillatory responses: separation of visuotactile congruency effect and corresponding electroencephalogram activities. Neuroreport 20:289–293PubMedCrossRefGoogle Scholar
  35. Kanayama N, Sato A, Ohira H (2007) Crossmodal effect with rubber hand illusion and gamma-band activity. Psychophysiology 44:392–402PubMedCrossRefGoogle Scholar
  36. Kayser C, Petkov CI, Logothetis NK (2008) Visual modulation of neurons in auditory cortex. Cereb Cortex 18:1560–1574PubMedCrossRefGoogle Scholar
  37. Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48:373–384PubMedCrossRefGoogle Scholar
  38. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279–292PubMedCrossRefGoogle Scholar
  39. Lewis JW, Wightman FL, Brefczynski JA, Phinney RE, Binder JR, DeYoe EA (2004) Human brain regions involved in recognizing environmental sounds. Cereb Cortex 14:1008–1021PubMedCrossRefGoogle Scholar
  40. Lutzenberger W, Pulvermüller F, Elbert T, Birbaumer N (1995) Visual stimulation alters local 40-Hz responses in humans: an EEG-study. Neurosci Lett 183:39–42PubMedCrossRefGoogle Scholar
  41. Lutzenberger W, Ripper B, Busse L, Birbaumer N, Kaiser J (2002) Dynamics of gamma-band activity during an audiospatial working memory task in humans. J Neurosci 22:5630–5638PubMedGoogle Scholar
  42. Maier JX, Chandrasekaran C, Ghazanfar AA (2008) Integration of bimodal looming signals through neuronal coherence in the temporal lobe. Curr Biol 18:963–968PubMedCrossRefGoogle Scholar
  43. McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748PubMedCrossRefGoogle Scholar
  44. Merabet LB, Swisher JD, McMains SA, Halko MA, Amedi A, Pascual-Leone A, Somers DC (2007) Combined activation and deactivation of visual cortex during tactile sensory processing. J Neurophysiol 97:1633–1641PubMedCrossRefGoogle Scholar
  45. Meredith MA (2002) On the neuronal basis for multisensory convergence: a brief overview. Cogn Brain Res 14:31–40CrossRefGoogle Scholar
  46. Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389–391PubMedCrossRefGoogle Scholar
  47. Mesulam MM (1998) From sensation to cognition. Brain 121:1013–1052PubMedCrossRefGoogle Scholar
  48. Mishra J, Martinez A, Sejnowski TJ, Hillyard SA (2007) Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. J Neurosci 27:4120–4131PubMedCrossRefGoogle Scholar
  49. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn Brain Res 14:115–128CrossRefGoogle Scholar
  50. Müller MM, Keil A (2004) Neuronal synchronization and selective color processing in the human brain. J Cogn Neurosci 16:503–522PubMedCrossRefGoogle Scholar
  51. Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ (2008) The effect of prior visual information on recognition of speech and sounds. Cereb Cortex 18:598–609PubMedCrossRefGoogle Scholar
  52. Osipova D, Takashima A, Oostenveld R, Fernandez G, Maris E, Jensen O (2006) Theta and gamma oscillations predict encoding and retrieval of declarative memory. J Neurosci 26:7523–7531PubMedCrossRefGoogle Scholar
  53. Sakowitz OW, Quiroga RQ, Schurmann M, Basar E (2001) Bisensory stimulation increases gamma-responses over multiple cortical regions. Cogn Brain Res 11:267–279CrossRefGoogle Scholar
  54. Sakowitz OW, Quian Quiroga R, Schurmann M, Basar E (2005) Spatio-temporal frequency characteristics of intersensory components in audiovisually evoked potentials. Cogn Brain Res 23:316–326CrossRefGoogle Scholar
  55. Schneider TR, Debener S, Oostenveld R, Engel AK (2008) Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming. Neuroimage 42:1244–1254PubMedCrossRefGoogle Scholar
  56. Schroeder CE, Smiley J, Fu KG, McGinnis T, O'Connell MN, Hackett TA (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50:5–17PubMedCrossRefGoogle Scholar
  57. Senkowski D, Talsma D, Herrmann CS, Woldorff MG (2005) Multisensory processing and oscillatory gamma responses: effects of spatial selective attention. Exp Brain Res 166:411–426PubMedCrossRefGoogle Scholar
  58. Senkowski D, Molholm S, Gomez-Ramirez M, Foxe JJ (2006) Oscillatory beta activity predicts response speed during a multisensory audiovisual reaction time task: a high-density electrical mapping study. Cereb Cortex 16:1556–1565PubMedCrossRefGoogle Scholar
  59. Senkowski D, Schneider TR, Foxe JJ, Engel AK (2008) Crossmodal binding through neural coherence: implications for multisensory processing. Trends Neurosci 31:401–409PubMedCrossRefGoogle Scholar
  60. Senkowski D, Schneider TR, Tandler F, Engel AK (2009) Gamma-band activity reflects multisensory matching in working memory. Exp Brain Res 198:363–372PubMedCrossRefGoogle Scholar
  61. Senkowski D, Talsma D, Grigutsch M, Herrmann CS, Woldorff MG (2007) Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia 45:561–571PubMedCrossRefGoogle Scholar
  62. Shams L, Kamitani Y, Shimojo S (2000) What you see is what you hear. Nature 408:788PubMedCrossRefGoogle Scholar
  63. Singer W, Engel AK, Kreiter A, Munk MHJ, Neuenschwander S, Roelfsema PR (1997) Neuronal assemblies: necessity, signature and detectability. Trends Cogn Sci 1:252–261PubMedCrossRefGoogle Scholar
  64. Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Cambridge, MAGoogle Scholar
  65. Tallon-Baudry C, Bertrand O, Delpuech C, Pernier J (1996) Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J Neurosci 16:4240–4249PubMedGoogle Scholar
  66. Tallon-Baudry C, Bertrand O, Peronnet F, Pernier J (1998) Induced gamma-band activity during the delay of a visual short-term memory task in humans. J Neurosci 18:4244–4254PubMedGoogle Scholar
  67. Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80:1006–1010PubMedGoogle Scholar
  68. Widmann A, Gruber T, Kujala T, Tervaniemi M, Schroger E (2007) Binding symbols and sounds: evidence from event-related oscillatory gamma-band activity. Cereb Cortex 17:2696–2702PubMedCrossRefGoogle Scholar
  69. Womelsdorf T, Schoffelen JM, Oostenveld R, Singer W, Desimone R, Engel AK, Fries P (2007) Modulation of neuronal interactions through neuronal synchronization. Science 316:1609–1612PubMedCrossRefGoogle Scholar
  70. Wyart V, Tallon-Baudry C (2008) Neural dissociation between visual awareness and spatial attention. J Neurosci 28:2667–2679PubMedCrossRefGoogle Scholar
  71. Yuval-Greenberg S, Deouell LY (2007) What you see is not (always) what you hear: induced gamma band responses reflect cross-modal interactions in familiar object recognition. J Neurosci 27:1090–1096PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2010

Authors and Affiliations

  1. 1.Institute of Medical Psychology, Faculty of MedicineGoethe UniversityFrankfurt am MainGermany

Personalised recommendations