Experimental Brain Research

, Volume 166, Issue 3–4, pp 411–426 | Cite as

Multisensory processing and oscillatory gamma responses: effects of spatial selective attention

  • Daniel Senkowski
  • Durk Talsma
  • Christoph S. Herrmann
  • Marty G. WoldorffEmail author
Research Article


Here we describe an EEG study investigating the interactions between multisensory (audio-visual) integration and spatial attention, using oscillatory gamma-band responses (GBRs). The results include a comparison with previously reported event-related potential (ERP) findings from the same paradigm. Unisensory-auditory (A), unisensory-visual (V), and multisensory (AV) stimuli were presented to the left and right hemispaces while subjects attended to a designated side to detect deviant target stimuli in either sensory modality. For attended multisensory stimuli we observed larger evoked GBRs approximately 40–50 ms post-stimulus over medial-frontal brain areas compared with those same multisensory stimuli when unattended. Further analysis indicated that the integration effect and its attentional enhancement may be caused in part by a stimulus-triggered phase resetting of ongoing gamma-band responses. Interestingly, no such early interaction effects (<90 ms) could be found in the ERP waveforms, suggesting that oscillatory GBRs may be more sensitive than ERPs to these early latency attention effects. Moreover, no GBR attention effects could be found for the unisensory auditory or unisensory visual stimuli, suggesting that attention particularly affects the integrative processing of audiovisual stimuli at these early latencies.


Audiovisual Binding Bimodal EEG ERP 



We express our appreciation to Maren Grigutsch for software development and helpful comments on the manuscript and to Tineke Grent-’t Jong for her technical assistance. This study was supported by NIH grants R01 MH 64015 and P01 NS41328 (Project No. 2) to Marty Woldorff. Daniel Senkowski was funded by the German Research Foundation (DFG, grant HE3353/1) and the Max Planck Society.


  1. Başar E, Başar-Eroglu C, Karakaş S, Schürmann M (2001) Gamma, alpha, delta, and theta oscillations govern cognitive processes. Int J Psychophysiol 39:241–248CrossRefGoogle Scholar
  2. Bertrand O, Tallon-Baudry C (2000) Oscillatory gamma activity in humans: a possible role for object representation. Int J Psychophysiol 38:211–223CrossRefPubMedGoogle Scholar
  3. Bhattacharya J, Shams L, Shimojo S (2002) Sound-induced illusory flash perception: role of gamma band responses. Neuroreport 13:1727–1730CrossRefPubMedGoogle Scholar
  4. Brandt ME (1997) Visual and auditory evoked phase resetting of the alpha EEG. Int J Psychophysiol 26:285f–298fCrossRefGoogle Scholar
  5. Buckner RL, Goodman J, Burock M, Rotte M, Koutstaal W, Schacter D, Rosen B, Dale AM (1998) Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI. Neuron 20:285–296CrossRefPubMedGoogle Scholar
  6. Burock MA, Buckner RL, Woldorff MG, Rosen BR, Dale AM (1998) Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport 9:3735–3739PubMedGoogle Scholar
  7. Busch NA, Debener S, Kranczioch C, Engel AK, Herrmann CS (2004) Size matters: effects of stimulus size, duration and eccentricity on the visual gamma-band response. Clin Neurophysiol 115:1810–1820CrossRefPubMedGoogle Scholar
  8. Busse L, Woldorff MG (2003) Omitted stimulus response and its implications for fast-rate event-related fMRI designs. Neuroimage 18:856–864CrossRefPubMedGoogle Scholar
  9. Callan D, Callan AM, Kroos C, Vatikiotis-Bateson E (2001) Multimodal contribution to speech perception revealed by independent component analysis: a single-sweep EEG case study. Brain Res Cogn Brain Res 10:349–353Google Scholar
  10. Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649–657CrossRefPubMedGoogle Scholar
  11. Eimer M, van Velzen J, Driver J (2002) Cross-modal interactions between audition, touch, and vision in endogenous spatial attention: ERP evidence on preparatory states and sensory modulations. Psychophysiology 39:437–449CrossRefPubMedGoogle Scholar
  12. Fort A, Delpuech C, Pernier J, Giard MH (2002) Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb Cortex 12:1031–1039PubMedGoogle Scholar
  13. Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490PubMedGoogle Scholar
  14. Gruber T, Müller MM, Keil A, Elbert T (1999) Selective visual-spatial attention alters induced gamma band responses in the human EEG. Clin Neurophysiol 110:2074–2085CrossRefPubMedGoogle Scholar
  15. Hackley SA, Woldorff M, Hillyard SA (1990) Cross-modal selective attention effects on retinal, myogenic, brainstem, and cerebral evoked potentials. Psychophysiology 27:195–208PubMedGoogle Scholar
  16. Haig AR, De Pascalis V, Gordon E (1999) Peak gamma latency correlated with reaction time in a conventional oddball paradigm. Clin Neurophysiol 110:158–165CrossRefPubMedGoogle Scholar
  17. Hawkins HL, Hillyard SA, Luck SJ, Mouloua M, Downing CJ, Woodward DP (1990) Visual attention modulates signal detectability. J Exp Psychol Hum Percept Perform 16:802–811CrossRefPubMedGoogle Scholar
  18. Herrmann CS, Knight RT (2001) Mechanisms of human attention: event-related potentials and oscillations. Neurosci Biobehav Rev 25:465–476CrossRefPubMedGoogle Scholar
  19. Herrmann CS, Mecklinger A (2001) Gamma activity in human EEG is related to high-speed memory comparisons during object selective attention. Vis Cogn 8:593–608CrossRefGoogle Scholar
  20. Herrmann CS, Mecklinger A, Pfeifer E (1999) Gamma responses and ERPs in a visual classification task. Clin Neurophysiol 110:636–642CrossRefPubMedGoogle Scholar
  21. Herrmann CS, Munk MH, Engel AK (2004) Cognitive functions of gamma-band activity: memory match and utilization. Trends Cogn Sci 8:347–355CrossRefPubMedGoogle Scholar
  22. Hillyard SA, Vogel EK, Luck SJ (1999) Sensory gain control (amplification) as a mechanism of selective attention: electrophysiological and neuroimaging evidence. In: Humphreys GW, Duncan J, Treisman A (eds) Attention, space and action. Oxford University Press, New York, pp 31–53Google Scholar
  23. Kaiser J, Lutzenberger W (2003) Induced gamma-band activity and human brain function. Neuroscientist 9:475–484CrossRefPubMedGoogle Scholar
  24. Kaiser J, Hertrich I, Ackermann H, Mathiak K, Lutzenberger W (2004) Hearing lips: gamma-band activity during audiovisual speech perception. Cereb Cortex 23:551–560Google Scholar
  25. Karakaş S, Başar E (1998) Early gamma response is sensory in origin: a conclusion based on cross-comparison of results from multiple experimental paradigms. Int J Psychophysiol 31:13–31CrossRefPubMedGoogle Scholar
  26. Karakaş S, Erzengin OU, Başar E (2000) The genesis of human event-related responses explained through the theory of oscillatory neural assemblies. Neurosci Lett 285:45–48CrossRefPubMedGoogle Scholar
  27. Keil A, Müller MM, Gruber T, Wienbruch C, Elbert T (2001) Human large-scale oscillatory brain activity during an operant shaping procedure. Brain Res Cogn Brain Res 12:397–407CrossRefPubMedGoogle Scholar
  28. Kolev V, Yordanova Y, Başar E (1998) Phase locking of oscillatory responses an informative approach for studying evoked brain activity. In: Başar E (ed) Brain function and oscillations. I. brain oscillations: principles and approaches. Springer, Berlin Heidelberg New York, pp 123–128Google Scholar
  29. Macaluso E, Frith CD, Driver J (2002) Supramodal effects of covert spatial orienting triggered by visual or tactile events. J Cogn Neurosci 14:382–401CrossRefGoogle Scholar
  30. Makeig S, Westerfield M, Jung TP, Enghoff S, Townsend J, Courchesne E, Sejnowski TJ (2002) Dynamic brain sources of visual evoked responses. Science 295:690–694Google Scholar
  31. Mangun GR, Hillyard SA (1991) Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. J Exp Psychol Hum Percept Perform 17:1057–1074CrossRefPubMedGoogle Scholar
  32. Mardia KV, Jupp PE (2000) Directional statistics. Willey, Chichester New YorkGoogle Scholar
  33. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115–128PubMedGoogle Scholar
  34. Müller MM, Gruber T, Keil A (2000) Modulation of induced gamma band activity in the human EEG by attention and visual information processing. Int J Psychophysiol 38:283–299CrossRefPubMedGoogle Scholar
  35. Posada A, Hugues E, Franck N, Vianin P, Kilner J (2003) Augmentation of induced visual gamma activity by increased task complexity. Eur J Neurosci 18:2351–2356CrossRefPubMedGoogle Scholar
  36. Quian Quiroga R, Kraskov A, Kreuz T, Grassberger P (2002) Performance of different synchronization measures in real data: a case study on electroencephalographic signals. Phys Rev E 66:041904CrossRefGoogle Scholar
  37. Raij T, Uutela K, Hari R (2000) Audiovisual integration of letters in the human brain. Neuron 28:617–625CrossRefPubMedGoogle Scholar
  38. Sakowitz OW, Quiroga RQ, Schürmann M, Başar E (2001) Bisensory stimulation increases gamma-responses over multiple cortical regions. Brain Res Cogn Brain Res 11:267–279CrossRefPubMedGoogle Scholar
  39. Schroeder CE, Foxe JJ (2002) The timing of laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14:187–198CrossRefPubMedGoogle Scholar
  40. Schroeder CE, Smiley J, Fu KG, McGinnis T, O’Connel MN, Hackett TA (2003) Anatomical and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiology 18:2351–2356Google Scholar
  41. Senkowski D, Herrmann CS (2002) Effects of task difficulty on evoked gamma activity and ERPs in a visual discrimination task. Clin Neurophysiol 113:1742–1753CrossRefPubMedGoogle Scholar
  42. Sokolov A, Pavlova M, Lutzenberger W, Birbaumer N (2004) Reciprocal modulation of neuromagnetic induced gamma activity by attention in the human visual and auditory cortex. Neuroimage 22:521–529CrossRefPubMedGoogle Scholar
  43. Strüber D, Başar-Eroglu C, Hoff E, Stadler M (2000) Reversal-rate dependent differences in the EEG gamma-band during multistable visual perception. Int J Psychophysiol 38:243–252CrossRefPubMedGoogle Scholar
  44. Tallon-Baudry C, Bertrand O (1999) Oscillatory gamma activity in humans and its role in object representation. Trends Cogn Sci 3:151–162CrossRefPubMedGoogle Scholar
  45. Tallon-Baudry C, Bertrand O, Delpuech C, Pernier J (1996) Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J Neurosci 16:4240–4249PubMedGoogle Scholar
  46. Tallon-Baudry C, Bertrand O, Peronnet F, Pernier J (1998) Induced gamma-band activity during the delay of a visual short-term memory task in humans. J Neurosci 18:4244–4254PubMedGoogle Scholar
  47. Talsma D, Kok A (2001) Nonspatial intermodal selective attention is mediated by sensory brain areas: evidence from event-related potentials. Psychophysiology 38:736–751CrossRefPubMedGoogle Scholar
  48. Talsma D, Kok A (2002) Intermodal spatial attention differs between vision and audition: an event-related potential analysis. Psychophysiology 39:689–706CrossRefPubMedGoogle Scholar
  49. Talsma D, Woldorff MG (2005a) Methods for the estimation and removal of artifacts and overlap in ERP waveforms. In: Handy T (ed) Event-related potentials: a methods handbook. MIT press, Cambridge MAGoogle Scholar
  50. Talsma D, Woldorff MG (2005b) Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. J Cogn Neurosci 17:1098−1114CrossRefPubMedGoogle Scholar
  51. Teder-Sälejärvi WA, McDonald JJ, Di Russo F, Hillyard SA (2002) An analysis of audiovisual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res Cogn Brain Res 14:106–114CrossRefPubMedGoogle Scholar
  52. Tiitinen H, Sinkkonen J, Reinikainen K, Alho K, Lavikainen J, Näätänen R (1993) Selective attention enhances the auditory 40-Hz transient response in humans. Nature 364:59–60Google Scholar
  53. Wallace MT, Stein BE (1997) Development of multisensory neurons and multisensory integration in cat superior colliculus. J Neurosci 17:2429–2444PubMedGoogle Scholar
  54. Wallace MT, Stein BE (2001) Sensory and multisensory responses in newborn monkey superior colliculus. J Neurosci 21:8886–8894PubMedGoogle Scholar
  55. Woldorff MG (1993) Distortion of ERP averages due to overlap from temporally adjacent ERPs: analysis and correction. Psychophysiology 30:98–119PubMedGoogle Scholar
  56. Woldorff MG, Hillyard SA (1991) Modulation of early auditory processing during selective listening to rapidly presented tones. Electroenceph Clin Neurophysiol 79:170–191CrossRefPubMedGoogle Scholar
  57. Woldorff M, Hansen JC, Hillyard SA (1987) Evidence for effects of selective attention in the mid-latency range of the human auditory event-related potential. Electroenceph Clin Neurophysiol 40(Suppl):146–154Google Scholar
  58. Woldorff MG, Gallen CC, Hampson SA, Hillyard SA, Pantev C, Sobel D, Bloom FE (1993) Modulation of early sensory processing in human auditory cortex during auditory selective attention. Proc Natl Acad Sci USA 90:8722–8726PubMedGoogle Scholar
  59. Woldorff MG, Fox P, Matzke M, Lancaster JL, Veeraswamy S, Zamarripa F, Seabolt M, Glass T, Gao JH, Martin CC, Jerabek P (1997) Retinotopic organization of early visual spatial attention effects as revealed by PET and ERP. Hum Brain Mapp 5:280–286CrossRefGoogle Scholar
  60. Woldorff MG, Liotti M, Seabolt M, Busse JL, Lancaster JL, Fox PT (2002) The temporal dynamics of the effects in occipital cortex of visual-spatial selective attention. Brain Res Cogn Brain Res 15:1–15CrossRefPubMedGoogle Scholar
  61. Woldorff MG, Hazlett CJ, Fichtenholtz MH, Weissman DH, Dale AM, Song AW (2004) Functional parcellation of attentional control regions of the brain. J Cogn Neurosci 16:149–165CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2005

Authors and Affiliations

  • Daniel Senkowski
    • 1
    • 2
    • 3
  • Durk Talsma
    • 1
    • 4
  • Christoph S. Herrmann
    • 5
  • Marty G. Woldorff
    • 1
    Email author
  1. 1.Center for Cognitive NeuroscienceDuke UniversityDurhamUSA
  2. 2.Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
  3. 3.Nathan Kline Institute for Psychiatric ResearchThe Cognitive Neurophysiology LaboratoryOrangeburgUSA
  4. 4.Clinical NeuropsychologyVrije Universiteit AmsterdamThe Netherlands
  5. 5.Institute for PsychologyOtto von Guericke University of MagdeburgGermany

Personalised recommendations