Skip to main content
Log in

Auditory facilitation of visual-target detection persists regardless of retinal eccentricity and despite wide audiovisual misalignments

Experimental Brain Research Aims and scope Submit manuscript

Abstract

It is well established that sounds can enhance visual-target detection, but the mechanisms that govern these cross-sensory effects, as well as the neural pathways involved, are largely unknown. Here, we tested behavioral predictions stemming from the neurophysiologic and neuroanatomic literature. Participants detected near-threshold visual targets presented either at central fixation or peripheral to central fixation that were sometimes paired with sounds that originated from widely misaligned locations (up to 104° from the visual target). Our results demonstrate that co-occurring sounds improve the likelihood of visual-target detection (1) regardless of retinal eccentricity and (2) despite wide audiovisual misalignments. With regard to the first point, these findings suggest that auditory facilitation of visual-target detection is unlikely to operate through previously described corticocortical pathways from auditory cortex that predominantly terminate in regions of visual cortex that process peripheral visual space. With regard to the second point, auditory facilitation of visual-target detection seems to operate through a spatially non-specific modulation of visual processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

References

  • Arndt PA, Colonius H (2003) Multimodal visual-somatosensory integration in saccadic integration: evidence from a visual-auditory focused attention task. Exp Brain Res 150:417–426

    PubMed  Google Scholar 

  • Busch NA, VanRullen R (2010) Spontaneous EEG oscillations reveal periodic sampling of visual attention. Proc Natl Acad Sci USA 107:16048–16053

    Article  PubMed  CAS  Google Scholar 

  • Busch NA, Dubois J, VanRullen R (2009) The phase of ongoing EEG oscillations predicts visual perception. J Neurosci 29:7869–7876

    Article  PubMed  CAS  Google Scholar 

  • Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration in humans by application of electrophysiological criteria to the BOLD effect. NeuroImage 14:427–438

    Article  PubMed  CAS  Google Scholar 

  • Cappe C, Barone P (2005) Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur J Neurosci 22:2886–2902

    Article  PubMed  Google Scholar 

  • Carrasco M, Ling S, Read S (2004) Attention alters appearance. Nat Neurosci 7:308–313

    Article  PubMed  CAS  Google Scholar 

  • Coull JT, Nobre AC (1998) Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J Neurosci 18:7426–7435

    PubMed  CAS  Google Scholar 

  • Falchier A, Clavagnier S, Barone P, Kennedy H (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749–5759

    PubMed  CAS  Google Scholar 

  • Falchier A, Schroeder CE, Hackett TA, Lakatos P, Nascimento-Silva S, Ulbert I, Karmos G, Smiley JF (2010) Projections from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cereb Cortex 20:1529–1538

    Article  PubMed  Google Scholar 

  • Fiebelkorn IC, Foxe JJ, Butler JS, Mercier MR, Snyder AC, Molholm S (2010) Ready, set, reset: stimulus-locked periodicity in behavioral performance demonstrates the consequences of cross-sensory phase reset. Under Review

  • Foxe JJ, Schroeder CE (2005) The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16:419–423

    Article  PubMed  Google Scholar 

  • Foxe JJ, Simpson GV, Ahlfors SP (1998) Cued shifts of intermodal attention: Parieto-occipital 10 Hz activity reflects anticipatory state of visual attention mechanisms. Neuroreport 9:3929–3933

    Article  PubMed  CAS  Google Scholar 

  • Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res Cogn Brain Res 10:77–83

    Article  PubMed  CAS  Google Scholar 

  • Frens MA, Van Opstal AJ, Van der Willigen RF (1995) Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Percept Psychophys 57:802–816

    Article  PubMed  CAS  Google Scholar 

  • Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490

    Article  PubMed  CAS  Google Scholar 

  • Gondan M, Niederhaus B, Rosler F, Roder B (2005) Multisensory processing in redundant-target effect: a behavioral and event-related potential study. Percept Psychophys 67:713–726

    Article  PubMed  Google Scholar 

  • Gonzalez Andino SL, Murray MM, Foxe JJ, Grave de Peralta Menendez R (2005) How single-trial electrical neuroimaging contributes to multisensory research. Exp Brain Res 166:298–304

    Article  PubMed  Google Scholar 

  • Green DM, Swets JA (1966) Signal detection and psychophysics. Wiley, New York

    Google Scholar 

  • Hackley SA, Schankin A, Wohlschlaeger A, Wascher E (2007) Localization of temporal preparation effects via trisected reaction time. Psychophysiology 44:334–338

    Article  PubMed  Google Scholar 

  • Kaplan E (1991) The receptive field structure of retinal ganglion cells in cat and monkey. In: Leventhal AG (ed) Vision and visual dysfunction. CRC, Boston, pp 10–40

    Google Scholar 

  • Kayser C, Montemurro MA, Logothetis NK, Panzeri S (2009) Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns. Neuron 61:597–608

    Article  PubMed  CAS  Google Scholar 

  • Keetels M, Vroomen J (2005) The role of spatial disparity and hemifield in audio-visual temporal order judgments. Exp Brain Res 167:635–640

    Article  PubMed  Google Scholar 

  • Lakatos P, Chen CM, O’Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279–292

    Article  PubMed  CAS  Google Scholar 

  • Lakatos P, O’Connell MN, Barczak A, Mills A, Javitt DC, Schroeder CE (2009) The leading sense: supramodal control of neurophysiological context by attention. Neuron 64:419–430

    Article  PubMed  CAS  Google Scholar 

  • Lippert M, Logothetis NK, Kayser C (2007) Improvement of visual contrast detection by a simultaneous sound. Brain Res 1173:102–109

    Article  PubMed  CAS  Google Scholar 

  • Macmillan NA, Creelman CD (1991) Detection theory: a user’s guide. Cambridge University Press, Cambridge

    Google Scholar 

  • Mathewson KE, Gratton G, Fabiani M, Beck DM, Ro T (2009) To see or not to see: prestimulus alpha phase predicts visual awareness. J Neurosci 29:2725–2732

    Article  PubMed  CAS  Google Scholar 

  • Meredith MA, Stein BE (1986) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365:350–354

    Article  PubMed  CAS  Google Scholar 

  • Merigan W, Maunsell J (1993) How parallel are the primate visual pathways? Annu Rev Neurosci 5:347–352

    Google Scholar 

  • Miniussi C, Wilding EL, Coull JT, Nobre AC (1999) Orienting attention in time: modulation of brain potentials. Brain 122:1507–1518

    Article  PubMed  Google Scholar 

  • Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn Brain Res 14:115–129

    Article  Google Scholar 

  • Montemurro MA, Rasch MJ, Murayama Y, Logothetis NK, Panzeri S (2008) Phase-of-firing coding of natural stimuli in primary visual cortex. Curr Biol 18:375–380

    Article  PubMed  CAS  Google Scholar 

  • Monto S, Palva S, Voipio J, Palva JM (2008) Very slow EEG fluctuations predict the dynamics of stimulus detection and oscillation amplitudes in humans. J Neurosci 28:8268–8272

    Article  PubMed  CAS  Google Scholar 

  • Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15L:963–974

    Google Scholar 

  • Noesselt T, Bergmann D, Hake M, Heinze HJ, Fendrich R (2008) Sound increases the saliency of visual events. Brain Res 1220:157–163

    Article  PubMed  CAS  Google Scholar 

  • Noesselt T, Tyll S, Boehler CN, Budinger E, Heinze HJ, Driver J (2010) Sound-induced enhancement of low-intensity vision: multisensory influences on human sensory-specific cortices and thalamic bodies related to perceptual enhancement of visual detection sensitivity. J Neurosci 30:13609–13623

    Article  PubMed  CAS  Google Scholar 

  • Odgaard EC, Arieh Y, Marks LE (2003) Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys 65:123–132

    Article  PubMed  Google Scholar 

  • Rockland KS, Ojima H (2003) Multisensory convergence in calcarine visual areas in macaque monkey. Inter J Psychophysiol 50:19–26

    Article  Google Scholar 

  • Schroeder CE, Molholm S, Lakatos P, Ritter W, Foxe JJ (2004) Human-Simian correspondence in the early cortical processing of multisensory cues. Cogni Process 5:140–151

    Google Scholar 

  • Snyder AC, Foxe JJ (2010) Anticipatory attentional suppression of visual features indexed by oscillatory alpha-band power increases: a high-density electrical mapping study. J Neurosci 30:4024–4032

    Article  PubMed  CAS  Google Scholar 

  • Stein BE (1998) Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp Brain Res 123:124–135

    Article  PubMed  CAS  Google Scholar 

  • Stein BE, London N, Wilkonson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8:497–506

    Article  Google Scholar 

  • Teder-Sälejärvi WA, Di Russo F, McDonald JJ, Hillyard SA (2005) Effects of spatial congruity on audio-visual multimodal integration. J Cogn Neurosci 17:1396–1409

    Article  PubMed  Google Scholar 

  • Van der Burg E, Olivers CNL, Bronkhorst AW, Theeuwes J (2008a) Pip and pop: auditory signals improve spatial visual search. J Exp Psychol Hum Percept Perform 34:1053–1065

    Article  PubMed  Google Scholar 

  • Van der Burg E, Olivers CNL, Bronkhorst AW, Theeuwes J (2008b) Audiovisual events capture attention: evidence from temporal order judgments. J Vis 8:1–10

    PubMed  Google Scholar 

  • VanRullen R, Reddy L, Koch C (2006) The continuous wagon wheel illusion is associated with changes in EEG power around 13 Hz. J Neurosci 26:502–507

    Article  PubMed  CAS  Google Scholar 

  • VanRullen R, Carlson T, Cavanagh P (2007) The blinking spotlight of attention. Proc Natl Acad Sci USA 104:19204–19209

    Article  PubMed  CAS  Google Scholar 

  • Vroomen J, Keetels M (2006) The spatial constraint in intersensory pairing: no role in temporal ventriloquism. J Exp Pschol Hum Percept Perform 32:1063–1071

    Article  Google Scholar 

  • Wang Y, Celebrini S, Trotter Y, Barone P (2008) Visuo-auditory interactions in the primary visual cortex of the behaving monkey: electrophysiological evidence. BMC Neurosci 9:79

    Article  PubMed  Google Scholar 

  • Whittingstall K, Logothetis NK (2009) Frequency-band coupling in surface EEG reflects spiking activity in monkey visual cortex. Neuron 64:281–289

    Article  PubMed  CAS  Google Scholar 

  • Worden MS, Foxe JJ, Wang N, Simpson GV (2000) Anticipatory biasing of visuospatial attention indexed by retinotopically specific alpha-band electroencephalography increases over occipital cortex. J Neurosci 20:RC63

    Google Scholar 

Download references

Acknowledgments

This work was supported by a National Science Foundation grant BCS0642584 to JJF and a National Institute of Mental Health grant MH-085322 to SM and JJF.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sophie Molholm.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fiebelkorn, I.C., Foxe, J.J., Butler, J.S. et al. Auditory facilitation of visual-target detection persists regardless of retinal eccentricity and despite wide audiovisual misalignments. Exp Brain Res 213, 167–174 (2011). https://doi.org/10.1007/s00221-011-2670-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-011-2670-7

Keywords

Navigation