Abstract
Our brain relies on neural mechanisms of selective attention and converging sensory processing to efficiently cope with rich and unceasing multisensory inputs. One prominent assumption holds that audio-visual synchrony can act as a strong attractor for spatial attention. Here, we tested for a similar effect of audio-visual synchrony on feature-selective attention. We presented two superimposed Gabor patches that differed in colour and orientation. On each trial, participants were cued to selectively attend to one of the two patches. Over time, spatial frequencies of both patches varied sinusoidally at distinct rates (3.14 and 3.63 Hz), giving rise to pulse-like percepts. A simultaneously presented pure tone carried a frequency modulation at the pulse rate of one of the two visual stimuli to introduce audio-visual synchrony. Pulsed stimulation elicited distinct time-locked oscillatory electrophysiological brain responses. These steady-state responses were quantified in the spectral domain to examine individual stimulus processing under conditions of synchronous versus asynchronous tone presentation and when respective stimuli were attended versus unattended. We found that both, attending to the colour of a stimulus and its synchrony with the tone, enhanced its processing. Moreover, both gain effects combined linearly for attended in-sync stimuli. Our results suggest that audio-visual synchrony can attract attention to specific stimulus features when stimuli overlap in space.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
An X, Hohne J, Ming D, Blankertz B (2014) Exploring combinations of auditory and visual stimuli for gaze-independent brain–computer interfaces. PLoS One 9:e111070. doi:10.1371/journal.pone.0111070
Andersen SK, Hillyard SA, Müller MM (2008) Attention facilitates multiple stimulus features in parallel in human visual cortex. Curr Biol 18:1006–1009
Atmaca S, Stadler W, Keitel A, Ott DV, Lepsien J, Prinz W (2013) Prediction processes during multiple object tracking (MOT): involvement of dorsal and ventral premotor cortices. Brain Behav 3:683–700. doi:10.1002/brb3.180
Bidet-Caulet A, Fischer C, Besle J, Aguera PE, Giard MH, Bertrand O (2007) Effects of selective attention on the electrophysiological representation of concurrent sounds in the human auditory cortex. J Neurosci 27:9252–9261
Busse L, Roberts KC, Crist RE, Weissman DH, Woldorff MG (2005) The spread of attention across modalities and space in a multisensory object. Proc Natl Acad Sci USA 102:18751–18756. doi:10.1073/pnas.0507704102
Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134:9–21
Donohue SE, Green JJ, Woldorff MG (2015) The effects of attention on the temporal integration of multisensory stimuli. Front Integr Neurosci 9:32. doi:10.3389/fnint.2015.00032
Eimer M, Grubert A (2014) The gradual emergence of spatially selective target processing in visual search: from feature-specific to object-based attentional control. J Exp Psychol Hum Percept Perform 40:1819–1831. doi:10.1037/a0037387
Franconeri SL, Pylyshyn ZW, Scholl BJ (2012) A simple proximity heuristic allows tracking of multiple objects through occlusion. Atten Percept Psychophys 74:691–702. doi:10.3758/s13414-011-0265-9
Fujisaki W, Nishida S (2005) Temporal frequency characteristics of synchrony-asynchrony discrimination of audio-visual signals. Exp Brain Res 166:455–464. doi:10.1007/s00221-005-2385-8
Fujisaki W, Nishida S (2008) Top-down feature-based selection of matching features for audio-visual synchrony discrimination. Neurosci Lett 433:225–230. doi:10.1016/j.neulet.2008.01.031
Fujisaki W, Nishida S (2009) Audio-tactile superiority over visuo-tactile and audio-visual combinations in the temporal resolution of synchrony perception. Exp Brain Res 198:245–259. doi:10.1007/s00221-009-1870-x
Giani AS, Ortiz E, Belardinelli P, Kleiner M, Preissl H, Noppeney U (2012) Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses. Neuroimage 60:1478–1489. doi:10.1016/j.neuroimage.2012.01.114
Herrmann B, Schlichting N, Obleser J (2014) Dynamic range adaptation to spectral stimulus statistics in human auditory cortex. J Neurosci 34:327–331. doi:10.1523/JNEUROSCI.3974-13.2014
Jarmasz J, Hollands JG (2009) Confidence intervals in repeated-measures designs: the number of observations principle. Can J Exp Psychol 63:124–138. doi:10.1037/a0014164
Jenkins J 3rd, Rhone AE, Idsardi WJ, Simon JZ, Poeppel D (2011) The elicitation of audiovisual steady-state responses: multi-sensory signal congruity and phase effects. Brain Topogr 24:134–148. doi:10.1007/s10548-011-0174-1
Keitel C, Schröger E, Saupe K, Müller MM (2011) Sustained selective intermodal attention modulates processing of language-like stimuli. Exp Brain Res 213(2–3):321–327
Keitel C, Maess B, Schroger E, Muller MM (2013) Early visual and auditory processing rely on modality-specific attentional resources. Neuroimage 70:240–249. doi:10.1016/j.neuroimage.2012.12.046
Keitel C, Quigley C, Ruhnau P (2014) Stimulus-driven brain oscillations in the alpha range: Entrainment of intrinsic rhythms or frequency-following response? J Neurosci 34:10137–10140. doi:10.1523/Jneurosci.1904-14.2014
Kim YJ, Grabowecky M, Paller KA, Muthu K, Suzuki S (2007) Attention induces synchronization-based response gain in steady-state visual evoked potentials. Nat Neurosci 10:117–125
Kim YJ, Grabowecky M, Paller KA, Suzuki S (2011) Differential roles of frequency-following and frequency-doubling visual responses revealed by evoked neural harmonics. J Cogn Neurosci 23:1875–1886. doi:10.1162/jocn.2010.21536
Lakatos P, O’Connell MN, Barczak A, Mills A, Javitt DC, Schroeder CE (2009) The leading sense: supramodal control of neurophysiological context by attention. Neuron 64:419–430. doi:10.1016/j.neuron.2009.10.014
Leek MR (2001) Adaptive procedures in psychophysical research. Percept Psychophys 63:1279–1292
Macaluso E, Driver J (2003) Multimodal spatial representations in the human parietal cortex: evidence from functional imaging. Adv Neurol 93:219–233
Maunsell JH, Treue S (2006) Feature-based attention in visual cortex. Trends Neurosci 29:317–322. doi:10.1016/j.tins.2006.04.001
Müller MM (2014) Neural mechanisms of feature-based attention. In: Mangun GR (ed) Cognitive electrophysiology of attention: signals of the mind. Elsevier, Amsterdam, pp 123–135
Müller MM, Picton TW, Valdes-Sosa P, Riera J, Teder-Salejarvi WA, Hillyard SA (1998) Effects of spatial selective attention on the steady-state visual evoked potential in the 20–28 Hz range. Brain Res Cogn Brain Res 6:249–261
Müller MM, Malinowski P, Gruber T, Hillyard SA (2003) Sustained division of the attentional spotlight. Nature 424:309–312
Müller MM, Andersen S, Trujillo NJ, Valdes-Sosa P, Malinowski P, Hillyard SA (2006) Feature-selective attention enhances color signals in early visual areas of the human brain. Proc Natl Acad Sci USA 103:14250–14254
Nolan H, Whelan R, Reilly RB (2010) FASTER: fully automated statistical thresholding for EEG artifact rejection. J Neurosci Methods 192:152–162. doi:10.1016/j.jneumeth.2010.07.015
Nozaradan S, Peretz I, Mouraux A (2012) Steady-state evoked potentials as an index of multisensory temporal binding. Neuroimage 60:21–28. doi:10.1016/j.neuroimage.2011.11.065
Pastor MA, Valencia M, Artieda J, Alegre M, Masdeu JC (2007) Topography of cortical activation differs for fundamental and harmonic frequencies of the steady-state visual-evoked responses. An EEG and PET H215O study. Cereb Cortex 17:1899–1905. doi:10.1093/cercor/bhl098
Porcu E, Keitel C, Muller MM (2013) Concurrent visual and tactile steady-state evoked potentials index allocation of inter-modal attention: a frequency-tagging study. Neurosci Lett 556:113–117. doi:10.1016/j.neulet.2013.09.068
Porcu E, Keitel C, Muller MM (2014) Visual, auditory and tactile stimuli compete for early sensory processing capacities within but not between senses. Neuroimage 97:224–235. doi:10.1016/j.neuroimage.2014.04.024
Regan D (1989) Human brain electrophysiology: evoked potentials and evoked magnetic fields in science and medicine. Elsevier, New York
Ross B, Picton TW, Herdman AT, Pantev C (2004) The effect of attention on the auditory steady-state response. Neurol Clin Neurophysiol 2004:22
Saenz M, Buracas GT, Boynton GM (2002) Global effects of feature-based attention in human visual cortex. Nat Neurosci 5:631–632. doi:10.1038/nn876
Saupe K, Schröger E, Andersen SK, Müller MM (2009a) Neural mechanisms of intermodal sustained selective attention with concurrently presented auditory and visual stimuli. Front Hum Neurosci 3:58
Saupe K, Widmann A, Bendixen A, Müller MM, Schröger E (2009b) Effects of intermodal attention on the auditory steady-state response and the event-related potential. Psychophysiology 46:321–327
Schall S, Quigley C, Onat S, König P (2009) Visual stimulus locking of EEG is modulated by temporal congruency of auditory stimuli. Exp Brain Res 198:137–151
Snyder AC, Foxe JJ (2010) Anticipatory attentional suppression of visual features indexed by oscillatory alpha-band power increases: a high-density electrical mapping study. J Neurosci 30:4024–4032. doi:10.1523/jneurosci.5684-09.2010
Talsma D, Senkowski D, Soto-Faraco S, Woldorff MG (2010) The multifaceted interplay between attention and multisensory integration. Trends Cogn Sci 14:400–410
Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12:97–136
Treue S, Martinez Trujillo JC (1999) Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399:575–579
Van der Burg E, Olivers CN, Bronkhorst AW, Theeuwes J (2008) Pip and pop: nonspatial auditory signals improve spatial visual search. J Exp Psychol Hum Percept Perform 34:1053–1065
Van der Burg E, Cass J, Olivers CN, Theeuwes J, Alais D (2010) Efficient visual search from synchronized auditory signals requires transient audiovisual events. PLoS One 5:e10664. doi:10.1371/journal.pone.0010664
Van der Burg E, Talsma D, Olivers CN, Hickey C, Theeuwes J (2011) Early multisensory interactions affect the competition among multiple visual objects. Neuroimage 55:1208–1218. doi:10.1016/j.neuroimage.2010.12.068
Van der Burg E, Cass J, Alais D (2014) Window of audio-visual simultaneity is unaffected by spatio-temporal visual clutter. Sci Rep 4:5098. doi:10.1038/srep05098
Vroomen J, Keetels M (2010) Perception of intersensory synchrony: a tutorial review. Atten Percept Psychophys 72:871–884. doi:10.3758/APP.72.4.871
Wagner G, Boynton RM (1972) Comparison of four methods of heterochromatic photometry. J Opt Soc Am 62:1508–1515
Werner S, Noppeney U (2011) The contributions of transient and sustained response codes to audiovisual integration. Cereb Cortex 21:920–931
Winkler I, Denham S, Mill R, Bohm TM, Bendixen A (2012) Multistability in auditory stream segregation: a predictive coding view. Philos Trans R Soc Lond B Biol Sci 367:1001–1012. doi:10.1098/rstb.2011.0359
Wolfe JM (1994) Guided search 2.0—a revised model of visual-search. Psychon Bull Rev 1:202–238. doi:10.3758/Bf03200774
Yi DJ, Turk-Browne NB, Flombaum JI, Kim MS, Scholl BJ, Chun MM (2008) Spatiotemporal object continuity in human ventral visual cortex. Proc Natl Acad Sci USA 105:8840–8845. doi:10.1073/pnas.0802525105
Acknowledgments
Work was supported by the Deutsche Forschungsgemeinschaft (Grant No. MU972/21-1). Data presented here were recorded at the Institut für Psychologie, Universität Leipzig. The authors appreciate the assistance of Renate Zahn and Emanuele Porcu in data collection. The experimental stimulation was realized using Cogent 2000 developed by the Cogent 2000 team at the Functional Imaging Laboratory and the Institute of Cognitive Neuroscience and Cogent Graphics developed by John Romaya at the Laboratory of Neurobiology at the Wellcome Department of Imaging Neuroscience.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
None declared.
Rights and permissions
About this article
Cite this article
Keitel, C., Müller, M.M. Audio-visual synchrony and feature-selective attention co-amplify early visual processing. Exp Brain Res 234, 1221–1231 (2016). https://doi.org/10.1007/s00221-015-4392-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-015-4392-8