Skip to main content
Download PDF

On audiovisual spatial synergy: The fragility of the phenomenon

Perception & Psychophysics volume 67, pages 444–457 (2005)Cite this article

Abstract

Recent literature has highlighted the importance and ubiquity of cross-modal links in spatial attention, whereby shifts in attention in one modality often induce corresponding shifts in other modalities. We attempted to provide further evidence for the case of audiovisual links during sustained endogenous attention by addressing several potential methodological confounds in previous demonstrations. However, we failed repeatedly to reproduce the phenomenon of spatial synergies between auditory and visual attention, found by Driver and Spence (1994) and frequently cited to support the automatic nature of cross-modal attention links. We discuss the results in light of recent evidence about cross-modal spatial links during sustained attention and support the idea that such links can weaken or even disappear under certain circumstances, such as during periods of sustained attention. The implication is that individuals can select inputs from different modalities from different locations more easily than previously had been thought.

References

  • Blauert, J. (1997).Spatial hearing (Rev. ed.). Cambridge, MA: MIT Press.

    Google Scholar 

  • Braun, J. (1998). Vision and attention: The role of training.Nature,393, 424–425.

    PubMed  Article  Google Scholar 

  • Brown, G. D. A. (1984). A frequency count of 190,000 words in the London—Lund Corpus of English Conversation.Behavior Research Methods, Instruments, & Computers,16, 502–532.

    Article  Google Scholar 

  • Buchtel, H. A., &Butter, C. M. (1988). Spatial attention shifts: Implications for the role of polysensory mechanisms.Neuropsychologia,26, 499–509.

    PubMed  Article  Google Scholar 

  • Butter, C. M., Buchtel, H. A., &Santucci, R. (1989). Spatial attentional shifts: Further evidence for the role of polysensory mechanisms using visual and tactile stimuli.Neuropsychologia,27, 1231–1240.

    PubMed  Article  Google Scholar 

  • Driver, J. (1996). Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading.Nature,381, 66–68.

    PubMed  Article  Google Scholar 

  • Driver, J., &Spence, C. J. (1994). Spatial synergies between auditory and visual attention. In C. Umiltà & M. Moscovitch (Eds.),Attention and performance XV: Conscious and nonconscious information processing (pp. 311–331). Cambridge, MA: MIT Press, Bradford Books.

    Google Scholar 

  • Driver, J., &Spence, C. (1998). Attention and the crossmodal construction of space.Trends in Cognitive Sciences,2, 254–262.

    Article  Google Scholar 

  • Driver, J., &Spence, C. (2004). Crossmodal spatial attention: Evidence from human performance. In J. Driver & C. Spence (Eds.),Crossmodal space and crossmodal attention (pp. 179–220). Oxford: Oxford University Press.

    Google Scholar 

  • Duncan, J., Martens, S., &Ward, R. (1997). Restricted attentional capacity within but not between sensory modalities.Nature,387, 808–810.

    PubMed  Article  Google Scholar 

  • Eimer, M. (1996). ERP modulations indicate the selective processing of visual stimuli as a result of transient and sustained spatial attention.Psychophysiology,33, 13–21.

    PubMed  Article  Google Scholar 

  • Eimer, M. (1999). Can attention be directed to opposite directions in different modalities? An ERP study.Clinical Neurophysiology,110, 1252–1259.

    PubMed  Article  Google Scholar 

  • Eimer, M., &Driver, J. (2000). An event-related brain potential study of cross-modal links in spatial attention between vision and touch.Psychophysiology,37, 697–705.

    PubMed  Article  Google Scholar 

  • Eimer, M., &Van Velzen, J. (2002). Cross modal links in spatial attention are mediated by supramodal control processes: Evidence from event-related potentials.Psychophysiology,39, 437–449.

    PubMed  Article  Google Scholar 

  • Eimer, M., Van Velzen, J., &Driver, J. (2002) Cross-modal interactions between audition, touch, and vision in endogenous spatial attention: ERP evidence on preparatory states and sensory modulations.Journal of Cognitive Neuroscience,14, 254–271.

    PubMed  Article  Google Scholar 

  • Farah, M. J., Wong, A. B., Monheit, M. A., &Morrow, L. A. (1989). Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal?Neuropsychologia,27, 461–470.

    PubMed  Article  Google Scholar 

  • Gopher, D. (1973). Eye-movement patterns in selective listening tasks of focused attention.Perception & Psychophysics,14, 259–264.

    Google Scholar 

  • Grant, K. W., &Seitz, P. F. (1998). Measures of auditory—visual integration in nonsense syllables and sentences.Journal of the Acoustical Society of America,104, 2438–2450.

    PubMed  Article  Google Scholar 

  • Grant, K. W., &Seitz, P. F. (2000). The use of visible speech cues for improving auditory detection of spoken sentences.Journal of the Acoustical Society of America,108, 1197–1208.

    PubMed  Article  Google Scholar 

  • Hillyard, S. A., Simpson, G. V., Woods, D. L., Van Voorhis, S., &Münte, T. F. (1984). Event-related brain potentials and selective attention to different modalities. In F. Reinoso-Suárez & C. Aimone-Marsan (Eds.),Cortical integration (pp. 395–414). New York: Raven.

    Google Scholar 

  • Hötting, K., Rösler, F., &Röder, B. (2002). Crossmodal and intermodal attention modulates event-related brain potentials to tactile and auditory stimuli.Experimental Brain Research,148, 26–37.

    Google Scholar 

  • Howell, D. C. (1982).Statistical methods for psychology (3rd. ed.). Belmont, CA: Duxbury.

    Google Scholar 

  • LaBerge, D. (1995).Attentional processing. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Leech, S. (1999, October).Can the ventriloquism illusion facilitate audiovisual speech perception? Poster presented at the 1st annual meeting of the International Multisensory Research Forum (crossmodal attention and multisensory integration), Oxford.

  • Lloyd, D. M., Merat, N., McGlone, F., &Spence, C. (2003). Crossmodal links between audition and touch in covert endogenous spatial attention.Perception & Psychophysics,65, 901–924.

    Article  Google Scholar 

  • Pashler, H. E. (1997).The psychology of attention. Cambridge, MA: MIT Press.

    Google Scholar 

  • Posner, M. I. (1980). Orienting of attention.Quarterly Journal of Experimental Psychology,32, 3–25.

    PubMed  Article  Google Scholar 

  • Reisberg, D. (1978). Looking where you listen: Visual cues and auditory attention.Acta Psychologica,42, 331–341.

    PubMed  Article  Google Scholar 

  • Reisberg, D., Scheiber, R., &Potemken, L. (1981). Eye position and the control of auditory attention.Journal of Experimental Psychology: Human Perception & Performance,7, 318–323.

    Article  Google Scholar 

  • Soto-Faraco, S., Spence, C., Fairbank, K., Kingstone, A., Hillstrom, A. P., &Shapiro, K. (2002). A crossmodal attentional blink between vision and touch.Psychonomic Bulletin & Review,9, 731–738.

    Article  Google Scholar 

  • Spence, C., &Driver, J. (1996). Audiovisual links in endogenous covert spatial attention.Journal of Experimental Psychology: Human Perception & Performance,22, 1005–1030.

    Article  Google Scholar 

  • Spence, C., &Driver, J. (1997). Audiovisual links in exogenous covert spatial orienting.Perception & Psychophysics,59, 1–22.

    Google Scholar 

  • Spence, C., &McDonald, J. (2004). The crossmodal consequences of the exogenous spatial orienting of attention. In G. Calvert, C. Spence, & B. E. Stein (Eds.),The handbook of multisensory processes (pp. 3–25). Cambridge, MA: MIT Press.

    Google Scholar 

  • Spence, C., McDonald, J., &Driver, J. (2004). Exogenous spatial cuing studies of human crossmodal attention and multisensory integration. In J. Driver and C. Spence (Eds.),Crossmodal space and crossmodal attention (pp. 277–320). Oxford: Oxford University Press.

    Google Scholar 

  • Spence, C., Pavani, F., &Driver, J. (2000). Crossmodal links between vision and touch in covert endogenous spatial attention.Journal of Experimental Psychology: Human Perception & Performance,26, 1298–1319.

    Article  Google Scholar 

  • Spence, C., Ranson, J., &Driver, J. (2000). Crossmodal selective attention: On the difficulty of ignoring sounds at the locus of visual attention.Perception & Psychophysics,62, 410–424.

    Google Scholar 

  • Spence, C., &Read, L. (2003). Talking while driving: On the difficulty of splitting attention between eye and ear.Psychological Science,14, 251–256.

    PubMed  Article  Google Scholar 

  • Sperling, G. (1960). The information available in brief visual presentations.Psychological Monographs,74 (11, Whole No. 498).

    Google Scholar 

  • Sturm, W., de Simone, A., Krause, B. J., Specht, K., Hesselmann, V., Radermacher, I., Herzog, H., Tellmann, L., Muller-Gartner, H. W., &Willmes, K. (1999). Functional anatomy of intrinsic alertness: Evidence for a fronto-parietal-thalamic-brainstem network in the right hemisphere.Neuropsychologia,37, 797–805.

    PubMed  Article  Google Scholar 

  • Sumby, W. H., &Pollack, I. (1954). Visual contribution to speech intelligibility in noise.Journal of the Acoustical Society of America,26, 212–215.

    Article  Google Scholar 

  • Teder-Sälejärvi, W. A., Münte, T. F., Sperlich, F., &Hillyard, S. A. (1999). Intra-modal and cross-modal spatial attention to auditory and visual stimuli: An event-related brain potential study.Cognitive Brain Research,8, 327–343.

    PubMed  Article  Google Scholar 

  • Ward, L. M., McDonald, J. J., &Lin, D. (2000). On asymmetries in cross-modal spatial attention orienting.Perception & Psychophysics,62, 1258–1264.

    Google Scholar 

  • Wickens, C. D. (1980). The structure of attentional resources. In R. S. Nickerson (Ed.),Attention and performance VIII (pp. 239–257). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Wickens, C. D. (1984). Processing resources in attention. In R. Parasuraman & D. R. Davies (Eds.),Varieties of attention (pp. 63–102). San Diego: Academic Press.

    Google Scholar 

  • Wolters, N. C. W., &Schiano, D. J. (1989). On listening where we look: The fragility of a phenomenon.Perception & Psychophysics,45, 184–186.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universitat de Barcelona, Barcelona, Spain

    Salvador Soto-Faraco

  2. University of British Columbia, Vancouver, British Columbia, Canada

    Sharon Morein-Zamir & Alan Kingstone

Authors
  1. Salvador Soto-Faraco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sharon Morein-Zamir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan Kingstone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salvador Soto-Faraco.

Additional information

This work was supported by a postdoctoral award from the Killam Trust to S.S.-F., by a predoctoral scholarship from the University of British Columbia to S.M.-Z., and by grants from the National Science and Engineering Research Council of Canada, the Human Frontier Science Programme, and the Michael Smith Foundation for Medical Sciences to A.K.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Soto-Faraco, S., Morein-Zamir, S. & Kingstone, A. On audiovisual spatial synergy: The fragility of the phenomenon. Perception & Psychophysics 67, 444–457 (2005). https://doi.org/10.3758/BF03193323

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.3758/BF03193323

Keywords

  • Visual Attention
  • Spatial Attention
  • Visual Task
  • Rapid Serial Visual Presentation
  • Auditory Attention