Perception & Psychophysics

, Volume 62, Issue 2, pp 321–332

The ventriloquist effect does not depend on the direction of deliberate visual attention

  • Paul Bertelson
  • Jean Vroomen
  • Béatrice De Gelder
  • Jon Driver
Article
  • 891 Downloads

Abstract

It is well known that discrepancies in the location of synchronized auditory and visual events can lead to mislocalizations of the auditory source, so-called ventriloquism. In two experiments, we tested whether such cross-modal influences on auditory localization depend on deliberate visual attention to the biasing visual event. In Experiment 1, subjects pointed to the apparent source of sounds in the presence or absence of a synchronous peripheral flash. They also monitored for target visual events, either at the location of the peripheral flash or in a central location. Auditory localization was attracted toward the synchronous peripheral flash, but this was unaffected by where deliberate visual attention was directed in the monitoring task. In Experiment 2, bilateral flashes were presented in synchrony with each sound, to provide competing visual attractors. When these visual events were equally salient on the two sides, auditory localization was unaffected by which side subjects monitored for visual targets. When one flash was larger than the other, auditory localization was slightly but reliably attracted toward it, but again regardless of where visual monitoring was required. We conclude that ventriloquism largely reflects automatic sensory interactions, with little or no role for deliberate spatial attention.

References

  1. Bermant, R. I., &Welch, R. B. (1976). The effect of degree of visualauditory stimulus separation and eye position upon the spatial interaction of vision and audition.Perceptual & Motor Skill,43, 487–493.Google Scholar
  2. Bertelson, P. (1994). The cognitive architecture behind auditory-visual interaction in scene analysis and speech identification.Current Psychology of Cognition,13, 69–75.Google Scholar
  3. Bertelson, P. (1998). Starting from the ventriloquist: The perception of multimodal events. In M. Sabourin, F. I. M. Craik, & M. Robert (Eds.),Advances in psychological science: Vol. I. Biological and cognitive aspects (pp. 419–439). Hove, U.K.: Psychology Press.Google Scholar
  4. Bertelson, P., &Aschersleben, G. (1998). Automatic visual bias of perceived auditory location.Psychonomic Bulletin & Review,5, 482–489.CrossRefGoogle Scholar
  5. Bertelson, P., &Radeau, M. (1976). Ventriloquism, sensory interaction, and response bias: Remarks on the paper by Choe, Welch, Gilford, and Juola.Perception & Psychophysics,19, 531–535.CrossRefGoogle Scholar
  6. Bertelson, P., &Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance.Perception & Psychophysics,29, 578–584.CrossRefGoogle Scholar
  7. Bertelson, P., & Radeau, M. (1987, September). Adaptation to auditory-visual conflict: Have top-down influences been overestimated here also? Paper presented at the 2nd Meeting of the European Society for Cognitive Psychology, Madrid.Google Scholar
  8. Bertelson, P., Vroomen, J., Wiegeraad, G., & De Gelder, B. (1994, September). Exploring the relation between McGurk interference and ventriloquism.In International Congress on Spoken Language Processing (pp. 559–562).Google Scholar
  9. Canon, L. K. (1970). Intermodality inconsistency of input and directed attention as determinants of the nature of adaptation.Journal of Experimental Psychology,84, 141–147.CrossRefPubMedGoogle Scholar
  10. Cave, K. R., &Wolfe, J. M. (1990). Modelling the role of parallel processing in visual search.Cognitive Psychology,22, 225–271.CrossRefPubMedGoogle Scholar
  11. Choe, C. S., Welch.R. B., Gilford, R. M., &Juola, J. F. (1975). The “ventriloquist effect”: Visual dominance or response bias?Perception & Psychophysics,18, 55–60.CrossRefGoogle Scholar
  12. Driver, J. (1996). Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading.Nature,381, 66–68.CrossRefPubMedGoogle Scholar
  13. Driver, J., &Spence, C. J. (1994). Cross-modal synergies in attention. In C. Umiltà & M. Moscovitch (Eds.),Attention and performance XXV: Conscious and nonconscious information processing (pp. 311–331 ). Cambridge, MA: MIT Press, Bradford Books.Google Scholar
  14. Egeth, H. E., &Yantis, S. (1997). Visual attention: Control, representation, and time course.Annual Review of Psychology,48, 269–297.CrossRefPubMedGoogle Scholar
  15. Held, R. (1965). Plasticity in sensorimotor systems.Scientific American,213, 84–94.CrossRefPubMedGoogle Scholar
  16. Jack, C. E., &Thurlow, W. R. (1973). Effects of degree of visual association and angle of displacement on the”ventriloquism”effect.Perceptual & Motor Skills,38, 967–979.Google Scholar
  17. Jackson, C. V. (1953). Visual factors in auditory localization.Quarterly Journal of Experimental Psychology,5, 52–65.CrossRefGoogle Scholar
  18. Jonides.J., &Yantis, S. (1988). Uniqueness of abrupt visual onset in capturing attention.Perception & Psychophysics,43, 346–354.CrossRefGoogle Scholar
  19. Klemm, O. (1909). Localisation von Sinneneindrücken bei disparate Nebenreizen [Localization of sense impressions with discordant additional stimulation].Psychologische Studien,5, 73–161.Google Scholar
  20. Krueger, L. (1989). Cognitive impenetrability of perception.Behavioural & Brain Sciences,12, 769–770.CrossRefGoogle Scholar
  21. Pashler, H. (1998).The psychology of attention. Cambridge, MA: MIT Press.Google Scholar
  22. Pick, H. L., Jr.,Warren, D. H., &Hay, J. C. (1969). Sensory conflict in judgments of spatial direction.Perception & Psychophysics,6, 203–205.CrossRefGoogle Scholar
  23. Posner, M. I. (1980). Orienting of attention.Quarterly Journal of Experimental Psychology,32, 3–25.CrossRefPubMedGoogle Scholar
  24. Pylyshyn, Z. W. (1999). Is vision continuous with cognition? The case for cognitive impenetrability of visual perception.Behavioural & Brain Sciences,22, 341–423.Google Scholar
  25. Radeau, M. (1974). Adaptation au déplacement prismatique sur la base d’une discordance entre la vision et l’audition [Adaptation to prismatic displacement on the basis of discordance between vision and audition],L’Année Psychologique,74, 23–34.CrossRefPubMedGoogle Scholar
  26. Radeau, M. (1985). Signal intensity, task context, and auditory-visual interaction.Perception,14, 571–577.CrossRefPubMedGoogle Scholar
  27. Radeau, M. (1992). Cognitive impenetrability in auditory-visual interaction. In J. Alegria, D. Holender, J. Morais, & M. Radeau (Eds.),Analytic approaches to human cognition (pp. 41–55). Amsterdam: Elsevier.Google Scholar
  28. Radeau, M. (1994). Auditory-visual interaction and modularity.Current Psychology of Cognition,13, 3–51.PubMedGoogle Scholar
  29. Radeau, M., &Bertelson, P. (1974). The after-effects of ventriloquism.Quarterly Journal of Experimental Psychology,26, 63–71.CrossRefPubMedGoogle Scholar
  30. Radeau, M., &Bertelson, P. (1976). The effect of a textured visual field on modality dominance in a ventriloquism situation.Perception & Psychophysics,20, 227–235.CrossRefGoogle Scholar
  31. Radeau.M., &Bertelson, P. (1977). Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations.Perception & Psychophysics,22, 137–146.CrossRefGoogle Scholar
  32. Radeau, M., &Bertelson.P. (1987). Auditory-visual interaction and the timing of inputs: Thomas (1941) revisited.Psychological Research,49, 17–22.CrossRefPubMedGoogle Scholar
  33. Reisberg, D. (1978). Looking where you listen: Visual cues and auditory attention.Acta Psychologica,72, 331–341.CrossRefGoogle Scholar
  34. Spence, C, &Driver, J. (1996). Audiovisual links in endogenous covert spatial attention.Journal of Experimental Psychology: Human Perception & Performance,22, 1005–1030.CrossRefGoogle Scholar
  35. Spence, C, &Driver, J. (1997a). Audiovisual links in exogenous covert spatial orientation.Perception & Psychophysics,59, 1–22.CrossRefGoogle Scholar
  36. Spence, C, &Driver, J. (1997b). On measuring selective attention to an expected sensory modality.Perception & Psychophysics,59, 389–403.CrossRefGoogle Scholar
  37. Thomas, G. J. (1941 ). Experimental study of the influence of vision on sound localisation.Journal of Experimental Psychology,28, 167–177.CrossRefGoogle Scholar
  38. Treisman, A. (1988). Features and objects: The fourteenth Bartlett memorial lecture.Quarterlv Journal of Experimental Psvchology,40A, 201–238.Google Scholar
  39. Treisman, A., &Gelade, G. (1980). A feature-integration theory of attention.Cognitive Psychology,12, 97–136.CrossRefPubMedGoogle Scholar
  40. Warren, D. H., Welch, R. B., &Mccarthy, T. J. (1981). The role of visual-auditory”compellingness”in the ventriloquism effect: Implications for transitivity among the spatial senses.Perception & Psychophysics,30, 557–564.CrossRefGoogle Scholar
  41. Welch, R. B. (1994). The dissection of intersensory bias: Weighting for Radeau.Current Psychology of Cognition,13, 117–123.PubMedGoogle Scholar
  42. Welch, R. B., &Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy.Psychological Bulletin,88, 638–667.CrossRefPubMedGoogle Scholar
  43. Welch, R. B., &Warren, D. H. (1984). Intersensory interactions. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.),Handbook of perception and human performance: Vol. 1. Sensory processes and perception (chap. 25, pp. 1–36). New York: Wiley.Google Scholar
  44. Witkin, H. A., Wapner, S., &Leventhal, T. (1952). Sound localization with conflicting visual and auditory cues.Journal of Experimental Psychology,43, 58–67.CrossRefPubMedGoogle Scholar
  45. Yantis, S., &Jonides, J. (1990). Abrupt visual onsets and selective attention: Voluntary versus automatic allocation.Journal of Experimental Psychology: Human Perception & Performance,16, 121–134.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2000

Authors and Affiliations

  • Paul Bertelson
    • 1
    • 2
  • Jean Vroomen
    • 2
  • Béatrice De Gelder
    • 1
    • 2
  • Jon Driver
    • 3
  1. 1.Free University of BrusselsBrusselsBelgium
  2. 2.Tilburg UniversityTilburgThe Netherlands
  3. 3.University College LondonLondonEngland

Personalised recommendations