Skip to main content
SpringerLink
Log in

Spatial and frequency specificity of the ventriloquism aftereffect revisited

  • Original Article
  • Published:
Psychological Research Aims and scope Submit manuscript

Abstract

Exposure to audiovisual stimuli with a consistent spatial misalignment seems to result in a recalibration of unisensory auditory spatial representations. The previous studies have suggested that this so-called ventriloquism aftereffect is confined to the trained region of space, but yielded inconsistent results as to whether or not recalibration generalizes to untrained sound frequencies. Here, we reassessed the spatial and frequency specificity of the ventriloquism aftereffect by testing whether auditory spatial perception can be independently recalibrated for two different sound frequencies and/or at two different spatial locations. Recalibration was confined to locations within the trained hemifield, suggesting that spatial representations were independently adjusted for the two hemifields. The frequency specificity of the ventriloquism aftereffect depended on the presence or the absence of conflicting audiovisual adaptation stimuli within the same hemifield. Moreover, adaptation of two different sound frequencies in opposite directions (leftward vs. rightward) resulted in a selective suppression of leftward recalibration, even when the adapting stimuli were presented in different hemifields. Thus, instead of representing a fixed stimulus-driven process, cross-modal recalibration seems to critically depend on the sensory context and takes into account inconsistencies in the cross-modal input.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257–262.

    Article  PubMed  Google Scholar 

  • Altmann, C. F., Henning, M., Döring, M. K., & Kaiser, J. (2008). Effects of feature-selective attention on auditory pattern and location processing. NeuroImage, 41, 69–79.

    Article  PubMed  Google Scholar 

  • Bertelson, P., & Aschersleben, G. (1998). Automatic visual bias of perceived auditory location. Psychonomic Bulletin and Review, 5, 482–489.

    Article  Google Scholar 

  • Bertelson, P., Frissen, I., Vroomen, J., & de Gelder, B. (2006). The aftereffects of ventriloquism: Patterns of spatial generalization. Perception and Psychophysics, 68, 428–436.

    Article  PubMed  Google Scholar 

  • Bertelson, P., & Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception and Psychophysics, 29, 578–584.

    Article  PubMed  Google Scholar 

  • Bonath, B., Noesselt, T., Krauel, K., Tyll, S., Tempelmann, C., & Hillyard, S. A. (2014). Audio-visual synchrony modulates the ventriloquist illusion and its neural/spatial representation in the auditory cortex. NeuroImage, 98, 425–434.

    Article  PubMed  Google Scholar 

  • Bonath, B., Noesselt, T., Martinez, A., Mishra, J., Schwiecker, K., Heinze, H.-J., & Hillyard, S. A. (2007). Neural basis of the ventriloquist illusion. Current Biology, 17, 1697–1703.

    Article  PubMed  Google Scholar 

  • Bruns, P., Liebnau, R., & Röder, B. (2011). Cross-modal training induces changes in spatial representations early in the auditory processing pathway. Psychological Science, 22, 1120–1126.

    Article  PubMed  Google Scholar 

  • Bruns, P., Maiworm, M., & Röder, B. (2014). Reward expectation influences audiovisual spatial integration. Attention Perception and Psychophysics, 76, 1815–1827.

    Article  Google Scholar 

  • Bruns, P., & Röder, B. (2015). Sensory recalibration integrates information from the immediate and the cumulative past. Scientific Reports, 5, 12739.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bruns, P., Spence, C., & Röder, B. (2011). Tactile recalibration of auditory spatial representations. Experimental Brain Research, 209, 333–344.

    Article  PubMed  Google Scholar 

  • Callan, A., Callan, D., & Ando, H. (2015). An fMRI study of the ventriloquism effect. Cerebral Cortex, 25, 4248–4258.

    Article  PubMed  Google Scholar 

  • Chen, L., & Vroomen, J. (2013). Intersensory binding across space and time: A tutorial review. Attention Perception and Psychophysics, 75, 790–811.

    Article  Google Scholar 

  • Dyson, B. J., & Quinlan, P. T. (2004). Stimulus processing constraints in audition. Journal of Experimental Psychology Human Perception and Performance, 30, 1117–1131.

    Article  PubMed  Google Scholar 

  • Eramudugolla, R., Kamke, M. R., Soto-Faraco, S., & Mattingley, J. B. (2011). Perceptual load influences auditory space perception in the ventriloquist aftereffect. Cognition, 118, 62–74.

    Article  PubMed  Google Scholar 

  • Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8, 162–169.

    Article  PubMed  Google Scholar 

  • Formisano, E., Kim, D.-S., Di Salle, F., van de Moortele, P.-F., Ugurbil, K., & Goebel, R. (2003). Mirror-symmetric tonotopic maps in human primary auditory cortex. Neuron, 40, 859–869.

    Article  PubMed  Google Scholar 

  • Frissen, I., Vroomen, J., & de Gelder, B. (2012). The aftereffects of ventriloquism: The time course of the visual recalibration of auditory localization. Seeing and Perceiving, 25, 1–14.

    Article  PubMed  Google Scholar 

  • Frissen, I., Vroomen, J., de Gelder, B., & Bertelson, P. (2003). The aftereffects of ventriloquism: Are they sound-frequency specific? Acta Psychologica, 113, 315–327.

    Article  PubMed  Google Scholar 

  • Frissen, I., Vroomen, J., de Gelder, B., & Bertelson, P. (2005). The aftereffects of ventriloquism: Generalization across sound-frequencies. Acta Psychologica, 118, 93–100.

    Article  PubMed  Google Scholar 

  • Heron, J., Roach, N. W., Hanson, J. V. M., McGraw, P. V., & Whitaker, D. (2012). Audiovisual time perception is spatially specific. Experimental Brain Research, 218, 477–485.

    Article  PubMed  PubMed Central  Google Scholar 

  • Heron, J., Roach, N. W., Whitaker, D., & Hanson, J. V. M. (2010). Attention regulates the plasticity of multisensory timing. European Journal of Neuroscience, 31, 1755–1762.

    Article  PubMed  Google Scholar 

  • Humphries, C., Liebenthal, E., & Binder, J. R. (2010). Tonotopic organization of human auditory cortex. NeuroImage, 50, 1202–1211.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ikumi, N., & Soto-Faraco, S. (2014). Selective attention modulates the direction of audio-visual temporal recalibration. PLoS One, 9, e99311.

    Article  PubMed  PubMed Central  Google Scholar 

  • King, A. J. (2009). Visual influences on auditory spatial learning. Philosophical Transactions of the Royal Society B Biological Sciences, 364, 331–339.

    Article  Google Scholar 

  • Kopčo, N., Lin, I.-F., Shinn-Cunningham, B. G., & Groh, J. M. (2009). Reference frame of the ventriloquism aftereffect. Journal of Neuroscience, 29, 13809–13814.

    Article  PubMed  Google Scholar 

  • Lewald, J. (2002). Rapid adaptation to auditory-visual spatial disparity. Learning and Memory, 9, 268–278.

    Article  PubMed  Google Scholar 

  • Lewald, J., Ehrenstein, W. H., & Guski, R. (2001). Spatio-temporal constraints for auditory-visual integration. Behavioural Brain Research, 121, 69–79.

    Article  PubMed  Google Scholar 

  • Magezi, D. A., & Krumbholz, K. (2010). Evidence for opponent-channel coding of interaural time differences in human auditory cortex. Journal of Neurophysiology, 104, 1997–2007.

    Article  PubMed  PubMed Central  Google Scholar 

  • Magosso, E., Cona, F., & Ursino, M. (2013). A neural network model can explain ventriloquism aftereffect and its generalization across sound frequencies. BioMed Research International, 2013, 475427.

    Article  PubMed  PubMed Central  Google Scholar 

  • Magosso, E., Cuppini, C., & Ursino, M. (2012). A neural network model of ventriloquism effect and aftereffect. PLoS One, 7, e42503.

    Article  PubMed  PubMed Central  Google Scholar 

  • Maiworm, M., Bellantoni, M., Spence, C., & Röder, B. (2012). When emotional valence modulates audiovisual integration. Attention Perception and Psychophysics, 74, 1302–1311.

    Article  Google Scholar 

  • McAlpine, D., & Grothe, B. (2003). Sound localization and delay lines—do mammals fit the model? Trends in Neurosciences, 26, 347–350.

    Article  PubMed  Google Scholar 

  • Middlebrooks, J. C., & Green, D. M. (1991). Sound localization by human listeners. Annual Review of Psychology, 42, 135–159.

    Article  PubMed  Google Scholar 

  • Miller, L. M., & Recanzone, G. H. (2009). Populations of auditory cortical neurons can accurately encode acoustic space across stimulus intensity. Proceedings of the National Academy of Sciences of the USA, 106, 5931–5935.

    Article  PubMed  Google Scholar 

  • Mondor, T. A., Zatorre, R. J., & Terrio, N. A. (1998). Constraints on the selection of auditory information. Journal of Experimental Psychology Human Perception and Performance, 24, 66–79.

    Article  Google Scholar 

  • Mullette-Gillman, O. A., Cohen, Y. E., & Groh, J. M. (2005). Eye-centered, head-centered, and complex coding of visual and auditory targets in the intraparietal sulcus. Journal of Neurophysiology, 94, 2331–2352.

    Article  PubMed  Google Scholar 

  • Phillips, D. P., & Hall, S. E. (2005). Psychophysical evidence for adaptation of central auditory processors for interaural differences in time and level. Hearing Research, 202, 188–199.

    Article  PubMed  Google Scholar 

  • Radeau, M., & Bertelson, P. (1974). The after-effects of ventriloquism. Quarterly Journal of Experimental Psychology, 26, 63–71.

    Article  PubMed  Google Scholar 

  • Recanzone, G. H. (1998). Rapidly induced auditory plasticity: The ventriloquism aftereffect. Proceedings of the National Academy of Sciences of the USA, 95, 869–875.

    Article  PubMed  Google Scholar 

  • Recanzone, G. H. (2009). Interactions of auditory and visual stimuli in space and time. Hearing Research, 258, 89–99.

    Article  PubMed  PubMed Central  Google Scholar 

  • Recanzone, G. H., & Sutter, M. L. (2008). The biological basis of audition. Annual Review of Psychology, 59, 119–142.

    Article  PubMed  Google Scholar 

  • Roseboom, W., & Arnold, D. H. (2011). Twice upon a time: Multiple concurrent temporal recalibrations of audiovisual speech. Psychological Science, 22, 872–877.

    Article  PubMed  Google Scholar 

  • Roseboom, W., Kawabe, T., & Nishida, S. (2013). Audio-visual temporal recalibration can be constrained by content cues regardless of spatial overlap. Frontiers in Psychology, 4, 189.

    Article  PubMed  PubMed Central  Google Scholar 

  • Salminen, N. H., May, P. J. C., Alku, P., & Tiitinen, H. (2009). A population rate code of auditory space in the human cortex. PLoS One, 4, e7600.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sarlat, L., Warusfel, O., & Viaud-Delmon, I. (2006). Ventriloquism aftereffects occur in the rear hemisphere. Neuroscience Letters, 404, 324–329.

    Article  PubMed  Google Scholar 

  • Shams, L., & Seitz, A. R. (2008). Benefits of multisensory learning. Trends in Cognitive Sciences, 12, 411–417.

    Article  PubMed  Google Scholar 

  • Shams, L., Wozny, D. R., Kim, R., & Seitz, A. R. (2011). Influences of multisensory experience on subsequent unisensory processing. Frontiers in Psychology, 2, 264.

    Article  PubMed  PubMed Central  Google Scholar 

  • Shrem, T., & Deouell, L. Y. (2014). Frequency-dependent auditory space representation in the human planum temporale. Frontiers in Human Neuroscience, 8, 524.

    Article  PubMed  PubMed Central  Google Scholar 

  • Slutsky, D. A., & Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. NeuroReport, 12, 7–10.

    Article  PubMed  Google Scholar 

  • Stecker, G. C., Harrington, I. A., & Middlebrooks, J. C. (2005). Location coding by opponent neural populations in the auditory cortex. PLoS Biology, 3, e78.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stricanne, B., Andersen, R. A., & Mazzoni, P. (1996). Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. Journal of Neurophysiology, 76, 2071–2076.

    Article  PubMed  Google Scholar 

  • Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14, 400–410.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tardif, E., Spierer, L., Clarke, S., & Murray, M. M. (2008). Interactions between auditory ‘what’ and ‘where’ pathways revealed by enhanced near-threshold discrimination of frequency and position. Neuropsychologia, 46, 958–966.

    Article  PubMed  Google Scholar 

  • Van der Burg, E., Awh, E., & Olivers, C. N. L. (2013). The capacity of audiovisual integration is limited to one item. Psychological Science, 24, 345–351.

    Article  PubMed  PubMed Central  Google Scholar 

  • Vigneault-MacLean, B. K., Hall, S. E., & Phillips, D. P. (2007). The effects of lateralized adaptors on lateral position judgements of tones within and across frequency channels. Hearing Research, 224, 93–100.

    Article  PubMed  Google Scholar 

  • Werner-Reiss, U., Kelly, K. A., Trause, A. S., Underhill, A. M., & Groh, J. M. (2003). Eye position affects activity in primary auditory cortex of primates. Current Biology, 13, 554–562.

    Article  PubMed  Google Scholar 

  • Woods, T. M., & Recanzone, G. H. (2004). Visually induced plasticity of auditory spatial perception in macaques. Current Biology, 14, 1559–1564.

    Article  PubMed  Google Scholar 

  • Wozny, D. R., & Shams, L. (2011). Recalibration of auditory space following milliseconds of cross-modal discrepancy. Journal of Neuroscience, 31, 4607–4612.

    Article  PubMed  Google Scholar 

  • Zaidel, A., Turner, A. H., & Angelaki, D. E. (2011). Multisensory calibration is independent of cue reliability. Journal of Neuroscience, 31, 13949–13962.

    Article  PubMed  Google Scholar 

  • Zierul, B., Röder, B., Tempelmann, C., Bruns, P., & Noesselt, T. (2017). The role of auditory cortex in the spatial ventriloquism aftereffect. NeuroImage, 162, 257–268.

    Article  PubMed  Google Scholar 

  • Zwiers, M. P., van Opstal, A. J., & Paige, G. D. (2003). Plasticity in human sound localization induced by compressed spatial vision. Nature Neuroscience, 6, 175–181.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research was supported by Grants from the German Research Foundation (DFG) [BR 4913/2-1 to P.B. and TRR 169 A1 to B.R.]. We thank Philipp Dehmel, Alexander Gornik and Samantha Schröder for help running participants.

Author information

Authors and Affiliations

  1. Biological Psychology and Neuropsychology, University of Hamburg, Von-Melle-Park 11, 20146, Hamburg, Germany

    Patrick Bruns & Brigitte Röder

  2. Department of Cognitive, Linguistic and Psychological Sciences, Brown University, 190 Thayer Street, Providence, RI, 02912, USA

    Patrick Bruns

Authors
  1. Patrick Bruns
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brigitte Röder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Patrick Bruns.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bruns, P., Röder, B. Spatial and frequency specificity of the ventriloquism aftereffect revisited. Psychological Research 83, 1400–1415 (2019). https://doi.org/10.1007/s00426-017-0965-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00426-017-0965-4

Search

Navigation