Advertisement

Spatial and frequency specificity of the ventriloquism aftereffect revisited

  • Patrick BrunsEmail author
  • Brigitte Röder
Original Article

Abstract

Exposure to audiovisual stimuli with a consistent spatial misalignment seems to result in a recalibration of unisensory auditory spatial representations. The previous studies have suggested that this so-called ventriloquism aftereffect is confined to the trained region of space, but yielded inconsistent results as to whether or not recalibration generalizes to untrained sound frequencies. Here, we reassessed the spatial and frequency specificity of the ventriloquism aftereffect by testing whether auditory spatial perception can be independently recalibrated for two different sound frequencies and/or at two different spatial locations. Recalibration was confined to locations within the trained hemifield, suggesting that spatial representations were independently adjusted for the two hemifields. The frequency specificity of the ventriloquism aftereffect depended on the presence or the absence of conflicting audiovisual adaptation stimuli within the same hemifield. Moreover, adaptation of two different sound frequencies in opposite directions (leftward vs. rightward) resulted in a selective suppression of leftward recalibration, even when the adapting stimuli were presented in different hemifields. Thus, instead of representing a fixed stimulus-driven process, cross-modal recalibration seems to critically depend on the sensory context and takes into account inconsistencies in the cross-modal input.

Notes

Acknowledgements

This research was supported by Grants from the German Research Foundation (DFG) [BR 4913/2-1 to P.B. and TRR 169 A1 to B.R.]. We thank Philipp Dehmel, Alexander Gornik and Samantha Schröder for help running participants.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257–262.CrossRefPubMedGoogle Scholar
  2. Altmann, C. F., Henning, M., Döring, M. K., & Kaiser, J. (2008). Effects of feature-selective attention on auditory pattern and location processing. NeuroImage, 41, 69–79.CrossRefPubMedGoogle Scholar
  3. Bertelson, P., & Aschersleben, G. (1998). Automatic visual bias of perceived auditory location. Psychonomic Bulletin and Review, 5, 482–489.CrossRefGoogle Scholar
  4. Bertelson, P., Frissen, I., Vroomen, J., & de Gelder, B. (2006). The aftereffects of ventriloquism: Patterns of spatial generalization. Perception and Psychophysics, 68, 428–436.CrossRefPubMedGoogle Scholar
  5. Bertelson, P., & Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception and Psychophysics, 29, 578–584.CrossRefPubMedGoogle Scholar
  6. Bonath, B., Noesselt, T., Krauel, K., Tyll, S., Tempelmann, C., & Hillyard, S. A. (2014). Audio-visual synchrony modulates the ventriloquist illusion and its neural/spatial representation in the auditory cortex. NeuroImage, 98, 425–434.CrossRefPubMedGoogle Scholar
  7. Bonath, B., Noesselt, T., Martinez, A., Mishra, J., Schwiecker, K., Heinze, H.-J., & Hillyard, S. A. (2007). Neural basis of the ventriloquist illusion. Current Biology, 17, 1697–1703.CrossRefPubMedGoogle Scholar
  8. Bruns, P., Liebnau, R., & Röder, B. (2011). Cross-modal training induces changes in spatial representations early in the auditory processing pathway. Psychological Science, 22, 1120–1126.CrossRefPubMedGoogle Scholar
  9. Bruns, P., Maiworm, M., & Röder, B. (2014). Reward expectation influences audiovisual spatial integration. Attention Perception and Psychophysics, 76, 1815–1827.CrossRefGoogle Scholar
  10. Bruns, P., & Röder, B. (2015). Sensory recalibration integrates information from the immediate and the cumulative past. Scientific Reports, 5, 12739.CrossRefPubMedPubMedCentralGoogle Scholar
  11. Bruns, P., Spence, C., & Röder, B. (2011). Tactile recalibration of auditory spatial representations. Experimental Brain Research, 209, 333–344.CrossRefPubMedGoogle Scholar
  12. Callan, A., Callan, D., & Ando, H. (2015). An fMRI study of the ventriloquism effect. Cerebral Cortex, 25, 4248–4258.CrossRefPubMedPubMedCentralGoogle Scholar
  13. Chen, L., & Vroomen, J. (2013). Intersensory binding across space and time: A tutorial review. Attention Perception and Psychophysics, 75, 790–811.CrossRefGoogle Scholar
  14. Dyson, B. J., & Quinlan, P. T. (2004). Stimulus processing constraints in audition. Journal of Experimental Psychology Human Perception and Performance, 30, 1117–1131.CrossRefPubMedGoogle Scholar
  15. Eramudugolla, R., Kamke, M. R., Soto-Faraco, S., & Mattingley, J. B. (2011). Perceptual load influences auditory space perception in the ventriloquist aftereffect. Cognition, 118, 62–74.CrossRefPubMedGoogle Scholar
  16. Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8, 162–169.CrossRefPubMedGoogle Scholar
  17. Formisano, E., Kim, D.-S., Di Salle, F., van de Moortele, P.-F., Ugurbil, K., & Goebel, R. (2003). Mirror-symmetric tonotopic maps in human primary auditory cortex. Neuron, 40, 859–869.CrossRefPubMedGoogle Scholar
  18. Frissen, I., Vroomen, J., & de Gelder, B. (2012). The aftereffects of ventriloquism: The time course of the visual recalibration of auditory localization. Seeing and Perceiving, 25, 1–14.CrossRefPubMedGoogle Scholar
  19. Frissen, I., Vroomen, J., de Gelder, B., & Bertelson, P. (2003). The aftereffects of ventriloquism: Are they sound-frequency specific? Acta Psychologica, 113, 315–327.CrossRefPubMedGoogle Scholar
  20. Frissen, I., Vroomen, J., de Gelder, B., & Bertelson, P. (2005). The aftereffects of ventriloquism: Generalization across sound-frequencies. Acta Psychologica, 118, 93–100.CrossRefPubMedGoogle Scholar
  21. Heron, J., Roach, N. W., Hanson, J. V. M., McGraw, P. V., & Whitaker, D. (2012). Audiovisual time perception is spatially specific. Experimental Brain Research, 218, 477–485.CrossRefPubMedPubMedCentralGoogle Scholar
  22. Heron, J., Roach, N. W., Whitaker, D., & Hanson, J. V. M. (2010). Attention regulates the plasticity of multisensory timing. European Journal of Neuroscience, 31, 1755–1762.CrossRefPubMedPubMedCentralGoogle Scholar
  23. Humphries, C., Liebenthal, E., & Binder, J. R. (2010). Tonotopic organization of human auditory cortex. NeuroImage, 50, 1202–1211.CrossRefPubMedPubMedCentralGoogle Scholar
  24. Ikumi, N., & Soto-Faraco, S. (2014). Selective attention modulates the direction of audio-visual temporal recalibration. PLoS One, 9, e99311.CrossRefPubMedPubMedCentralGoogle Scholar
  25. King, A. J. (2009). Visual influences on auditory spatial learning. Philosophical Transactions of the Royal Society B Biological Sciences, 364, 331–339.CrossRefGoogle Scholar
  26. Kopčo, N., Lin, I.-F., Shinn-Cunningham, B. G., & Groh, J. M. (2009). Reference frame of the ventriloquism aftereffect. Journal of Neuroscience, 29, 13809–13814.CrossRefPubMedPubMedCentralGoogle Scholar
  27. Lewald, J. (2002). Rapid adaptation to auditory-visual spatial disparity. Learning and Memory, 9, 268–278.CrossRefPubMedPubMedCentralGoogle Scholar
  28. Lewald, J., Ehrenstein, W. H., & Guski, R. (2001). Spatio-temporal constraints for auditory-visual integration. Behavioural Brain Research, 121, 69–79.CrossRefPubMedGoogle Scholar
  29. Magezi, D. A., & Krumbholz, K. (2010). Evidence for opponent-channel coding of interaural time differences in human auditory cortex. Journal of Neurophysiology, 104, 1997–2007.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Magosso, E., Cona, F., & Ursino, M. (2013). A neural network model can explain ventriloquism aftereffect and its generalization across sound frequencies. BioMed Research International, 2013, 475427.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Magosso, E., Cuppini, C., & Ursino, M. (2012). A neural network model of ventriloquism effect and aftereffect. PLoS One, 7, e42503.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Maiworm, M., Bellantoni, M., Spence, C., & Röder, B. (2012). When emotional valence modulates audiovisual integration. Attention Perception and Psychophysics, 74, 1302–1311.CrossRefGoogle Scholar
  33. McAlpine, D., & Grothe, B. (2003). Sound localization and delay lines—do mammals fit the model? Trends in Neurosciences, 26, 347–350.CrossRefPubMedGoogle Scholar
  34. Middlebrooks, J. C., & Green, D. M. (1991). Sound localization by human listeners. Annual Review of Psychology, 42, 135–159.CrossRefPubMedGoogle Scholar
  35. Miller, L. M., & Recanzone, G. H. (2009). Populations of auditory cortical neurons can accurately encode acoustic space across stimulus intensity. Proceedings of the National Academy of Sciences of the USA, 106, 5931–5935.CrossRefPubMedPubMedCentralGoogle Scholar
  36. Mondor, T. A., Zatorre, R. J., & Terrio, N. A. (1998). Constraints on the selection of auditory information. Journal of Experimental Psychology Human Perception and Performance, 24, 66–79.CrossRefGoogle Scholar
  37. Mullette-Gillman, O. A., Cohen, Y. E., & Groh, J. M. (2005). Eye-centered, head-centered, and complex coding of visual and auditory targets in the intraparietal sulcus. Journal of Neurophysiology, 94, 2331–2352.CrossRefPubMedGoogle Scholar
  38. Phillips, D. P., & Hall, S. E. (2005). Psychophysical evidence for adaptation of central auditory processors for interaural differences in time and level. Hearing Research, 202, 188–199.CrossRefPubMedGoogle Scholar
  39. Radeau, M., & Bertelson, P. (1974). The after-effects of ventriloquism. Quarterly Journal of Experimental Psychology, 26, 63–71.CrossRefPubMedGoogle Scholar
  40. Recanzone, G. H. (1998). Rapidly induced auditory plasticity: The ventriloquism aftereffect. Proceedings of the National Academy of Sciences of the USA, 95, 869–875.CrossRefPubMedPubMedCentralGoogle Scholar
  41. Recanzone, G. H. (2009). Interactions of auditory and visual stimuli in space and time. Hearing Research, 258, 89–99.CrossRefPubMedPubMedCentralGoogle Scholar
  42. Recanzone, G. H., & Sutter, M. L. (2008). The biological basis of audition. Annual Review of Psychology, 59, 119–142.CrossRefPubMedGoogle Scholar
  43. Roseboom, W., & Arnold, D. H. (2011). Twice upon a time: Multiple concurrent temporal recalibrations of audiovisual speech. Psychological Science, 22, 872–877.CrossRefPubMedGoogle Scholar
  44. Roseboom, W., Kawabe, T., & Nishida, S. (2013). Audio-visual temporal recalibration can be constrained by content cues regardless of spatial overlap. Frontiers in Psychology, 4, 189.CrossRefPubMedPubMedCentralGoogle Scholar
  45. Salminen, N. H., May, P. J. C., Alku, P., & Tiitinen, H. (2009). A population rate code of auditory space in the human cortex. PLoS One, 4, e7600.CrossRefPubMedPubMedCentralGoogle Scholar
  46. Sarlat, L., Warusfel, O., & Viaud-Delmon, I. (2006). Ventriloquism aftereffects occur in the rear hemisphere. Neuroscience Letters, 404, 324–329.CrossRefPubMedGoogle Scholar
  47. Shams, L., & Seitz, A. R. (2008). Benefits of multisensory learning. Trends in Cognitive Sciences, 12, 411–417.CrossRefPubMedGoogle Scholar
  48. Shams, L., Wozny, D. R., Kim, R., & Seitz, A. R. (2011). Influences of multisensory experience on subsequent unisensory processing. Frontiers in Psychology, 2, 264.CrossRefPubMedPubMedCentralGoogle Scholar
  49. Shrem, T., & Deouell, L. Y. (2014). Frequency-dependent auditory space representation in the human planum temporale. Frontiers in Human Neuroscience, 8, 524.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Slutsky, D. A., & Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. NeuroReport, 12, 7–10.CrossRefPubMedGoogle Scholar
  51. Stecker, G. C., Harrington, I. A., & Middlebrooks, J. C. (2005). Location coding by opponent neural populations in the auditory cortex. PLoS Biology, 3, e78.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Stricanne, B., Andersen, R. A., & Mazzoni, P. (1996). Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. Journal of Neurophysiology, 76, 2071–2076.CrossRefPubMedGoogle Scholar
  53. Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14, 400–410.CrossRefPubMedPubMedCentralGoogle Scholar
  54. Tardif, E., Spierer, L., Clarke, S., & Murray, M. M. (2008). Interactions between auditory ‘what’ and ‘where’ pathways revealed by enhanced near-threshold discrimination of frequency and position. Neuropsychologia, 46, 958–966.CrossRefPubMedGoogle Scholar
  55. Van der Burg, E., Awh, E., & Olivers, C. N. L. (2013). The capacity of audiovisual integration is limited to one item. Psychological Science, 24, 345–351.CrossRefPubMedPubMedCentralGoogle Scholar
  56. Vigneault-MacLean, B. K., Hall, S. E., & Phillips, D. P. (2007). The effects of lateralized adaptors on lateral position judgements of tones within and across frequency channels. Hearing Research, 224, 93–100.CrossRefPubMedGoogle Scholar
  57. Werner-Reiss, U., Kelly, K. A., Trause, A. S., Underhill, A. M., & Groh, J. M. (2003). Eye position affects activity in primary auditory cortex of primates. Current Biology, 13, 554–562.CrossRefPubMedGoogle Scholar
  58. Woods, T. M., & Recanzone, G. H. (2004). Visually induced plasticity of auditory spatial perception in macaques. Current Biology, 14, 1559–1564.CrossRefPubMedGoogle Scholar
  59. Wozny, D. R., & Shams, L. (2011). Recalibration of auditory space following milliseconds of cross-modal discrepancy. Journal of Neuroscience, 31, 4607–4612.CrossRefPubMedPubMedCentralGoogle Scholar
  60. Zaidel, A., Turner, A. H., & Angelaki, D. E. (2011). Multisensory calibration is independent of cue reliability. Journal of Neuroscience, 31, 13949–13962.CrossRefPubMedPubMedCentralGoogle Scholar
  61. Zierul, B., Röder, B., Tempelmann, C., Bruns, P., & Noesselt, T. (2017). The role of auditory cortex in the spatial ventriloquism aftereffect. NeuroImage, 162, 257–268.CrossRefPubMedGoogle Scholar
  62. Zwiers, M. P., van Opstal, A. J., & Paige, G. D. (2003). Plasticity in human sound localization induced by compressed spatial vision. Nature Neuroscience, 6, 175–181.CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Biological Psychology and NeuropsychologyUniversity of HamburgHamburgGermany
  2. 2.Department of Cognitive, Linguistic and Psychological SciencesBrown UniversityProvidenceUSA

Personalised recommendations