Advertisement

Experimental Brain Research

, Volume 166, Issue 3–4, pp 538–547 | Cite as

Low-level integration of auditory and visual motion signals requires spatial co-localisation

  • Georg F. MeyerEmail author
  • Sophie M. Wuerger
  • Florian Röhrbein
  • Christoph Zetzsche
Research Article

Abstract

It is well known that the detection thresholds for stationary auditory and visual signals are lower if the signals are presented bimodally rather than unimodally, provided the signals coincide in time and space. Recent work on auditory–visual motion detection suggests that the facilitation seen for stationary signals is not seen for motion signals. We investigate the conditions under which motion perception also benefits from the integration of auditory and visual signals. We show that the integration of cross-modal local motion signals that are matched in position and speed is consistent with thresholds predicted by a neural summation model. If the signals are presented in different hemi-fields, move in different directions, or both, then behavioural thresholds are predicted by a probability-summation model. We conclude that cross-modal signals have to be co-localised and co-incident for effective motion integration. We also argue that facilitation is only seen if the signals contain all localisation cues that would be produced by physical objects.

Keywords

Visual Signal Visual Motion Motion Signal Linear Summation Bimodal Stimulus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work was supported by the EU TMR projects SPHEAR and HOARSE and by the Royal Society. We are grateful to the subjects who took part in the experiments.

References

  1. Alais D, Burr D (2004) No direction-specific bimodal facilitation for audiovisual motion detection. Cogn Brain Res 19:185–194CrossRefGoogle Scholar
  2. Blauert J (1983) Spatial hearing. MIT Press, Cambridge, MAGoogle Scholar
  3. Frassinetti F, Bolognini N, Ladavas E (2002) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332–343CrossRefPubMedGoogle Scholar
  4. Graham N (1989) Visual pattern analyzers. Oxford University Press, LondonGoogle Scholar
  5. Hofbauer M, Wuerger SM, Meyer GF, Roehrbein M, Schill K, Zetzsche C (2004) Catching audio-visual mice: predicting the arrival time of auditory–visual motion signals. Cogn Affect Behav Neurosci 4:241–250PubMedGoogle Scholar
  6. Howard RJ, Brammer M, Wright I, Woodruff PW, Bullmore ET (1996) A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain. Curr Biol 6:1015–1019CrossRefPubMedGoogle Scholar
  7. Lewis JW, Beauchamp MS, DeYoe EA (2000) A comparison of visual and auditory motion processing in human cerebral cortex. Cereb Cortex 10:873–888Google Scholar
  8. McDonald JJ, Teder-Sälejärvi WA, Hillyard SA (2000) Involuntary orienting to sound improves visual perception. Nature 407:906–908CrossRefPubMedGoogle Scholar
  9. Meese T, Andersen SJ (2002) Spiral mechanisms are required to account for summation of complex motion components. Vision Res 42:1073–1080CrossRefPubMedGoogle Scholar
  10. Meredith MA, Stein BE (1996) Spatial determinants of multisensory integration in cat superior colliculus. J Neurophysiol 75:1843–1857PubMedGoogle Scholar
  11. Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurones. I. Temporal factors. J Neurosci 10:3215–3229Google Scholar
  12. Meyer G, Wuerger S (2001) Cross-modal integration of auditory and visual motion signals. Neuroreport 12:2557–2600CrossRefPubMedGoogle Scholar
  13. Mullen KT, Sankeralli MJ (1998) Evidence for the stochastic independence of the blue-yellow, red-green and luminance detection mechanisms revealed by subthreshold summation. Vision Res 39:733–745CrossRefGoogle Scholar
  14. Quick RF (1974) A vector magnitude model of contrast detection. Kybernetik 16:65–67CrossRefPubMedGoogle Scholar
  15. Röhrbein F, Zetzsche C (2000) Auditory–visual interactions and the covariance structure generated by relative movements in natural environments. In: Guidati G, Hunt H, Heiss A (eds) Proceedings of the 7th International Congress on Sound and Vibration. International Institute of Acoustics and Vibration. Kramer Technology Publishing, Munich, pp 2427–2434Google Scholar
  16. Sanabria D, Soto-Faraco S, Spence C (2004) Exploring the role of visual perceptual grouping on the audiovisual integration of motion. Neuroreport 15:2745–2749PubMedGoogle Scholar
  17. Soto-Faraco S, Lyons J, Gazzaniga M, Spence C, Kingstone A (2002) The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. Cogn Brain Res 14:139–146CrossRefGoogle Scholar
  18. Soto-Faraco S, Kingstone A, Spence C (2003) Multisensory contributions to the perception of motion. Neuropsychologia 41:1847–1862CrossRefPubMedGoogle Scholar
  19. Soto-Faraco S, Spence C, Kingstone A (2004) Moving multisensory research along: motion perception across sensory modalities. Curr Direct Psychol Sci 13:29–32CrossRefGoogle Scholar
  20. Spence C, Driver J (1996) Audiovisual links in endogenous covert spatial attention. J Exp Psychol Hum Percept Perform 22(4):1005–1030CrossRefPubMedGoogle Scholar
  21. Spence C, Driver J (1997) Audiovisual links in exogenous covert spatial orienting. Percept Psychophys 59:1–22PubMedGoogle Scholar
  22. Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Cambridge, MAGoogle Scholar
  23. Tyler CW, Chen C-C (2000) Signal detection theory in the 2AFC paradigm: attention, channel uncertainty and probability summation. Vision Res 40:3121–3144CrossRefPubMedGoogle Scholar
  24. Wallace MT, Stein BE (1997) Development of multisensory neurons and multisensory integration in cat superior colliculus. J Neurosci 17:2429–2444PubMedGoogle Scholar
  25. Wallace MT, Stein BE (2001) Sensory and multisensory responses in the newborn monkey superior colliculus. J Neurosci 21:8886–8894PubMedGoogle Scholar
  26. Wallace MT, Meredith MA, Stein BE (1992) Integration of multiple sensory modalities in cat cortex. Exp Brain Res 91:484–488CrossRefPubMedGoogle Scholar
  27. Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80:1006–1010Google Scholar
  28. Watson AB, Pelli D (1983) QUEST: a Bayesian adaptive psychometric method. Percept Psychophys 33:113–120PubMedGoogle Scholar
  29. Wuerger SM, Hofbauer M, Meyer GF (2003) The integration of auditory and visual motion signals at threshold. Percept Psychophys 65:1188–1196PubMedGoogle Scholar

Copyright information

© Springer-Verlag 2005

Authors and Affiliations

  • Georg F. Meyer
    • 1
    Email author
  • Sophie M. Wuerger
    • 1
  • Florian Röhrbein
    • 2
    • 3
  • Christoph Zetzsche
    • 2
  1. 1.Centre for Cognitive Neuroscience, School of PsychologyUniversity of LiverpoolLiverpoolUK
  2. 2.Cognitive Neuroinformatics, School of Mathematics and Computer ScienceBremen UniversityBremenGermany
  3. 3.HONDA Research Institute EuropeOffenbachGermany

Personalised recommendations