Experimental Brain Research

, Volume 165, Issue 4, pp 505–514 | Cite as

Spatiotemporal interactions between audition and touch depend on hand posture

  • Daniel Sanabria
  • Salvador Soto-Faraco
  • Charles Spence
Research Article


We report two experiments designed to assess the consequences of posture change on audiotactile spatiotemporal interactions. In Experiment 1, participants had to discriminate the direction of an auditory stream (consisting of the sequential presentation of two tones from different spatial positions) while attempting to ignore a task-irrelevant tactile stream (consisting of the sequential presentation of two vibrations, one to each of the participant’s hands). The tactile stream presented to the participants’ hands was either spatiotemporally congruent or incongruent with respect to the sounds. A significant decrease in performance in incongruent trials compared with congruent trials was demonstrated when the participants adopted an uncrossed-hands posture but not when their hands were crossed over the midline. In Experiment 2, we investigated the ability of participants to discriminate the direction of two sequentially presented tactile stimuli (one presented to each hand) as a function of the presence of congruent vs incongruent auditory distractors. Here, the crossmodal effect was stronger in the crossed-hands posture than in the uncrossed-hands posture. These results demonstrate the reciprocal nature of audiotactile interactions in spatiotemporal processing, and highlight the important role played by body posture in modulating such crossmodal interactions.


Crossmodal interactions Spatiotemporal processing Audition Touch Posture change 


  1. Adelstein BD, Begault DR, Anderson MR, Wenzel EM (2003) Sensitivity to haptic-audio asynchrony. Proceedings of the Fifth International Conference on Multimodal Interfaces, New York. The Association for Computing Machinery Press, pp 73–76Google Scholar
  2. Alais D, Burr D (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14:257–262CrossRefPubMedGoogle Scholar
  3. Andersen TS, Tiippana K, Sams M (2004) Factors influencing audiovisual fission and fusion illusions. Cognit Brain Res 21:301–308CrossRefGoogle Scholar
  4. Battaglia PW, Jacobs RA, Aslin, RN (2003) Bayesian integration of visual and auditory signals for spatial localization. J Opt Soc Am 20:1391–1397Google Scholar
  5. Bresciani JP, Ernst MO, Drewing K, Bouyer G, Maury V, Kheddar A (2005) Feeling what you hear: auditory signals can modulate tactile tap perception. Exp Brain Res 162:172–180CrossRefPubMedGoogle Scholar
  6. Burton H, Sinclair R (1996) Somatosensory cortex and tactile perceptions. In: Kruger L (ed) Pain and touch. Academic, San Diego, pp 105–177Google Scholar
  7. Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile ‘capture’ of audition. Percept Psychophys 64:616–630PubMedGoogle Scholar
  8. Eimer M, Cockburn D, Smedley B, Driver J (2001) Cross-modal links in endogenous spatial attention are mediated by common external locations: evidence from event-related brain potentials. Exp Brain Res 139:398–411CrossRefPubMedGoogle Scholar
  9. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433CrossRefPubMedGoogle Scholar
  10. Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169CrossRefPubMedGoogle Scholar
  11. Foxe JJ, Morocz IA, Murray MM, Higgings BA, Javitt DC, Schroeder CE (2000) Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cogn Brain Res 10:77–83CrossRefGoogle Scholar
  12. Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM (2002) Auditory–somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540–543PubMedGoogle Scholar
  13. Fu KMG, Johnston TA, Shah AS, Arnold L, Smiley J, Hackett TA, Garraghty PE, Schroeder CE (2003) Auditory cortical neurons respond to somatosensory stimulation. J Neurosci 23:7510–7505PubMedGoogle Scholar
  14. Gobbelé R, Schürrmann M, Forss N, Juottonen K, Buchner H, Hari R (2003) Activation of the human posterior and tempoparietal cortices during audiotactile interaction. Neuroimage 20:503–511CrossRefPubMedGoogle Scholar
  15. Graziano MSA, Ghandi S (2000) Location of the polysensory zone in the precentral gyrus of anesthetized monkeys. Exp Brain Res 135:259–266CrossRefPubMedGoogle Scholar
  16. Graziano MSA, Gross CG, Taylor CSR, Moore T (2004) A system of multimodal areas in the primate brain. In: Spence C, Driver J (eds) Crossmodal space and crossmodal attention. Oxford University Press, Oxford, pp 51–69Google Scholar
  17. Hötting K, Röder B (2004) Hearing cheats touch, but less in congenitally blind than in sighted individuals. Psychol Sci 15:60–64CrossRefPubMedGoogle Scholar
  18. Kim D-H, Cruse H (2001) Two kinds of body representation are used to control hand movements following tactile stimulation. Exp Brain Res 139:76–91CrossRefPubMedGoogle Scholar
  19. Lloyd DM, Merat N, McGlone F, Spence C (2003) Crossmodal links between audition and touch in covert endogenous spatial attention. Percept Psychophys 65:901–924PubMedGoogle Scholar
  20. Lütkenhöner B, Lammertmann C, Simöes C, Hari R (2002) Magnetoencephalographic correlates of audiotactile interaction. Neuroimage 15:509–522CrossRefPubMedGoogle Scholar
  21. Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Shroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory–somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex (in press)Google Scholar
  22. Pallier C, Dupoux E, Jeanin X (1997) EXPE: an expandable programming language for psychological experiments. Behav Res Methods Instrum Comput 29:322–327Google Scholar
  23. Penfield W, Rasmussen T (1950) The cerebral cortex of man. Macmillan, New YorkGoogle Scholar
  24. Pouget A, Deneve S, Duhamel JR (2004) A computational neural theory of multisensory spatial representations. In: Spence C, Driver J (eds) Crossmodal space and crossmodal attention. Oxford University Press, Oxford, pp 123–141Google Scholar
  25. Röder B, Spence C, Rösler F (2000) Inhibition of return and oculomotor control in the blind. Neuroreport 11:3043–3045PubMedGoogle Scholar
  26. Röder B, Rösler F, Spence C (2004) Early vision impairs tactile perception in the blind. Curr Biol 14:121–124PubMedGoogle Scholar
  27. Sanabria D, Soto-Faraco S, Chan J, Spence C (2004a) When does intramodal perceptual grouping affect multisensory integration? Cogn Affect Behav Neurosci 4:218–229PubMedGoogle Scholar
  28. Sanabria D, Soto-Faraco S, Chan J, Spence C (2004b) Exploring the role of visual perceptual grouping on the audiovisual integration of motion. Neuroreport 15:2745–2749PubMedGoogle Scholar
  29. Sanabria D, Soto-Faraco S, Chan J, Spence C (2005) Intramodal perceptual grouping modulates crossmodal interaction: evidence from the crossmodal dynamic capture task. Neurosci Lett 377:59–64CrossRefPubMedGoogle Scholar
  30. Schroeder CE, Foxe JJ (2002) Timing and laminar profile of converging inputs in multisensory areas of the macaque neocortex. Cogn Brain Res 14:195–207CrossRefGoogle Scholar
  31. Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322–1327PubMedGoogle Scholar
  32. Schroeder CE, Smiley J, Fu KMG, McGinnis T, O’Connell MN, Hackett TA (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50:5–18CrossRefPubMedGoogle Scholar
  33. Shams L, Kamitani Y, Shimojo S (2000) What we see is what you hear. Nature 408:788CrossRefPubMedGoogle Scholar
  34. Sherrick CE (1976) The antagonisms of hearing and touch. In: Hirsh SK, Eldredge DH, Hirsh IJ, Silverman SR (eds) Hearing and Davis: essays honouring Hallowell Davis. Washington University Press, St. Louis, pp 149–158Google Scholar
  35. Shore D, Spry E, Spence C (2002) Confusing the mind by crossing the hands. Cogn Brain Res 14:153–163CrossRefGoogle Scholar
  36. Soto-Faraco S, Lyons J, Gazzaniga M, Spence C, Kingstone A (2002) The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. Cogn Brain Res 14:139–146CrossRefGoogle Scholar
  37. Soto-Faraco S, Spence C, Kingstone A (2004a) Congruency effects between auditory and tactile motion: extending the crossmodal dynamic capture effect. Cogn Affect Behav Neurosci 4:208–217PubMedGoogle Scholar
  38. Soto-Faraco S, Spence C, Kingstone A (2004b) Crossmodal dynamic capture: congruency effects of motion perception across sensory modalities. J Exp Psychol: HPP 30:330–345Google Scholar
  39. Soto-Faraco S, Spence C, Kingstone A (2005) Assessing automaticity in the audio-visual integration of motion. Acta Psychol 118:71–92CrossRefGoogle Scholar
  40. Spence C, Walton M (2005) On the inability to ignore touch when responding to vision in the crossmodal congruency task. Acta Psychol 118:47–70CrossRefGoogle Scholar
  41. Spence C, Baddeley R, Zampini M, James R, Shore D (2003) Multisensory temporal order judgments: when two locations are better than one. Percept Psychophys 65:318–328PubMedGoogle Scholar
  42. Spence C, Nicholls MER, Gillespie N, Driver J (1998) Crossmodal links in exogenous covert spatial orienting between touch, audition, and vision. Percept Psychophys 60:544–557PubMedGoogle Scholar
  43. Strybel TZ, Vatakis A (2004) A comparison of auditory and visual apparent motion presented individually and with crossmodal moving distractors. Perception 33:1033–1048CrossRefPubMedGoogle Scholar
  44. Taylor MM, Creelman CD (1967) PEST: efficient estimates on probability functions. J Acoust Soc Am 41:782–787Google Scholar
  45. Wada M, Yamamoto S, Kitazawa S (2004) Effects of handedness on tactile temporal order judgments. Neuropsychologica 42:1887–1895CrossRefGoogle Scholar
  46. Welch RB, Warren DH (1980) Immediate perceptual response to intersensory discrepancy. Psychol Bull 3:638–667Google Scholar
  47. Yamamoto S, Kitazawa S (2001) Reversal of subjective temporal order due to arm crossing. Nat Neurosci 4:759–765CrossRefPubMedGoogle Scholar
  48. Zampini M, Brown T, Shore DI, Maravita A, Röder B, Spence C (2005) Audiotactile temporal order judgments. Acta Psychol 118:277–291CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2005

Authors and Affiliations

  • Daniel Sanabria
    • 1
  • Salvador Soto-Faraco
    • 1
    • 2
  • Charles Spence
    • 1
  1. 1.Department of Experimental PsychologyUniversity of OxfordOxfordUK
  2. 2.Cognitive Neuroscience Group-Parc CientíficUniversitat de BarcelonaSpain

Personalised recommendations