Experimental Brain Research

, 189:133 | Cite as

Multisensory enhancement: gains in choice and in simple response times

  • David HechtEmail author
  • Miriam Reiner
  • Avi Karni
Research Article


Human observers can detect combinations of multisensory signals faster than each of the corresponding signals presented separately. In simple detection tasks, this facilitation in response times may reflect an enhancement in the perceptual processing stage or/and in the motor response stage. The current study compared the multisensory enhancements obtained in simple and choice response times (SRT and CRT, respectively) in bi- and tri-sensory (audio–visual–haptic) signal combinations using an identical experimental setup that differed only in the tasks—detecting the signals (SRT) or reporting the signals’ location (CRT). Our measurements show that RTs were faster in the multisensory combinations conditions compared to the single stimulus conditions and that the absolute multisensory gains were larger in CRT than in SRT. These results can be interpreted in two ways. According to a serial stages model, the larger multisensory gains in CRT may suggest that when combinations of multisensory signals are presented, an additional enhancement occurs in the cognitive processing stages engaged in the CRT, beyond the enhancement in the perceptual and motor stages common to both SRT and CRT. Alternatively, the results suggest that multisensory enhancement reflect task-dependent interactions within and between multiple processing levels rather than facilitated processing modules. Thus, the larger absolute multisensory gains in CRT may reflect the inverse effectiveness principle, and Bayesian statistics, in that the maximal multisensory enhancements occur in the more difficult (less precise) uni-sensory conditions, i.e., in the CRT.


Multisensory enhancement Simple reaction time Choice reaction time Inverse effectiveness 



This research was funded by the EU project IMMERSENCE—Multi-modal immersion into interactive virtual environments. We thank Mr. Gad Halevy for his help in programming the system for the experiment, and Mrs. Tatiana Gelfeld for her help in producing the CDF plots.


  1. Balas M (2006) Context and experience as factors in motor interference and motor representations: learning a finger opposition sequence and handwriting—behavioral and fMRI studies. Ph.D. thesis, University of Haifa, IsraelGoogle Scholar
  2. Bays PM, Wolpert DM (2007) Computational principles of sensorimotor control that minimize uncertainty and variability. J Physiol 278(2):387–396Google Scholar
  3. Delazer M, Lochy A, Jenner C, Domahs F, Benke T (2002) When writing 0 (zero) is easier than writing O (o): a neuropsychological case study of agraphia. Neuropsychologia 40(12):2167–2177PubMedCrossRefGoogle Scholar
  4. Diederich A, Colonius H (2004) Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66(8):1388–1404PubMedGoogle Scholar
  5. Diederich A, Colonius H, Bockhorst D, Tabeling S (2003) Visual–tactile spatial interaction in saccade generation. Exp Brain Res 148(3):328–37PubMedGoogle Scholar
  6. Diederich A, Colonius H (1987) Intersensory facilitation in the motor component? Psychol Res 49(1):23–29CrossRefGoogle Scholar
  7. Donders FC (1969) Over de snelheid van psychische processen (On the speed of psychological processes). In: Koster W (ed and trans) Attention and performance: II. North-Holland, Amsterdam (Original work published 1868)Google Scholar
  8. Doyle MC, Snowden RJ (2001) Identification of visual stimuli is improved by accompanying auditory stimuli: the role of eye movements and sound location. Perception 30(7):795–810PubMedCrossRefGoogle Scholar
  9. Forster B, Cavina-Pratesi C, Aglioti S, Berlucchi G (2002) Redundant target effect and intersensory facilitation from visual–tactile interactions in simple reaction time. Exp Brain Res 143(4):480–487PubMedCrossRefGoogle Scholar
  10. Fort A, Delpuech C, Pernier J, Giard MH (2002) Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb Cortex 12(10):1031–1039PubMedCrossRefGoogle Scholar
  11. Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res Cogn Brain Res 10(1):77–83PubMedCrossRefGoogle Scholar
  12. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25(20):5004–5012PubMedCrossRefGoogle Scholar
  13. Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11(5):473–490PubMedCrossRefGoogle Scholar
  14. Giray M, Ulrich R (1993) Motor coactivation revealed by response force in divided and focused attention. J Exp Psychol Hum Percept Perform 19(6):1278–91PubMedCrossRefGoogle Scholar
  15. Gobbelé R, Schürmann M, Forss N, Juottonen K, Buchner H, Hari R (2003) Activation of the human posterior and temporoparietal cortices during audiotactile interaction. Neuroimage 20(1):503–511PubMedCrossRefGoogle Scholar
  16. Gottfried JA, Dolan RJ (2003) The nose smells what the eye sees: crossmodal visual facilitation of human olfactory perception. Neuron 39(2):375–386PubMedCrossRefGoogle Scholar
  17. Graziano MSA (2001) A system of multimodal areas in the primate brain. Neuron 29:4–6PubMedCrossRefGoogle Scholar
  18. Graziano MSA, Reiss LAJ, Gross CG (1999) A neuronal representation of the location of nearby sounds. Nature 397:428–430PubMedCrossRefGoogle Scholar
  19. Grill-Spector K, Kanwisher N (2005) Visual recognition. As soon as you know it is there, you know what it is. Psychol Sci 16(2):152–160PubMedCrossRefGoogle Scholar
  20. Hauptmann B, Karni A (2002) From primed to learn: the saturation of repetition priming and the induction of long-term memory. Brain Res Cogn Brain Res 13(3):313–322PubMedCrossRefGoogle Scholar
  21. Hecht D, Reiner M, Karni A (2008) Enhancement of response times to bi- and tri-modal sensory stimuli during active movements. Exp Brain Res 185(4):655–665PubMedCrossRefGoogle Scholar
  22. Heller MA (1982) Visual and tactual texture perception: intersensory cooperation. Percept Psychophys 31(4):339–344PubMedGoogle Scholar
  23. Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289–293PubMedCrossRefGoogle Scholar
  24. Honoré J, Bourdeaud’hui M, Sparrow L (1989) Reduction of cutaneous reaction time by directing eyes towards the source of stimulation. Neuropsychologia 27(3):367–371PubMedCrossRefGoogle Scholar
  25. Jiang W, Wallace MT, Jiang H, Vaughan JW, Stein BE (2001) Two cortical areas mediate multisensory integration in superior colliculus neurons. J Neurophysiol 85(2):506–522PubMedGoogle Scholar
  26. Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48(2):373–384PubMedCrossRefGoogle Scholar
  27. Körding KP (2007) Decision theory: what “should” the nervous system do? Science 318(5850):606–610PubMedCrossRefGoogle Scholar
  28. Körding KP, Wolpert DM (2006) Bayesian decision theory in sensorimotor control. Trends Cog Sci 10(7):320–326CrossRefGoogle Scholar
  29. Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT (2006) Enhanced multisensory integration in older adults. Neurobiol Aging 27(8):1155–1163PubMedCrossRefGoogle Scholar
  30. Lederman SJ, Klatzky RL, Hamilton C, Morgan T (2002) Integrating multimodal information about surface texture via a probe: relative contributions of haptic and touch-produced sound sources. In: Proceedings of the 10th annual symposium on haptic interfaces for teleoperators and virtual environment systems (IEEE VR ‘02), Orlando, Florida, USA, pp 97–104Google Scholar
  31. Luce RD (1991) Response times. Oxford University PressGoogle Scholar
  32. Mack ML, Gauthier I, Sadr J, Palmeri TJ (2008) Object detection and basic-level categorization: sometimes you know it is there before you know what it is. Psychon B Rev 15(1):28–35Google Scholar
  33. Mack ML, Wong ACN, Gauthier I, Palmeri TJ (2006) The time course of visual object detection and categorization. J Vis 6(6):608. doi: 10.1167/6.6.608 Google Scholar
  34. Miller JO, Low KA (2001) Motor processes in simple, go/no-go, and choice reaction time tasks: a psychophysiological analysis. J Exp Psychol Hum Percept Perform 27(2):266–289PubMedCrossRefGoogle Scholar
  35. Miller J, Ulrich R, Lamarre Y (2001) Locus of the redundant-signals effect in bimodal divided attention: a neurophysiological analysis. Percept Psychophys 63(3):555–562PubMedGoogle Scholar
  36. Miller J (1986) Time course of coactivation in bimodal divided attention. Percept Psychophys 40:331–343PubMedGoogle Scholar
  37. Miller J (1982) Divided attention: evidence for coactivation with redundant signals. Cognit Psychol 14(2):247–279PubMedCrossRefGoogle Scholar
  38. Molholm S, Ritter W, Javitt DC, Foxe JJ (2004) Multisensory visual–auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 14(4):452–65PubMedCrossRefGoogle Scholar
  39. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14(1):115–28PubMedCrossRefGoogle Scholar
  40. Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15(7):963–974PubMedCrossRefGoogle Scholar
  41. Nagy A, Eördegh G, Paróczy Z, Márkus Z, Benedek G (2006) Multisensory integration in the basal ganglia. Eur J Neurosci 24(3):917–924PubMedCrossRefGoogle Scholar
  42. Nagy A, Paróczy Z, Norita M, Benedek G (2005) Multisensory responses and receptive field properties of neurons in the substantia nigra and in the caudate nucleus. Eur J Neurosci 22(2):419–424PubMedCrossRefGoogle Scholar
  43. Nickerson RS (1973) Intersensory facilitation of reaction time: energy summation or preparation enhancement? Psychol Rev 80(6):489–509PubMedCrossRefGoogle Scholar
  44. Odgaard EC, Arieh Y, Marks LE (2004) Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation. Cogn Affect Behav Neurosci 4(2):127–132PubMedGoogle Scholar
  45. Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9(1):97–113PubMedCrossRefGoogle Scholar
  46. Orliaquet JP, Kandel S, Boë LJ (1997) Visual perception of motor anticipation in cursive handwriting: influence of spatial and movement information on the prediction of forthcoming letters. Perception 26(7):905–12CrossRefGoogle Scholar
  47. Perrault TJ Jr, Vaughan JW, Stein BE, Wallace MT (2003) Neuron-specific response characteristics predict the magnitude of multisensory integration. J Neurophysiol 90(6):4022–4026PubMedCrossRefGoogle Scholar
  48. Plat FM, Praamstra P, Horstink MW (2000) Redundant-signals effects on reaction time, response force, and movement-related potentials in Parkinson’s disease. Exp Brain Res 130(4):533–539PubMedCrossRefGoogle Scholar
  49. Raab DH (1962) Statistical facilitation of simple reaction times. Trans NY Acad Sci 24:574–590Google Scholar
  50. Rowland BA, Quessy S, Stanford TR, Stein BE (2007a) Multisensory integration shortens physiological response latencies. J Neurosci 27(22):5879–84PubMedCrossRefGoogle Scholar
  51. Rowland BA, Stanford TR, Stein BE (2007b) A Bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus. Exp Brain Res 180(1):153–61PubMedCrossRefGoogle Scholar
  52. Sagi D, Julesz B (1985) ‘Where’ and ‘what’ in vision. Science 228(4704):1217–9PubMedCrossRefGoogle Scholar
  53. Sagi D, Julesz B (1984) Detection versus discrimination of visual orientation. Perception 13(5):619–628PubMedCrossRefGoogle Scholar
  54. Schmidt RF, Schady WJ, Torebjork HE (1990a) Gating of tactile input from the hand. I. Effects of finger movement. Exp Brain Res 79(1):97–102PubMedGoogle Scholar
  55. Schmidt RF, Torebjork HE, Schady WJ (1990b) Gating of tactile input from the hand. II. Effects of remote movements and anaesthesia. Exp Brain Res 79(1):103–108PubMedGoogle Scholar
  56. Sekuler R, Sekuler AB, Lau R (1997) Sound alters visual motion perception. Nature 385(6614):308PubMedCrossRefGoogle Scholar
  57. Seki K, Perlmutter SI, Fetz EE (2003) Sensory input to primate spinal cord is presynaptically inhibited during voluntary movement. Nat Neurosci 6:1309–16PubMedCrossRefGoogle Scholar
  58. Shimojo S, Shams L (2001) Sensory modalities are not separate modalities: plasticity and interactions. Curr Opin Neurobiol 11(4):505–509PubMedCrossRefGoogle Scholar
  59. Smith EE (1968) Choice reaction time: an analysis of major theoretical positions. Psychol Bull 69(2):77–110PubMedCrossRefGoogle Scholar
  60. Sosnik R, Flash T, Hauptmann B, Karni A (2007) The acquisition and implementation of the smoothness maximization motion strategy is dependent on spatial accuracy demands. Exp Brain Res 176(2):311–331PubMedCrossRefGoogle Scholar
  61. Sosnik R, Hauptmann B, Karni A, Flash T (2004) When practice leads to co-articulation: the evolution of geometrically defined movement primitives. Exp Brain Res 156(4):422–438PubMedCrossRefGoogle Scholar
  62. Sparks DL (1986) Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. Physiol Rev 66(1):118–171PubMedGoogle Scholar
  63. Stein BE, Jiang W, Stanford TR (2004) Multisensory integration in single neurons of the midbrain. In: Calvert G, Spence C, Stein BE (eds) The handbook of multisensory processes. MIT, Cambridge, pp 243–264Google Scholar
  64. Stein BE, London N, Wilkinson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8:497–506CrossRefGoogle Scholar
  65. Stein BE, Meredith MA (1993) The merging of the senses. MIT, CambridgeGoogle Scholar
  66. Sternberg S (1969) The discovery of processing stages: extensions of Donders’ method. Acta Psychol 30:276–315CrossRefGoogle Scholar
  67. Taylor-Clarke M, Kennett S, Haggard P (2002) Vision modulates somatosensory cortical processing. Curr Biol 12(3):233–236PubMedCrossRefGoogle Scholar
  68. Ulrich R, Miller J, Schröter H (2007) Testing the race model inequality: an algorithm and computer programs. Behav Res Meth Instr 39(2):291–302Google Scholar
  69. Ulrich R, Mattes S, Miller JO (1999) Donders’s assumption of pure insertion: an evaluation on the basis of response dynamics. Acta Psychol 102(1):43–75CrossRefGoogle Scholar
  70. Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80(2):1006–1110PubMedGoogle Scholar
  71. Wallace MT, Meredith MA, Stein BE (1993) Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J Neurophysiol 69:1797–1809PubMedGoogle Scholar
  72. Watanabe K (2001) Crossmodal interaction in humans. Ph.D. thesis, California Institute of Technology, PasadenaGoogle Scholar
  73. Williams SR, Chapman CE (2002) Time course and magnitude of movement-related gating of tactile detection in humans. III. Effect of motor task. J Neurophysiol 88(4):1968–1979PubMedCrossRefGoogle Scholar
  74. Williams SR, Chapman CE (2000) Time course and magnitude of movement-related gating of tactile detection in humans. II. Effects of stimulus intensity on detection and scaling of tactile stimuli. J Neurophysiol 84(2):863–875PubMedGoogle Scholar

Copyright information

© Springer-Verlag 2008

Authors and Affiliations

  1. 1.The Touch Laboratory, Gutwirth Building, Department of Education in Technology and ScienceTechnion—Israel Institute of TechnologyHaifaIsrael
  2. 2.The Brain-Behavior Research Center, Department of Neurobiology and Ethology, Faculty of Science and Science EducationUniversity of HaifaHaifaIsrael

Personalised recommendations