Skip to main content
SpringerLink
Log in

Multisensory enhancement: gains in choice and in simple response times

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

Human observers can detect combinations of multisensory signals faster than each of the corresponding signals presented separately. In simple detection tasks, this facilitation in response times may reflect an enhancement in the perceptual processing stage or/and in the motor response stage. The current study compared the multisensory enhancements obtained in simple and choice response times (SRT and CRT, respectively) in bi- and tri-sensory (audio–visual–haptic) signal combinations using an identical experimental setup that differed only in the tasks—detecting the signals (SRT) or reporting the signals’ location (CRT). Our measurements show that RTs were faster in the multisensory combinations conditions compared to the single stimulus conditions and that the absolute multisensory gains were larger in CRT than in SRT. These results can be interpreted in two ways. According to a serial stages model, the larger multisensory gains in CRT may suggest that when combinations of multisensory signals are presented, an additional enhancement occurs in the cognitive processing stages engaged in the CRT, beyond the enhancement in the perceptual and motor stages common to both SRT and CRT. Alternatively, the results suggest that multisensory enhancement reflect task-dependent interactions within and between multiple processing levels rather than facilitated processing modules. Thus, the larger absolute multisensory gains in CRT may reflect the inverse effectiveness principle, and Bayesian statistics, in that the maximal multisensory enhancements occur in the more difficult (less precise) uni-sensory conditions, i.e., in the CRT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Balas M (2006) Context and experience as factors in motor interference and motor representations: learning a finger opposition sequence and handwriting—behavioral and fMRI studies. Ph.D. thesis, University of Haifa, Israel

  • Bays PM, Wolpert DM (2007) Computational principles of sensorimotor control that minimize uncertainty and variability. J Physiol 278(2):387–396

    Google Scholar 

  • Delazer M, Lochy A, Jenner C, Domahs F, Benke T (2002) When writing 0 (zero) is easier than writing O (o): a neuropsychological case study of agraphia. Neuropsychologia 40(12):2167–2177

    Article  PubMed  CAS  Google Scholar 

  • Diederich A, Colonius H (2004) Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66(8):1388–1404

    PubMed  Google Scholar 

  • Diederich A, Colonius H, Bockhorst D, Tabeling S (2003) Visual–tactile spatial interaction in saccade generation. Exp Brain Res 148(3):328–37

    PubMed  Google Scholar 

  • Diederich A, Colonius H (1987) Intersensory facilitation in the motor component? Psychol Res 49(1):23–29

    Article  Google Scholar 

  • Donders FC (1969) Over de snelheid van psychische processen (On the speed of psychological processes). In: Koster W (ed and trans) Attention and performance: II. North-Holland, Amsterdam (Original work published 1868)

  • Doyle MC, Snowden RJ (2001) Identification of visual stimuli is improved by accompanying auditory stimuli: the role of eye movements and sound location. Perception 30(7):795–810

    Article  PubMed  CAS  Google Scholar 

  • Forster B, Cavina-Pratesi C, Aglioti S, Berlucchi G (2002) Redundant target effect and intersensory facilitation from visual–tactile interactions in simple reaction time. Exp Brain Res 143(4):480–487

    Article  PubMed  Google Scholar 

  • Fort A, Delpuech C, Pernier J, Giard MH (2002) Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb Cortex 12(10):1031–1039

    Article  PubMed  Google Scholar 

  • Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res Cogn Brain Res 10(1):77–83

    Article  PubMed  CAS  Google Scholar 

  • Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25(20):5004–5012

    Article  PubMed  CAS  Google Scholar 

  • Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11(5):473–490

    Article  PubMed  CAS  Google Scholar 

  • Giray M, Ulrich R (1993) Motor coactivation revealed by response force in divided and focused attention. J Exp Psychol Hum Percept Perform 19(6):1278–91

    Article  PubMed  CAS  Google Scholar 

  • Gobbelé R, Schürmann M, Forss N, Juottonen K, Buchner H, Hari R (2003) Activation of the human posterior and temporoparietal cortices during audiotactile interaction. Neuroimage 20(1):503–511

    Article  PubMed  Google Scholar 

  • Gottfried JA, Dolan RJ (2003) The nose smells what the eye sees: crossmodal visual facilitation of human olfactory perception. Neuron 39(2):375–386

    Article  PubMed  CAS  Google Scholar 

  • Graziano MSA (2001) A system of multimodal areas in the primate brain. Neuron 29:4–6

    Article  PubMed  CAS  Google Scholar 

  • Graziano MSA, Reiss LAJ, Gross CG (1999) A neuronal representation of the location of nearby sounds. Nature 397:428–430

    Article  PubMed  CAS  Google Scholar 

  • Grill-Spector K, Kanwisher N (2005) Visual recognition. As soon as you know it is there, you know what it is. Psychol Sci 16(2):152–160

    Article  PubMed  Google Scholar 

  • Hauptmann B, Karni A (2002) From primed to learn: the saturation of repetition priming and the induction of long-term memory. Brain Res Cogn Brain Res 13(3):313–322

    Article  PubMed  CAS  Google Scholar 

  • Hecht D, Reiner M, Karni A (2008) Enhancement of response times to bi- and tri-modal sensory stimuli during active movements. Exp Brain Res 185(4):655–665

    Article  PubMed  Google Scholar 

  • Heller MA (1982) Visual and tactual texture perception: intersensory cooperation. Percept Psychophys 31(4):339–344

    PubMed  CAS  Google Scholar 

  • Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289–293

    Article  PubMed  CAS  Google Scholar 

  • Honoré J, Bourdeaud’hui M, Sparrow L (1989) Reduction of cutaneous reaction time by directing eyes towards the source of stimulation. Neuropsychologia 27(3):367–371

    Article  PubMed  Google Scholar 

  • Jiang W, Wallace MT, Jiang H, Vaughan JW, Stein BE (2001) Two cortical areas mediate multisensory integration in superior colliculus neurons. J Neurophysiol 85(2):506–522

    PubMed  CAS  Google Scholar 

  • Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48(2):373–384

    Article  PubMed  CAS  Google Scholar 

  • Körding KP (2007) Decision theory: what “should” the nervous system do? Science 318(5850):606–610

    Article  PubMed  CAS  Google Scholar 

  • Körding KP, Wolpert DM (2006) Bayesian decision theory in sensorimotor control. Trends Cog Sci 10(7):320–326

    Article  Google Scholar 

  • Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT (2006) Enhanced multisensory integration in older adults. Neurobiol Aging 27(8):1155–1163

    Article  PubMed  Google Scholar 

  • Lederman SJ, Klatzky RL, Hamilton C, Morgan T (2002) Integrating multimodal information about surface texture via a probe: relative contributions of haptic and touch-produced sound sources. In: Proceedings of the 10th annual symposium on haptic interfaces for teleoperators and virtual environment systems (IEEE VR ‘02), Orlando, Florida, USA, pp 97–104

  • Luce RD (1991) Response times. Oxford University Press

  • Mack ML, Gauthier I, Sadr J, Palmeri TJ (2008) Object detection and basic-level categorization: sometimes you know it is there before you know what it is. Psychon B Rev 15(1):28–35

    Google Scholar 

  • Mack ML, Wong ACN, Gauthier I, Palmeri TJ (2006) The time course of visual object detection and categorization. J Vis 6(6):608. http://journalofvision.org/6/6/608/. doi:10.1167/6.6.608

    Google Scholar 

  • Miller JO, Low KA (2001) Motor processes in simple, go/no-go, and choice reaction time tasks: a psychophysiological analysis. J Exp Psychol Hum Percept Perform 27(2):266–289

    Article  PubMed  CAS  Google Scholar 

  • Miller J, Ulrich R, Lamarre Y (2001) Locus of the redundant-signals effect in bimodal divided attention: a neurophysiological analysis. Percept Psychophys 63(3):555–562

    PubMed  CAS  Google Scholar 

  • Miller J (1986) Time course of coactivation in bimodal divided attention. Percept Psychophys 40:331–343

    PubMed  CAS  Google Scholar 

  • Miller J (1982) Divided attention: evidence for coactivation with redundant signals. Cognit Psychol 14(2):247–279

    Article  PubMed  CAS  Google Scholar 

  • Molholm S, Ritter W, Javitt DC, Foxe JJ (2004) Multisensory visual–auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 14(4):452–65

    Article  PubMed  Google Scholar 

  • Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14(1):115–28

    Article  PubMed  Google Scholar 

  • Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15(7):963–974

    Article  PubMed  Google Scholar 

  • Nagy A, Eördegh G, Paróczy Z, Márkus Z, Benedek G (2006) Multisensory integration in the basal ganglia. Eur J Neurosci 24(3):917–924

    Article  PubMed  Google Scholar 

  • Nagy A, Paróczy Z, Norita M, Benedek G (2005) Multisensory responses and receptive field properties of neurons in the substantia nigra and in the caudate nucleus. Eur J Neurosci 22(2):419–424

    Article  PubMed  Google Scholar 

  • Nickerson RS (1973) Intersensory facilitation of reaction time: energy summation or preparation enhancement? Psychol Rev 80(6):489–509

    Article  PubMed  CAS  Google Scholar 

  • Odgaard EC, Arieh Y, Marks LE (2004) Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation. Cogn Affect Behav Neurosci 4(2):127–132

    PubMed  Google Scholar 

  • Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9(1):97–113

    Article  PubMed  CAS  Google Scholar 

  • Orliaquet JP, Kandel S, Boë LJ (1997) Visual perception of motor anticipation in cursive handwriting: influence of spatial and movement information on the prediction of forthcoming letters. Perception 26(7):905–12

    Article  Google Scholar 

  • Perrault TJ Jr, Vaughan JW, Stein BE, Wallace MT (2003) Neuron-specific response characteristics predict the magnitude of multisensory integration. J Neurophysiol 90(6):4022–4026

    Article  PubMed  Google Scholar 

  • Plat FM, Praamstra P, Horstink MW (2000) Redundant-signals effects on reaction time, response force, and movement-related potentials in Parkinson’s disease. Exp Brain Res 130(4):533–539

    Article  PubMed  CAS  Google Scholar 

  • Raab DH (1962) Statistical facilitation of simple reaction times. Trans NY Acad Sci 24:574–590

    CAS  Google Scholar 

  • Rowland BA, Quessy S, Stanford TR, Stein BE (2007a) Multisensory integration shortens physiological response latencies. J Neurosci 27(22):5879–84

    Article  PubMed  CAS  Google Scholar 

  • Rowland BA, Stanford TR, Stein BE (2007b) A Bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus. Exp Brain Res 180(1):153–61

    Article  PubMed  Google Scholar 

  • Sagi D, Julesz B (1985) ‘Where’ and ‘what’ in vision. Science 228(4704):1217–9

    Article  PubMed  CAS  Google Scholar 

  • Sagi D, Julesz B (1984) Detection versus discrimination of visual orientation. Perception 13(5):619–628

    Article  PubMed  CAS  Google Scholar 

  • Schmidt RF, Schady WJ, Torebjork HE (1990a) Gating of tactile input from the hand. I. Effects of finger movement. Exp Brain Res 79(1):97–102

    PubMed  CAS  Google Scholar 

  • Schmidt RF, Torebjork HE, Schady WJ (1990b) Gating of tactile input from the hand. II. Effects of remote movements and anaesthesia. Exp Brain Res 79(1):103–108

    PubMed  CAS  Google Scholar 

  • Sekuler R, Sekuler AB, Lau R (1997) Sound alters visual motion perception. Nature 385(6614):308

    Article  PubMed  CAS  Google Scholar 

  • Seki K, Perlmutter SI, Fetz EE (2003) Sensory input to primate spinal cord is presynaptically inhibited during voluntary movement. Nat Neurosci 6:1309–16

    Article  PubMed  CAS  Google Scholar 

  • Shimojo S, Shams L (2001) Sensory modalities are not separate modalities: plasticity and interactions. Curr Opin Neurobiol 11(4):505–509

    Article  PubMed  CAS  Google Scholar 

  • Smith EE (1968) Choice reaction time: an analysis of major theoretical positions. Psychol Bull 69(2):77–110

    Article  PubMed  CAS  Google Scholar 

  • Sosnik R, Flash T, Hauptmann B, Karni A (2007) The acquisition and implementation of the smoothness maximization motion strategy is dependent on spatial accuracy demands. Exp Brain Res 176(2):311–331

    Article  PubMed  Google Scholar 

  • Sosnik R, Hauptmann B, Karni A, Flash T (2004) When practice leads to co-articulation: the evolution of geometrically defined movement primitives. Exp Brain Res 156(4):422–438

    Article  PubMed  Google Scholar 

  • Sparks DL (1986) Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. Physiol Rev 66(1):118–171

    PubMed  CAS  Google Scholar 

  • Stein BE, Jiang W, Stanford TR (2004) Multisensory integration in single neurons of the midbrain. In: Calvert G, Spence C, Stein BE (eds) The handbook of multisensory processes. MIT, Cambridge, pp 243–264

    Google Scholar 

  • Stein BE, London N, Wilkinson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8:497–506

    Article  Google Scholar 

  • Stein BE, Meredith MA (1993) The merging of the senses. MIT, Cambridge

    Google Scholar 

  • Sternberg S (1969) The discovery of processing stages: extensions of Donders’ method. Acta Psychol 30:276–315

    Article  Google Scholar 

  • Taylor-Clarke M, Kennett S, Haggard P (2002) Vision modulates somatosensory cortical processing. Curr Biol 12(3):233–236

    Article  PubMed  CAS  Google Scholar 

  • Ulrich R, Miller J, Schröter H (2007) Testing the race model inequality: an algorithm and computer programs. Behav Res Meth Instr 39(2):291–302

    Google Scholar 

  • Ulrich R, Mattes S, Miller JO (1999) Donders’s assumption of pure insertion: an evaluation on the basis of response dynamics. Acta Psychol 102(1):43–75

    Article  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80(2):1006–1110

    PubMed  CAS  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1993) Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J Neurophysiol 69:1797–1809

    PubMed  CAS  Google Scholar 

  • Watanabe K (2001) Crossmodal interaction in humans. Ph.D. thesis, California Institute of Technology, Pasadena

  • Williams SR, Chapman CE (2002) Time course and magnitude of movement-related gating of tactile detection in humans. III. Effect of motor task. J Neurophysiol 88(4):1968–1979

    Article  PubMed  Google Scholar 

  • Williams SR, Chapman CE (2000) Time course and magnitude of movement-related gating of tactile detection in humans. II. Effects of stimulus intensity on detection and scaling of tactile stimuli. J Neurophysiol 84(2):863–875

    PubMed  CAS  Google Scholar 

Download references

Acknowledgments

This research was funded by the EU project IMMERSENCE—Multi-modal immersion into interactive virtual environments. We thank Mr. Gad Halevy for his help in programming the system for the experiment, and Mrs. Tatiana Gelfeld for her help in producing the CDF plots.

Author information

Authors and Affiliations

  1. The Touch Laboratory, Gutwirth Building, Department of Education in Technology and Science, Technion—Israel Institute of Technology, Haifa, 32000, Israel

    David Hecht & Miriam Reiner

  2. The Brain-Behavior Research Center, Department of Neurobiology and Ethology, Faculty of Science and Science Education, University of Haifa, Haifa, 31905, Israel

    David Hecht & Avi Karni

Authors
  1. David Hecht
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miriam Reiner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Avi Karni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Hecht.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hecht, D., Reiner, M. & Karni, A. Multisensory enhancement: gains in choice and in simple response times. Exp Brain Res 189, 133–143 (2008). https://doi.org/10.1007/s00221-008-1410-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-008-1410-0

Keywords

Search

Navigation