Multisensory enhancement: gains in choice and in simple response times
- 578 Downloads
Human observers can detect combinations of multisensory signals faster than each of the corresponding signals presented separately. In simple detection tasks, this facilitation in response times may reflect an enhancement in the perceptual processing stage or/and in the motor response stage. The current study compared the multisensory enhancements obtained in simple and choice response times (SRT and CRT, respectively) in bi- and tri-sensory (audio–visual–haptic) signal combinations using an identical experimental setup that differed only in the tasks—detecting the signals (SRT) or reporting the signals’ location (CRT). Our measurements show that RTs were faster in the multisensory combinations conditions compared to the single stimulus conditions and that the absolute multisensory gains were larger in CRT than in SRT. These results can be interpreted in two ways. According to a serial stages model, the larger multisensory gains in CRT may suggest that when combinations of multisensory signals are presented, an additional enhancement occurs in the cognitive processing stages engaged in the CRT, beyond the enhancement in the perceptual and motor stages common to both SRT and CRT. Alternatively, the results suggest that multisensory enhancement reflect task-dependent interactions within and between multiple processing levels rather than facilitated processing modules. Thus, the larger absolute multisensory gains in CRT may reflect the inverse effectiveness principle, and Bayesian statistics, in that the maximal multisensory enhancements occur in the more difficult (less precise) uni-sensory conditions, i.e., in the CRT.
KeywordsMultisensory enhancement Simple reaction time Choice reaction time Inverse effectiveness
This research was funded by the EU project IMMERSENCE—Multi-modal immersion into interactive virtual environments. We thank Mr. Gad Halevy for his help in programming the system for the experiment, and Mrs. Tatiana Gelfeld for her help in producing the CDF plots.
- Balas M (2006) Context and experience as factors in motor interference and motor representations: learning a finger opposition sequence and handwriting—behavioral and fMRI studies. Ph.D. thesis, University of Haifa, IsraelGoogle Scholar
- Bays PM, Wolpert DM (2007) Computational principles of sensorimotor control that minimize uncertainty and variability. J Physiol 278(2):387–396Google Scholar
- Donders FC (1969) Over de snelheid van psychische processen (On the speed of psychological processes). In: Koster W (ed and trans) Attention and performance: II. North-Holland, Amsterdam (Original work published 1868)Google Scholar
- Lederman SJ, Klatzky RL, Hamilton C, Morgan T (2002) Integrating multimodal information about surface texture via a probe: relative contributions of haptic and touch-produced sound sources. In: Proceedings of the 10th annual symposium on haptic interfaces for teleoperators and virtual environment systems (IEEE VR ‘02), Orlando, Florida, USA, pp 97–104Google Scholar
- Luce RD (1991) Response times. Oxford University PressGoogle Scholar
- Mack ML, Gauthier I, Sadr J, Palmeri TJ (2008) Object detection and basic-level categorization: sometimes you know it is there before you know what it is. Psychon B Rev 15(1):28–35Google Scholar
- Raab DH (1962) Statistical facilitation of simple reaction times. Trans NY Acad Sci 24:574–590Google Scholar
- Stein BE, Jiang W, Stanford TR (2004) Multisensory integration in single neurons of the midbrain. In: Calvert G, Spence C, Stein BE (eds) The handbook of multisensory processes. MIT, Cambridge, pp 243–264Google Scholar
- Stein BE, Meredith MA (1993) The merging of the senses. MIT, CambridgeGoogle Scholar
- Ulrich R, Miller J, Schröter H (2007) Testing the race model inequality: an algorithm and computer programs. Behav Res Meth Instr 39(2):291–302Google Scholar
- Watanabe K (2001) Crossmodal interaction in humans. Ph.D. thesis, California Institute of Technology, PasadenaGoogle Scholar