Multisensory temporal integration: task and stimulus dependencies
- 1.8k Downloads
The ability of human sensory systems to integrate information across the different modalities provides a wide range of behavioral and perceptual benefits. This integration process is dependent upon the temporal relationship of the different sensory signals, with stimuli occurring close together in time typically resulting in the largest behavior changes. The range of temporal intervals over which such benefits are seen is typically referred to as the temporal binding window (TBW). Given the importance of temporal factors in multisensory integration under both normal and atypical circumstances such as autism and dyslexia, the TBW has been measured with a variety of experimental protocols that differ according to criterion, task, and stimulus type, making comparisons across experiments difficult. In the current study, we attempt to elucidate the role that these various factors play in the measurement of this important construct. The results show a strong effect of stimulus type, with the TBW assessed with speech stimuli being both larger and more symmetrical than that seen using simple and complex non-speech stimuli. These effects are robust across task and statistical criteria and are highly consistent within individuals, suggesting substantial overlap in the neural and cognitive operations that govern multisensory temporal processes.
KeywordsAudiovisual Multisensory integration Temporal binding window Psychophysics Temporal perception Speech perception Synchrony Multimodal
This research was funded in part through a grant from NIDCD awarded to Mark Wallace and Stephen Camarata, NIH # R34 DC010927, as well as an NIDCD grant awarded to Ryan Stevenson, NIH 1F32 DC011993. We also acknowledge the help of Raquel Zemtsov, Juliane Kreuger Fister, and Justin Siemann with assistance running subjects, Zachary Barnett for technical assistance, and Lena Quinto for the speech stimuli.
- James TW, Stevenson RA, Kim S (2009) Assessing multisensory integration with additive factors and functional MRI. The International Society for Psychophysics, DublinGoogle Scholar
- James TW, Stevenson RA, Kim S (2012) Inverse effectiveness in multisensory processing. In: Stein BE (ed) The new handbook of multisensory processes. MIT Press, Cambridge, MAGoogle Scholar
- Wallace MH, Murray MM (eds) (2011) Frontiers in the neural basis of multisensory processes. Taylor & Francis, LondonGoogle Scholar
- Woynaroski TG, Kwakye LD, Foss-Feig JH, Stevenson RA, Stone WL, Wallace MT (2013) Multisensory speech perception in high-functioning children with autism spectrum disorders. J Autism Dev Disord. doi: JADD-D-12-00456R1