Semantic incongruity influences response caution in audio-visual integration
- 354 Downloads
Multisensory stimulus combinations trigger shorter reaction times (RTs) than individual single-modality stimuli. It has been suggested that this inter-sensory facilitation effect is found exclusively for semantically congruent stimuli, because incongruity would prevent multisensory integration. Here we provide evidence that the effect of incongruity is due to a change in response caution rather than prevention of stimulus integration. In two experiments, participants performed two-alternative forced-choice decision tasks in which they categorized auditory stimuli, visual stimuli or audio-visual stimulus pairs. The pairs were either semantically congruent (e.g. ambulance image and horn sound) or incongruent (e.g. ambulance image and bell sound). Shorter RTs and violations of the race model inequality on congruent trials are in accordance with previous studies. However, Bayesian hierarchical drift diffusion analyses contradict former co-activation-based explanations of the effects of congruency. Instead, they show that longer RTs on incongruent compared to congruent trials are most likely the result of an incongruity caution effect—more cautious response behaviour in face of semantically incongruent sensory input. Further, they show that response caution can be adjusted on a trial-by-trial basis depending on incoming information. Finally, stimulus modality influenced non-cognitive components of the response. We suggest that the combined stimulus energy from simultaneously presented stimuli reduces encoding time.
KeywordsAudio-visual integration Multisensory Drift diffusion model Semantic congruency Reaction times
Thomas Otto and Michael Herzog provided helpful comments on an earlier version of this manuscript and valuable advice for the data analysis. We also would like to thank Halina Sutter and Marina Wunderlin for their help in carrying out the experiments.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- Alais D, Burr D, (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14(3):257–262Google Scholar
- Assink N, Lubbe R, Fox J-P, Wang Y, Pierre BE, Rudas I (2015) Does time pressure induce tunnel vision? An examination with the Eriksen Flanker Task by applying the Hierarchical Drift Diffusion Model. Proceedings of the international conference on neural networks-fuzzy systems, pp 30–40Google Scholar
- Colonius H, Diederich A (2006) The race model inequality: Interpreting a geometric measure of the amount of violation. Psychol Rev 113(1):148–154Google Scholar
- Diederich A (1995) Intersensory Facilitation of Reaction Time: evaluation of counter and diffusion coactivation models. J Math Psychol 39(2):197–215Google Scholar
- Gondan M, Heckel A (2008) Testing the race inequality: A simple correction procedure for fast guesses. J Math Psychol 52(5):322–325Google Scholar
- Gondan M, Blurton SP, Hughes F, Greenlee MW (2011) Effects of spatial and selective attention on basic multisensory integration. J Exp Psychol: Hum Percept Perform 37(6):1887–1897Google Scholar
- Hackley SA, Valle-Inclán F (1998) Automatic alerting does not speed late motoric processes in a reaction-time task. Nature 391(6669):786–788Google Scholar
- Hughes HC, Reuter-Lorenz PA, Nozawa G, Fendrich R (1994) Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J Exp Psychol: Hum Percept Perform 20(1):131–153Google Scholar
- Miller J (2016) Statistical facilitation and the redundant signals effect: what are race and coactivation models? Atten, Percept, Psychophys 78(2):516–519Google Scholar
- Miller J, Franz V, Ulrich R (1999) Effects of auditory stimulus intensity on response force in simple, go/no-go, and choice RT tasks. Percept Psychophys 61(1):107–119Google Scholar
- Schwarz W (1994) Diffusion, superposition, and the redundant-targets effect. J Math Psychol 38(4):504–520Google Scholar
- Stahl J, Rammsayer TH (2005) Accessory stimulation in the time course of visuomotor information processing: Stimulus intensity effects on reaction time and response force. Acta Psychologica 120(1):1–18Google Scholar
- Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9(4):255–266Google Scholar
- Todd JW (1912) Reaction to multiple stimuli. Archives of psychology. The Science Press, Lancaster. doi: 10.1037/13053-000
- Voss A, Nagler M, Lerche V (2013) Diffusion models in experimental psychology. Exp Psychol 60(6):385–402Google Scholar