Psychological Research

, Volume 75, Issue 2, pp 77–94 | Cite as

On quantifying multisensory interaction effects in reaction time and detection rate

Original Article

Abstract

Both mean reaction time (RT) and detection rate (DR) are important measures for assessing the amount of multisensory interaction occurring in crossmodal experiments, but they are often applied separately. Here we demonstrate that measuring multisensory performance using either RT or DR alone misses out on important information. We suggest an integration of RT and DR into a single measure of multisensory performance: the first index (MRE*) is based on an arithmetic combination of RT and DR, the second (MPE) is constructed from parameters derived from fitting a sequential sampling model to RT and DR data simultaneously. Our approach is illustrated by data from two audio–visual experiments. In the first, a redundant targets detection experiment using stimuli of different intensity, both measures yield similar pattern of results supporting the “principle of inverse effectiveness”. The second experiment, introducing stimulus onset asynchrony and differing instructions (focused attention vs. redundant targets task) further supports the usefulness of both indices. Statistical properties of both measures are investigated via bootstrapping procedures.

Keywords

Audio–visual interaction Redundant target paradigm Inverse effectiveness Sequential sampling model Inverse efficiency scores Multisensory performance enhancement 

Notes

Acknowledgments

This research was supported by Deutsche Forschungsgemeinschaft (DFG) Grant No. Di 506/8-1 to A.D. and by SFB/TR31 “Active Hearing”, Teilprojekt B4 to H.C.

References

  1. Amlôt, R., Walker, R., Driver, J., & Spence, C. (2003). Multimodal visual–somatosensory integration in saccade generation. Neuropsychologica, 41, 1–15.CrossRefGoogle Scholar
  2. Arieh, Y., & Marks, L. E. (2008). Cross-modal interaction between vision and hearing: A speed–accuracy analysis. Perception & Psychophysics, 70, 412–421.CrossRefGoogle Scholar
  3. Arndt, P. A., & Colonius, H. (2003). Two stages in crossmodal saccadic integration: Evidence from a visual–auditory focused attention task. Experimental Brain Research, 150, 417–426.Google Scholar
  4. Baird, J. C. (1984). Information theory and information processing. Information Processing and Management, 20, 373–381.CrossRefGoogle Scholar
  5. Bernstein, I. H., Chu, P. K., Briggs, P., & Schurman, D. L. (1973). Stimulus intensity and foreperiod effects in intersensory facilitation. Journal of Experimental Psychology, 25, 171–181.CrossRefGoogle Scholar
  6. Bernstein, I. H., Clark, M. E., & Edelstein, B. A. (1969a). Effects of an auditory signal on visual reaction time. Journal of Experimental Psychology, 80(3), 567–569.CrossRefPubMedGoogle Scholar
  7. Bernstein, I. H., Clark, M. H., & Edelstein, B. A. (1969b). Intermodal effects in choice reaction time. Journal of Experimental Psychology, 81(2), 405–407.CrossRefGoogle Scholar
  8. Bernstein, I. H., & Edelstein, B. A. (1971). Effects of some variation in auditory input upon visual choice reaction time. Journal of Experimental Psychology, 87(2), 241–247.CrossRefPubMedGoogle Scholar
  9. Bernstein, I. H., Rose, R., & Ashe, V. M. (1970). Energy integration in intersensory facilitation. Journal of Experimental Psychology, 86(2), 196–203.CrossRefPubMedGoogle Scholar
  10. Bolognini, N., Frassinetti, F., Serino, A., & Làdavas, E. (2005). “acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs. Experimental Brain Research, 160, 273–282.CrossRefGoogle Scholar
  11. Colonius, H., & Diederich, A. (2004). Multisensory interaction in saccadic reaction time: A time-window-of-integration model. Journal of Cognitive Neuroscience, 16(6), 1–10.CrossRefGoogle Scholar
  12. Corneil, B. D., Van Wanrooij, M., Munoz, D. P., & Van Opstal, A. J. (2002). Auditory–visual interactions subserving goal-directed saccades in a complex scene. Journal of Neurophysiology, 88(1), 438–454.PubMedGoogle Scholar
  13. Diederich, A. (1992). Intersensory facilitation: Race, superposition, and diffusion models for reaction time to multiple stimuli (Dissertation). Frankfurt: Verlag Peter Lang.Google Scholar
  14. Diederich, A. (1995). Intersensory facilitation of reaction time: Evaluation of counter and diffusion coactivation models. Journal of Mathematical Psychology, 39, 197–215.CrossRefGoogle Scholar
  15. Diederich, A. (1997). Dynamic stochastic models for decision making with time constraints. Journal of Mathematical Psychology, 41, 260–274.CrossRefPubMedGoogle Scholar
  16. Diederich, A. (2008). A further test of sequential-sampling models that account for payoff effects on response bias in perceptual decision tasks. Perception & Psychophysics, 70, 229–256.CrossRefGoogle Scholar
  17. Diederich, A., & Busemeyer, J. R. (2003). Simple matrix methods for analyzing diffusion models of choice probability, choice response time and simple response time. Journal of Mathematical Psychology, 47, 304–322.CrossRefGoogle Scholar
  18. Diederich, A., & Busemeyer, J. R. (2006). Modeling the effects of payoff on response bias in a perceptual discrimination task: Threshold-bound, drift-rate-change, or two-stage-processing hypothesis. Perception & Psychophysics, 68, 194–207.CrossRefGoogle Scholar
  19. Diederich, A., & Colonius, H. (1987). Intersensory facilitation in the motor component? A reaction time analysis. Psychological Research, 49, 23–29.CrossRefGoogle Scholar
  20. Diederich, A., & Colonius, H. (2004). Bimodal and trimodal multisensory enhancement: Effects of stimulus onset and intensity on reaction time. Perception & Psychophysics, 66(8), 1388–1404.CrossRefGoogle Scholar
  21. Efron, B., & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistical Science, 1, 54–75.CrossRefGoogle Scholar
  22. Eimer, M. (2001). Crossmodal links in spatial attention between vision, audition, and touch: evidence from event-related brain potentials. Neuropsychologia, 39, 1292–1303.CrossRefPubMedGoogle Scholar
  23. Fitts, P. M. (1966). Cognitive aspects of information processing. III. Set for speed versus accuracy. Journal of Experimental Psychology, 71, 849–857.CrossRefPubMedGoogle Scholar
  24. Frassinetti, F., Bolognini, N., & Làdavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Experimental Brain Research, 147, 332–343.CrossRefGoogle Scholar
  25. Frens, M. A., Van Opstal, A. J., & Van der Willigen, R. F. (1995). Spatial and temporal factors determine auditory–visual interaction in human saccadic eye movements. Perception & Psychophysics, 57(6), 802–816.CrossRefGoogle Scholar
  26. Gielen, S. C. A. M., Schmidt, R. A., & Van den Heuvel, P. J. M. (1983). On the nature of intersensory facilitation of reaction time. Perception & Psychophysics, 34(2), 161–168.CrossRefGoogle Scholar
  27. Gillmeister, H., & Eimer, M. (2007). Tactile enhancement of auditory detection and perceived loudness. Brain Research, 1160, 58–68.CrossRefPubMedGoogle Scholar
  28. Giray, M., & Ulrich, R. (1993). Motor coactivation revealed by response force in divided and focused attention. Journal of Experimental Psychology: Human Perception and Performance, 19(6), 1278–1291.CrossRefPubMedGoogle Scholar
  29. Gomez, P., Ratcliff, R., & Perea, M. (2007). A model of the go/no-go task. Journal of Experimental Psychology: General, 136, 389–413.CrossRefGoogle Scholar
  30. Harrington, L.K., & Peck, C.K. (1998). Spatial disparity affects visual–auditory interactions in human sensorimotor processing. Experimental Brain Research, 122, 247–252.CrossRefGoogle Scholar
  31. Hershenson, M. (1962). Reaction time as a measure of intersensory facilitation. Journal of Experimental Psychology, 63(3), 289–293.CrossRefPubMedGoogle Scholar
  32. Hick, W. (1952). On the rate of gain of information. Quarterly Journal of Experimental Psycholology, 4, 11–26.CrossRefGoogle Scholar
  33. Hilgard, E. R. (1933). Reinforcement and inhibition of eyelid reflexes. Journal of General Psychology, 8, 85–113.CrossRefGoogle Scholar
  34. Holmes, N. P. (2009). The principle of inverse effectiveness in multisensory integration: Some statistical considerations. Brain Topography, 21, 168–176.CrossRefPubMedGoogle Scholar
  35. Jepma, M., Wagenmakers, E.-J., Band, G. P. H., & Nieuwenhuis, S. (2008). The effects of accessory stimuli on information processing: Evidence from electrophysiology and a diffusion model analysis. Journal of Cognitive Neuroscience, 21, 847–864.CrossRefGoogle Scholar
  36. Kitagawa, N., & Spence, C. (2005). Investigating the effect of a transparent barrier on the crossmodal congruency effect. Experimental Brain Research, 161, 62–71.CrossRefGoogle Scholar
  37. Lovelace, C. T., Stein, B. E., & Wallace, M. T. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cognitive Brain Research, 17, 447–453.CrossRefPubMedGoogle Scholar
  38. Luce, R. D. (1986). Response times: Their role in inferring elementary mental organisation. Oxford Psychology Series; no. 8. New York: Oxford University Press.Google Scholar
  39. Luce, R. D. (2003). Whatever happened to information theory in psychology? Review of General Psychology, 7, 183–188.CrossRefGoogle Scholar
  40. Meredith, M. A., Nemitz, J. W., & Stein, B. E. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. Journal of Neuroscience, 7(10), 3215–3229.PubMedGoogle Scholar
  41. Meredith, M. A. & Stein, B. E. (1986). Visual, auditory and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–662.PubMedGoogle Scholar
  42. Miller, J. O. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14, 247–279.CrossRefPubMedGoogle Scholar
  43. Miller, J. O. (1986). Time course of coactivation in bimodal divided attention. Perception & Psychophysics, 40(5), 331–343.CrossRefGoogle Scholar
  44. Miller, J. O., Ulrich, R., & Lamarre, Y. (2001). Locus of the redundant-signals effect in bimodal divided attention: a neurophysiological analysis. Perception & Psychophysics, 63(3), 555–562.CrossRefGoogle Scholar
  45. Mordkoff, J. T., Miller, J., & Roch, A.-C. (1996). Absence of coactivation in the motor component: evidence from psychophysiological measures of target detection. JJournal of Experimental Psychology: Human Perception and Performance, 22(1), 25–41.CrossRefPubMedGoogle Scholar
  46. Morrell, E. K. (1968). Temporal characteristics of sensory interaction in choice reaction times. Journal of Experimental Psychology, 77(1), 14–18.CrossRefPubMedGoogle Scholar
  47. Nickerson, R. S. (1973). Intersensory facilitation of reaction time: Energy summation or preparation enhancement. Psychological Review, 80, 489–509.CrossRefPubMedGoogle Scholar
  48. Perrault, T. J., Vaughan, J. W., Stein, B. E., & Wallace, M. T. (2005). Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J Neurophysiology, 93, 2575–2586.CrossRefGoogle Scholar
  49. Proctor, R. W., & Vu, K.-P.L. (2006). The cognitive revolution at age 50: Has the promise of the human information-processing approach been fulfilled? International Journal of Human–Computer Interaction, 21, 253–284.CrossRefGoogle Scholar
  50. Rach, S., & Diederich, A. (2006). Visual–tactile integration: does stimulus duration influence the relative amount of response enhancement? Experimental Brain Research, 173, 514–520.CrossRefGoogle Scholar
  51. Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111(2), 333–367.CrossRefPubMedGoogle Scholar
  52. Röder, B., Kusmierek, A., Spence, C., & Schicke, T. (2007). Developmental vision determines the reference frame for the multisensory control of action. PNAS, 104, 4753–4758.CrossRefPubMedGoogle Scholar
  53. Schwarz, W. (1994). Diffusion, superposition, and the redundant-targets effect. Journal of Mathematical Psychology, 38, 504–520.CrossRefGoogle Scholar
  54. Shore, D. I., Barnes, M. E., & Spence, C. (2006). Temporal aspects of the visuotactile congruency effect. Neuroscience Letters, 392, 96–100.CrossRefPubMedGoogle Scholar
  55. Spence, C., McGlone, F. P., Kettenmann, B., & Kobal, G. (2001). Attention to olfaction: A psychophysical investigation. Experimental Brain Research, 138, 432–437.CrossRefGoogle Scholar
  56. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. London, Cambridge: The MIT Press.Google Scholar
  57. Townsend, J. T., & Ashby, F. G. (1983). Stochastic modelling of elementary psychological processes. New York: Cambridge Univ Press.Google Scholar
  58. Townsend, J. T., & Honey, C. J. (2007). Consequences of base time for redundant signals experiments. Journal of Mathematical Psychology, 51, 242–265.CrossRefPubMedGoogle Scholar
  59. Walker, R., Deubel, H., Schneider, W. X., & Findlay, J. M. (1997). Effect of remote distractors on saccade programming: Evidence for an extended fixation zone. Journal of Neurophysiology, 78, 1108–1119.PubMedGoogle Scholar
  60. Wichmann, F. A., & Hill, N. J. (2001). The psychometric function: II. Bootstrap-based confidence intervals and sampling. Perception & Psychophysics, 63, 1314–1329.CrossRefGoogle Scholar
  61. Woodworth, R. S., & Schlosberg, H. (1956). Experimental psychology (rev. ed.). New York: Holt.Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of OldenburgOldenburgGermany
  2. 2.Jacobs University BremenBremenGermany

Personalised recommendations