Abstract
Data from a sustained monitoring experiment involving auditory, visual and combined audio-visual signal recognition were used to assess the predictive validity of five models of bisensory information processing. Satisfactory predictions of the dual-mode performance levels were made only by two models, neither of which assumes that the auditory and visual systems operate independently, and correlations which attest to this nonindependence are presented. One of these models explicitly assumes that the two systems are associated so that their judgments tend to coincide; the other assumes that the visual system “alerts” the auditory system to the presence of a signal. Both models accurately predict the levels of d’ and β in the dual-mode condition, and the “alerting” one also accounts for the observed reduction in response latencies.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Reference Notes
1. Loveless, N. E.Signal detection with simultaneous visual and auditory presentation. Air Ministry, F.P.R.C. Report No. 1027, 1957.
2. Kinchla, R. A.A comparison of sequential effects in detection and recognition. Experimental Psychology Series, Psychology Department, New York University, Technical Report No. 1, 1966.
Atkinson, R. C. A variable sensitivity theory of signal detection.Psychological Review, 1963,70, 91–106.
Atkinson, R. C., Bower, G. H., &Crothers, E. J.An introduction to mathematical learning theory. New York: Wiley, 1965
Atkinson, R. C., Carterrette, E. C., &Kinchla, R. A. The effect of information feedback upon psychophysical judgments.Psychonomic Science, 1964,1, 83–84.
Baker, R. A., Ware, J. R., &Sipowicz, R. R. Vigilance: A comparison in auditory, visual and combined audio-visual tasks.Canadian Journal of Psychology, 1962,16, 192–198.
Bernstein, I. H. Can we see and hear at the same time? Some recent studies of intersensory facilitation of reaction time. In A. F. Sanders (Ed.), Attention and performance III.Acta Psychologica, 1970,33, 21-35.
Bernstein, I. H., Rose, R., &Astor, V. Preparatory state effects in intersensory facilitation.Psychonomic Science, 1970,19, 113–114.
Broadbent, D. E., &Gregory, M. Vigilance considered as a statistical decision.British Journal of Psychology, 1963,54, 309–323.
Brown, A. E., &Hopkins, H. K. Interaction of auditory and visual sense modalities.Journal of Acoustical Society of America, 1967,41, 1–6.
Buckner, D. N., &McGrath, J. J. A comparison of performances on single and dual sensory mode vigilance tasks. In D. N. Buckner & J. J. McGrath (Eds.),Vigilance: a symposium. New York: McGraw-Hill, 1963. Pp. 53–69.
Colquhoun, W. P. Evaluation of auditory, visual, and dual-mode displays for prolonged sonar monitoring in repeated sessions.Human Factors, 1975,17, 425–437.
Colquhoun, W. P., &Baddeley, A. D. Influence of signal probability during pretraining on vigilance decrement.Journal of Experimental Psychology, 1967,73, 153–155.
Corcoras, D. W. J.Pattern recognition. Harmondsworth: Penguin Books, 1971.
Corcoran, D. W. J., &Weening, D. L. On the combination of evidence from the eye and ear.Ergonomics, 1969,12, 383–394.
Creelman, C. D., &Donaldson, W. ROC curves for discrimination of linear extent.Journal of Experimental Psychology, 1968,77, 514–516.
Davenport, W. G. Vigilance for simultaneous auditory and cutaneous signals.Canadian Journal of Psychology, 1969,23, 93–100.
Dougherty, W. G., Jones, G. R., &Engel, G. R. Sensory integration of auditory and visual information.Canadian Journal of Psychology, 1971,25, 476–485.
Eijkman, E., &Vendrik, A. J. H. Can a sensory system be specified by its internal noise?Journal of the Acoustical Society of America, 1965,37, 1102–1109.
Fidell, S. Sensory function in multimodal signal detection.Journal of the Acoustical Society of America, 1970,47, 1009–1015.
Freeman, P. R.Table of d’ and β. Cambridge: University Press, 1973.
Garner, W. R., &Morton, J. Perceptual independence: Definitions, models, and experimental paradigms.Psychological Bulletin, 1969,72, 233–259.
Green, D. M., &Swets, J. A.Signal detection theory and psychophysics. New York: Wiley, 1966.
Gunn, W. J., &Loeb, M. Correlation of performance in detecting visual and auditory signals.American Journal of Psychology, 1967,80, 236–242.
Hershenson, M. Reaction time as a measure of intersensory facilitation.Journal of Experimental Psychology, 1962,63, 289–293.
Howarth, C. I., &Treisman, M. The effects of warning interval on the electric phosphere to auditory thresholds.Quarterly Journal of Experimental Psychology, 1958,10, 130–141.
Kinchla, R. A., &Atkinson, R. C. The effects of false information feedback upon psychophysical judgments.Psychonomic Science, 1964,1, 317–318.
Kohfeld, D. L. Effects of the intensity of auditory and visual ready signals on simple reaction time.Journal of Experimental Psychology, 1969,82, 88–95.
Loveless, N. E., Brebner, J., &Hamilton, P. Bisensory presentation of information.Psychological Bulletin, 1970,73, 161–199.
Mackworth, J. F.Vigilance and attention. Harmondsworth: Penguin Books, 1970.
Morrell, L. K. Temporal characteristics of sensory interaction in choice reaction times.Journal of Experimental Psychology, 1968,77, 14–18.
Neisser, U.Cognitive psychology. New York: Appleton-Century-Crofts, 1966.
Nickerson, R. S. Intersensory facilitation of reaction time: Energy summation or preparation enhancement.Psychological Review, 1973,80, 489–509.
Osborn, W. C., Sheldon, R W., &Barker, R. A. Vigilance performance under conditions of redundant and nonredundant signal presentation.Journal of Applied Psychology, 1963,47, 130–134.
Parks, T. E., &Kellicutt, M. H. The probability-matching decision rule in the visual discrimination of order.Perception & Psychophysics, 1968,3, 355–360.
Pollack, I. Interaction between auditory and visual information sources in word recognition.Language and Speech, 1964,7, 76–83.
Raab, D. H. Statistical facilitation of single reaction times.Transactions of the New York Academy of Sciences, 1961, Session II,24, No. 5.
Todd, J. W. Reaction to multiple stimuli.Archives of Psychology, New York, 1912,3, 1–65.
Treisman, M. The effect of one stimulus on the threshold for another: an application of signal detectability theory.British Journal of Mathematical and Statistical Psychology, 1964,17, 15–35.
Tulving, E., Mandler, G., &Baumal, R. Interaction of two sources of information in tachistoscopic word recognition.Canadian Journal of Psychology, 1904,18, 62–71.
Tyler, D. M., Waag,,W, L., &Halcomb, C. G. Monitoring performance across sense modes: An individual difference approach.Human Factors, 1972,14, 539–547.
Author information
Authors and Affiliations
Additional information
This work has been carried out with the support of the Procurement Executive, U. K Ministry of Defence.
Rights and permissions
About this article
Cite this article
Craig, A., Colquhoun, W.P. & Corcoran, D.W.J. Combining evidence presented simultaneously to the eye and the ear: A comparison of some predictive models. Perception & Psychophysics 19, 473–484 (1976). https://doi.org/10.3758/BF03211215
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.3758/BF03211215