Abstract
When an audio—visual event is perceived in the natural environment, a physical delay will always occur between the arrival of the leading visual component and that of the trailing auditory component. This natural timing relationship suggests that the point of subjective simultaneity (PSS) should occur at an auditory delay greater than or equal to 0 msec. A review of the literature suggests that PSS estimates derived from a temporal order judgment (TOJ) task differ from those derived from a synchrony judgment (SJ) task, with (unnatural) auditory-leading PSS values reported mainly for the TOJ task. We report data from two stimulus types that differed in terms of complexity— namely, (1) a flash and a click and (2) a bouncing ball and an impact sound. The same participants judged the temporal order and synchrony of both stimulus types, using three experimental methods: (1) a TOJ task with two response categories (“audio first” or “video first”), (2) an SJ task with two response categories (“synchronous” or “asynchronous”; SJ2), and (3) an SJ task with three response categories (“audio first,” “synchronous,” or “video first”; SJ3). Both stimulus types produced correlated PSS estimates with the SJ tasks, but the estimates from the TOJ procedure were uncorrelated with those obtained from the SJ tasks. These results suggest that the SJ task should be preferred over the TOJ task when the primary interest is in perceived audio—visual synchrony.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Allan, L. G. (1975). The relationship between judgments of successiveness and judgments of order. Perception & Psychophysics, 18, 29–36.
Allan, L. G., & Kristofferson, A. B. (1974). Successiveness discrimination: Two models. Perception & Psychophysics, 15, 37–46.
Arrighi, R., Alais, D., & Burr, D. (2006). Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. Journal of Vision, 6, 260–268.
Aschersleben, G. (1999). On the asymmetry of the temporal contiguity window: Commentary on Lewkowicz. In G. Aschersleben, T. Bachmann, & J. Müsseler (Eds.), Cognitive contributions to the perception of spatial and temporal events (pp. 421–424). Amsterdam: Elsevier.
Aschersleben, G., & Müsseler, J. (1999). Dissociations in the timing of stationary and moving stimuli. Journal of Experimental Psychology: Human Perception & Performance, 25, 1709–1720.
Bald, L., Berrien, F. K., Price, J. B., & Sprague, R. O. (1942). Errors in perceiving the temporal order of auditory and visual stimuli. Journal of Applied Psychology, 26, 382–388.
Dinnerstein, A. J., & Zlotogura, P. (1968). Intermodal perception of temporal order and motor skills: Effects of age. Perceptual & Motor Skills, 26, 987–1000.
Dixon, N. F., & Spitz, L. (1980). The detection of auditory visual desynchrony. Perception, 9, 719–721.
Enoki, K., Washikita, K., & Yamada, M. (2006). Detection threshold of asynchrony between auditory and visual stimuli for various motion patterns of a ball. In Proceedings of the 9th Western Pacific Acoustics Conference (WESPAC IX) [CD-ROM]. Seoul: Acoustical Society of Korea.
Erlebacher, A., & Sekuler, R. (1971). Response frequency equalization: A bias model for psychophysics. Perception & Psychophysics, 9, 315–320.
Exner, S. (1875). Experimentelle Untersuchung der einfachsten psychischen Processe: III. Der persönlichen Gleichung zweiter Theil. Archiv für die gesammte Physiologie des Menschen & der Thiere, 11, 403–432.
Fujisaki, W., Shimojo, S., Kashino, M., & Nishida, S. (2004). Recalibration of audiovisual simultaneity. Nature Neuroscience, 7, 773–778.
Hamlin, A. J. (1895). On the least observable interval between stimuli addressed to disparate senses and to different organs of the same sense. American Journal of Psychology, 6, 564–575.
Hirsh, I. J., & Fraisse, P. (1964). Simultanéité et succession de stimuli hétérogènes. L'Année Psychologique, 64, 1–19.
Hirsh, I. J., & Sherrick, C. E., Jr. (1961). Perceived order in different sense modalities. Journal of Experimental Psychology, 62, 423–432.
Hollier, M. P., & Rimell, A. N. (1998, September). An experimental investigation into multi-modal synchronization sensitivity for perceptual model development. Paper presented at the 105th Convention of the Audio Engineering Society, San Francisco, CA.
Jaśkowski, P. (1993). Selective attention and temporal-order judgment. Perception, 22, 681–689.
Jaśkowski, P., Jaroszyk, F., & Hojan-Jezierska, D. (1990). Temporal-order judgments and reaction time for stimuli of different modalities. Psychological Research, 52, 35–38.
Juola, J. F., van Eijk, R. L. J., Kohlrausch, A., & van de Par, S. (2007, July). Synchrony detection and temporal order judgments for bimodal stimuli. Paper presented at the joint meeting of the Experimental Psychology (U.K.) & Psychonomic (U.S.A.) Societies, Edinburgh, Scotland.
Keetels, M., & Vroomen, J. (2005). The role of spatial disparity and hemifields in audio—visual temporal order judgments. Experimental Brain Research, 167, 635–640.
King, A. J., & Palmer, A. R. (1985). Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Experimental Brain Research, 60, 492–500.
Lewkowicz, D. J. (1996). Perception of auditory—visual temporal synchrony in human infants. Journal of Experimental Psychology: Human Perception & Performance, 22, 1094–1106.
Ludbrook, J. (1998). Multiple comparison procedures updated. Clinical & Experimental Pharmacology & Physiology, 25, 1032–1037.
McGrath, M., & Summerfield, Q. (1985). Intermodal timing relations and audio—visual speech recognition by normal-hearing adults. Journal of the Acoustical Society of America, 77, 678–685.
Mitrani, L., Shekerdjiiski, S., & Yakimoff, N. (1986). Mechanisms and asymmetries in visual perception of simultaneity and temporal order. Biological Cybernetics, 54, 159–165.
Myung, I. J. (2003). Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology, 47, 90–100.
Neumann, O., & Niepel, M. (2004). Timing of “perception” and perception of “time.” In C. Kaernbach, E. Schröger, & H. Müller (Eds.), Psychophysics beyond sensation: Laws and invariants of human cognition (pp. 245–269). Mahwah, NJ: Erlbaum.
Parducci, A., & Haugen, R. (1967). The frequency principle for comparative judgments. Perception & Psychophysics, 2, 81–82.
Pöppel, E. (1988). Mindworks: Time and conscious experience [Trans. of Grenzen des Bewuβtseins by T. Artin]. Boston: Harcourt Brace Jovanovich.
Rihs, S. (1995). The influence of audio on perceived picture quality and subjective audio—video delay tolerance. In R. Hamberg & H. de Ridder (Eds.), Advanced methods for the evaluation of television picture quality: Proceedings of the MOSAIC Workshop (pp. 133–137). Eindhoven, The Netherlands: Institute for Perception Research.
Rutschmann, J., & Link, R. (1964). Perception of temporal order of stimuli differing in sense mode and simple reaction time. Perceptual & Motor Skills, 18, 345–352.
Schneider, K. A., & Bavelier, D. (2003). Components of visual prior entry. Cognitive Psychology, 47, 333–366.
Sekuler, R., & Erlebacher, A. (1971). The invalidity of “invalid results from the method of constant stimuli”: A common artifact in the methods of psychophysics. Perception & Psychophysics, 9, 309–311.
Shore, D. I., Gray, K., Spry, E., & Spence, C. (2005). Spatial modulation of tactile temporal-order judgments. Perception, 34, 1251–1262.
Shore, D. I., Spry, E., & Spence, C. (2002). Confusing the mind by crossing the hands. Cognitive Brain Research, 14, 153–163.
Smeele, P. M. T. (1994). Perceiving speech: Integrating auditory and visual speech. Unpublished PhD thesis, Technische Universiteit Delft, The Netherlands. Retrieved April 11, 2007, from repository.tudelft.nl/ file/257315/201176.
Smith, W. F. (1933). The relative quickness of visual and auditory perception. Journal of Experimental Psychology, 16, 239–257.
Spence, C., Baddeley, R., Zampini, M., James, R., & Shore, D. I. (2003). Multisensory temporal order judgments: When two locations are better than one. Perception & Psychophysics, 65, 318–328.
Spence, C., Shore, D. I., & Klein, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130, 799–832.
Stekelenburg, J. J., & Vroomen, J. (2007). Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience, 19, 1964–1973.
Stelmach, L. B., & Herdman, C. M. (1991). Directed attention and perception of temporal order. Journal of Experimental Psychology: Human Perception & Performance, 17, 539–550.
Sternberg, S., & Knoll, R. L. (1973). The perception of temporal order: Fundamental issues and a general model. In S. Kornblum (Ed.), Attention and performance IV (pp. 629–685). New York: Academic Press.
Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanland, M., et al. (2001). When is now? Perception of simultaneity. Proceedings of the Royal Society B, 268, 31–38.
Teatini, G., Farnè, M., Verzella, F., & Berruecos, P., Jr. (1976). Perception of temporal order: Visual and auditory stimuli. Giornale Italiano di Psicologia, 3, 157–164.
Tune, G. S. (1964). Response preferences: A review of some relevant literature. Psychological Bulletin, 61, 286–302.
Van de Par, S., Kohlrausch, A., & Juola, J. F. (2002). Some methodological aspects for measuring asynchrony detection in audio—visual stimuli. In A. Calvo-Manzano, A. Perez-Lopez, & J. S. Santiago (Eds.), Proceedings of the Forum Acusticum 2002 [CD-ROM]. Porto, Portugal: Universidade do Porto, Faculdade de Engenharia.
Van Eijk, R. L. J., Kohlrausch, A., van de Par, S., & Juola, J. F. (2006, June). The influence of psychophysical procedure and stimulus type on estimates of human performance in detecting audio—visual asynchrony. Paper presented at the 7th International Multisensory Research Forum (IMRF), University of Dublin, Trinity College, Dublin, Ireland.
Van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45, 598–607.
Vatakis, A., Navarra, J., Soto-Faraco, S., & Spence, C. (2008). Audiovisual temporal adaptation of speech: Temporal order versus simultaneity judgments. Experimental Brain Research, 185, 521–529.
Vatakis, A., & Spence, C. (2006a). Audiovisual synchrony perception for music, speech, and object actions. Brain Research, 1111, 134–142.
Vatakis, A., & Spence, C. (2006b). Audiovisual synchrony perception for speech and music assessed using a temporal order judgment task. Neuroscience Letters, 393, 40–44.
Vatakis, A., & Spence, C. (2007). Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli. Perception & Psychophysics, 69, 744–756.
Vroomen, J., Keetels, M., de Gelder, B., & Bertelson, P. (2004). Recalibration of temporal order perception by exposure to audio— visual asynchrony. Cognitive Brain Research, 22, 32–35.
Whipple, G. M. (1899). On nearly simultaneous clicks and flashes. American Journal of Psychology, 10, 280–286.
Wichmann, F. A., & Hill, N. J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 1293–1313.
Zampini, M., Brown, T., Shore, D. I., Maravita, A., Röder, B., & Spence, C. (2005). Audiotactile temporal order judgments. Acta Psychologica, 118, 277–291.
Zampini, M., Guest, S., Shore, D. I., & Spence, C. (2005). Audio— visual simultaneity judgments. Perception & Psychophysics, 67, 531–544.
Zampini, M., Shore, D. I., & Spence, C. (2003a). Audiovisual temporal order judgments. Experimental Brain Research, 152, 198–210.
Zampini, M., Shore, D. I., & Spence, C. (2003b). Multisensory temporal order judgments: The role of hemispheric redundancy. International Journal of Psychophysiology, 50, 165–180.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
van Eijk, R.L.J., Kohlrausch, A., Juola, J.F. et al. Audiovisual synchrony and temporal order judgments: Effects of experimental method and stimulus type. Perception & Psychophysics 70, 955–968 (2008). https://doi.org/10.3758/PP.70.6.955
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/PP.70.6.955