Advertisement

When the Perception of a Synchronous World Is—Mostly—Just an Illusion

  • Nadia Paraskevoudi
  • Argiro Vatakis
Chapter

Abstract

Creating a coherent representation of our environment is highly dependent on integrating sensory signals into unified multisensory percepts. These percepts are not only multisensory but also synchronous despite the physical and processing time differences of the various signals. How perceptual synchrony is attained by the human brain is one of the most perplexing questions yet to be answered, with major implications about the philosophy and psychology of time. Although many issues still remain largely unknown, evidence suggests that synchrony perception is modulated by stimulus features, the individual’s age, and prior experience and exhibits abnormalities in pathology. In this chapter, we review the factors modulating our ability to align sensory inputs in time and summarize recent findings on the behavioral and neural correlates of perceptual synchrony.

References

  1. Adhikari, B. M., Goshorn, E. S., Lamichhane, B., & Dhamala, M. (2013). Temporal-order judgment of audiovisual events involves network activity between parietal and prefrontal cortices. Brain Connectivity, 3(5), 536–545.  https://doi.org/10.1089/brain.2013.0163 CrossRefGoogle Scholar
  2. Alais, D., Ho, T., Han, S., & Van der Burg, E. (2017). A matched comparison across three different sensory pairs of cross-modal temporal recalibration from sustained and transient adaptation. I-Perception, 8(4), 2041669517718697.  https://doi.org/10.1177/2041669517718697 CrossRefGoogle Scholar
  3. Balz, J., Roa Romero, Y., Keil, J., Krebber, M., Niedeggen, M., Gallinat, J., & Senkowski, D. (2016). Beta/Gamma oscillations and event-related potentials indicate aberrant multisensory processing in schizophrenia. Frontiers in Psychology, 7, 1896.  https://doi.org/10.3389/fpsyg.2016.01896 CrossRefGoogle Scholar
  4. Barnett-Cowan, M., & Harris, L. R. (2009). Perceived timing of vestibular stimulation relative to touch, light and sound. Experimental Brain Research, 198(2–3), 221–231.  https://doi.org/10.1007/s00221-009-1779-4 CrossRefGoogle Scholar
  5. Barnett-Cowan, M., & Harris, L. R. (2011). Temporal processing of active and passive head movement. Experimental Brain Research, 214(1), 27–35.  https://doi.org/10.1007/s00221-011-2802-0 CrossRefGoogle Scholar
  6. Basharat, A., Adams, M. S., Staines, W. R., & Barnett-Cowan, M. (2018). Simultaneity and temporal order judgments are coded differently and change with age: An event-related potential study. Frontiers in Integrative Neuroscience, 12, 15.  https://doi.org/10.3389/fnint.2018.00015 CrossRefGoogle Scholar
  7. Baumann, O., Vromen, J. M. G., Cheung, A., McFadyen, J., Ren, Y., & Guo, C. C. (2018). Neural correlates of temporal complexity and synchrony during audiovisual correspondence detection. eNeuro, 5(1).  https://doi.org/10.1523/ENEURO.0294-17.2018 CrossRefGoogle Scholar
  8. Baumgarten, T. J., Königs, S., Schnitzler, A., & Lange, J. (2017). Subliminal stimuli modulate somatosensory perception rhythmically and provide evidence for discrete perception. Scientific Reports, 7, 43937.  https://doi.org/10.1038/srep43937 CrossRefGoogle Scholar
  9. Baumgarten, T. J., Schnitzler, A., & Lange, J. (2015). Beta oscillations define discrete perceptual cycles in the somatosensory domain. Proceedings of the National Academy of Sciences of the United States of America, 112(39), 12187–12192.  https://doi.org/10.1073/pnas.1501438112 CrossRefGoogle Scholar
  10. Bedard, G., & Barnett-Cowan, M. (2016). Impaired timing of audiovisual events in the elderly. Experimental Brain Research, 234(1), 331–340.  https://doi.org/10.1007/s00221-015-4466-7 CrossRefGoogle Scholar
  11. Benedetto, A., Burr, D. C., & Morrone, M. C. (2018). Perceptual oscillation of audiovisual time simultaneity. eNeuro, 5(3).  https://doi.org/10.1523/ENEURO.0047-18.2018 CrossRefGoogle Scholar
  12. Benedetto, A., & Morrone, M. C. (2017). Saccadic suppression is embedded within extended oscillatory modulation of sensitivity. Journal of Neuroscience, 37(13), 3661–3670.  https://doi.org/10.1523/JNEUROSCI.2390-16.2016 CrossRefGoogle Scholar
  13. Benedetto, A., Spinelli, D., & Morrone, M. C. (2016). Rhythmic modulation of visual contrast discrimination triggered by action. Proceedings of the Royal Society B: Biological Sciences, 283(1831), 20160692.  https://doi.org/10.1098/rspb.2016.0692 CrossRefGoogle Scholar
  14. Bhat, J., Miller, L. M., Pitt, M. A., & Shahin, A. J. (2015). Putative mechanisms mediating tolerance for audiovisual stimulus onset asynchrony. Journal of Neurophysiology, 113(5), 1437–1450.  https://doi.org/10.1152/jn.00200.2014 CrossRefGoogle Scholar
  15. Binder, M. (2015). Neural correlates of audiovisual temporal processing—Comparison of temporal order and simultaneity judgments. Neuroscience, 300, 432–447.  https://doi.org/10.1016/j.neuroscience.2015.05.011 CrossRefGoogle Scholar
  16. Brooks, C. J., Chan, Y. M., Anderson, A. J., & McKendrick, A. M. (2018). Audiovisual temporal perception in aging: The role of multisensory integration and age-related sensory loss. Frontiers in Human Neuroscience, 12, 192.  https://doi.org/10.3389/fnhum.2018.00192 CrossRefGoogle Scholar
  17. Bushara, K. O., Grafman, J., & Hallett, M. (2001). Neural correlates of auditory-visual stimulus onset asynchrony detection. Journal of Neuroscience, 21(1), 300–304.  https://doi.org/10.1523/JNEUROSCI.21-01-00300.2001 CrossRefGoogle Scholar
  18. Cecere, R., Gross, J., Willis, A., & Thut, G. (2017). Being first matters: Topographical representational similarity analysis of ERP signals reveals separate networks for audiovisual temporal binding depending on the leading sense. Journal of Neuroscience, 37(21), 5274–5287.  https://doi.org/10.1523/JNEUROSCI.2926-16.2017 CrossRefGoogle Scholar
  19. Cecere, R., Rees, G., & Romei, V. (2015). Individual differences in alpha frequency drive crossmodal illusory perception. Current Biology, 25(2), 231–235.  https://doi.org/10.1016/j.cub.2014.11.034 CrossRefGoogle Scholar
  20. Cecere, R., Romei, V., Bertini, C., & Làdavas, E. (2014). Crossmodal enhancement of visual orientation discrimination by looming sounds requires functional activation of primary visual areas: A case study. Neuropsychologia, 56, 350–358.  https://doi.org/10.1016/j.neuropsychologia.2014.02.008 CrossRefGoogle Scholar
  21. Chan, Y. M., Pianta, M. J., & McKendrick, A. M. (2014a). Older age results in difficulties separating auditory and visual signals in time. Journal of Vision, 14(11), 1–11.  https://doi.org/10.1167/14.11.13 CrossRefGoogle Scholar
  22. Chan, Y. M., Pianta, M. J., & McKendrick, A. M. (2014b). Reduced audiovisual recalibration in the elderly. Frontiers in Aging Neuroscience, 6, 226.  https://doi.org/10.3389/fnagi.2014.00226 CrossRefGoogle Scholar
  23. Chen, Y. C., Lewis, T. L., Shore, D. I., Spence, C., & Maurer, D. (2018). Developmental changes in the perception of visuotactile simultaneity. Journal of Experimental Child Psychology, 173, 304–317.  https://doi.org/10.1016/j.jecp.2018.04.014 CrossRefGoogle Scholar
  24. Condello, G., Forte, R., Falbo, S., Shea, J. B., Di Baldassarre, A., Capranica, L., & Pesce, C. (2017). Steps to health in cognitive aging: Effects of physical activity on spatial attention and executive control in the elderly. Frontiers in Human Neuroscience, 11, 107.  https://doi.org/10.3389/fnhum.2017.00107 CrossRefGoogle Scholar
  25. Conrey, B., & Pisoni, D. B. (2006). Auditory-visual speech perception and synchrony detection for speech and nonspeech signals. The Journal of the Acoustical Society of America, 119(6), 4065–4073.  https://doi.org/10.1121/1.2195091 CrossRefGoogle Scholar
  26. Couth, S., Gowen, E., & Poliakoff, E. (2016). Investigating the spatial and temporal modulation of visuotactile interactions in older adults. Experimental Brain Research, 234(5), 1233–1248.  https://doi.org/10.1007/s00221-015-4431-5 CrossRefGoogle Scholar
  27. Cuppini, C., Shams, L., Magosso, E., & Ursino, M. (2017). A biologically inspired neurocomputational model for audiovisual integration and causal inference. Computational Neuroscience, 46(9), 2481–2498.  https://doi.org/10.1111/ejn.13725 CrossRefGoogle Scholar
  28. de Boer-Schellekens, L., & Vroomen, J. (2014). Multisensory integration compensates loss of sensitivity of visual temporal order in the elderly. Experimental Brain Research, 232(1), 253–262.  https://doi.org/10.1007/s00221-013-3736-5 CrossRefGoogle Scholar
  29. de Dieuleveult, A. L., Siemonsma, P. C., van Erp, J. B. F., & Brouwer, A.-M. (2017). Effects of aging in multisensory integration: A systematic review. Frontiers in Aging Neuroscience, 9, 80.  https://doi.org/10.3389/fnagi.2017.00080 CrossRefGoogle Scholar
  30. De Niear, M. A., Gupta, P. B., Baum, S. H., & Wallace, M. T. (2018). Perceptual training enhances temporal acuity for multisensory speech. Neurobiology of Learning and Memory, 147, 9–17.  https://doi.org/10.1016/j.nlm.2017.10.016 CrossRefGoogle Scholar
  31. De Niear, M. A., Koo, B., & Wallace, M. T. (2016). Multisensory perceptual learning is dependent upon task difficulty. Experimental Brain Research, 234(11), 3269–3277.  https://doi.org/10.1007/s00221-016-4724-3 CrossRefGoogle Scholar
  32. De Niear, M. A., Noel, J.-P., & Wallace, M. T. (2017). The impact of feedback on the different time courses of multisensory temporal recalibration. Neural Plasticity, 2017, 3478742.  https://doi.org/10.1155/2017/3478742 CrossRefGoogle Scholar
  33. Dean, C. L., Eggleston, B. A., Gibney, K. D., Aligbe, E., Blackwell, M., & Kwakye, L. D. (2017). Auditory and visual distractors disrupt multisensory temporal acuity in the crossmodal temporal order judgment task. PLoS One, 12(7), e0179564.  https://doi.org/10.1371/journal.pone.0179564 CrossRefGoogle Scholar
  34. DeLoss, D. J., Pierce, R. S., & Andersen, G. J. (2013). Multisensory integration, aging, and the sound-induced flash illusion. Psychology and Aging, 28(3), 802–812.  https://doi.org/10.1037/a0033289 CrossRefGoogle Scholar
  35. Diederich, A., Colonius, H., & Schomburg, A. (2008). Assessing age-related multisensory enhancement with the time-window-of-integration model. Neuropsychologia, 46(10), 2556–2562.  https://doi.org/10.1016/j.neuropsychologia.2008.03.026 CrossRefGoogle Scholar
  36. Doesburg, S. M., Emberson, L. L., Rahi, A., Cameron, D., & Ward, L. M. (2008). Asynchrony from synchrony: Long-range gamma-band neural synchrony accompanies perception of audiovisual speech asynchrony. Experimental Brain Research, 185(1), 11–20.  https://doi.org/10.1007/s00221-007-1127-5 CrossRefGoogle Scholar
  37. Efron, R. (1963). The effect of handedness on the perception of simultaneity and temporal order. Brain, 86(2), 261.  https://doi.org/10.1093/brain/86.2.261 CrossRefGoogle Scholar
  38. Eg, R., & Behne, D. M. (2015). Perceived synchrony for realistic and dynamic audiovisual events. Frontiers in Psychology, 6, 736.  https://doi.org/10.3389/fpsyg.2015.00736 CrossRefGoogle Scholar
  39. Eg, R., Griwodz, C., Halvorsen, P., & Behne, D. (2015). Audiovisual robustness: Exploring perceptual tolerance to asynchrony and quality distortion. Multimedia Tools and Applications, 74(2), 345–365.  https://doi.org/10.1007/s11042-014-2136-6 CrossRefGoogle Scholar
  40. Foss-Feig, J. H., Kwakye, L. D., Cascio, C. J., Burnette, C. P., Kadivar, H., Stone, W. L., & Wallace, M. T. (2010). An extended multisensory temporal binding window in autism spectrum disorders. Experimental Brain Research, 203(2), 381–389.  https://doi.org/10.1007/s00221-010-2240-4 CrossRefGoogle Scholar
  41. Foxe, J. J., Molholm, S., Del Bene, V. A., Frey, H.-P., Russo, N. N., Blanco, D., … Ross, L. A. (2015). Multisensory speech integration deficits in high-functioning school-aged children with autism spectrum disorder (ASD) and their resolution during early adolescence. Cerebral Cortex, 25(2), 298–312.  https://doi.org/10.1093/cercor/bht213 CrossRefGoogle Scholar
  42. Fujisaki, W., & Nishida, S. (2009). Audio-tactile superiority over visuo-tactile and audio-visual combinations in the temporal resolution of synchrony perception. Experimental Brain Research, 198(2–3), 245–259.  https://doi.org/10.1007/s00221-009-1870-x CrossRefGoogle Scholar
  43. Gao, F., Edden, R. A. E., Li, M., Puts, N. A. J., Wang, G., Liu, C., … Barker, P. B. (2013). Edited magnetic resonance spectroscopy detects an age-related decline in brain GABA levels. NeuroImage, 78, 75–82.  https://doi.org/10.1016/j.neuroimage.2013.04.012 CrossRefGoogle Scholar
  44. Gori, M. (2015). Multisensory integration and calibration in children and adults with and without sensory and motor disabilities. Multisensory Research, 28(1–2), 71–99.  https://doi.org/10.1163/22134808-00002478 CrossRefGoogle Scholar
  45. Gotow, N., & Kobayakawa, T. (2017). Simultaneity judgment using olfactory-visual, visual-gustatory, and olfactory–gustatory combinations. PLoS ONE, 12(4), e0174958.  https://doi.org/10.1371/journal.pone.0174958 CrossRefGoogle Scholar
  46. Halliday, A., & Mingay, R. (1964). On the resolution of small time intervals and the effect of conduction delays on the judgement of simultaneity. Quarterly Journal of Experimental Psychology, 16(1), 37–41.  https://doi.org/10.1080/17470216408416344 CrossRefGoogle Scholar
  47. Harrar, V., & Harris, L. (2008). The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity. Experimental Brain Research, 186(4), 517–524.  https://doi.org/10.1007/s00221-007-1253-0 CrossRefGoogle Scholar
  48. Hermosillo, R., Ritterband-Rosenbaum, A., & van Donkelaar, P. (2011). Predicting future sensorimotor states influences current temporal decision making. Journal of Neuroscience, 31(27), 10019–10022.  https://doi.org/10.1523/JNEUROSCI.0037-11.2011 CrossRefGoogle Scholar
  49. Hillock, A. R., Powers, A. R., & Wallace, M. T. (2011). Binding of sights and sounds: Age-related changes in multisensory temporal processing. Neuropsychologia, 49(3), 461–467.  https://doi.org/10.1016/j.neuropsychologia.2010.11.041 CrossRefGoogle Scholar
  50. Hillock-Dunn, A., & Wallace, M. T. (2012). Developmental changes in the multisensory temporal binding window persist into adolescence. Developmental Science, 15(5), 688–696.  https://doi.org/10.1111/j.1467-7687.2012.01171.x CrossRefGoogle Scholar
  51. Hogendoorn, H. (2016). Voluntary saccadic eye movements ride the attentional rhythm. Journal of Cognitive Neuroscience, 28(10), 1625–1635.  https://doi.org/10.1162/jocn_a_00986 CrossRefGoogle Scholar
  52. Hoover, A. E. N., & Harris, L. R. (2012). Detecting delay in visual feedback of an action as a monitor of self recognition. Experimental Brain Research, 222(4), 389–397.  https://doi.org/10.1007/s00221-012-3224-3 CrossRefGoogle Scholar
  53. Hoover, A. E. N., & Harris, L. R. (2016). Inducing ownership over an “other” perspective with a visuo-tactile manipulation perception. Experimental Brain Research, 234(12), 3633–3639.  https://doi.org/10.1007/s00221-016-4760-z CrossRefGoogle Scholar
  54. Jaskowski, P. (1999). Reaction time and temporal-order judgment as measures of perceptual latency: The problem of dissociations. In G. Aschersleben, T. Bachmann, & J. Musseler (Eds.), Cognitive contributions to the perception of spatial and temporal events (pp. 265–282). North-Holland: Elsevier Science B.V.CrossRefGoogle Scholar
  55. Jicol, C., Proulx, M. J., Pollick, F. E., & Petrini, K. (2018). Long-term music training modulates the recalibration of audiovisual simultaneity. Experimental Brain Research, 236(7), 1869–1880.  https://doi.org/10.1007/s00221-018-5269-4 CrossRefGoogle Scholar
  56. Keetels, M., & Vroomen, J. (2012). Perception of synchrony between the senses. In M. M. Murray & M. T. Wallace (Eds.), The neural bases of multisensory processes (pp. 1–27). Boca Raton, FL: CRC Press.Google Scholar
  57. Keys, R. T., Rich, A. N., & Zopf, R. (2018). Multisensory temporal processing in own-body contexts: Plausibility of hand ownership does not improve visuo-tactile asynchrony detection. Experimental Brain Research, 236(5), 1431–1443.  https://doi.org/10.1007/s00221-018-5232-4 CrossRefGoogle Scholar
  58. Kostaki, M., & Vatakis, A. (2018). Temporal order and synchrony judgments: A primer for students. In A. Vatakis, F. Balci, M. Di Luca, & A. Correa (Eds.), Timing and time perception: Procedures, measures, and applications (pp. 233–262). Leiden: Brill.Google Scholar
  59. Krueger Fister, J., Stevenson, R. A., Nidiffer, A. R., Barnett, Z. P., & Wallace, M. T. (2016). Stimulus intensity modulates multisensory temporal processing. Neuropsychologia, 88, 92–100.  https://doi.org/10.1016/j.neuropsychologia.2016.02.016 CrossRefGoogle Scholar
  60. Kwakye, L. D., Foss-Feig, J. H., Cascio, C. J., Stone, W. L., & Wallace, M. T. (2010). Altered auditory and multisensory temporal processing in autism spectrum disorders. Frontiers in Integrative Neuroscience, 4, 129.  https://doi.org/10.3389/fnint.2010.00129 CrossRefGoogle Scholar
  61. Lakatos, P., Chen, C. M., O’Connell, M., Mills, A., & Schroeder, C. E. (2007). Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron, 53(2), 279–292.  https://doi.org/10.1016/j.neuron.2006.12.011 CrossRefGoogle Scholar
  62. Lakatos, P., Karmos, G., Mehta, A. D., Ulbert, I., & Schroeder, C. E. (2008). Entrainment of neuronal oscillations as a mechanism of attentional selection. Science, 320(5872), 110–113.  https://doi.org/10.1126/science.1154735 CrossRefGoogle Scholar
  63. Landau, A. N., & Fries, P. (2012). Attention samples stimuli rhythmically. Current Biology, 22(11), 1000–1004.  https://doi.org/10.1016/j.cub.2012.03.054 CrossRefGoogle Scholar
  64. Lange, J., Kapala, K., Krause, H., Baumgarten, T. J., & Schnitzler, A. (2018). Rapid temporal recalibration to visuo-tactile stimuli. Experimental Brain Research, 236(2), 347–354.  https://doi.org/10.1007/s00221-017-5132-z CrossRefGoogle Scholar
  65. Laurienti, P. J., Burdette, J. H., Maldjian, J. A., & Wallace, M. T. (2006). Enhanced multisensory integration in older adults. Neurobiology of Aging, 27(8), 1155–1163.  https://doi.org/10.1016/j.neurobiolaging.2005.05.024 CrossRefGoogle Scholar
  66. Love, S. A., Petrini, K., Cheng, A., & Pollick, F. E. (2013). A psychophysical investigation of differences between synchrony and temporal order judgments. PLoS One, 8(1), e54798.  https://doi.org/10.1371/journal.pone.0054798 CrossRefGoogle Scholar
  67. Magnotti, J. F., Ma, W. J., & Beauchamp, M. S. (2013). Causal inference of asynchronous audiovisual speech. Frontiers in Psychology, 4, 798.  https://doi.org/10.3389/fpsyg.2013.00798 CrossRefGoogle Scholar
  68. Mahoney, J. R., Holtzer, R., & Verghese, J. (2014). Visual-somatosensory integration and balance: Evidence for psychophysical integrative differences in aging. Multisensory Research, 27(1), 17–42.  https://doi.org/10.1163/22134808-00002444 CrossRefGoogle Scholar
  69. Martin, B., Giersch, A., Huron, C., & van Wassenhove, V. (2013). Temporal event structure and timing in schizophrenia: Preserved binding in a longer now. Neuropsychologia, 51(2), 358–371.  https://doi.org/10.1016/j.neuropsychologia.2012.07.002 CrossRefGoogle Scholar
  70. McGovern, D. P., Roudaia, E., Stapleton, J., McGinnity, T. M., & Newell, F. N. (2014). The sound-induced flash illusion reveals dissociable age-related effects in multisensory integration. Frontiers in Aging Neuroscience, 6, 250.  https://doi.org/10.3389/fnagi.2014.00250 CrossRefGoogle Scholar
  71. Merriman, N. A., Whyatt, C., Setti, A., Craig, C., & Newell, F. N. (2015). Successful balance training is associated with improved multisensory function in fall-prone older adults. Computers in Human Behavior, 45, 192–203.  https://doi.org/10.1016/j.chb.2014.12.017 CrossRefGoogle Scholar
  72. Mishra, J., & Gazzaley, A. (2014). Attentional control of multisensory integration is preserved in aging. In G. R. Mangun (Ed.), Cognitive electrophysiology of attention. Signals of the mind (pp. 190–203). San Diego, CA: Academic Press.CrossRefGoogle Scholar
  73. Mitrani, L., Shekerdjiiski, S., & Yakimoff, N. (1986). Mechanisms and asymmetries in visual perception of simultaneity and temporal order. Biological Cybernetics, 54(3), 159–165.Google Scholar
  74. Miyazaki, M., Kadota, H., Matsuzaki, K. S., Takeuchi, S., Sekiguchi, H., Aoyama, T., & Kochiyama, T. (2016). Dissociating the neural correlates of tactile temporal order and simultaneity judgements. Scientific Reports, 6, 23323.  https://doi.org/10.1038/srep23323 CrossRefGoogle Scholar
  75. Mozolic, J. L., Hugenschmidt, C. E., Peiffer, A. M., & Laurienti, P. J. (2012). Multisensory integration and aging. In M. Murray & M. Wallace (Eds.), The neural bases of multisensory processes (pp. 381–392). Boca Raton, FL: CRC Press.Google Scholar
  76. Murray, M. M., & Spierer, L. (2009). Auditory spatio-temporal brain dynamics and their consequences for multisensory interactions in humans. Hearing Research, 258(1–2), 121–133.  https://doi.org/10.1016/j.heares.2009.04.022 CrossRefGoogle Scholar
  77. Musacchia, G., & Schroeder, C. E. (2009). Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hearing Research, 258(1–2), 72–79.  https://doi.org/10.1016/j.heares.2009.06.018 CrossRefGoogle Scholar
  78. Navarra, J., Fernández-Prieto, I., & Garcia-Morera, J. (2013). Realigning thunder and lightning: Temporal adaptation to spatiotemporally distant events. PLoS One, 8(12), e84278.  https://doi.org/10.1371/journal.pone.0084278 CrossRefGoogle Scholar
  79. Navarra, J., Vatakis, A., Zampini, M., Soto-Faraco, S., Humphreys, W., & Spence, C. (2005). Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Research, 25(2), 499–507.  https://doi.org/10.1016/j.cogbrainres.2005.07.009 CrossRefGoogle Scholar
  80. Neumann, O., & Niepel, M. (2004). Timing of “perception” and perception of “time”. In C. Kaernbach, E. Schroger, & H. Muller (Eds.), Psychophysics beyond sensation: Laws and invariants of human cognition (pp. 245–270). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar
  81. Noel, J. P., De Niear, M., Lazzara, N., & Wallace, M. T. (2017). Uncoupling between multisensory temporal function and non-verbal turn-taking in Autism Spectrum Disorder. IEEE Transactions on Cognition and Developmental Systems, 10, 973–982.  https://doi.org/10.1109/TCDS.2017.2778141 CrossRefGoogle Scholar
  82. Noel, J. P., Lytle, M., Cascio, C., & Wallace, M. T. (2017). Disrupted integration of exteroceptive and interoceptive signaling in autism spectrum disorder. Autism Research, 11(1), 194–205.  https://doi.org/10.1002/aur.1880 CrossRefGoogle Scholar
  83. Noel, J.-P., Stevenson, R. A., & Wallace, M. T. (2018). Atypical audiovisual temporal function in autism and schizophrenia: Similar phenotype, different cause. European Journal of Neuroscience, 47(10), 1230–1241.  https://doi.org/10.1111/ejn.13911 CrossRefGoogle Scholar
  84. Noel, J.-P., Wallace, M. T., Orchard-Mills, E., Alais, D., & Van der Burg, E. (2015). True and perceived synchrony are preferentially associated with particular sensory pairings. Scientific Reports, 5, 17467.  https://doi.org/10.1038/srep17467 CrossRefGoogle Scholar
  85. Paraskevoudi, N., Balcı, F., & Vatakis, A. (2018). “Walking” through the sensory, cognitive, and temporal degradations of healthy aging. Annals of the New York Academy of Sciences, 1426(1), 72–92.  https://doi.org/10.1111/nyas.13734 CrossRefGoogle Scholar
  86. Peiffer, A. M., Mozolic, J. L., Hugenschmidt, C. E., & Laurienti, P. J. (2007). Age-related multisensory enhancement in a simple audiovisual detection task. NeuroReport, 18(10), 1077–1081.  https://doi.org/10.1097/WNR.0b013e3281e72ae7 CrossRefGoogle Scholar
  87. Perrodin, C., Kayser, C., Logothetis, N. K., & Petkov, C. I. (2015). Natural asynchronies in audiovisual communication signals regulate neuronal multisensory interactions in voice-sensitive cortex. Proceedings of the National Academy of Sciences of the United States of America, 112(1), 273–278.  https://doi.org/10.1073/pnas.1412817112 CrossRefGoogle Scholar
  88. Pöppel, E. (1988). Mindworks. Time and conscious experience. San Diego: Harcourt Brace Jovanovich.Google Scholar
  89. Pöppel, E. (2004). Lost in time: A historical frame, elementary processing units and the 3-second window. Acta Neurobiologiae Experimentalis, 64(3), 295–301.Google Scholar
  90. Proverbio, A. M., Attardo, L., Cozzi, M., & Zani, A. (2015). The effect of musical practice on gesture/sound pairing. Frontiers in Psychology, 6, 376.  https://doi.org/10.3389/fpsyg.2015.00376 CrossRefGoogle Scholar
  91. Ren, Y., Yang, W., Nakahashi, K., Takahashi, S., & Wu, J. (2017). Audiovisual integration delayed by stimulus onset asynchrony between auditory and visual stimuli in older adults. Perception, 6(2), 205–218.  https://doi.org/10.1177/0301006616673850 CrossRefGoogle Scholar
  92. Roefs, J. A. J. (1963). Perception lag as a function of stimulus luminance. Vision Research, 3, 81–91.  https://doi.org/10.1016/0042-6989(63)90070-1 CrossRefGoogle Scholar
  93. Rohde, M., & Ernst, M. O. (2012). To lead and to lag—Forward and backward recalibration of perceived visuo-motor simultaneity. Frontiers in Psychology, 3, 599.  https://doi.org/10.3389/fpsyg.2012.00599 CrossRefGoogle Scholar
  94. Rohde, M., & Ernst, M. O. (2016). Time, agency, and sensory feedback delays during action. Current Opinion in Behavioral Sciences, 8, 193–199.  https://doi.org/10.1016/j.cobeha.2016.02.029 CrossRefGoogle Scholar
  95. Ronconi, L., Oosterhof, N. N., Bonmassar, C., & Melcher, D. (2017). Multiple oscillatory rhythms determine the temporal organization of perception. Proceeding of the National Academy of Sciences of the United States of America, 114, 13435–13440.  https://doi.org/10.1073/pnas.1714522114 CrossRefGoogle Scholar
  96. Roy, C., Dalla Bella, S., & Lagarde, J. (2017). To bridge or not to bridge the multisensory time gap: Bimanual coordination to sound and touch with temporal lags. Experimental Brain Research, 235(1), 135–151.  https://doi.org/10.1007/s00221-016-4776-4 CrossRefGoogle Scholar
  97. Samaha, J., & Postle, B. R. (2015). The speed of alpha-band oscillations predicts the temporal resolution of visual perception. Current Biology, 25(2), 2985–2990.  https://doi.org/10.1016/j.cub.2015.10.007 CrossRefGoogle Scholar
  98. Sanford, A. J. (1971). Effects of changes in the intensity of white noise on simultaneity judgements and simple reaction time. Quarterly Journal of Experimental Psychology, 23(3), 296–303.  https://doi.org/10.1080/14640746908401824 CrossRefGoogle Scholar
  99. Schroeder, C. E., Lakatos, P., Kajikawa, Y., Partan, S., & Puce, A. (2008). Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences, 12(3), 106–113.  https://doi.org/10.1016/j.tics.2008.01.002 CrossRefGoogle Scholar
  100. Schwartz, J. L., & Savariaux, C. (2014). No, there is no 150 ms lead of visual speech on auditory speech, but a range of audiovisual asynchronies varying from small audio lead to large audio lag. PLoS Computational Biology, 10(7), e1003743.  https://doi.org/10.1371/journal.pcbi.1003743 CrossRefGoogle Scholar
  101. Senkowski, D., Talsma, D., Grigutsch, M., Herrmann, C. S., & Woldorff, M. G. (2007). Good times for multisensory integration: Effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia, 45(3), 561–571.  https://doi.org/10.1016/j.neuropsychologia.2006.01.013 CrossRefGoogle Scholar
  102. Setti, A., Burke, K. E., Kenny, R., & Newell, F. N. (2013). Susceptibility to a multisensory speech illusion in older persons is driven by perceptual processes. Frontiers in Psychology, 4, 575.  https://doi.org/10.3389/fpsyg.2013.00575 CrossRefGoogle Scholar
  103. Setti, A., Burke, K. E., & Kenny, R. A. (2011). Is inefficient multisensory processing associated with falls in older people? Experimental Brain Research, 209(3), 375–384.  https://doi.org/10.1007/s00221-011-2560-z CrossRefGoogle Scholar
  104. Setti, A., Stapleton, J., Leahy, D., Walsh, C., Kenny, R. A., & Newell, F. N. (2014). Improving the efficiency of multisensory integration in older adults: Audio-visual temporal discrimination training reduces susceptibility to the sound-induced flash illusion. Neuropsychologia, 61, 259–268.  https://doi.org/10.1016/j.neuropsychologia.2014.06.027 CrossRefGoogle Scholar
  105. Shahin, A. J., Shen, S., & Kerlin, J. R. (2017). Tolerance for audiovisual asynchrony is enhanced by the spectrotemporal fidelity of the speaker’s mouth movements and speech. Language, Cognition and Neuroscience, 32(9), 1102–1118.  https://doi.org/10.1080/23273798.2017.1283428 CrossRefGoogle Scholar
  106. Simon, D. M., Nidiffer, A. R., & Wallace, M. T. (2018). Single trial plasticity in evidence accumulation underlies rapid recalibration to asynchronous audiovisual speech. Scientific Reports, 8, 12499.  https://doi.org/10.1038/s41598-018-30414-9 CrossRefGoogle Scholar
  107. Simon, D. M., Noel, J.-P., & Wallace, M. T. (2017). Event related potentials index rapid recalibration to audiovisual temporal asynchrony. Frontiers in Integrative Neuroscience, 11, 8.  https://doi.org/10.3389/fnint.2017.00008 CrossRefGoogle Scholar
  108. Simon, D. M., & Wallace, M. T. (2017). Integration and temporal processing of asynchronous audiovisual speech. Journal of Cognitive Neuroscience, 30(3), 319–337.  https://doi.org/10.1162/jocn_a_01205 CrossRefGoogle Scholar
  109. Smith, W. F. (1933). The relative quickness of visual and auditory perception. Journal of Experimental Psychology, 16(2), 239–257.  https://doi.org/10.1037/h0071379 CrossRefGoogle Scholar
  110. Stapleton, J., Setti, A., Doheny, E. P., Kenny, R. A., & Newell, F. N. (2014). A standing posture is associated with increased susceptibility to the sound-induced flash illusion in fall-prone older adults. Experimental Brain Research, 32(2), 423–434.  https://doi.org/10.1007/s00221-013-3750-7 CrossRefGoogle Scholar
  111. Stefanics, G., Hangya, B., Hernádi, I., Winkler, I., Lakatos, P., & Ulbert, I. (2010). Phase entrainment of human delta oscillations can mediate the effects of expectation on reaction speed. Journal of Neuroscience, 30(41), 13578–13585.  https://doi.org/10.1523/JNEUROSCI.0703-10.2010 CrossRefGoogle Scholar
  112. Stevenson, R. A., Bushmakin, M., Kim, S., Wallace, M. T., Puce, A., & James, T. W. (2012). Inverse effectiveness and multisensory interactions in visual event-related potentials with audiovisual speech. Brain Topography, 25(3), 308–326.  https://doi.org/10.1007/s10548-012-0220-7 CrossRefGoogle Scholar
  113. Stevenson, R. A., Park, S., Cochran, C., McIntosh, L. G., Noel, J.-P., Barense, M. D., … Wallace, M. T. (2017). The associations between multisensory temporal processing on symptoms of schizophrenia. Schizophrenia Research, 179, 97–103.  https://doi.org/10.1016/j.schres.2016.09.035 CrossRefGoogle Scholar
  114. Stevenson, R. A., Segers, M., Ferber, S., Barense, M. D., & Wallace, M. T. (2016). Keeping time in the brain: Autism spectrum disorder and audiovisual temporal processing. Autism Research, 9(7), 720–738.  https://doi.org/10.1002/aur.1566 CrossRefGoogle Scholar
  115. Stevenson, R. A., Segers, M., Ncube, B. L., Black, K. R., Bebko, M., Ferber, S., & Barense, M. D. (2017). The cascading influence of low-level multisensory processing on speech perception in autism. Autism, 22(5), 609–624.  https://doi.org/10.1177/1362361317704413 CrossRefGoogle Scholar
  116. Stevenson, R. A., Siemann, J. K., Schneider, B. C., Eberly, H. E., Woynaroski, T. G., Camarata, S. M., & Wallace, M. T. (2014). Multisensory temporal integration in autism spectrum disorders. Journal of Neuroscience, 34(3), 691–697.  https://doi.org/10.1523/jneurosci.3615-13.2014 CrossRefGoogle Scholar
  117. Stevenson, R. A., Toulmin, J. K., Youm, A., Besney, R. M., Schulz, S. E., Barense, M. D., & Ferber, S. (2017). Increases in the autistic trait of attention to detail are associated with decreased multisensory temporal adaptation. Scientific Reports, 7(1), 14354.  https://doi.org/10.1038/s41598-017-14632-1 CrossRefGoogle Scholar
  118. Stevenson, R. A., & Wallace, M. T. (2013). Multisensory temporal integration: Task and stimulus dependencies. Experimental Brain Research. Experimentelle Hirnforschung. Experimentation Cerebrale, 227(2), 249–261.  https://doi.org/10.1007/s00221-013-3507-3 CrossRefGoogle Scholar
  119. Stevenson, R. A., Wilson, M. M., Powers, A. R., & Wallace, M. T. (2013). The effects of visual training on multisensory temporal processing. Experimental Brain Research, 225(4), 479–489.  https://doi.org/10.1007/s00221-012-3387-y CrossRefGoogle Scholar
  120. Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanland, M., … Porter, N. R. (2001). When is now? Perception of simultaneity. Proceedings. Biological Sciences/The Royal Society, 268(1462), 31–38.  https://doi.org/10.1098/rspb.2000.1326 CrossRefGoogle Scholar
  121. Su, L., Wyble, B., Zhou, L., Wang, K., Wang, Y., Cheung, E. F. C., … Chan, R. C. K. (2015). Temporal perception deficits in schizophrenia: Integration is the problem, not deployment of attentions. Scientific Reports, 5, 9745.  https://doi.org/10.1038/srep09745 CrossRefGoogle Scholar
  122. Takahashi, T., Kansaku, K., Wada, M., Shibuya, S., & Kitazawa, S. (2013). Neural correlates of tactile temporal-order judgment in humans: An fMRI study. Cerebral Cortex, 23(8), 1952–1964.  https://doi.org/10.1093/cercor/bhs179 CrossRefGoogle Scholar
  123. Takayama, H., Ogawa, N., Yamamoto, M., Asanuma, M., Hirata, H., & Ota, Z. (1992). Age-related changes in cerebrospinal fluid γ-Aminobutyric acid concentration. European Journal of Clinical Chemistry and Clinical Biochemistry, 30(5), 271–274.  https://doi.org/10.1515/cclm.1992.30.5.271 CrossRefGoogle Scholar
  124. Teramoto, W., Honda, K., Furuta, K., & Sekiyama, K. (2017). Visuotactile interaction even in far sagittal space in older adults with decreased gait and balance functions. Experimental Brain Research, 235(8), 2391–2405.  https://doi.org/10.1007/s00221-017-4975-7 CrossRefGoogle Scholar
  125. Tiippana, K., & Salmela, V. R. (2018). Stimulus duration has little effect on auditory, visual and audiovisual temporal order judgement. Experimental Brain Research, 236(5), 1273–1282.  https://doi.org/10.1007/s00221-018-5218-2 CrossRefGoogle Scholar
  126. Tomassini, A., Ambrogioni, L., Medendorp, W. P., & Maris, E. (2017). Theta oscillations locked to intended actions rhythmically modulate perception. eLife, 6, e25618.  https://doi.org/10.7554/eLife.25618 CrossRefGoogle Scholar
  127. Tomassini, A., Spinelli, D., Jacono, M., Sandini, G., & Morrone, M. C. (2015). Rhythmic oscillations of visual contrast sensitivity synchronized with action. Journal of Neuroscience, 35(18), 7019–7029.  https://doi.org/10.1523/JNEUROSCI.4568-14.2015 CrossRefGoogle Scholar
  128. Tsilionis, E., & Vatakis, A. (2016). Multisensory binding: Is the contribution of synchrony and semantic congruency obligatory? Current Opinion in Behavioral Sciences, 8, 7–13.  https://doi.org/10.1016/j.cobeha.2016.01.002 CrossRefGoogle Scholar
  129. Turi, M., Karaminis, T., Pellicano, E., & Burr, D. (2016). No rapid audiovisual recalibration in adults on the autism spectrum. Scientific Reports, 6, 21756.  https://doi.org/10.1038/srep21756 CrossRefGoogle Scholar
  130. Ursino, M., Cuppini, C., & Magosso, E. (2017). Multisensory Bayesian inference depends on synapse maturation during training: Theoretical analysis and neural modeling implementation. Neural Computation, 29(3), 735–782.  https://doi.org/10.1162/NECO_a_00935 CrossRefGoogle Scholar
  131. Van der Burg, E., Alais, D., & Cass, J. (2013). Rapid recalibration to audiovisual asynchrony. Journal of Neuroscience, 33(37), 14633–14637.  https://doi.org/10.1523/JNEUROSCI.1182-13.2013 CrossRefGoogle Scholar
  132. Van der Burg, E., & Goodbourn, P. T. (2015). Rapid, generalized adaptation to asynchronous audiovisual speech. Proceedings of the Royal Society B: Biological Sciences, 282(1804), 20143083.  https://doi.org/10.1098/rspb.2014.3083 CrossRefGoogle Scholar
  133. Van der Burg, E., Orchard-Mills, E., & Alais, D. (2015). Rapid temporal recalibration is unique to audiovisual stimuli. Experimental Brain Research, 233(1), 53–59.  https://doi.org/10.1007/s00221-014-4085-8 CrossRefGoogle Scholar
  134. Van Eijk, R. L., Kohlrausch, A., Juola, J. F., & van de Par, S. (2008). Audiovisual synchrony and temporal order judgments: Effects of experimental method and stimulus type. Perception & Psychophysics, 70(6), 955–968.  https://doi.org/10.3758/PP.70.6.955 CrossRefGoogle Scholar
  135. van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, 102(4), 1181–1186.  https://doi.org/10.1073/pnas.0408949102 CrossRefGoogle Scholar
  136. van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45(3), 598–607.  https://doi.org/10.1016/j.neuropsychologia.2006.01.001 CrossRefGoogle Scholar
  137. VanRullen, R. (2016). Perceptual cycles. Trends in Cognitive Sciences, 20(10), 723–735.  https://doi.org/10.1016/j.tics.2016.07.006 CrossRefGoogle Scholar
  138. Vatakis, A. (2013). The role of stimulus properties and cognitive processes in the quality of the multisensory perception of synchrony. In L. Albertazzi (Ed.), The Wiley-Blackwell handbook of experimental phenomenology: Visual perception of shape, space and appearance (pp. 243–263). Chichester: Wiley-Blackwell.CrossRefGoogle Scholar
  139. Vatakis, A., & Bakou, A. E. (2015). Distorted multisensory experiences of order and simultaneity. In A. Vatakis & M. Allman (Eds.), Time distortions in mind: Temporal processing in clinical populations (pp. 1–36). Leiden: Brill.Google Scholar
  140. Vatakis, A., Navarra, J., Soto-Faraco, S., & Spence, C. (2008). Audiovisual temporal adaptation of speech: Temporal order versus simultaneity judgments. Experimental Brain Research, 185, 521–529.  https://doi.org/10.1007/s00221-007-1168-9 CrossRefGoogle Scholar
  141. Vatakis, A., & Spence, C. (2006). Audiovisual synchrony perception for music, speech, and object actions. Brain Research, 1111(1), 134–142.  https://doi.org/10.1016/j.brainres.2006.05.078 CrossRefGoogle Scholar
  142. Venezia, J. H., Thurman, S. M., Matchin, W., George, S. E., & Hickok, G. (2016). Timing in audiovisual speech perception: A mini review and new psychophysical data. Attention, Perception & Psychophysics, 78(2), 583–601.  https://doi.org/10.3758/s13414-015-1026-y CrossRefGoogle Scholar
  143. Wu, P.-Y., Chen, Y.-C., Yeh, S.-L., Wu, M.-T., & Tang, P.-F. (2015). Multisensory integration ability correlates with spatial working memory and functional mobility in cognitively normal middle-aged and older adults. Physiotherapy, 101(1), e1663–e1664.  https://doi.org/10.1016/j.physio.2015.03.061 CrossRefGoogle Scholar
  144. Wutz, A., Muschter, E., van Koningsbruggen, M. G., Weisz, N., & Melcher, D. (2016). Temporal integration windows in neural processing and perception aligned to saccadic eye movements. Current Biology, 26(13), 1659–1668.  https://doi.org/10.1016/j.cub.2016.04.070 CrossRefGoogle Scholar
  145. Yarrow, K., Shapiro, S., DiCosta, S., & Arnold, D. H. (2014). A 2AFC simultaneity-judgment task to estimate the point of subjective simultaneity. Procedia Social and Behavioral Sciences, 126, 66.  https://doi.org/10.3389/fpsyg.2016.00416 CrossRefGoogle Scholar
  146. Zhou, H. Y., Cai, X. L., Weigl, M., Bang, P., Cheung, E. F. C., & Chan, R. C. K. (2018). Multisensory temporal binding window in autism spectrum disorders and schizophrenia spectrum disorders: A systematic review and meta-analysis. Neuroscience & Biobehavioral Reviews, 86, 66–76.  https://doi.org/10.1016/j.neubiorev.2017.12.013 CrossRefGoogle Scholar
  147. Zopf, R., Friedman, J., & Williams, M. A. (2015). The plausibility of visual information for hand ownership modulates multisensory synchrony perception. Experimental Brain Research, 233(8), 2311–2321.  https://doi.org/10.1007/s00221-015-4300-2 CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Authors and Affiliations

  • Nadia Paraskevoudi
    • 1
    • 2
  • Argiro Vatakis
    • 3
    • 4
  1. 1.Brainlab, Cognitive Neuroscience Research Group, Department of Clinical Psychology and PsychobiologyUniversity of BarcelonaBarcelonaSpain
  2. 2.Institute of Neurosciences, University of BarcelonaBarcelonaSpain
  3. 3.Cognitive Systems Research InstituteAthensGreece
  4. 4.Department of PsychologyPanteion University of Social and Political SciencesAthensGreece

Personalised recommendations