Multisensory integration and behavioral stability

Abstract

Information coming from multiple senses, as compared to a single one, typically enhances our performance. The multisensory improvement has been extensively examined in perception studies, as well as in tasks involving a motor response like a simple reaction time. However, how this effect extends to more complex behavior, typically involving the coordination of movements, such as bimanual coordination or walking, is still unclear. A critical element in achieving motor coordination in complex behavior is its stability. Reaching a stable state in the coordination pattern allows to sustain complex behavior over time (e.g., without interruption or negative consequences, like falling). This study focuses on the relation between stability in the coordination of movement patterns, like walking, and multisensory improvement. Participants walk with unimodal and audio-tactile metronomes presented either at their preferred rate or at a slower walking rate, the instruction being to synchronize their steps to the metronomes. Walking at a slower rate makes gait more variable than walking at the preferred rate. Interestingly however, the multimodal stimuli enhance the stability of motor coordination but only in the slower condition. Thus, the reduced stability of the coordination pattern (at a slower gait rate) prompts the sensorimotor system to capitalize on multimodal stimulation. These findings provide evidence of a new link between multisensory improvement and behavioral stability, in the context of ecological sensorimotor task.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Albouy, P., Lévêque, Y., Hyde, K. L., Bouchet, P., Tillmann, B., & Caclin, A. (2015). Boosting pitch encoding with audiovisual interactions in congenital amusia. Neuropsychologia,67, 111–120. https://doi.org/10.1016/j.neuropsychologia.2014.12.006.

    Article  PubMed  Google Scholar 

  2. Ammirante, P., Patel, A. D., & Russo, F. A. (2016). Synchronizing to auditory and tactile metronomes: a test of the auditory-motor enhancement hypothesis. Psychonomic Bulletin and Review. https://doi.org/10.3758/s13423-016-1067-9.

    Article  PubMed  Google Scholar 

  3. Armstrong, A., & Issartel, J. (2014). Sensorimotor synchronization with audio-visual stimuli: limited multisensory integration. Experimental Brain Research,232(11), 3453–3463. https://doi.org/10.1007/s00221-014-4031-9.

    Article  PubMed  Google Scholar 

  4. Aschersleben, G., & Bertelson, P. (2003). Temporal ventriloquism: crossmodal interaction on the time dimension. 2. Evidence from sensorimotor synchronization. International Journal of Psychophysiology,50, 157–163. https://doi.org/10.1016/S0167-8760.

    Article  PubMed  Google Scholar 

  5. Blais, M., Albaret, J.-M., & Tallet, J. (2015). Is there a link between sensorimotor coordination and inter-manual coordination? Differential effects of auditory and/or visual rhythmic stimulations. Experimental Brain Research,233(11), 3261–3269. https://doi.org/10.1007/s00221-015-4394-6.

    Article  PubMed  Google Scholar 

  6. Blais, M., Martin, E., Albaret, J.-M., & Tallet, J. (2014). Preservation of perceptual integration improves temporal stability of bimanual coordination in the elderly: An evidence of age-related brain plasticity. Behavioural Brain Research,275, 34–42. https://doi.org/10.1016/j.bbr.2014.08.043.

    Article  PubMed  Google Scholar 

  7. Bolognini, N., Convento, S., Rossetti, A., & Merabet, L. B. (2013). Multisensory processing after a brain damage: Clues on post-injury crossmodal plasticity from neuropsychology. Neuroscience and Biobehavioral Reviews,37(3), 269–278. https://doi.org/10.1016/j.neubiorev.2012.12.006.

    Article  PubMed  Google Scholar 

  8. Bressler, S., & Kelso, J. (2001). Cortical coordination dynamics. Trends in Cognitive Sciences,5(1), 26–36. https://doi.org/10.1016/S1364-6613(00)01564-3.

    Article  PubMed  Google Scholar 

  9. Bruijn, S. M., Meijer, O. G., Beek, P. J., & Van Dieën, J. H. (2013). Assessing the stability of human locomotion : a review of current measures. Journal of the Royal Society, Interface/the Royal Society. https://doi.org/10.1098/rsif.2012.0999.

    Article  Google Scholar 

  10. Bruijn, S. M., van Dieën, J. H., Meijer, O. G., & Beek, P. J. (2009). Is slow walking more stable? Journal of Biomechanics,42(10), 1506–1512. https://doi.org/10.1016/j.jbiomech.2009.03.047.

    Article  PubMed  Google Scholar 

  11. Caclin, A., Bouchet, P., Djoulah, F., Pirat, E., Pernier, J., & Giard, M. H. (2011). Auditory enhancement of visual perception at threshold depends on visual abilities. Brain Research,1396, 35–44. https://doi.org/10.1016/j.brainres.2011.04.016.

    Article  PubMed  Google Scholar 

  12. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale: Erlbaum Associates.

    Google Scholar 

  13. Colonius, H., & Diederich, A. (2004). Multisensory interaction in saccadic reaction time: a time-window-of-integration model. Journal of Cognitive Neuroscience,16(6), 1000–1009. https://doi.org/10.1162/0898929041502733.

    Article  PubMed  Google Scholar 

  14. Dakos, V., Van Nes, E. H., D’Odorico, P., & Scheffer, M. (2012). Robustness of variance and autocorrelation as indicators of critical slowing down. Ecology,93(2), 264–271.

    Article  Google Scholar 

  15. Dalla Bella, S. (2018). Music and movement: Towards a translational approach. Neurophysiologie Clinique, 48(6), 377–386.

    Article  Google Scholar 

  16. Dingwell, J., & Marin, L. (2006). Kinematic variability and local dynamic stability of upper body motions when walking at different speeds. Journal of Biomechanics,39(3), 444–452. https://doi.org/10.1016/j.jbiomech.2004.12.014.

    Article  PubMed  Google Scholar 

  17. Elliott, M. T., Wing, A. M., & Welchman, A. E. (2010). Multisensory cues improve sensorimotor synchronisation. European Journal of Neuroscience,31(10), 1828–1835. https://doi.org/10.1111/j.1460-9568.2010.07205.x.

    Article  PubMed  Google Scholar 

  18. Elliott, Mark T., Wing, A. M., & Welchman, A. E. (2011). The effect of ageing on multisensory integration for the control of movement timing. Experimental Brain Research,213(2–3), 291–298. https://doi.org/10.1007/s00221-011-2740-x.

    Article  PubMed  Google Scholar 

  19. Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature,415(6870), 429–433. https://doi.org/10.1038/415429a.

    Article  PubMed  Google Scholar 

  20. Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences,8(4), 162–169. https://doi.org/10.1016/j.tics.2004.02.002.

    Article  PubMed  Google Scholar 

  21. Frassinetti, F., Bolognini, N., Bottari, D., Bonora, A., & Làdavas, E. (2005). Audiovisual integration in patients with visual deficit. Journal of Cognitive Neuroscience,17(9), 1442–1452. https://doi.org/10.1162/0898929054985446.

    Article  PubMed  Google Scholar 

  22. Gardiner, C. W. (2003). Handbook of stochastic methods for physics, chemistry and the natural sciences. Berlin: Springer.

    Google Scholar 

  23. Giard, M. H., & Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience,11(5), 473–490. https://doi.org/10.1162/089892999563544.

    Article  PubMed  Google Scholar 

  24. Haken, H., Kelso, J. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological Cybernetics,51(5), 347–356.

    Article  Google Scholar 

  25. Hershenson, M. (1962). Reaction time as a measure of intersensory facilitation. Journal of Experimental Psychology,63(3), 289.

    Article  Google Scholar 

  26. Huys, R., Perdikis, D., & Jirsa, V. K. (2014). Functional architectures and structured flows on manifolds: A dynamical framework for motor behavior. Psychological Review,121(3), 302.

    Article  Google Scholar 

  27. Jordan, K., Challis, J. H., & Newell, K. M. (2007). Walking speed influences on gait cycle variability. Gait and Posture,26(1), 128–134. https://doi.org/10.1016/j.gaitpost.2006.08.010.

    Article  PubMed  Google Scholar 

  28. Kato, M., & Konishi, Y. (2006). Auditory dominance in the error correction process: A synchronized tapping study. Brain Research,1084(1), 115–122. https://doi.org/10.1016/j.brainres.2006.02.019.

    Article  PubMed  Google Scholar 

  29. Kelso, J. A. S. (1995). Dynamic patterns: the self-organization of brain and behavior. Cambridge: MIT Press.

    Google Scholar 

  30. Kelso, J. A. S., Fink, P. W., DeLaplain, C. R., & Carson, R. G. (2001). Haptic information stabilizes and destabilizes coordination dynamics. Proceedings of the Royal Society B: Biological Sciences, 268(1472), 1207–1213. https://doi.org/10.1098/rspb.2001.1620.

    Article  PubMed  Google Scholar 

  31. Lagarde, J., & Kelso, J. A. S. (2006). Binding of movement, sound and touch: multimodal coordination dynamics. Experimental Brain Research,173(4), 673–688. https://doi.org/10.1007/s00221-006-0410-1.

    Article  PubMed  Google Scholar 

  32. Lagarde, J., Zelic, G., & Mottet, D. (2012). Segregated audio-tactile events destabilize the bimanual coordination of distinct rhythms. Experimental Brain Research,219(3), 409–419. https://doi.org/10.1007/s00221-012-3103-y.

    Article  PubMed  Google Scholar 

  33. Large, E. W., & Jones, M. R. (1999). The dynamics of attending: How people track time-varying events. Psychological Review,106(1), 119.

    Article  Google Scholar 

  34. Lovelace, C., Stein, B., & Wallace, M. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cognitive Brain Research,17(2), 447–453.

    Article  Google Scholar 

  35. Meredith, M., & Stein, B. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–662. https://doi.org/10.1152/jn.1986.56.3.640.

    Article  PubMed  Google Scholar 

  36. Meredith, M., Nemitz, J., & Stein, B. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. The Journal of Neuroscience, 7(10), 3215. https://doi.org/10.1523/JNEUROSCI.07-10-03215.1987.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Murray, M. M., Molholm, S., Michel, C. M., Heslenfeld, D. J., Ritter, W., Javitt, D. C., & Foxe, J. J. (2005). Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cerebral Cortex,15(7), 963–974. https://doi.org/10.1093/cercor/bhh197.

    Article  PubMed  Google Scholar 

  38. Nelson, W., Hettinger, L., Cunningham, J., Brickman, B., Haas, M., & McKinley, R. (1998). Effects of localized auditory information on visual target detection performance using a helmet-mounted display. Human Factors,40(3), 452–460.

    Article  Google Scholar 

  39. Repp, B. H. (2005). Sensorimotor synchronization: A review of the tapping literature. Psychonomic Bulletin and Review,12(6), 969–992. https://doi.org/10.3758/BF03206433.

    Article  PubMed  Google Scholar 

  40. Repp, B. H., & Penel, A. (2002). Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. Journal of Experimental Psychology Human Perception and Performance,28(5), 1085–1099. https://doi.org/10.1037/0096-1523.28.5.1085.

    Article  PubMed  Google Scholar 

  41. Repp, B., & Penel, A. (2004). Rhythmic movement is attracted more strongly to auditory than to visual rhythms. Psychological Research Psychologische Forschung. https://doi.org/10.1007/s00426-003-0143-8.

    Article  PubMed  Google Scholar 

  42. Repp, B. H., & Su, Y.-H. (2013). Sensorimotor synchronization: A review of recent research (2006–2012). Psychonomic Bulletin and Review,20(3), 403–452. https://doi.org/10.3758/s13423-012-0371-2.

    Article  PubMed  Google Scholar 

  43. Rowland, B. A., Quessy, S., Stanford, T. R., & Stein, B. E. (2007). Multisensory integration shortens physiological response latencies. Journal of Neuroscience,27(22), 5879–5884. https://doi.org/10.1523/JNEUROSCI.4986-06.2007.

    Article  PubMed  Google Scholar 

  44. Roy, C., Dalla Bella, S., & Lagarde, J. (2017a). To bridge or not to bridge the multisensory time gap: bimanual coordination to sound and touch with temporal lags. Experimental Brain Research,235(1), 135–151. https://doi.org/10.1007/s00221-016-4776-4.

    Article  PubMed  Google Scholar 

  45. Roy, C., Lagarde, J., Dotov, D., & Dalla Bella, S. (2017b). Walking to a multisensory beat. Brain and Cognition, 113, 172–183. https://doi.org/10.1016/j.bandc.2017.02.002.

    Article  PubMed  Google Scholar 

  46. Schöner, G., Haken, H., & Kelso, J. (1986). A stochastic theory of phase transitions in human hand movement. Biological Cybernetics,53(4), 247–257. https://doi.org/10.1007/BF00336995.

    Article  PubMed  Google Scholar 

  47. Schöner, G., & Kelso, J. A. (1988). Dynamic pattern generation in behavioral and neural systems. Science, 239(4847), 1513–1520.

    Article  Google Scholar 

  48. Sejdić, E., Fu, Y., Pak, A., Fairley, J. A., & Chau, T. (2012). The effects of rhythmic sensory cues on the temporal dynamics of human gait. PLoS One. https://doi.org/10.1371/journal.pone.0043104.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Sowiński, J., & Dalla Bella, S. (2013). Poor synchronization to the beat may result from deficient auditory-motor mapping. Neuropsychologia,51(10), 1952–1963. https://doi.org/10.1016/j.neuropsychologia.2013.06.027.

    Article  PubMed  Google Scholar 

  50. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge: MIT Press.

    Google Scholar 

  51. Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nature Reviews Neuroscience,9(4), 255–266. https://doi.org/10.1038/nrn2331.

    Article  PubMed  Google Scholar 

  52. Strogatz, S. H. (2018). Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. Boca Raton: CRC Press.

    Google Scholar 

  53. Terrier, P., & Dériaz, O. (2012). Persistent and anti-persistent pattern in stride-to-stride variability of treadmill walking: influence of rhythmic auditory cueing. Human Movement Science,31(6), 1585–1597. https://doi.org/10.1016/j.humov.2012.05.004.

    Article  PubMed  Google Scholar 

  54. Varlet, M., Marin, L., Issartel, J., Schmidt, R. C., & Bardy, B. G. (2012). Continuity of visual and auditory rhythms influences sensorimotor coordination. PLoS One. https://doi.org/10.1371/journal.pone.0044082.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Vroomen, J., & Keetels, M. (2010). Perception of intersensory synchrony: A tutorial review. Attention, Perception and Psychophysics,72(4), 871–884. https://doi.org/10.3758/APP.72.4.871.

    Article  PubMed  Google Scholar 

  56. Warren, W. H. (2006). The dynamics of perception and action. Psychological Review,113(2), 358.

    Article  Google Scholar 

  57. Wright, R. L., & Elliott, M. T. (2014). Stepping to phase-perturbed metronome cues: multisensory advantage in movement synchrony but not correction. Frontiers in Human Neuroscience,8(September), 1–7. https://doi.org/10.3389/fnhum.2014.00724.

    Article  Google Scholar 

  58. Yamasaki, M., Sasaki, T., Tsuzuki, S., & Torii, M. (1984). Stereotyped pattern of lower limb movement during level and grade walking on treadmill. The Annals of Physiological Anthropology,3(4), 291–296. https://doi.org/10.1248/cpb.37.3229.

    Article  PubMed  Google Scholar 

  59. Zelic, Gregory, Mottet, D., & Lagarde, J. (2012). Behavioral impact of unisensory and multisensory audio-tactile events: Pros and cons for interlimb coordination in juggling. PLoS One,7(2), e32308. https://doi.org/10.1371/journal.pone.0032308.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Zelic, G., Mottet, D., & Lagarde, J. (2016). Perceptuo-motor compatibility governs multisensory integration in bimanual coordination dynamics. Experimental Brain Research,234(2), 463–474. https://doi.org/10.1007/s00221-015-4476-5.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Charlotte Roy.

Ethics declarations

Conflict of interest

The authors Charlotte Roy, Simone Dalla Bella, Simon Pla, and Julien Lagarde declare that they have not conflict of interest.

Ethical approval

The study was in accordance with the ethical standards of the national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Roy, C., Dalla Bella, S., Pla, S. et al. Multisensory integration and behavioral stability. Psychological Research (2019). https://doi.org/10.1007/s00426-019-01273-4

Download citation