Abstract
Music training is associated with better beat processing in the auditory modality. However, it is unknown how rhythmic training that emphasizes visual rhythms, such as dance training, might affect beat processing, nor whether training effects in general are modality specific. Here we examined how music and dance training interacted with modality during audiovisual integration and synchronization to auditory and visual isochronous sequences. In two experiments, musicians, dancers, and controls completed an audiovisual integration task and an audiovisual target-distractor synchronization task using dynamic visual stimuli (a bouncing figure). The groups performed similarly on the audiovisual integration tasks (Experiments 1 and 2). However, in the finger-tapping synchronization task (Experiment 1), musicians were more influenced by auditory distractors when synchronizing to visual sequences, while dancers were more influenced by visual distractors when synchronizing to auditory sequences. When participants synchronized with whole-body movements instead of finger-tapping (Experiment 2), all groups were more influenced by the visual distractor than the auditory distractor. Taken together, this study highlights how training is associated with audiovisual processing, and how different types of visual rhythmic stimuli and different movements alter beat perception and production outcome measures. Implications for the modality appropriateness hypothesis are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adams, W. J. (2016). The development of audio-visual integration for temporal judgements. PLoS Computational Biology, 12(4), e1004865. https://doi.org/10.1371/journal.pcbi.1004865
Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14(3), 257–262. https://doi.org/10.1016/j.cub.2004.01.029
Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15(9), 839–843. https://doi.org/10.1016/j.cub.2005.03.046
Bangert, M., & Schlaug, G. (2006). Specialization of the specialized in features of external human brain morphology. European Journal of Neuroscience, 24, 1832–1834. https://doi.org/10.1111/j.1460-9568.2006.05031.x
Bläsing, B., Calvo-Merino, B., Cross, E. S., Jola, C., Honisch, J., & Stevens, C. J. (2012). Neurocognitive control in dance perception and performance. Acta Psychologica, 139(2), 300–308. https://doi.org/10.1016/j.actpsy.2011.12.005
Bonda, E., Petrides, M., Ostry, D., & Evans, A. (1996). Specific involvement of human parietal systems and the amygdala in the perception of biological motion. Journal of Neuroscience, 16(11), 3737–3744.
Buchanan, J. J., Zihlman, K., Ryu, Y. U., & Wright, D. L. (2007). Learning and transfer of a relative phase pattern and a joint amplitude ratio in a rhythmic multijoint arm movement. Journal of Motor Behavior, 39(1), 49–67. https://doi.org/10.3200/JMBR.39.1.49-67
Burger, B., Thompson, M. R., Luck, G., Saarikallio, S., Toiviainen, P. (2013). Influences of rhythm- and timbre-related musical features on characteristics of music-induced movement. Frontiers in Psychology, 4, https://doi.org/10.3389/fpsyg.2013.00183
Calvo-Merino, B., Glaser, D. E., Grèzes, J., Passingham, R. E., & Haggard, P. (2005). Action observation and acquired skills: an fRMI study with expert dancers. Cerebral Cortex, 15(8), 1243–1249. https://doi.org/10.1093/cercor/bhi007
Chen, Y., Repp, B. H., & Patel, A. D. (2002). Spectral decomposition of variability in synchronization and continuation tapping: comparisons between auditory and visual pacing feedback conditions. Human Movement Science, 21, 515–532. https://doi.org/10.1016/s0167-9457(02)00138-0
Cooper, G., & Meyer, L. B. (1960). The rhythmic structure of music. The University of Chicago Press.
Cross, E. S., de C Hamilton, A. F., & Grafton, S. T. (2006). Building a motor simulation de novo: observation of dance by dancers. Neuroimage, 31(3), 1257–1267. https://doi.org/10.1016/j.neuroimage.2006.01.003
de Boer-Schellekens, L., Eussen, M., & Vroomen, J. (2013). Diminished sensitivity of audiovisual temporal order in autism spectrum disorder. Frontiers in Integrative Neuroscience, 7, 8. https://doi.org/10.3389/fnint.2013.00008
Déry, C., Campbell, N. K. J., Lifshitz, M., & Raz, A. (2014). Suggestion overrides automatic audiovisual integration. Consciousness and Cognition, 24, 33–37. https://doi.org/10.1016/j.concog.2013.12.010
Dixon, N. F., & Spitz, L. (1980). The detection of auditory visual desynchrony. Perception, 9(6), 719–721. https://doi.org/10.1068/p090719
Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001). A cortical area selective for visual processing of the human body. Social Neuroscience, 293(5539), 2470–2473. https://doi.org/10.1126/science.1063414
Downing, P. E., Peelen, M. V., Wiggett, A. J., & Tew, B. D. (2006). The role of the extrastraite body area in action perception. Science, 1(1), 52–62. https://doi.org/10.1080/17470910600668854
Drake, C. (1998). Psychological processes involved in the temporal organization of complex auditory sequences: universal and acquired processes. Music Perception, 16(1), 11–26. https://doi.org/10.2307/40285774
E-Prime (Version 2.0) [Computer software]. Pennsylvania: Psychology Software Tools.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.
Faulkenberry, T. J., Ly, A., & Wagenmakers, E.-J. (2020). Bayesian inference in numerical cognition: a tutorial using JASP. Journal of Numerical Cognition, 6(2), 231–259. https://doi.org/10.5964/jnc.v6i2.288
Fernandes, L. F., & de Barros, R. M. (2012). Grip pattern and finger coordination differences between pianists and non-pianists. Journal of Electromyography and Kinesiology, 22, 412–418. https://doi.org/10.1016/j.jelekin.2012.02.007
Fiedler, A., O’Sullivan, J. L., Schröter, H., Miller, J., & Ulrich, R. (2011). Illusory double flashes can speed up responses like physical ones: evidence from the sound-induced flash illusion. Experimental Brain Research, 214(1), 113. https://doi.org/10.1007/s00221-011-2811-z
Gardner, T., Goulden, N., & Cross, E. S. (2015). Dynamic modulation of the action observation network by movement familiarity. The Journal of neuroscience : the official journal of the Society for Neuroscience, 35(4), 1561–1572. https://doi.org/10.1523/JNEUROSCI.2942-14.2015
Gentilucci, M., & Cattaneo, L. (2005). Automatic audiovisual integration in speech perception. Experimental Brain Research, 167(1), 66–75. https://doi.org/10.1007/s00221-005-0008-z
Grahn, J. A. (2012). See what I hear? Beat perception in auditory and visual rhythms. Experimental Brain Research, 220(1), 51–61. https://doi.org/10.1007/s00221-012-3114-8
Grahn, J. A., Henry, M. J., & McAuley, J. D. (2011). FMRI investigation of cross-modal interactions in beat perception: audition primes vision, but not vice versa. NeuroImage, 54, 1231–1243. https://doi.org/10.1016/j.neuroimage.2010.09.033
Grondin, S., & McAuley, J. D. (2009). Duration discrimination in crossmodal sequences. Perception, 38(10), 1542–1559. https://doi.org/10.1068/p6359
Grossman, E., Donnelly, M., Price, R., Pickens, D., Morgan, V., Neighbor, G., & Blake, R. (2000). Brain areas involve in perception of biological motion. Journal of Cognitive Neuroscience, 12(5), 711–720. https://doi.org/10.1162/089892900562417
Guttman, S. E., Gilroy, L. A., & Blake, R. (2005). Hearing what the eyes see: auditory encoding of visual temporal sequences. Psychological Science, 16(3), 228–235. https://doi.org/10.1111/j.0956-7976.2005.00808.x
Hartcher-O’Brien, J., Di Luca, M., & Ernst, M. O. (2014). The duration of uncertain times: audiovisual information about intervals is integrated in a statistically optimal fashion. PLoS ONE, 9(4), e96134. https://doi.org/10.1371/journal.pone.0089339
Honing, H., Merchant, H., Háden, G. P., Prado, L., & Bartolo, R. (2012). Rhesus monkeys (Macaca mulatta) detect rhythmic groups in music, but not the beat. PLoS ONE, 7(12), e51369. https://doi.org/10.1371/journal.pone.0051369
Hove, M. J., & Keller, P. E. (2010). Spatiotemporal relations and movement trajectories in visuomotor synchronization. Music Perception, 28(1), 15–26. https://doi.org/10.1525/mp.2010.28.1.15
Hove, M. J., Spivey, M. J., & Krumhansl, C. L. (2010). Compatibility of motion facilitates visuomotor synchronization. Journal of Experimental Psychology: Human Perception and Performance, 36(6), 1525–1534. https://doi.org/10.1037/a0019059
Hove, M. J., Iversen, J. R., Zhang, A., & Repp, B. H. (2013). Synchronization with competing visual and auditory rhythms: bouncing ball meets metronome. Psychological Research, 77(4), 388–398. https://doi.org/10.1007/s00426-012-0441-0
Innes-Brown, H., Barutchu, A., Shivdasani, M. N., Crewther, D. P., Grayden, D. B., & Paolini, A. G. (2011). Susceptibility to the flash-beep illusion is increased in children compared to adults. Developmental Science, 14(5), 1089–1099. https://doi.org/10.1111/j.1467-7687.2011.01059.x
Inui, N., & Ichihara, T. (2001). Comparison of the relation between timing and force control during finger-tapping sequences by pianists and non pianists. Motor Control, 5, 385–398. https://doi.org/10.1123/mcj.5.4.385
Iversen, J. R., Patel, A. D., Nicodemus, B., & Emmorey, K. (2015). Synchronization to auditory and visual rhythms in hearing and deaf individuals. Cognition, 134, 232–244. https://doi.org/10.1016/j.cognition.2014.10.018
Jäncke, L., Loose, R., Lutz, K., Specht, K., & Shah, N. J. (2000). Cortical activations during paced finger-tapping applying visual and auditory pacing stimuli. Cognitive Brain Research, 10(1–2), 51–66. https://doi.org/10.1016/S0926-6410(00)00022-7
Karpati, F. J., Giacosa, C., Foster, N. E. V., Penhune, V. B., & Hyde, K. L. (2016). Sensorimotor integration is enhanced in dancers and musicians. Experimental Brain Research, 234, 893–903. https://doi.org/10.1007/s00221-015-4524-1
Kato, M., & Konishi, Y. (2006). Auditory dominance in the error correction process: a synchronized tapping study. Brain Research, 1084(1), 115–122. https://doi.org/10.1016/j.brainres.2006.02.019
Kayser, C., Petkov, C. I., & Logothetis, N. K. (2008). Visual modulation of neurons in auditory cortex. Cerebral Cortex, 18(7), 1560–1574. https://doi.org/10.1093/cercor/bhm187
Keetels, M., & Vroomen, J. (2012). Perception of synchrony. In M. M. Murray & M. T. Wallace (Eds.), The Neural Bases of Multisensory Processes (9th ed.). CRC Press/Taylor & Francis.
Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What’s new in psychtoolbox-3. Perception, 36(14), 1–16.
Körding, K., & Wolpert, D. (2004). Bayesian integration in sensorimotor learning. Nature, 427, 244–247. https://doi.org/10.1038/nature02169
Ladda, A. M., Wallwork, S. B., & Lotze, M. (2020). Multimodal Sensory-Spatial Integration and Retrieval of Trained Motor Patterns for Body Coordination in Musicians and Dancers. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.576120
Lakens, D. (2022). Sample Size Justification. Collabra. Psychology, 8(1), 33267. https://doi.org/10.1525/collabra.33267
Large, E. W., & Palmer, C. (2002). Perceiving temporal regularity in music. Cognitive Science, 26, 1–37. https://doi.org/10.1207/s15516709cog2601_1
Lederman, S. J., & Klatzky, R. L. (2004). Multisensory texture perception. Handbook of multisensory processes, (pp. 107-122)
Lorås, H., Sigmundsson, H., Talcott, J. B., Öhberg, F., & Stensdotter, A. K. (2012). Timing continuous or discontinuous movements across effectors specified by different pacing modalities and intervals. Experimental Brain Research, 220(3–4), 335–347. https://doi.org/10.1007/s00221-012-3142-4
McAuley, J. D., & Henry, M. J. (2010). Modality effects in rhythm processing: auditory encoding of visual rhythms is neither obligatory nor automatic. Attention, Perception, & Psychophysics, 72(5), 1377–1389. https://doi.org/10.3758/app.72.5.1377
Merker, B., Madison, G., & Eckerdal, P. (2009). On the role and origin of isochrony in human rhythmic entrainment. Cortex, 45(1), 4–17. https://doi.org/10.1016/j.cortex.2008.06.011
Neumann, O., & Niepel, M. (2004). Timing of perception and perception of time. In C. Kaernbach, E. Schroger, & H. Müller (Eds.), Psychophysics Beyond Sensation: Laws and Invariants of Human Cognition (pp. 245–269). Lawrence Erlbaum Associates.
Nguyen, T., Sidhu, R. K., Everling, J. C., Wickett, M. C., Gibbings, A., & Grahn, J. A. (2022). Beat Perception and Production in Musicians and Dancers. Music Perception, 39(3), 229–248. https://doi.org/10.1525/mp.2022.39.3.229
Parncutt, R. (1994). A perceptual model of pulse salience and metrical accent in musical rhythms. Music Perception, 11(4), 409–464. https://doi.org/10.2307/40285633
Patel, A. D., Iversen, J. R., Chen, Y., & Repp, B. H. (2005). The influence of metricality and modality on synchronization with a beat. Experimental Brain Research, 163, 226–238. https://doi.org/10.1007/s00221-004-2159-8
Pau, S., Jahn, G., Sakreida, K., Domin, M., & Lotze, M. (2013). Encoding and recall of a finger sequence in experienced pianists compared with musically naïve controls: a combined behavioral and functional imaging study. Neuroimage, 64, 379–387. https://doi.org/10.1016/j.neuroimage.2012.09.012
Pilgramm, S., Lorey, B., Stark, R., Munzert, J., Vaitl, S., & Zentgraf, K. (2010). Differential activation of the lateral premotor cortex during action observation. BMC Neuroscience, 11, 89. https://doi.org/10.1186/1471-2202-11-89
Proverbio, A. M., Calbi, M., Manfredi, M., & Zani, A. (2014). Audio-visuomotor processing in the musician’s brain: an ERP study on professional violinists and clarinetists. Scientific Reports, 4, 5866. https://doi.org/10.1038/srep05866
Repp, B. H. (2003b). Rate limits in sensorimotor synchronization with auditory and visual sequences: the synchronization threshold and the benefits and costs of interval subdivision. Journal of Motor Behavior, 35(4), 355–370. https://doi.org/10.1080/00222890309603156
Repp, B. H. (2004). On the nature of phase attraction in sensorimotor synchronization with interleaved auditory sequences. Human Movement Science, 23(3–4), 389–413.
Repp, B. H. (2005). Sensorimotor synchronization: a review of the tapping literature. Psychonomic Bulletin and Review, 12(6), 969–992. https://doi.org/10.3758/BF03206433
Repp, B. H. (2010). Sensorimotor synchronization and perception of timing: effects of music training and task experience. Human Movement Science, 29(2), 200–213. https://doi.org/10.1016/j.humov.2009.08.002
Repp, B. H., & Penel, A. (2002). Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. Journal of Experimental Psychology: Human Perception and Performance, 28(5), 1085–1099. https://doi.org/10.1037/0096-1523.28.5.1085
Repp, B. H., & Penel, A. (2004). Rhythmic movement is attracted more strongly to auditory than to visual rhythms. Psychological Research, 68(4), 252–270. https://doi.org/10.1007/s00426-003-0143-8
Repp, B. H. (2003a). Phase attraction in sensorimotor synchronization with auditory sequences: effects of single and periodic distractors on synchronization accuracy. Journal of Experimental Psychology, 29(2). https://doi.org/10.1037/0096-1523.29.2.290
Shimada, S. (2010). Deactivation in the sensorimotor area during observation of a human agent performing robotic actions. Brain and Cognition, 72, 394–399. https://doi.org/10.1016/j.bandc.2009.11.005
Sofianidis, G., Hatzitaki, V., Grouios, G., Johannsen, L., & Wing, A. (2012). Somatosensory driven interpersonal synchrony during rhythmic sway. Human Movement Science, 31, 553–566. https://doi.org/10.1016/j.humov.2011.07.007
Su, Y.-H. (2014). Audiovisual beat induction in complex auditory rhythms: point-light figure movement as an effective visual beat. Acta Psychologica, 151, 40–50. https://doi.org/10.1016/j.actpsy.2014.05.016
Su, Y.-H., & Pöppel, E. (2012). Body movement enhances the extraction of temporal structures in auditory sequences. Psychological Research, 76(3), 373–382. https://doi.org/10.1007/s00426-011-0346-3
Thullier, F., & Moufti, H. (2004). Multi-joint coordination in ballet dancers. Neuroscience Letters, 369(1), 80–84. https://doi.org/10.1016/j.neulet.2004.08.011
Tillmann, B. (2008). Music cognition: learning, perception, expectations. In R. Kronland-Martinet, S. Ystad, & K. Jensen (Eds.), Computer Music Modeling and Retrieval (pp. 11–33). Springer.
Vaina, L. M., Solomon, J., Chowdhury, S., Sinha, P., & Belliveau, J. W. (2001). Functional neuroanatomy of biological motion perception in humans. Proceedings of the National Academy of Sciences of the United States of America, 98(20), 11656–11661. https://doi.org/10.1073/pnas.191374198
Vatakis, A., & Spence, C. (2006). Audiovisual synchrony perception for music, speech, and object actions. Brain Research, 1111(1), 134–142. https://doi.org/10.1016/j.brainres.2006.05.078
Verheul, M. H., & Geuze, R. H. (2004). Bimanual coordination and musical experience: the role of intrinsic dynamics and behavioral information. Motor Control, 8, 270–291. https://doi.org/10.1123/mcj.8.3.270
Vogt, S., Buccino, G., Wohlschläger, A. M., Canessa, N., Shah, N. J., Zilles, K., Eickhoff, S. B., Freund, H. J., Rizzolatti, G., & Fink, G. R. (2007). Prefrontal involvement in imitation learning of hand actions: effects of practice and expertise. Neuroimage, 37, 1371–1383. https://doi.org/10.1016/j.neuroimage.2007.07.005
Welch, R., & Warren, D. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638–667. https://doi.org/10.1037/0033-2909.88.3.638
Zampini, M., Guest, S., Shore, D. I., & Spence, C. (2005). Audio-visual simultaneity judgments. Perception & Psychophysics, 67(3), 531–544. https://doi.org/10.3758/BF03193329
Authors’ note
We have no known conflicts of interest to disclose.
Funding
NSERC Discovery Grant to JAG (RGPIN-2016-05834), James S. McDonnell Foundation Understand Human Cognition Scholar Award to JAG (DOI: https://doi.org/10.37717/220020403).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Practices Statement
The study’s design and analyses were not preregistered. Data were collected in 2014 and 2015 and cannot be shared because participants were not asked whether they consented to sharing their data. Study materials can be provided on demand.
Public significance statement
Prior rhythm training in either the music or the dance domains biases attention to either auditory or visual information. During a perceptual task, auditory stimuli were easier to attend to (and harder to ignore) than visual stimuli regardless of music or dance expertise. However, during a synchronization task using finger-tapping, dance training biased movement toward the visual modality and music training biased movement toward the auditory modality.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Nguyen, T., Lagacé-Cusiac, R., Everling, J.C. et al. Audiovisual integration of rhythm in musicians and dancers. Atten Percept Psychophys (2024). https://doi.org/10.3758/s13414-024-02874-x
Accepted:
Published:
DOI: https://doi.org/10.3758/s13414-024-02874-x