On Cross-Modal Perception of Musical Tempo and the Speed of Human Movement

  • Kathleya Afanador
  • Ellen Campana
  • Todd Ingalls
  • Dilip Swaminathan
  • Harvey Thornburg
  • Jodi James
  • Jessica Mumford
  • Gang Qian
  • StJepan Rajko
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4969)


Studies in crossmodal perception often use very simplified auditory and visual contexts. While these studies have been theoretically valuable, it is sometimes difficult to see how the findings can be ecologically valid or practically valuable. This study hypothesizes that a musical parameter (tempo) may affect the perception of a human movement quality (speed) and finds that although there are clear limitations, this may be a promising first step towards widening both the contexts in which cross-modal effects are studied and the application areas in which the findings can be used.


Dynamic Bayesian Network Music Perception Motion Capture Data McGurk Effect Shape Quality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alais, D., Burr, D.: The ventriloquist effect results from near-optimal bimodal integration. Current Biology 14(3), 257–262 (2004)Google Scholar
  2. 2.
    Arulampalam, S., Maskell, S., Gordon, N., Clapp, T.: Tutorial on particle filters for on-line nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing (2001)Google Scholar
  3. 3.
    Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
  4. 4.
    Bregman, A.S.: Auditory scene analysis. MIT Press, Cambridge (1990)Google Scholar
  5. 5.
    Bolivar, V.J., Cohen, A.J., Fentress, J.C.: Semantic and formal congruency in music and motion pictures: Effects the interpretation of visual action. Psychomusicology 2, 38–43 (1994)Google Scholar
  6. 6.
    Clarke, E.F.: Meaning and the specification of motion in music. Musicae Scientiae 5, 213–234 (2001)Google Scholar
  7. 7.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. John Wiley, Chichester (1999)Google Scholar
  8. 8.
    Eitan, Z., Granot, R.: How music moves. Music Perception 23(3), 221–248 (2006)CrossRefGoogle Scholar
  9. 9.
    Hodgins, P.: Relationships between score and choreography in 20th century music and dance: Music and metaphor. Mellen, London (1992)Google Scholar
  10. 10.
    Iwamiya, S.: Interactions between auditory and visual processing when listening to music in an audiovisual context. Psychomusicology 13, 133–154 (1994)Google Scholar
  11. 11.
    Jaynes, E.T.: Probability Theory: Logic of Science. Cambridge (2003)Google Scholar
  12. 12.
    Krumhansl, C.L., Schenck, D.L.: Can dance reflect the structural and expressive qualities of music?: A perceptual experiment on Balanchine’s choreography of Mozart’s Divertimento No. 5. Musicae Scientae 1(1), 63–85 (1997)Google Scholar
  13. 13.
    Laban, R.V.: The Language of Movement: A Guidebook to Choreutics: Plays, inc. (1974)Google Scholar
  14. 14.
    Laban, R.V.: Principles of dance and movement notation: With 114 basic movement graphs and their explanation. Macdonald & Evans, London (1956)Google Scholar
  15. 15.
    Lipscomb, S.D., Kendall, R.A.: Perceptual judgment of the relationship between musical and visual components in film. Psychomusicology 13, 60–98 (1994)Google Scholar
  16. 16.
    McGurk, H., MacDonald, J.W.: Hearing lips and seeing voices. Nature 264, 746–748 (1976)CrossRefGoogle Scholar
  17. 17.
    McNeill, W.H.: Keeping together in time: dance and drill in human history. Harvard University Press, London (1995)Google Scholar
  18. 18.
    Mitchell, R.W., Gallaher, M.C.: Embodying music: Matching music and dance in memory. Music Perception 19(1), 65–85 (2001)CrossRefGoogle Scholar
  19. 19.
    Moog, H.: The development of musical experience of children in the pre-school age. Psychology of Music 4, 38–45 (1976)CrossRefGoogle Scholar
  20. 20.
    Murphy, K.: Dynamic Bayesian Networks:Representation, Inference and Learning. PhD thesis, University of California, Berkeley (2002)Google Scholar
  21. 21.
    O’Leary, A., Rhodes, G.: Cross-modal effects on visual and auditory object perception. Perception & Psychophysics 35, 565–569 (1984)Google Scholar
  22. 22.
    Papousek, M.: Intuitive parenting: A hidden source of musical stimulation in infancy. In: Deliege, I., Sloboda, J. (eds.) Musical Beginnings: Origins and Development of Musical Competence, pp. 88–112. Oxford University Press, Oxford, New York, and Tokyo (1996)Google Scholar
  23. 23.
    Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann Publishers, San Mateo, Calif. (1988)Google Scholar
  24. 24.
    Phillips-Silver, J., Trainor, L.J.: Hearing what the body feels: Auditory encoding of rhythmic movement. Cognition (in press, 2007)Google Scholar
  25. 25.
    Phillips-Silver, J., Trainor, L.J.: Feeling the beat: movement influences infants rhythm perception. Science 308, 1430 (2005)CrossRefGoogle Scholar
  26. 26.
    Repp, B.H., Penel, A.: Auditory dominance in temporal processing: New evidence from synchronization with simultaneous visual and auditory sequences. Journal of Experimental Psychology: Human Perception and Performance 28(5), 1085–1099 (2002)CrossRefGoogle Scholar
  27. 27.
    Savitzky, A., Golay, M.J.E.: Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry, 1627–1639 (1964)Google Scholar
  28. 28.
    Sekuler, R., Sekuler, A.B., Lau, R.: Motion perception. Nature 385, 308 (1997)CrossRefGoogle Scholar
  29. 29.
    Shams, L., Kamitani, Y., Shimojo, S.: Visual illusion induced by sound. Cognitive Brain Research 14, 147–152 (2002)CrossRefGoogle Scholar
  30. 30.
    Thornburg, H.: Detection and Modeling of Transient Audio Signals with Prior Information, Stanford University, Department of Electrical Engineering (2005)Google Scholar
  31. 31.
    Vroomen, J., de Gelder, B.: Sound enhances visual perception: Cross-modal effects of auditory organization on vision. Journal of experimental psychology: Human perception and performance 26(5), 1583–1590 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Kathleya Afanador
    • 1
  • Ellen Campana
    • 1
  • Todd Ingalls
    • 1
  • Dilip Swaminathan
    • 1
  • Harvey Thornburg
    • 1
  • Jodi James
    • 1
  • Jessica Mumford
    • 1
  • Gang Qian
    • 1
  • StJepan Rajko
    • 1
  1. 1.Arts, Media and Engineering ProgramArizona State UniversityTempeUnited States

Personalised recommendations