A Statistical Approach to Analyzing Sound Tracings

  • Kristian Nymoen
  • Jim Torresen
  • Rolf Inge Godøy
  • Alexander Refsum Jensenius
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7172)

Abstract

This paper presents an experiment on sound tracing, meaning an experiment on how people relate motion to sound. 38 participants were presented with 18 short sounds, and instructed to move their hands in the air while acting as though the sound was created by their hand motion. The hand motion of the participants was recorded, and has been analyzed using statistical tests, comparing results between different sounds, between different subjects, and between different sound classes. We have identified several relationships between sound and motion which are present in the majority of the subjects. A clear distinction was found in onset acceleration for motion to sounds with an impulsive dynamic envelope compared to non-impulsive sounds. Furthermore, vertical movement has been shown to be related to sound frequency, both in terms of spectral centroid and pitch. Moreover, a significantly higher amount of overall acceleration was observed for non-pitched sounds as compared to pitched sounds.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Publication Manual of the American Psychological Association, 6th edn. American Psychological Association, Washington, DC (2010)Google Scholar
  2. 2.
    Caramiaux, B., Bevilacqua, F., Schnell, N.: Towards a Gesture-Sound Cross-Modal Analysis. In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS, vol. 5934, pp. 158–170. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Caramiaux, B., Bevilacqua, F., Schnell, N.: Sound selection by gestures. In: Proceedings of the International Conference on New Interfaces for Musical Expression, Oslo, Norway, pp. 329–330 (2011)Google Scholar
  4. 4.
    Cox, A.W.: The Metaphoric Logic of Musical Motion and Space. Ph.D. thesis, University of Oregon (1999)Google Scholar
  5. 5.
    Eitan, Z., Granot, R.Y.: How music moves: Musical parameters and listeners’ images of motion. Music Perception 23(3), 221–248 (2006)CrossRefGoogle Scholar
  6. 6.
    Eitan, Z., Timmers, R.: Beethoven’s last piano sonata and those who follow crocodiles: Cross-domain mappings of auditory pitch in a musical context. Cognition 114(3), 405–422 (2010)CrossRefGoogle Scholar
  7. 7.
    Gjerdingen, R., Perrott, D.: Scanning the dial: The rapid recognition of music genres. Journal of New Music Research 37(2), 93–100 (2008)CrossRefGoogle Scholar
  8. 8.
    Glette, K., Jensenius, A.R., Godøy, R.I.: Extracting action-sound features from a sound-tracing study. In: Proceedings of the Second Norwegian Artificial Intelligence Symposium, pp. 63–66 (2010)Google Scholar
  9. 9.
    Godøy, R.I.: Gestural-sonorous objects: Embodied extensions of Schaeffer’s conceptual apparatus. Organised Sound 11(2), 149–157 (2006)CrossRefGoogle Scholar
  10. 10.
    Godøy, R.I.: Chunking Sound for Musical Analysis. In: Ystad, S., Kronland-Martinet, R., Jensen, K. (eds.) CMMR 2008. LNCS, vol. 5493, pp. 67–80. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Godøy, R.I.: Gestural affordances of musical sound. In: Godøy, R.I., Leman, M. (eds.) Musical Gestures: Sound, Movement, and Meaning, ch. 5, pp. 103–125. Routledge (2010)Google Scholar
  12. 12.
    Godøy, R.I., Haga, E., Jensenius, A.R.: Exploring music-related gestures by sound-tracing. A preliminary study. In: Ng, K. (ed.) COST287-ConGAS 2nd Int. Symposium on Gesture Interfaces for Multimedia Systems, Leeds, pp. 27–33 (2006)Google Scholar
  13. 13.
    Godøy, R.I., Jensenius, A.R.: Body movement in music information retrieval. In: Proceedings of the 10th International Society for Music Information Retrieval Conference, Kobe, Japan (2009)Google Scholar
  14. 14.
    Husserl, E.: The Phenomenology of Internal Time Consciousness. Indiana University Press, Bloomington (1964, trans. Churchill, J.S.)Google Scholar
  15. 15.
    Jensenius, A.R.: Action–Sound: Developing Methods and Tools for Studying Music-Related Bodily Movement. Ph.D. thesis, University of Oslo (2007)Google Scholar
  16. 16.
    Jensenius, A.R.: Some video abstraction techniques for displaying body movement in analysis and performance. Leonardo Music Journal (forthcoming, 2012)Google Scholar
  17. 17.
    Kozak, M., Nymoen, K., Godøy, R.I.: The Effects of Spectral Features of Sound on Gesture Type and Timing. In: Efthimiou, E., Kouroupetroglou, G., Fotinea, S.-E. (eds.) GW 2011. LNCS(LNAI), vol. 7206, Springer, Heidelberg (to appear, 2012)Google Scholar
  18. 18.
    Lakoff, G., Johnson, M.: Metaphors We Live By. University of Chicago Press, Chicago (1980)Google Scholar
  19. 19.
    Leman, M.: Embodied Music Cognition and Mediation Technology. MIT Press, Cambridge (2007)Google Scholar
  20. 20.
    Leman, M.: Systematic musicology at the crossroads of modern music research. In: Schneider, A. (ed.) Systematic and Comparative Musicology: Concepts, Methods, Findings, Hamburger Jahrbuch für Musikwissenschaft, vol. 24, pp. 89–115. Peter Lang, Frankfurt (2008)Google Scholar
  21. 21.
    Merer, A., Ystad, S., Kronland-Martinet, R., Aramaki, M.: Semiotics of Sounds Evoking Motions: Categorization and Acoustic Features. In: Kronland-Martinet, R., Ystad, S., Jensen, K. (eds.) CMMR 2007. LNCS, vol. 4969, pp. 139–158. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Miller, G.A.: The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 63, 81–97 (1956)CrossRefGoogle Scholar
  23. 23.
    van Nort, D.: Instrumental listening: sonic gesture as design principle. Organised Sound 14(02), 177–187 (2009)CrossRefGoogle Scholar
  24. 24.
    Nymoen, K., Caramiaux, B., Kozak, M., Torresen, J.: Analyzing sound tracings: a multimodal approach to music information retrieval. In: Proceedings of the 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies, pp. 39–44. ACM, New York (2011)Google Scholar
  25. 25.
    Nymoen, K., Glette, K., Skogstad, S.A., Torresen, J., Jensenius, A.R.: Searching for cross-individual relationships between sound and movement features using an SVM classifier. In: Proceedings of the International Conference on New Interfaces for Musical Expression, Sydney, pp. 259–262 (2010)Google Scholar
  26. 26.
    Schaeffer, P., Reibel, G.: Solfège de l’objet sonore. ORTF, Paris, France (1967, Reedited by INA-GRM 1998)Google Scholar
  27. 27.
    Walker, B.N., Kramer, G.: Mappings and metaphors in auditory displays: An experimental assessment. ACM Trans. Appl. Percept. 2(4), 407–412 (2005)CrossRefGoogle Scholar
  28. 28.
    Wilson, M.: Six views of embodied cognition. Psychonomic Bulletin & Review 9(4), 625–636 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Kristian Nymoen
    • 1
  • Jim Torresen
    • 1
  • Rolf Inge Godøy
    • 2
  • Alexander Refsum Jensenius
    • 2
  1. 1.Department of InformaticsUniversity of OsloOsloNorway
  2. 2.Department of MusicologyUniversity of OsloOsloNorway

Personalised recommendations