Between Mapping, Sonification and Composition: Responsive Audio Environments in Live Performance

  • Christopher L. Salter
  • Marije A. J. Baalman
  • Daniel Moody-Grigsby
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4969)

Abstract

This paper describes recent work on a large-scale, interactive dance theater performance entitled Schwelle as a platform to pose critical questions around the conception, design and implementation of what is commonly labeled responsive audio environments. The authors first discuss some principal issues in the design of responsive audio environments specifically within the domain of stage performance, addressing existing human-computer interaction paradigms in three key areas: sensing, mapping and data sonification. Next, we explore larger questions of composition in relation to these key areas, suggesting that potential strategies cross three different domains: mapping within algorithmic composition, data sonification techniques, and time-based evolutionary processes emerging from dynamical systems theory. We then examine recent work on Schwelle, which employs real time, distributed sensor data to drive a continuous dynamical system-based composition engine. The project’s conceptual and technical challenges are discussed as well as audience evaluation and feedback from the first presentation in Berlin in February 2007, and the subsequent revisions for the second presentation in Montréal in May 2007. Finally, the paper concludes with a set of issues that may act as a framework for future research focused on compositional strategies for larger scale, distributed, network-based sensor environments.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    d’Inverno, M., Eacott, J., Lorstad, H., Oloffson, F.: The intelligent street, responsive sound environments for social interaction. In: ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, ACE 2004. ACM, New York (2004)Google Scholar
  2. 2.
    Miranda, E.R., McAlpine, K., Hoggar, S.: Dynamical systems and applications to music composition: A research report. In: Proceedings of Journees d’Informatique Musicale (JIM97). French Society for Musical Informatics (SFIM), Lyon, France (1997)Google Scholar
  3. 3.
    Son-o-house (2000-2004), http://www.noxarch.com
  4. 4.
    Mynatt, E., Back, M., Want, R., Baer, M., Ellis, J.: Designing audio aura. In: ACM SIGCHI Conference on Human Factors in Computing Systems. ACM, New York (1998)Google Scholar
  5. 5.
    Livingstone, D., Miranda, E.: Composition for ubiquitous responsive environments. In: Proceedings of International Computer Music Conference (ICMC 2004). International Computer Music Association - ICMA, Miami, USA (2004)Google Scholar
  6. 6.
    Kruger, M.: Responsive environments. In: Proceedings of the American Federation of Information Processing Societies, USA. AFIPS, vol. 46, pp. 423–429 (1977)Google Scholar
  7. 7.
  8. 8.
    Roads, C.: Microsound. MIT Press, Cambridge (2001)Google Scholar
  9. 9.
    Orio, N., Schnell, N., Wanderley, M.M.: Input devices for musical expression: Borrowing tools from hci. In: Workshop (NIME-01) during ACM CHI 2001, Seattle, USA (April 2001)Google Scholar
  10. 10.
    Gurevich, M., Verplank, B., Wilson, S.: Physical interaction design for music. In: Proceedings of International Computer Music Conference (ICMC 2003), Singapore (2003)Google Scholar
  11. 11.
    D’Arcangelo, G.: Creating a context for musical innovation: A nime curriculum. In: Proceedings of the 2002 Conference on New Instruments for Musical Expression (NIME 2002), Dublin, Ireland (2002)Google Scholar
  12. 12.
    Tanaka, A., Toeplitz, K.: Global string (1998), http://www.sensorband.com/atau/globalstring/
  13. 13.
    Schiemer, G., Havryliv, M.: Pocket gamelan: Tuneable trajectories for flying sources. In: Proceedings of the 2006 Conference on New Instruments for Musical Expression (NIME 2006), Paris, France (2006)Google Scholar
  14. 14.
    Miranda, E.R., Wanderley, M.M.: New Digital Musical Instruments: Control and Interaction Beyond the Keyboard. A-R Editions (2006)Google Scholar
  15. 15.
    Hunt, A., Wanderley, M.M.: Mapping performer parameters to synthesis engines. In: Organised Sound, pp. 97–108. Cambridge University Press, Cambridge (2002)Google Scholar
  16. 16.
    Hunt, A., Wanderley, M.M., Paradiso, M.: The importance of mapping in musical instrument design. Journal of New Music Research, 429–440 (2003)Google Scholar
  17. 17.
    Hunt, A., Kirk, R.: Mapping Strategies for Musical Performance. In: Trends in Gestural Control of Music. IRCAM-Centre Pompidou (2000)Google Scholar
  18. 18.
    Doornbusch, P.: Composers views on mapping in algorithmic composition. In: Organised Sound, pp. 145–156. Cambridge University Press, Cambridge (2002)Google Scholar
  19. 19.
    Chadabe, J.: The limitations of mapping as a structural descriptive in electronic musical instruments. In: Proceedings of the 2002 Conference on New Instruments for Musical Expression (NIME 2002), Dublin, Ireland (2002)Google Scholar
  20. 20.
    Downie, M.: Choreographing the Extended Agent: performance graphics for dance theater. PhD thesis, Massachusetts Institute of Technology (MIT), Media Laboratory (2005)Google Scholar
  21. 21.
    Varela, F.J., Thompson, E., Rosch, E.: The Embodied Mind: cognitive science and human experience. MIT Press, Cambridge (1991)Google Scholar
  22. 22.
    Scaletti, C.: Sound synthesis algorithms for auditory data representations. In: Kramer, G. (ed.) Auditory Display: Sonification, Audification, and Auditory Interfaces. Addison-Wesley, Reading (1994)Google Scholar
  23. 23.
    Campo, A.D.: Toward a sonification design space map. In: Proceedings of the ICAD 07-13th International Conference on Auditory Display, Montreal, Canada, June 26-29 (2007)Google Scholar
  24. 24.
    Bargar, R.: Pattern and Reference in Auditory Display. In: Kramer, G. (ed.) Auditory Display: Sonification, Audification, and Auditory Interfaces. Addison-Wesley, Reading (1994)Google Scholar
  25. 25.
    Gaver, W.: Using and Creating Auditory Icons. In: Kramer, G. (ed.) Auditory Display: Sonification, Audification, and Auditory Interfaces. Addison-Wesley, Reading (1994)Google Scholar
  26. 26.
    Polansky, L.: Manifestation and sonification (2002), http://eamusic.dartmouth.edu/~larry/sonification.html
  27. 27.
    Paine, G.: Reeds - a responsive sound installation. In: Proceedings of the ICAD 04-10th Meeting of the International Conference on Auditory Display, Sydney, Australia (2004)Google Scholar
  28. 28.
    de la Hunta, S.: Invisibility/corporeality (March 13, 2001), http://www.noemalab.org/sections/ideas/ideas_articles/delahunta.html/nettime.org
  29. 29.
  30. 30.
    Ryan, J., Salter, C.L.: Tgarden: Wearable instruments and augmented physicality. In: Proceedings of the 2003 Conference on New Instruments for Musical Expression (NIME 2003), Montreal, CA (2003)Google Scholar
  31. 31.
    Baalman, M.A., Moody-Grigsby, D., Salter, C.L.: Schwelle: Sensor augmented, adaptive sound design for live theater performance. In: Proceedings of NIME 2007 New Interfaces for Musical Expression, New York, NY, USA (2007)Google Scholar
  32. 32.
  33. 33.
    Franklin, G.F., Powell, J.D., Emami-Naeini, A.: Feedback Control of Dynamic Systems, 3rd edn. Addison-Wesley Publishing Company, Reading (1994)Google Scholar
  34. 34.
    Herbart, J.F.: De Attentionis Mensura causisque primariis (orig. published 1822). In: Kleinere Abhandlungen. E.J. Bonset, Amsterdam (1969)Google Scholar
  35. 35.
    Herbart, J.F.: Psychologische Bemerkungen zur Tonlehre (orig. published 1811). In: Kleinere Abhandlungen, E.J. Bonset, Amsterdam (1969)Google Scholar
  36. 36.
    Herbart, J.F.: Psychologische Untersuchungen. Erstes Heft’ (orig. published 1839). In: Kleinere Abhandlungen, E.J. Bonset, Amsterdam (1969)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Christopher L. Salter
    • 1
    • 2
  • Marije A. J. Baalman
    • 1
    • 3
  • Daniel Moody-Grigsby
    • 1
  1. 1.Design and Computation ArtsConcordia UniversityMontréalCanada
  2. 2.Interactive Performance and Sound, Hexagram InstituteMontréalCanada
  3. 3.Institute for Audio CommunicationTechnische Universität BerlinBerlinGermany

Personalised recommendations