Journal on Multimodal User Interfaces

, Volume 4, Issue 3–4, pp 147–156 | Cite as

Multi-modal musical environments for mixed-reality performance

  • Robert Hamilton
  • Juan-Pablo Caceres
  • Chryssie Nanou
  • Chris Platz
Original Paper


This article describes a series of multi-modal networked musical performance environments designed and implemented for concert presentation at the Torino-Milano (MiTo) Festival (Settembre musica, 2009, between 2009 and 2010. Musical works, controlled by motion and gestures generated by in-engine performer avatars will be discussed with specific consideration given to the multi-modal presentation of mixed-reality works, combining both software-based and real-world traditional musical instruments.


Music Virtual environments Mixed-reality Multi-modal 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barbosa A (2003) Displaced soundscapes: a survey of network systems for music and sonic art creation. Leonardo Music J 13:53–59 MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bencina R (2006) Oscpacka simple c++ osc packet manipulation library. URL
  3. 3.
    Bilinghurst M, Kato H (1999) Merging physical world and virtual world. In: Proceedings of the international symposium on mixed reality, Yokohama, Japan. URL Google Scholar
  4. 4.
    Cáceres JP, Chafe C (2009) JackTrip: under the hood of an engine for network audio. In: Proceedings of international computer music conference, International Computer Music Association, San Francisco, CA, pp 509–512 Google Scholar
  5. 5.
    Cáceres JP, Chafe C (2010) JackTrip: under the hood of an engine for network audio. J New Music Res 39(3):183–187. doi: 10.1080/09298215.2010.481361 CrossRefGoogle Scholar
  6. 6.
    Cáceres JP, Hamilton R, Iyer D, Chafe C, Wang G (2008) To the edge with china: explorations in network performance. In: ARTECH 2008: proceedings of the 4th international conference on digital arts, Porto, Portugal, pp 61–66 Google Scholar
  7. 7.
    Chadabe J (1996) Electric sound: the past and promise of electronic music. Prentice Hall, New York Google Scholar
  8. 8.
    Freeman JEA (2005) The architecture of auracle: a voice-controlled, networked sound instrument. In: Proceedings of international computer music conference, Barcelona Google Scholar
  9. 9.
    Furukawa K, Fujihata M, Muench W (2000) Small_fish. URL
  10. 10.
    Gresham-Lancaster S (1998) The aesthetics and history of the Hub: the effect of changing technology on network computer music. Leonardo Music J 8:39–44 CrossRefGoogle Scholar
  11. 11.
    Hamilton R (2008) q3osc: or how i learned to stop worrying and love the game. In: Proceedings of international computer music conference, Belfast, Northern Ireland Google Scholar
  12. 12.
    Hamilton R (2008) Maps and legends: designing fps-based interfaces for multi-user composition, improvisation and interactive performance. In: Computer music modeling and retrieval. Lecture notes in computer science, vol 13, pp 478–486 CrossRefGoogle Scholar
  13. 13.
    id software: Quake 3 (2000) URL
  14. 14.
    Malham D (1995) 3-D sound spatialization using ambisonic techniques. Comput Music J 4(19):58–70 CrossRefGoogle Scholar
  15. 15.
    Marinetti FT (1909) The futurist manifesto Google Scholar
  16. 16.
    Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77-D(12):58–70. URL Google Scholar
  17. 17.
    Mito: Settembre musica (2009) URL
  18. 18.
    Oliver J (2003) q3apd: making music with bots. URL
  19. 19.
    Oliver J, Pickles S (2003) q3apd. URL
  20. 20.
    Oliveros P, Chafe C, Dresser M (2009) Telematic music: six perspectives. Leonardo Music J 13:95–96 Google Scholar
  21. 21.
    Pickles S (2004) Fijuu 2. URL
  22. 22.
    Puckette M (1996) Pure data. In: Proceedings of international computer music conference, San Francisco, pp 269–272 Google Scholar
  23. 23.
    Sirikata (2009) A bsd licensed open source platform for games and virtual worlds. URL
  24. 24.
    SoundWIRE Group (2010) SoundWIRE research group at CCRMA, Stanford University. URL
  25. 25.
    Supercollider (2006) URL
  26. 26.
    Trueman D, Cook P, Smallwood S, Wang G (2006) Plork: Princeton laptop orchestra, year 1. In: Proceedings of international computer music conference Google Scholar
  27. 27.
    Unreal development kit (2010) URL
  28. 28.
    Wang G, Cook PR (2003) ChucK: a concurrent, on-the-fly, audio programming language. In: Proceedings of international computer music conference, Singapore Google Scholar
  29. 29.
    Wang G, Bryan N, Oh J, Hamilton R (2009) Stanford laptop orchestra (slork). In: Proceedings of international computer music conference, Montreal Google Scholar
  30. 30.
    Wright M (2005) Open sound control: an enabling technology for musical networking. Organ Sound 10:193–200 CrossRefGoogle Scholar
  31. 31.
    Xu A, Cooperstock JR (2000) Real-time streaming of multichannel audio data over Internet. In: Proceedings of the 108th convention of the audio engineering society, Paris, pp 627–641 Google Scholar

Copyright information

© OpenInterface Association 2011

Authors and Affiliations

  • Robert Hamilton
    • 1
  • Juan-Pablo Caceres
    • 1
  • Chryssie Nanou
    • 1
  • Chris Platz
    • 1
  1. 1.Center for Computer Research in Music and Acoustics (CCRMA)Stanford UniversityStanfordUSA

Personalised recommendations