A Percussion-Focussed Approach to Preserving Touch-Screen Improvisation

  • Charles MartinEmail author
  • Henry Gardner
Part of the Springer Series on Cultural Computing book series (SSCC)


Musical performances with touch-screen devices can be recorded by capturing a log of touch interactions. This object can serve as an archive or as a basis for other representations of the musical work. This chapter presents a protocol for recording ensemble touch-screen performances and details the processes for generating visualisations, gestural classifications, and graphical scores from these logs. Our experience of using these new representations to study a series of improvised ensemble performances with iPad-based digital musical instruments leads us to conclude that these new-media artefacts allow unique insights into ensemble interactions, comprehensive archiving of improvised performances, and the potential for re-synthesis into new performances and artworks.


Musical Performance Musical Work Musical Instrument Digital Interface Ensemble Interaction Touch Interaction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Apple Inc. (2015) Event handling guide for iOS. Apple Developer Documentation (published online).
  2. Bailey D (1993) Improvisation: its nature and practice in music. Da Capo, CambridgeGoogle Scholar
  3. Blades J (1992) Percussion instruments and their history. The Bold Strummer Ltd., WestportGoogle Scholar
  4. Borgo D (2006) Sync or swarm: musical improvisation and the complex dynamics of group creativity. In: Futatsugi K, Jouannaud JP, Meseguer J (eds) Algebra, meaning, and computation. Lecture notes in computer science, vol 4060. Springer, Berlin/Heidelberg, pp 1–24. doi:10.1007/11780274_1 CrossRefGoogle Scholar
  5. Breiman L (2001) Random forests. Mach Learn 45(1):5–32. doi:10.1023/A:1010933404324 MathSciNetCrossRefzbMATHGoogle Scholar
  6. Burtner M (2011) Syntax of snow for bells and amplified snow. Published digitally by the author. Google Scholar
  7. Cahn WL (2005) Creative music making. Routledge, New YorkGoogle Scholar
  8. Cook G (1997) Teaching percussion. Schirmer Books, New YorkGoogle Scholar
  9. Davies S (1991) The ontology of musical works and the authenticity of their performances. Noûs 25(1):21–41. doi:10.2307/2216091 CrossRefGoogle Scholar
  10. Davies S (2005) Themes in the philosophy of music. Oxford University Press, OxfordGoogle Scholar
  11. Evens A (2005) Sound ideas: music, machines, and experience. Theory out of bounds, vol 27. University of Minnesota Press, MinneapolisGoogle Scholar
  12. Feldman M (1965) The king of Denmark for solo percussionist. Edition Peters, GlendaleGoogle Scholar
  13. Fette I, Melnikov A (2011) The WebSocket protocol. RFC 6455 (Proposed Standard). doi:10.17487/RFC6455
  14. Fowler CB (1967) The museum of music: a history of mechanical instruments. Music Educ J 54(2):45–49. doi:10.2307/3391092 MathSciNetCrossRefGoogle Scholar
  15. Freed A, Schmeder A (2009) Features and future of open sound control version 1.1 for NIME. In: Proceedings of the international conference on new interfaces for musical expression. Carnegie Mellon University, Pittsburgh, pp 116–120. Google Scholar
  16. Hope C, Vickery L (2015) The Decibel scoreplayer – a digital tool for reading graphic notation. In: International conference on technologies for music notation and representation (TENOR 2015). IRCAM, ParisGoogle Scholar
  17. Kaltenbrunner M, Bovermann T, Bencina R, Costanza E (2005) TUIO – a protocol for table based tangible user interfaces. In: Proceedings of the 6th international workshop on gesture in human-computer interaction and simulation. Springer, Ile de BerderGoogle Scholar
  18. Kania A (2006) Making tracks: the ontology of rock music. J Aesthet Art Crit 64(4):401–414. doi:10.1111/j.1540-594X.2006.00219.x CrossRefGoogle Scholar
  19. Kapur A (2005) A history of robotic musical instruments. In: Proceedings of the international computer music conference. International Computer Music Association, San Francisco. Google Scholar
  20. Lazzaro J, Wawrzynek J (2004) An RTP payload format for MIDI. In: 117th Audio engineering society convention. Audio Engineering Society, New YorkGoogle Scholar
  21. MacDonald C (2009) Scoring the work: documenting practice and performance in variable media art. Leonardo 42(1):59–63. doi:10.1162/leon.2009.42.1.59 CrossRefGoogle Scholar
  22. Manovich L (2002) The language of new media. MIT Press, CambridgeGoogle Scholar
  23. Martin C (2012) Creating mobile computer music for percussionists: Snow Music. In: Hitchcock M, Taylor J (eds) Interactive: Australasian computer music conference 2012 conference proceedings. Australasian Computer Music Association, The BasinGoogle Scholar
  24. Martin C (2014) Making improvised music for iPad and percussion with Ensemble Metatone. In: Proceedings of the Australasian computer music conference ’14. Australasian Computer Music Association, Fitzroy.
  25. Martin C, Gardner H (2015) That syncing feeling: networked strategies for enabling ensemble creativity in iPad musicians. In: CreateWorld 2015: A Digital Arts Conference. Griffith University, BrisbaneGoogle Scholar
  26. Martin C, Gardner H, Swift B (2014a) Exploring percussive gesture on iPads with Ensemble Metatone. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ’14). ACM, New York, pp 1025–1028. doi:10.1145/2556288.2557226
  27. Martin C, Gardner H, Swift B (2014b) MetaTravels and MetaLonsdale: iPad apps for percussive improvisation. In: CHI ’14 extended abstracts on human factors in computing systems (CHI EA ’14). ACM, New York, pp 547–550. doi:10.1145/2559206.2574805
  28. Martin C, Gardner H, Swift B (2015) Tracking ensemble performance on touch-screens with gesture classification and transition matrices. In: Berdahl E, Allison J (eds) Proceedings of the international conference on new interfaces for musical expression. Louisiana State University, Baton Rouge, pp 359–364. Google Scholar
  29. Martin C, Hopgood C, Griffiths J, Lam Y (2014) Ensemble Metatone. Digital Audio Album available on Bandcamp.
  30. Mazzola G, Cherlin PB (2009) Flow, gesture, and spaces in free jazz. Computational music science. Springer, BerlinCrossRefGoogle Scholar
  31. Mey TD (1987) Musique de tables for three percussionists. Percussion Music Europe, TienenGoogle Scholar
  32. MIDI Manufacturers Association (1996) The complete MIDI 1.0 detailed specification. MIDI Manufacturers Association, Los AngelesGoogle Scholar
  33. Nunn T (1998) Wisdom of the impulse: on the nature of musical free improvisation. Self-publishedGoogle Scholar
  34. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830MathSciNetzbMATHGoogle Scholar
  35. Pressing J (1988) Improvisation: methods and models. In: Sloboda J (ed) Generative processes in music. Oxford University Press, OxfordGoogle Scholar
  36. Reas C, Fry B (2006) Processing: programming for the media arts. AI Soc 20(4):526–538. doi:10.1007/s00146-006-0050-9 CrossRefGoogle Scholar
  37. Rinehart R (2007) The media art notation system: documenting and preserving digital/media art. Leonardo 40(2):181–187. doi:10.1162/leon.2007.40.2.181 CrossRefGoogle Scholar
  38. Schick S (2006) The Percussionist’s art: same bed, different dreams. University of Rochester Press, RochesterGoogle Scholar
  39. Schmeder A, Freed A, Wessel D (2010) Best practices for open sound control. In: Proceedings of the Linux audio conference, vol 10.
  40. Smith SS, Goldstein T (1998) Inner-views. Perspect New Music 36(2):187–199. doi:10.2307/833528 CrossRefGoogle Scholar
  41. Stenström H (2009) Free ensemble improvisation. ArtMonitor, vol 13. Konstnärliga fakultetskansliet, University of Gothenburg, Gothenburg.
  42. Swift B, Sorensen A, Martin M, Gardner HJ (2014) Coding livecoding. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ’14). ACM, New York, pp 1021–1024. doi:10.1145/2556288.2557049 Google Scholar
  43. Victor B (2014) Seeing spaces. Available on the author’s website.
  44. Waisvisz M (1985) THE HANDS, a set of remote MIDI-controllers. In: Proceedings of the international computer music conference (ICMC ’85). International Computer Music Association, San Francisco, pp 313–318. Google Scholar
  45. Wobbrock JO, Wilson AD, Li Y (2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on user interface software and technology (UIST ’07). ACM, New York, pp 159–168. doi:10.1145/1294211.1294238 CrossRefGoogle Scholar
  46. Xenakis I (1975) Psappha for solo multi-percussionist. Universal Music Publishing – Durand Salabert Eschig, ParisGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Research School of Computer ScienceThe Australian National UniversityCanberraAustralia

Personalised recommendations