Free-Improvised Rehearsal-as-Research for Musical HCI

  • Charles P. MartinEmail author
  • Henry Gardner
Part of the Springer Series on Cultural Computing book series (SSCC)


The formal evaluation of new interfaces for musical expression (NIMEs) in their use by ensembles of musicians is a challenging problem in human-computer interaction (HCI). NIMEs are designed to support creative expressions that are often improvised and unexpected. In the collaborative setting of a musical ensemble, interactions are complex and it can be almost impossible to directly evaluate the impact of interface variations. The evaluation environment also needs to be carefully considered. In the wild, concert pressures and practicalities limit experimental control. In the laboratory, studies may not sufficiently reflect real-world usage to make their conclusions relevant. To address some of these issues, we propose a methodology of rehearsal-as-research to study free improvisation by ensembles of NIME performers. In this methodology, evaluation sessions are structured to mirror established practices for improvisation training and performance development. Such sessions can allow controlled, order-balanced studies with extensive data collection in the style of factorial HCI experiments while preserving the artistic setting of a rehearsal. Experiment design, questionnaires, and objective measures such as session duration will be discussed along with two case studies.



We thank Ben Swift, Michael Martin, and our study participants for assistance with the case studies listed in this chapter. Special thanks to Alec Hunter, the Canberra Experimental Music Studio, and the ANU Schools of Music and Art for support in the artistic goals of this research. This work was partially supported by The Research Council of Norway as a part of the Engineering Predictability with Embodied Cognition (EPEC) project, under grant agreement 240862.


  1. Barbosa J, Malloch J, Wanderley M, Huot S (2015) What does ‘evaluation’ mean for the NIME community? In: Berdahl E, Allison J (eds) Proceedings of the international conference on new interfaces for musical expression. Louisiana State University, Baton Rouge, LA, pp 156–161Google Scholar
  2. Bin SMA, Morreale F, Bryan-Kinns N, McPherson AP (2017) In-the-moment and beyond: Combining post-hoc and real-time data for the study of audience perception of electronic music performance. In: Bernhaupt R, Dalvi G, Joshi A, Balkrishan DK, O’Neill J, Winckler M (eds) Human-computer interaction—INTERACT 2017. Springer International Publishing, Cham, pp 263–281. Scholar
  3. Bradley JV (1958) Complete counterbalancing of immediate sequential effects in a Latin square design. J Am Stat Assoc 53:525–528. Scholar
  4. Cahn WL (2005) Creative music making. Routledge, New York, NYGoogle Scholar
  5. Corbin J, Strauss A (2008) Basics of qualitative research: techniques and procedures for developing grounded theory. SAGE Publications, LondonCrossRefGoogle Scholar
  6. Costello B, Muller L, Amitani S, Edmonds E (2005) Understanding the experience of interactive art: Iamascope in Betaspace. In: Proceedings of the second Australasian conference on interactive entertainment. Creativity & Cognition Studios Press, Sydney, pp 49–56Google Scholar
  7. Csikszentmihalyi M (1991) Flow: the psychology of optimal experience. Harper Collins, New York, NYGoogle Scholar
  8. Eisenberg J, Thompson WF (2003) A matter of taste: Evaluating improvised music. Creat Res J 15:287–296. Scholar
  9. Fiebrink R, Cook PR, Trueman D (2011) Human model evaluation in interactive supervised learning. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, pp 147–156.
  10. Gardner HJ, Martin MA (2007) Analyzing ordinal scales in studies of virtual environments: Likert or lump it! Presence: Teleoper Virtual Environ 16:439–446. Scholar
  11. Gelineck S, Serafin S (2012) Longitudinal evaluation of the integration of digital musical instruments into existing compositional work processes. J New Music Res 41:259–276. Scholar
  12. Gurevich M, Marquez-Borbon A, Stapleton P (2012) Playing with constraints: stylistic variation with a simple electronic instrument. Comput Music J 36:23–41. Scholar
  13. Hayden S, Windsor L (2007) Collaboration and the composer: case studies from the end of the 20th century. Tempo 61:28–39. Scholar
  14. Jordà S, Mealla S (2014) A methodological framework for teaching, evaluating and informing NIME design with a focus on mapping and expressiveness. In: Proceedings of the international conference on new interfaces for musical expression. Goldsmiths, University of London, London, pp 233–238Google Scholar
  15. Krüger S (2008) Ethnography in the performing arts: a student guide. The Higher Education Academy, YorkGoogle Scholar
  16. Lazar J, Feng J, Hochheiser H (2010) Research methods in human-computer interaction. Wiley, West SussexGoogle Scholar
  17. Martin C, Gardner H (2016) A percussion-focussed approach to preserving touch-screen improvisation. In: England D, Schiphorst T, Bryan-Kinns N (eds) Curating the digital: spaces for art and interaction. Springer, Cham, pp 51–72. Scholar
  18. Martin C, Gardner H, Swift B (2014) Exploring percussive gesture on iPads with Ensemble Metatone. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, pp 1025–1028.
  19. Martin C, Gardner H, Swift B, Martin M (2015) Music of 18 performances: evaluating apps and agents with free improvisation. In: Stevenson I (ed) Proceedings of the Australasian computer music conference. Australasian Computer Music Association, Fitzroy, pp 85–94Google Scholar
  20. Martin C, Gardner H, Swift B, Martin M (2016) Intelligent agents and networked buttons improve free-improvised ensemble music-making on touch-screens. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, pp 2295–2306.
  21. Neustaedter C, Sengers P (2012) Autobiographical design in HCI research: Designing and learning through use-it-yourself. In: Proceedings of the designing interactive systems conference. ACM, New York, NY, pp 514–523.
  22. O’Modhrain S (2011) A framework for the evaluation of digital musical instruments. Comput Music J 35:28–42. Scholar
  23. Shneiderman B (2007) Creativity support tools: accelerating discovery and innovation. Commun ACM 50:20–32. Scholar
  24. Skogstad SA, Nymoen K, Høvin M (2011) Comparing inertial and optical MoCap technologies for synthesis control. In: Proceedings of the 8th sound and music computing conference. SMC Network, BarcelonaGoogle Scholar
  25. Smith DT (2009) Development and validation of a rating scale for wind jazz improvisation performance. J Res Music Educ 57:217–235. Scholar
  26. Stowell D, Robertson A, Bryan-Kinns N, Plumbley MD (2009) Evaluation of live human-computer music-making: quantitative and qualitative approaches. Int J Hum Comput Stud 67:960–975. Scholar
  27. Unander-Scharin C, Unander-Scharin Å, Höök K (2014) The Vocal Chorder: empowering opera singers with a large interactive instrument. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, pp 1001–1010.
  28. Wanderley MM, Orio N (2002) Evaluation of input devices for musical expression: borrowing tools from HCI. Comput Music J 26:62–76. Scholar
  29. Williams EJ (1949) Experimental designs balanced for the estimation of residual effects of treatments. Aust J Chem 2:149–168. Scholar
  30. Wobbrock JO, Findlater L, Gergle D, Higgins JJ (2011) The aligned rank transform for nonparametric factorial analyses using only ANOVA procedures. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, NY, pp 143–146.
  31. Wobbrock JO, Kay M (2016) Nonparametric statistics in human-computer interaction. In: Robertson J, Kaptein M (eds) Modern statistical methods for HCI. Springer, Cham, pp 135–170. Scholar
  32. Xambó A, Hornecker E, Marshall P et al (2013) Let’s jam the reactable: peer learning during musical improvisation with a tabletop tangible interface. ACM Trans Comput-Hum Interact 20:36:1–36:34. Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of InformaticsUniversity of OsloOsloNorway
  2. 2.College of Engineering and Computer ScienceThe Australian National UniversityCanberraAustralia

Personalised recommendations