Video Analysis for Evaluating Music Interaction: Musical Tabletops

  • Anna XambóEmail author
  • Robin Laney
  • Chris Dobbyn
  • Sergi Jordà
Part of the Springer Series on Cultural Computing book series (SSCC)


There is little evaluation of musical tabletops for music performance, and current approaches tend to have little consideration of social interaction. However, in collaborative settings, social aspects such as coordination, communication, or musical engagement between collaborators are fundamental for a successful performance. After an overview of the use of video in music interaction research as a convenient method for understanding interaction between people and technology, we present three empirical examples of approaches to video analysis applied to musical tabletops; firstly, an exploratory approach to give informal insight towards understanding collaboration in new situations; secondly, a participatory design approach oriented to improve an interface design by getting feedback from the user experience; thirdly, a quantitative approach, towards understanding collaboration by considering frequencies of interaction events. The aim of this chapter is to provide a useful insight into how to evaluate musical tabletops using video as a data source. Furthermore, this overview can shed light on understanding shareable interfaces in a wider HCI context of group creativity and multi-player interaction.


Video Analysis Participatory Design Music Performance Audiovisual Material Audiovisual Medium 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We wish to thank Eva Hornecker, Paul Marshall and Gerard Roma for lively discussions about several ideas reflected in this chapter. We are also thankful to all the participants of the studies for giving their time and insights. Thanks to Jennifer Ferreira and Minh Tran for their help. And last but not least, thanks to the organisers and participants of the BCS HCI 2011 Workshop for the opportunity to meet and share thoughts with other researchers about this exciting new field of Music Interaction.


  1. Bau, O., Tanaka, A., & Mackay, W. E. (2008). The A20: Musical metaphors for interface design. In Proceedings of NIME’08 (pp. 91–96). Genoa.Google Scholar
  2. Berg, B. L. (2001). Qualitative research methods for the social sciences. Boston: Allyn and Bacon.Google Scholar
  3. Blaine, T., & Perkis, T. (2000). The Jam-O-Drum interactive music system: A study in interaction design. In Proceedings of DIS’00 (pp. 165–173). Brooklyn.Google Scholar
  4. Bødker, S., Grønbæk, K., & Kyng, M. (1993). Cooperative design: Techniques and experiences from the Scandinavian scene. In D. Schuler & A. Namioka (Eds.), Participatory design: Principles and practices. Hillsdale: CRC/Lawrence Erlbaum Associates.Google Scholar
  5. Bryan-Kinns, N., & Hamilton, F. (2012). Identifying mutual engagement. Behaviour and Information Technology, 31(2), 101–125.Google Scholar
  6. Coughlan, T., & Johnson, P. (2006). Interaction in creative tasks: Ideation, representation and evaluation in composition. In Proceedings of CHI’06 (pp. 531–540). Montreal.Google Scholar
  7. Cox, C., & Warner, D. (2004). Audio culture: Readings in modern music. London: International Publishing Group Ltd.Google Scholar
  8. Fiebrink, R., Morris, D., & Morris, M. R. (2009). Dynamic mapping of physical controls for tabletop groupware. In Proceedings of CHI’09 (pp. 471–480). Boston.Google Scholar
  9. Glaser, B. G., & Strauss, A. L. (1967). Discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine Transaction.Google Scholar
  10. Gresham-Lancaster, S. (1998). The aesthetics and history of the hub: The effects of changing technology on network computer music. LMJ, 8, 39–44.Google Scholar
  11. Hagedorn, J., Hailpern, J., & Karahalios, K. G. (2008). VCode and VData: Illustrating a new framework for supporting the video annotation workflow. In: Proceedings of AVI’08 (pp. 317–321). Naples.Google Scholar
  12. Heath, C., Hindmarsh, J., & Luff, P. (2010). Video in qualitative research. London: Sage.Google Scholar
  13. Hornecker, E. (2008). “I don’t understand it either, but it is cool” visitor interactions with a multi-touch table in a museum. In Proceedings of IEEE Tabletop 2008 (pp. 121–128). Amsterdam.Google Scholar
  14. Hornecker, E., Marshall, P., Dalton, N. S., & Rogers, Y. (2008). Collaboration and interference: Awareness with mice or touch input. In Proceedings of CSCW’08 (pp. 167–176). San Diego.Google Scholar
  15. Iwai, T. (1999). Composition on the table. In Proceedings of SIGGRAPH’99 (p. 10). Los Angeles.Google Scholar
  16. Jordà, S. (2008). On stage: The reactable and other musical tangibles go real. International Journal of Arts and Technology, 1(3/4), 268–287.CrossRefGoogle Scholar
  17. Jordà, S., Kaltenbrunner, M., Geiger, G., & Bencina, R. (2005). The reacTable*. In Proceedings of ICMC 2005 (pp. 579–582). Barcelona.Google Scholar
  18. Jordan, B. (1996). Ethnographic workplace studies and computer supported cooperative work. In D. Shapiro, M. Tauber, & R. Traunmüller (Eds.), The design of computer-supported cooperative work and groupware systems. Amsterdam: North Holland/Elsevier Science.Google Scholar
  19. Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the Learning Sciences, 4(1), 39–103.CrossRefGoogle Scholar
  20. Kiefer, C., Collins, N., & Fitzpatrick, G. (2008). HCI methodology for evaluating musical controllers: A case study. In Proceedings of NIME’08 (pp. 87–90). Genoa.Google Scholar
  21. Klügel, N., Friess, M. R., Groh, G., & Echtler, F. (2011). An approach to collaborative music composition. In Proceedings of NIME’11 (pp. 32–35). Oslo.Google Scholar
  22. Laney, R., Dobbyn, C., Xambó, A., Schirosa, M., Miell, D., Littleton, K., & Dalton, S. (2010). Issues and techniques for collaborative music making on multi-touch surfaces. In Proceedings of SMC 2010 (pp. 146–153).Google Scholar
  23. Lazar, J., Feng, J. H., & Hochheiser, H. (2009). Research methods in human-computer interaction. Chichester: Wiley.Google Scholar
  24. MacDougall, D. (2006). The corporeal image: Film, ethnography, and the senses. Princeton: Princeton University Press.Google Scholar
  25. Marshall, P., Fleck, R., Harris, A., Rick, J., Hornecker, E., Rogers, Y., Yuill, N., & Dalton, N. S. (2009). Fighting for control: Children’s embodied interactions when using physical and digital representations. In Proceedings of CHI’09 (pp. 4–9). Boston.Google Scholar
  26. Marshall, P., Morris, R., Rogers, Y., Kreitmayer, S., & Davies, M. (2011). Rethinking ‘multi-user’: An in-the-wild study of how groups approach a walk-up-and-use tabletop interface. In Proceedings of CHI’11 (pp. 3033–3042). Vancouver.Google Scholar
  27. Martin, P., & Bateson, P. (2007). Measuring behaviour: An introductory guide. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  28. Mealla, S., Väljamäe, A., Bosi, M., & Jordà, S. (2011). Listening to your brain: Implicit interaction in collaborative music performances. In Proceedings of NIME’11 (pp. 149–154). Oslo.Google Scholar
  29. Murchison, J. (2010). Ethnography essentials: Designing, conducting, and presenting your research. San Francisco: Wiley.Google Scholar
  30. Patten, J., Recht, B., & Ishii, H. (2002). Audiopad: A tag-based interface for musical performance. In Proceedings of NIME’02 (pp. 1–6).Google Scholar
  31. Pink, S. (2006). The future of visual anthropology: Engaging the senses. London: Routledge.Google Scholar
  32. Poupyrev, I., Lyons, M. J., Fels, S., & Blaine, T. (2001). New interfaces for musical expression. In Proceedings of CHI’01 EA (pp. 491–492).Google Scholar
  33. Rauh, A. (2009). Assessing usability and user experience in Tangible User Interfaces. A case study of the Reactable. Master thesis, Universitat Pompeu Fabra.Google Scholar
  34. Rick, J., Marshall, P., & Yuill, N. (2011). Beyond one-size-fits-all: How interactive tabletops support collaborative learning. In Proceedings of IDC’11 (pp. 109–117). Ann Arbor.Google Scholar
  35. Rogers, Y., Connelly, K., Tedesco, L., Hazlewood, W., Kurtz, A., Hall, R. E., Hursey, J., & Toscos, T. (2007). Why it’s worth the hassle: The value of in-situ studies when designing Ubicomp. In Proceedings of Ubicomp (pp. 336–353). Innsbruck.Google Scholar
  36. Ruby, J. (1980). Franz Boas and early camera study of behaviour. The Kinesis Report, 3(1), 6–11.Google Scholar
  37. Ruby, J. (2000). Picturing culture: Explorations of film and anthropology. Chicago: University of Chicago Press.Google Scholar
  38. Schuler, D., & Namioka, A. (1993). Participatory design: Principles and practices. Hillsdale: CRC/Lawrence Erlbaum Associates.Google Scholar
  39. Stowell, D., Plumbley, M. D., & Bryan-Kinns, N. (2008). Discourse analysis evaluation method for expressive musical interfaces. In Proceedings of NIME’08 (pp. 81–86). Genoa.Google Scholar
  40. Tuddenham, P., Kirk, D., & Izadi, S. (2010). Graspables revisited: Multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks. In Proceedings of CHI’10 (pp.2223–2232). Atlanta.Google Scholar
  41. Ullmer, B., & Ishii, H. (2000). Emerging frameworks for tangible user interfaces. IBM Systems Journal, 39(3–4), 915–931.CrossRefGoogle Scholar
  42. Wanderley, M. M., & Orio, N. (2002). Evaluation of input devices for musical expression: Borrowing tools from HCI. Computer Music Journal, 26(3), 62–76.CrossRefGoogle Scholar
  43. Xambó, A., Laney, R., & Dobbyn, C. (2011). TOUCHtr4ck: Democratic collaborative music. In Proceedings of TEI’11 (pp. 309–312). Funchal.Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Anna Xambó
    • 1
    Email author
  • Robin Laney
    • 1
  • Chris Dobbyn
    • 1
  • Sergi Jordà
    • 2
  1. 1.Music Computing Lab, Centre for Research in ComputingThe Open UniversityMilton KeynesUK
  2. 2.Music Technology GroupUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations