User Experience in an Interactive Music Virtual Reality System: An Exploratory Study

  • Thomas Deacon
  • Tony Stockman
  • Mathieu Barthet
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10525)


The Objects VR interface and study explores interactive music and virtual reality, focusing on user experience, understanding of musical functionality, and interaction issues. Our system offers spatio-temporal music interaction using 3D geometric shapes and their designed relationships. Control is provided by tracking of the hands, and the experience is rendered across a head-mounted display with binaural sound presented over headphones. The evaluation of the system uses a mixed methods approach based on semi-structured interviews, surveys and video-based interaction analysis. On average the system was positively received in terms of interview self-report, metrics for spatial presence and creative support. Interaction analysis and interview thematic analysis also revealed instances of frustration with interaction and levels of confusion with system functionality. Our results allow reflection on design criteria and discussion of implications for facilitating music engagement in virtual reality. Finally our work discusses the effectiveness of measures with respect to future evaluation of novel interactive music systems in virtual reality.


Creativity support Design research Flow Interaction analysis Interactive music systems Thematic analysis User experience Virtual reality 



We would like to give thanks to Stuart Cupit and the development team at Inition for guiding the initial technical development of the interface. This project was funded by the EPSRC and AHRC Centre for Doctoral Training in Media and Arts Technology (EP/L01632X/1), and the EU H2020 research and innovation project Audio Commons (688382).


  1. 1.
    Adams, A.T., Gonzalez, B., Latulipe, C.: SonicExplorer: fluid exploration of audio parameters. In: CHI, pp. 237–246 (2014)Google Scholar
  2. 2.
    Andersen, K., Gibson, D.: The instrument as the source of new in new music. In: Research Through Design, pp. 25–27 (2015)Google Scholar
  3. 3.
    Andolina, S., Klouche, K., Cabral, D., Ruotsalo, T., Jacucci, G.: InspirationWall: supporting idea generation through automatic information exploration. In: Proceedings of the C&C, pp. 8–11 (2015)Google Scholar
  4. 4.
    Barthet, M., Thalmann, F., Fazekas, G., Sandler, M., Wiggins, G.: Crossroads: interactive music systems transforming performance, production and listening. In: Proceedings of the CHI Workshop on Music and HCI (2016)Google Scholar
  5. 5.
    Bengler, B., Bryan-Kinns, N.: Designing collaborative musical experiences for broad audiences. In: Proceedings of the C&C, p. 234. ACM Press, New York (2013)Google Scholar
  6. 6.
    Berthaut, F., Desainte-Catherine, M., Hachet, M.: DRILE: an immersive environment for hierarchical live-looping. In: Proceedings of the NIME, pp. 192–197 (2010)Google Scholar
  7. 7.
    Blaine, T., Fels, S.: Collaborative musical experiences for novices. J. New Music Res. (2010)Google Scholar
  8. 8.
    Bowman, D.A., Johnson, D.B., Hodges, L.F.: Testbed evaluation of virtual environment interaction techniques. Presence: Teleoperators Virtual Environ. 10(1), 75–95 (2001)CrossRefGoogle Scholar
  9. 9.
    Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006)CrossRefGoogle Scholar
  10. 10.
    Byrne, D.: How Music Works. Canongate Books, Edinburgh (2012)Google Scholar
  11. 11.
    Cherry, E., Latulipe, C.: Quantifying the creativity support of digital tools through the creativity support index. ACM TOCHI 21(4), 1–25 (2014)CrossRefGoogle Scholar
  12. 12.
    Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience, 1st edn. Harper & Row, New York (1990)Google Scholar
  13. 13.
    Elblaus, L., Hansen, K.F., Unander-Scharin, C.: Artistically directed prototyping in development and in practice. J. New Music Res. 41(4, SI), 377–387 (2012)CrossRefGoogle Scholar
  14. 14.
    Forlizzi, J., Battarbee, K.: Understanding experience in interactive systems. In: Proceedings of the DIS, pp. 261–268 (2004)Google Scholar
  15. 15.
    Gelineck, S.: Virtual reality instruments capable of changing dimensions in real-time. In: Proceedings of the ENACTIVE (2005)Google Scholar
  16. 16.
    Godøy, R.I.: Gestural-sonorous objects: embodied extensions of Schaeffer’s conceptual apparatus. Organ. Sound 11(02), 149 (2006)CrossRefGoogle Scholar
  17. 17.
    Hamilton, R.: Mediated musical interactions in virtual environments. In: CHI Workshop on Music and HCI (2016)Google Scholar
  18. 18.
    Hartmann, T., Wirth, W., Schramm, H., Klimmt, C., Vorderer, P., Gysbers, A., Böcking, S., Ravaja, N., Laarni, J., Saari, T., et al.: The spatial presence experience scale. J. Media Psychol. (2015)Google Scholar
  19. 19.
    Heath, C.C., Luff, P.: Collaboration and control: crisis management and multimedia technology. CSCW 1(1–2), 69–94 (1990, 1992)Google Scholar
  20. 20.
    Houde, S., Hill, C.: What do prototypes prototype. In: Handbook of Human-computer Interaction, 2nd edn., pp. 367–381 (1997)Google Scholar
  21. 21.
    Jacob, M., Magerko, B.: Interaction-based authoring for scalable co-creative agents. In: Proceedings of the ICCC, pp. 236–243 (2015)Google Scholar
  22. 22.
    Johnston, A.: Beyond evaluation: linking practice and theory in new musical interface design. In: Proceedings of the NIME (2011)Google Scholar
  23. 23.
    Jordan, B., Henderson, A.: Interaction analysis: foundations and practice. J. Learn. Sci. 4(1), 39–103 (1995)CrossRefGoogle Scholar
  24. 24.
    Mulder, A., Fels, S.S., Mase, K.: Mapping virtual object manipulation to sound variation. IPSJ Sig Notes 97(122), 63–68 (1997)Google Scholar
  25. 25.
    O’hara, K., Harper, R., Mentis, H., Sellen, A., Taylor, A.: On the naturalness of touchless: putting the interaction back into NUI. ACM Trans. Comput.-Hum. Interact. 20(1), 1–25 (2013)CrossRefGoogle Scholar
  26. 26.
    Rodet, X., Lambert, J.P., Gaudy, T., Gosselin, F.: Study of haptic and visual interaction for sound and music control in the phase project. In: Proceedings of the NIME, pp. 109–114 (2005)Google Scholar
  27. 27.
    Ryan, W., Street, E., Siegel, M.A.: Evaluating interactive entertainment using breakdown: understanding embodied learning in video games. In: Proceedings of the DiGRA (2009)Google Scholar
  28. 28.
    Salisbury, J.H., Tomlinson, P.: Reconciling Csikszentmihalyi’s broader flow theory: with meaning and value in digital games. ToDIGRA 2(2), 55–77 (2016)Google Scholar
  29. 29.
    Schubert, T.W.: A new conception of spatial presence: once again, with feeling. Commun. Theory 19, 161–187 (2009)CrossRefGoogle Scholar
  30. 30.
    Sèdes, A., Guillot, P., Paris, E., Anne, S., Pierre, G., Eliott, P.: The HOA library, review and prospects. In: Proceedings of the ICMC, vol. 2014, pp. 855–860 (2014)Google Scholar
  31. 31.
    Seldess, Z.: MIAP: manifold-interface amplitude panning in Max/MSP and pure data. In: Proceedings of the AES, pp. 1–10 (2011)Google Scholar
  32. 32.
    Slater, M., Usoh, M.: Body centred interaction in immersive virtual environments. Artif. Life Virtual Real. 1, 1–22 (1994)Google Scholar
  33. 33.
    Slepian, M.L., Ambady, N.: Fluid movement and creativity. J. Exp. Psychol. 141(4), 625–629 (2012)CrossRefGoogle Scholar
  34. 34.
    Stowell, D., Robertson, A., Bryan-Kinns, N., Plumbley, M.: Evaluation of live human-computer music-making: quantitative and qualitative approaches. Int. J. Hum.-Comput Stud. 67(11), 960–975 (2009)CrossRefGoogle Scholar
  35. 35.
    Sutcliffe, A., Gault, B.: Heuristic evaluation of virtual reality applications. Interact. Comput. 16(4), 831–849 (2004)CrossRefGoogle Scholar
  36. 36.
    Vinayak, R.D., Piya, C., Ramani, K.: MobiSweep: exploring spatial design ideation using a smartphone as a hand-held reference plane. In: Proceedings of the TEI, pp. 12–20 (2016)Google Scholar
  37. 37.
    Wirth, W., Hartmann, T., Böcking, S., Vorderer, P., Klimmt, C., Schramm, H., Saari, T., Laarni, J., Ravaja, N., Gouveia, F.R., Biocca, F., Sacau, A., Jäncke, L., Baumgartner, T., Jäncke, P.: A process model of the formation of spatial presence experiences. Media Psychol. 9, 493–525 (2007)CrossRefGoogle Scholar
  38. 38.
    Wozniewski, M., Settel, Z., Cooperstock, J.: A spatial interface for audio and music production. In: Proceedings of the DAFX, pp. 18–21 (2006)Google Scholar
  39. 39.
    Zimmerman, J., Stolterman, E., Forlizzi, J.: An analysis and critique of research through design: towards a formalization of a research approach. In: Proceedings of the DIS, pp. 310–319 (2010)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Electronic Engineering and Computer Science, Centre for Digital Music (Room Eng 111)Queen Mary University of LondonLondonUK

Personalised recommendations