Journal on Multimodal User Interfaces

, Volume 11, Issue 1, pp 25–38 | Cite as

Gamified music improvisation with BilliArT: a multimodal installation with balls

  • T. Vets
  • L. Nijs
  • M. Lesaffre
  • B. Moens
  • F. Bressan
  • P. Colpaert
  • P. Lambert
  • R. Van de Walle
  • M. Leman
Original Paper


This paper presents the concept and the realisation of an interactive multimedia installation, called BilliArT, together with an explorative user study conducted on the data gathered during a public exhibition of the installation. The study concerns functional properties of the installation (e.g. usability, design quality) and subjective qualities of the sonic and visual feedback of the installation. The installation consists in a collaborative environment based on the carambole billiards game, which allows the users to engage in a user-driven machine-based jazz-inspired music improvisation, augmented with visual feedback. The installation is designed to promote the interaction among the users and the billiard game, stimulating the motivation to engage in the game by balancing predictable and unpredictable output, and reinforcing the feeling of reward, irrespective of their level of musical training. BilliArT introduces a new framework for expressive interaction related to the concepts of motivation and reward. The exploratory study proved the ability of the installation to activate the users’ sense of aesthetic reward, leading to a more active and satisfactory engagement in the game. Future studies may exploit these results to the advantage of the world of the arts, as well as of studies in human-computer interaction, interface design, and cultural heritage preservation.


Installation Art Music Augmented reality Machine improvisation Gaming Interaction 


  1. 1.
    GPU gems—chapter 38 (2015) Fast fluid dynamics simulation on the GPU. Accessed 6 Jan 2016
  2. 2.
    Adelson E, Bergen J (1991) Computational models of visual processing, chap. The Plenoptic Function and the Elements of Early Vision. MIT Press, Cambridge, pp 3–20Google Scholar
  3. 3.
    Alves R, Sousa L, Rodrigues J (2013) Poolliveaid: augmented reality pool table to assist inexperienced players. In: Baranoski G, Skala V (eds) Proceedings of the 21st international conference on computer graphics, Visualization and Computer Vision, Plzen, Czech Republic, pp 184–193Google Scholar
  4. 4.
    Biles J (1994) Genjam: a genetic algorithm for generating jazz solos. In: Proceedings of the international computer music conference. International Computer Music Association, San Francisco, pp 131–137Google Scholar
  5. 5.
    Blaine T (2005) The convergence of alternate controllers and musical interfaces in interactive entertainment. In: Proceedings of the 2005 conference on New interfaces for musical expression. National University of Singapore, Singapore, pp 27–33Google Scholar
  6. 6.
    Blaine T, Fels S (2003) Collaborative musical experiences for novices. J New Music Res 32(4):411–428CrossRefGoogle Scholar
  7. 7.
    Blinn JF (1982) A generalization of algebraic surface drawing. ACM Trans Graphics 1(3):235–256CrossRefGoogle Scholar
  8. 8.
    Brattico E, Brattico P, Jacobsen T (2009) The origins of the aesthetic enjoyment of music–a review of the literature. Music Sci 13(2 suppl):15–39Google Scholar
  9. 9.
    Brent W (2011) DILib: control data parsing for digital musical instrument design. In: Proceedings of the 4th international pure data convention. Weimar, pp 176–180Google Scholar
  10. 10.
    Bressan F, Canazza S (2014) The challenge of preserving Interactive Sound Art: A multi-level approach. Int J Arts Technol 7(4):294–315CrossRefGoogle Scholar
  11. 11.
    Collins N (2003) Generative music and laptop performance. Contemp Music Rev 22(4):67–79CrossRefGoogle Scholar
  12. 12.
    Csikszentmihalyi M (1990) Flow. The psychology of optimal experience. Harper & Row, New YorkGoogle Scholar
  13. 13.
    Edmonds E, Muller L, Connell M (2006) On creative engagement. Visual Commun 5(3):307–322CrossRefGoogle Scholar
  14. 14.
    Hedges SA (1978) Dice music in the eighteenth century. Music & Letters 59(2):180–187Google Scholar
  15. 15.
    Hudak P, Berger J (1995) A model of performance, interaction, and improvisation. In: Proceedings of international computer music conference. International Computer Music Association, San Francisco, pp 1–8Google Scholar
  16. 16.
    Kaplan F, Oudeyer PY (2007) In search of the neural circuits of intrinsic motivation. Front Neurosci 1(1):225CrossRefGoogle Scholar
  17. 17.
    Kato J, Nakashima T (2013) Openpool: community-based prototyping of digitally-augmented billiard table. In: GCCE ’13: Proceedings of the 2nd IEEE global conference on consumer electronics. IEEE, Tokyo, pp 175–176Google Scholar
  18. 18.
    Klügel N, Frieß MR, Groh G, Echtler F (2011) An approach to collaborative music composition. In: Proceedings of the international conference on new interfaces for musical expression, Oslo, pp 32–35Google Scholar
  19. 19.
    Leman M (2008) Embodied music cognition and mediation technology. MIT Press, CambridgeGoogle Scholar
  20. 20.
    Leman M (2016) The expressive moment. MIT Press, CambridgeGoogle Scholar
  21. 21.
    Leman M, Demey M, Lesaffre M, van Noorden L, Moelants D (2009) Concepts, technology, and assessment of the social music game “sync-in-team”. In: International conference on computational science and engineering. CSE’09, vol 4. IEEE Computer Society, Washington, pp 837–842Google Scholar
  22. 22.
    Maes PJ, Nijs L, Leman M (2015) A conceptual framework for music-based interaction systems. In: Bader R (ed) Springer handbook in systematic musicology. Springer, Berlin, HeidelbergGoogle Scholar
  23. 23.
    Maes PJ, Leman M, Palmer C, Wanderley M (2014) Action-based effects on music perception. Front Pshychol 4(1008) (2014). doi: 10.3389/fpsyg.2013.01008
  24. 24.
    Martens R, Gulikers J, Bastiaens T (2004) The impact of intrinsic motivation on e-learning in authentic computer tasks. J Comput Assist Learn 20(5):368–376CrossRefGoogle Scholar
  25. 25.
    Martin DC (1989) Retroreflective sheet material and method of making same. US Patent 4,801,193Google Scholar
  26. 26.
    Nijs L, Coussement P, Moens B, Amelinck D, Lesaffre M, Leman M (2012) Interacting with the music paint machine: relating the constructs of flow experience and presence. Interact Comput 24(4):237–250CrossRefGoogle Scholar
  27. 27.
    Papadopoulos G, Wiggins G (1999) Ai methods for algorithmic composition: a survey, a critical view and future prospects. In: AISB symposium on musical creativity. Edinburgh, pp 110–117Google Scholar
  28. 28.
    Pease A, Winterstein D, Colton S (2001) Evaluating machine creativity. In: Workshop on creative systems, 4th international conference on case based reasoning, Vancouver, pp 129–137Google Scholar
  29. 29.
    Perkins DN (1991) What constructivism demands of the learner. Educ Technol 31(9):19–21Google Scholar
  30. 30.
    Puckette M et al (1996) Pure data: another integrated computer music environment. In: Proceedings of the Second Intercollege Computer Music Concerts, pp 37–41Google Scholar
  31. 31.
    Reeves WT (1983) Particle systems—a technique for modeling a class of fuzzy objects. In: ACM SIGGRAPH Computer Graphics, vol 17. ACM, pp 359–375Google Scholar
  32. 32.
    Rocchesso D, Polotti P (2008) Designing continuous multisensory interaction. In: CHI Workshop - Sonic Interaction Design, Firenze, pp 3–9Google Scholar
  33. 33.
    Ryan RM, Deci EL (2000) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 25(1):54–67CrossRefGoogle Scholar
  34. 34.
    Saenen IP, De Bock S, Abdou E, Lambert P, Van de Walle R, Vets T, Lesaffre M, Demey M, Leman M (2014) Billiart-ar carom billiards. In: HCI International 2014-Posters’ Extended Abstracts. Springer, New York, pp 636–641Google Scholar
  35. 35.
    Sargaana U, Farahani HS, Lee JW, Ryu J, Woo W (2005) Collaborative billiards: towards the ultimate gaming experience. In: Entertainment Computing-ICEC. Springer, New York, pp 357–367Google Scholar
  36. 36.
    Schwarz D (2005) Current research in concatenative sound synthesis. In: Proceedings of the international computer music conference (ICMC). Barcelona, pp 1–4Google Scholar
  37. 37.
    Smith K (2006) Reversible-jump markov chain monte carlo multi-object tracking tutorial. Communication IDIAP-COM-06-07. IDIAP Research Institute, MartignyGoogle Scholar
  38. 38.
    Tormey A (1974) Indeterminacy and identity in art. The Monist, pp 203–215Google Scholar
  39. 39.
    Van Krevelen D, Poelman R (2010) A survey of augmented reality technologies, applications and limitations. Int J Virtual Real 9(2):1Google Scholar
  40. 40.
    Wolff C (1968-71) Prose collection. Lebanon: Frog Peak Music reprint. Accessed 6 March 2016
  41. 41.
    Wright M (1997) Open sound control-a new protocol for communicationg with sound synthesizers. In: Proceedings of the 1997 International Computer Music Conference, pp 101–104Google Scholar

Copyright information

© SIP 2016

Authors and Affiliations

  • T. Vets
    • 1
  • L. Nijs
    • 1
  • M. Lesaffre
    • 1
  • B. Moens
    • 1
  • F. Bressan
    • 2
  • P. Colpaert
    • 3
  • P. Lambert
    • 3
  • R. Van de Walle
    • 3
  • M. Leman
    • 1
  1. 1.IPEM, Department of MusicologyGhent UniversityGhentBelgium
  2. 2.Department of Information EngineeringUniversity of PadovaPaduaItaly
  3. 3.Multimedia Lab, Faculty of Engineering and ArchitectureGhent UniversityLedeberg-GhentBelgium

Personalised recommendations