Skip to main content
SpringerLink
Log in

Gamified music improvisation with BilliArT: a multimodal installation with balls

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

This paper presents the concept and the realisation of an interactive multimedia installation, called BilliArT, together with an explorative user study conducted on the data gathered during a public exhibition of the installation. The study concerns functional properties of the installation (e.g. usability, design quality) and subjective qualities of the sonic and visual feedback of the installation. The installation consists in a collaborative environment based on the carambole billiards game, which allows the users to engage in a user-driven machine-based jazz-inspired music improvisation, augmented with visual feedback. The installation is designed to promote the interaction among the users and the billiard game, stimulating the motivation to engage in the game by balancing predictable and unpredictable output, and reinforcing the feeling of reward, irrespective of their level of musical training. BilliArT introduces a new framework for expressive interaction related to the concepts of motivation and reward. The exploratory study proved the ability of the installation to activate the users’ sense of aesthetic reward, leading to a more active and satisfactory engagement in the game. Future studies may exploit these results to the advantage of the world of the arts, as well as of studies in human-computer interaction, interface design, and cultural heritage preservation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://www.youtube.com/watch?v=U2h_XPKxGQU.

  2. http://www.timvets.net/music/egi.php?page=music.

  3. http://www.timvets.net/installation/citylights1.php?page=installation_citylights.

  4. http://www.ipem.ugent.be/node/282.

  5. http://www.naturalpoint.com/optitrack/.

  6. http://www.naturalpoint.com/optitrack/products/natnet-sdk.

  7. http://www.amd.com/en-us/products/graphics/desktop/7000/7900.

  8. http://www.geisswerks.com/ryan/BLOBS/blobs.html.

  9. http://openframeworks.cc.

  10. https://www.opengl.org.

  11. http://glslsandbox.com.

  12. http://duesenbergusa.com/guitars/starplayertv/index.html.

  13. http://freesound.org.

  14. http://www.timvets.net/video/grains.php?page=software.

References

  1. GPU gems—chapter 38 (2015) Fast fluid dynamics simulation on the GPU. https://developer.nvidia.com/gpugems/GPUGems/gpugems_pref01.html. Accessed 6 Jan 2016

  2. Adelson E, Bergen J (1991) Computational models of visual processing, chap. The Plenoptic Function and the Elements of Early Vision. MIT Press, Cambridge, pp 3–20

  3. Alves R, Sousa L, Rodrigues J (2013) Poolliveaid: augmented reality pool table to assist inexperienced players. In: Baranoski G, Skala V (eds) Proceedings of the 21st international conference on computer graphics, Visualization and Computer Vision, Plzen, Czech Republic, pp 184–193

  4. Biles J (1994) Genjam: a genetic algorithm for generating jazz solos. In: Proceedings of the international computer music conference. International Computer Music Association, San Francisco, pp 131–137

  5. Blaine T (2005) The convergence of alternate controllers and musical interfaces in interactive entertainment. In: Proceedings of the 2005 conference on New interfaces for musical expression. National University of Singapore, Singapore, pp 27–33

  6. Blaine T, Fels S (2003) Collaborative musical experiences for novices. J New Music Res 32(4):411–428

    Article  Google Scholar 

  7. Blinn JF (1982) A generalization of algebraic surface drawing. ACM Trans Graphics 1(3):235–256

    Article  Google Scholar 

  8. Brattico E, Brattico P, Jacobsen T (2009) The origins of the aesthetic enjoyment of music–a review of the literature. Music Sci 13(2 suppl):15–39

    Google Scholar 

  9. Brent W (2011) DILib: control data parsing for digital musical instrument design. In: Proceedings of the 4th international pure data convention. Weimar, pp 176–180

  10. Bressan F, Canazza S (2014) The challenge of preserving Interactive Sound Art: A multi-level approach. Int J Arts Technol 7(4):294–315

    Article  Google Scholar 

  11. Collins N (2003) Generative music and laptop performance. Contemp Music Rev 22(4):67–79

    Article  Google Scholar 

  12. Csikszentmihalyi M (1990) Flow. The psychology of optimal experience. Harper & Row, New York

  13. Edmonds E, Muller L, Connell M (2006) On creative engagement. Visual Commun 5(3):307–322

    Article  Google Scholar 

  14. Hedges SA (1978) Dice music in the eighteenth century. Music & Letters 59(2):180–187

  15. Hudak P, Berger J (1995) A model of performance, interaction, and improvisation. In: Proceedings of international computer music conference. International Computer Music Association, San Francisco, pp 1–8

  16. Kaplan F, Oudeyer PY (2007) In search of the neural circuits of intrinsic motivation. Front Neurosci 1(1):225

    Article  Google Scholar 

  17. Kato J, Nakashima T (2013) Openpool: community-based prototyping of digitally-augmented billiard table. In: GCCE ’13: Proceedings of the 2nd IEEE global conference on consumer electronics. IEEE, Tokyo, pp 175–176

  18. Klügel N, Frieß MR, Groh G, Echtler F (2011) An approach to collaborative music composition. In: Proceedings of the international conference on new interfaces for musical expression, Oslo, pp 32–35

  19. Leman M (2008) Embodied music cognition and mediation technology. MIT Press, Cambridge

  20. Leman M (2016) The expressive moment. MIT Press, Cambridge

  21. Leman M, Demey M, Lesaffre M, van Noorden L, Moelants D (2009) Concepts, technology, and assessment of the social music game “sync-in-team”. In: International conference on computational science and engineering. CSE’09, vol 4. IEEE Computer Society, Washington, pp 837–842

  22. Maes PJ, Nijs L, Leman M (2015) A conceptual framework for music-based interaction systems. In: Bader R (ed) Springer handbook in systematic musicology. Springer, Berlin, Heidelberg

    Google Scholar 

  23. Maes PJ, Leman M, Palmer C, Wanderley M (2014) Action-based effects on music perception. Front Pshychol 4(1008) (2014). doi:10.3389/fpsyg.2013.01008

  24. Martens R, Gulikers J, Bastiaens T (2004) The impact of intrinsic motivation on e-learning in authentic computer tasks. J Comput Assist Learn 20(5):368–376

    Article  Google Scholar 

  25. Martin DC (1989) Retroreflective sheet material and method of making same. US Patent 4,801,193

  26. Nijs L, Coussement P, Moens B, Amelinck D, Lesaffre M, Leman M (2012) Interacting with the music paint machine: relating the constructs of flow experience and presence. Interact Comput 24(4):237–250

    Article  Google Scholar 

  27. Papadopoulos G, Wiggins G (1999) Ai methods for algorithmic composition: a survey, a critical view and future prospects. In: AISB symposium on musical creativity. Edinburgh, pp 110–117

  28. Pease A, Winterstein D, Colton S (2001) Evaluating machine creativity. In: Workshop on creative systems, 4th international conference on case based reasoning, Vancouver, pp 129–137

  29. Perkins DN (1991) What constructivism demands of the learner. Educ Technol 31(9):19–21

    Google Scholar 

  30. Puckette M et al (1996) Pure data: another integrated computer music environment. In: Proceedings of the Second Intercollege Computer Music Concerts, pp 37–41

  31. Reeves WT (1983) Particle systems—a technique for modeling a class of fuzzy objects. In: ACM SIGGRAPH Computer Graphics, vol 17. ACM, pp 359–375

  32. Rocchesso D, Polotti P (2008) Designing continuous multisensory interaction. In: CHI Workshop - Sonic Interaction Design, Firenze, pp 3–9

  33. Ryan RM, Deci EL (2000) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 25(1):54–67

    Article  Google Scholar 

  34. Saenen IP, De Bock S, Abdou E, Lambert P, Van de Walle R, Vets T, Lesaffre M, Demey M, Leman M (2014) Billiart-ar carom billiards. In: HCI International 2014-Posters’ Extended Abstracts. Springer, New York, pp 636–641

  35. Sargaana U, Farahani HS, Lee JW, Ryu J, Woo W (2005) Collaborative billiards: towards the ultimate gaming experience. In: Entertainment Computing-ICEC. Springer, New York, pp 357–367

  36. Schwarz D (2005) Current research in concatenative sound synthesis. In: Proceedings of the international computer music conference (ICMC). Barcelona, pp 1–4

  37. Smith K (2006) Reversible-jump markov chain monte carlo multi-object tracking tutorial. Communication IDIAP-COM-06-07. IDIAP Research Institute, Martigny

  38. Tormey A (1974) Indeterminacy and identity in art. The Monist, pp 203–215

  39. Van Krevelen D, Poelman R (2010) A survey of augmented reality technologies, applications and limitations. Int J Virtual Real 9(2):1

    Google Scholar 

  40. Wolff C (1968-71) Prose collection. Lebanon: Frog Peak Music reprint. http://www.frogpeak.org/unbound/wolff/wolff_prose_collection.pdf. Accessed 6 March 2016

  41. Wright M (1997) Open sound control-a new protocol for communicationg with sound synthesizers. In: Proceedings of the 1997 International Computer Music Conference, pp 101–104

Download references

Author information

Authors and Affiliations

  1. IPEM, Department of Musicology, Ghent University, Sint-Pietersnieuwstraat 41, Ghent, Belgium

    T. Vets, L. Nijs, M. Lesaffre, B. Moens & M. Leman

  2. Department of Information Engineering, University of Padova, via G. Gradenigo 6/B, 35131, Padua, Italy

    F. Bressan

  3. Multimedia Lab, Faculty of Engineering and Architecture, Ghent University, Gaston Crommenlaan 8 bus 201, 9050, Ledeberg-Ghent, Belgium

    P. Colpaert, P. Lambert & R. Van de Walle

Authors
  1. T. Vets
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Nijs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Lesaffre
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. B. Moens
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. F. Bressan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. P. Colpaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. P. Lambert
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. R. Van de Walle
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. M. Leman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. Vets.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vets, T., Nijs, L., Lesaffre, M. et al. Gamified music improvisation with BilliArT: a multimodal installation with balls. J Multimodal User Interfaces 11, 25–38 (2017). https://doi.org/10.1007/s12193-016-0224-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-016-0224-9

Keywords

Search

Navigation