Skip to main content
SpringerLink
Log in
Book cover

Electronic Visualisation in Arts and Culture pp 153–165Cite as

mConduct: A Multi-sensor Interface for the Capture and Analysis of Conducting Gesture

  • Chapter
  • First Online:

Part of the book series: Springer Series on Cultural Computing ((SSCC))

Abstract

The art of conducting has a long and well-established history, as both a technical and expressive art form, using physical gesture to convey musical intent and expression. Conducting relies on visual communication to direct the individual instrumentalists of an ensemble as a single, coherent unit. The aim of this project is to capture and analyse the hand gestures of conducting in order to provide real-time, interactive multimodal feedback. The system encompasses a number of application contexts including gesture visualisation for conducting analysis, pedagogy and preservation. This chapter presents the design and development of the interface involving hardware sensors and software analysis modules, and discusses the application of visualisation for conducting. Visualisation software has been designed to produce a sculpture of the conductor’s gesture, to reflect the individual conductor’s style and technique. The chapter concludes with latest findings, future directions and the impact the research may have outside the realm of gesture communication application.

This chapter is an updated and extended version of the following paper, published here with kind permission of the Chartered Institute for IT (BCS) and of EVA London Conferences: Armitage et al., “mConduct: A multi-sensor interface for the capture and analysis of conducting gesture.” In S. Dunn, J. P. Bowen, and K. Ng (eds.). EVA London 2012 Conference Proceedings. Electronic Workshops in Computing (eWiC), British Computer Society, 2012. http://www.bcs.org/ewic/eva2012 (accessed 26 May 2013).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Nakara, T. M. (2000). Inside the conductor’s jacket: Analysis, interpretation and musical synthesis of expressive gesture. Ph.D. thesis. Department of Media Arts and Sciences, MIT, Cambridge.

    Google Scholar 

  2. Green, E. (1981). The modern conductor. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  3. Boult, A. C. (1976). A handbook on the technique of conducting. Lakeland: Patterson.

    Google Scholar 

  4. Bowen, J. A. (2003). The rise of conducting. In The Cambridge Companion to Conducting. Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  5. Randhofer, R. (2004). By the rivers of Babylon: Echoes of the Babylonian past in the musical heritage of the Iraqi Jewish diaspora. Ethnomusicology Forum, 13, 21–45.

    Article  Google Scholar 

  6. Haïk-Vantoura, S. Chironomy in the ancient world, 17 December 2012. http://www.rakkav.com/biblemusic/pages/chironomy.htm. Accessed 22 Mar 2013.

  7. Marrin, T. (1996). Toward an understanding of musical gesture: Mapping expressive intention with the digital baton. M.Sc. thesis, MIT Media Lab, MIT, Cambridge.

    Google Scholar 

  8. Jacobson, B. (1979). Conductors on conducting. Frenchtown: Columbia Publishing.

    Google Scholar 

  9. Bianchi, F., & Smith, D. B. Virtual orchestra. http://www.virtualorchestra.com. Accessed 22 Mar 2013.

  10. Morita, H., Hashimoto, S., & Ohteru, S. (1991). A computer music system that follows a human conductor. Computer, 24(7), 44–53.

    Article  Google Scholar 

  11. Nakra, T. M. (2000). Inside the conductor’s jacket: Analysis, interpretation and musical synthesis of expressive gesture. Doctor of Philosophy, Massachusetts Institute of Technology, Boston.

    Google Scholar 

  12. Peng, L., & Gerhard, D. A wii-based gestural interface for computer-based conducting systems. In Proceedings of the 9th international conference on new interfaces for musical expression (NIME), Pittsburg, 4–6 June 2009.

    Google Scholar 

  13. Bradshaw, D., & Ng, K. (2008). Analyzing a conductors gestures with the Wiimote. In S. Dunn, S. Keene, G. Mallen, & J. P. Bowen (Eds.), Proceedings of electronic visualisation and the arts (EVA London), BCS, Electronic Workshops in Computing (eWiC).

    Google Scholar 

  14. Oliver, M., & Aczel, J. (2002). Theoretical models of the role of visualisation in learning in formal reasoning. Journal of Interactive Media in Education (JIME), 2. The Open University, UK.

    Google Scholar 

  15. Ng, K. (2011). Interactive multimedia for technology-enhanced learning with multimodal feedback. In J. Solis & K. Ng (Eds.), Musical robots and interactive multimodal systems (STAR 74) (Springer tracts in advanced robotics, Vol. 74, pp. 105–126). Berlin: Springer. Chapter 7.

    Chapter  Google Scholar 

  16. MacRitchie, J., Buck, B., & Bailey, N. J. Visualising musical structure through performance gesture. In Proceedings of the 10th international society for music information retrieval conference (ISMIR), Kobe, 26–30 October 2009.

    Google Scholar 

  17. Witt, H., Lawo, M., & Drugge, M. (2008). Visual feedback and different frames of reference: The impact of gesture interaction techniques for wearable computing. In G. H. ter Hofte, I. Mulder, & B. E. R. de Ruyter (Eds.), Proceedings of the 10th conference on human-computer interaction with mobile devices and services (Mobile HCI), Amsterdam, 2–5 September 2008, pp. 293–300, ACM International Conference Proceeding Series, ACM.

    Google Scholar 

  18. Garnett, G. E., Malvar-Ruiz, F., & Stoltzfus, F. (1999). Virtual conducting practice environment. In Proceedings of the international computer music conference (ICMC), (pp. 371–374).

    Google Scholar 

  19. Bakanas, P., Armitage, J., Balmer, J., Halpin, P., Hudspeth, K., & Ng, K. (2012). mConduct: Gesture transmission and reconstruction for distributed performance. In P. Nesi & R. Santucci (Eds.), International conference on information technologies for performing arts, media access and entertainment (ECLAP2012) (pp. 107–112). Florence: Firenze University Press.

    Google Scholar 

  20. Diebel, J. (2006). Representing attitude: Euler angles, unit quaternions, and rotation vectors. Technical Report, Stanford University, USA.

    Google Scholar 

  21. Madgwick, S. (2010). An efficient orientation filter for inertial and inertial/magnetic sensor arrays. Technical Report, Department of Mechanical Engineering, University of Bristol, UK.

    Google Scholar 

  22. Ng, K. (2004). Music via motion: Transdomain mapping of motion and sound for interactive performances. Proceedings of the IEEE, 92(4), 645–655.

    Article  Google Scholar 

  23. Kallio, S., Kela, J., Mäntyjärvi, J., & Plomp, J. (2006). Visualization of hand gestures for pervasive computing environments. In Proceedings of ACM working conference on advanced visual interfaces (AVI), (pp. 480–483). ACM.

    Google Scholar 

  24. Anglin, G. J., Vaez, H., & Cunningham, K. L. (2004). Visual representations and learning: The role of static and animated graphics. In D. H. Jonassen, (Ed.), Handbook of research on educational communications and technology (2nd ed., pp. 865–916). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  25. Arcavi, A. (2003). The role of visual representations in the learning of mathematics. Educational Studies in Mathematics, 52(3), 215–241.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, School of Electronic and Electrical Engineering & School of Music, ICSRiM – University of Leeds, Leeds, LS2 9JT, UK

    Joanne Armitage & Kia Ng

Authors
  1. Joanne Armitage
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kia Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kia Ng .

Editor information

Editors and Affiliations

  1. Department of Informatics, London South Bank University, London, United Kingdom

    Jonathan P. Bowen

  2. Department of Archaeology, University College London, London, United Kingdom

    Suzanne Keene

  3. School of Computing & School of Music, University of Leeds, Interdisciplinary Centre for Scientific Research in Music (ICSRiM), Leeds, United Kingdom

    Kia Ng

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Armitage, J., Ng, K. (2013). mConduct: A Multi-sensor Interface for the Capture and Analysis of Conducting Gesture. In: Bowen, J., Keene, S., Ng, K. (eds) Electronic Visualisation in Arts and Culture. Springer Series on Cultural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-5406-8_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5406-8_11

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5405-1

  • Online ISBN: 978-1-4471-5406-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation