A Customizable Recognizer for Orchestral Conducting Gestures Based on Neural Networks

  • Roberto De Prisco
  • Paolo Sabatino
  • Gianluca Zaccagnino
  • Rocco Zaccagnino
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6625)


In this paper we present a system for the recognition of orchestral conducting gestures using the Nintendo Wiimote controller. The system is based on a neural network. This is not the first of such systems; however compared to previous systems that use the Wiimote or other cheap hardware, it has several advantages: it is fully customizable, continuous and does not require third party commercial software.

The system has been implemented in Java and we have run several tests to evaluate its behavior.


Data Representation Gesture Recognition Remote Controller Continuous Recognition Single Gesture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bradshaw, D., Ng, K.: Tracking conductors hand movements using multiple wiimotes. In: Proceedings of the International Conference on Automated Solutions for Cross Media Content and Multi-channel Distribution (AXMEDIS 2008), Firenze, Italy, pp. 93–99. IEEE Computer Society Press, Los Alamitos (2008)CrossRefGoogle Scholar
  2. 2.
    Lee, J.C.: Hacking the Nintendo Wii remote. Pervasive Computing 7(3), 39–45 (2008)CrossRefGoogle Scholar
  3. 3.
    Kiefer, C., Collins, N., Fitzpatrick, G.: Evaluating the Wiimote as a musical controller. In: Proceedings of the International Computer Music Conference, Belfast, N. Ireland (2008)Google Scholar
  4. 4.
    Ilmonen, T., Takala, T.: Conductor following with artificial neural networks. In: Proceedings of the International Computer Music Conference, Bejing, China (1999)Google Scholar
  5. 5.
    Hofmann, F.G., Heyer, P., Hommel, G.: Velocity profile based recognition of dynamic gestures with discrete Hidden Markov Models. In: Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction, pp. 81–95. Springer, London (1998)CrossRefGoogle Scholar
  6. 6.
    Kolesnik, P., Wanderley, M.: Recognition, analysis and performance with expressive conducting gesture. In: Proceedings of the International Computer Music Conference, Miami, USA (2004)Google Scholar
  7. 7.
    Liang, H., Ouhyoung, M.: A real-time continuous gesture recognition system for sign language. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, p. 558 (1998)Google Scholar
  8. 8.
    Mathews, M.: The radio baton and the conductor program, or: Pitch-the most important and least expressive part of music. Computer Music Journal 15(4), 37–46 (1991)CrossRefGoogle Scholar
  9. 9.
    Miranda, E.R., Wanderley, M.: New digital musical instruments: Control and interaction beyond the keyboard. A-R Editions, Middleton (2006)Google Scholar
  10. 10.
    Mlich, J.: Wiimote gesture recognition. In: Proceedings of the 15th Conference and Competition STUDENT EEICT 2009, vol. 4, pp. 344–349 (2009)Google Scholar
  11. 11.
    Bruegge, B., Teschner, C., Lachenmaier, P., Fenzl, E., Schmidt, D., Bierbaum, S.: Pinocchio: Conducting a virtual symphony orchestra. In: Proceedings of the International Conference on Advances in Computer Entertainment Technology (ACE), Salzburg, Austria, pp. 294–295 (2007)Google Scholar
  12. 12.
    Schlömer, T., Poppinga, B., Henze, N., Boll, S.: Gesture recognition with a Wii controller. In: Proceedings of the 2nd International ACM Conference on Tangible and Embedded Interaction, New York, NY, USA, pp. 11–14 (2008)Google Scholar
  13. 13.
    Nakra, T.M., Ivanov, Y., Smaragdis, P., Ault, C.: The UBS virtual maestro: An interactive conducting system. In: Proceeding of 9th International Conference on New Interfaces for Musical Expression, Pittsburgh, PA, USA (2009)Google Scholar
  14. 14.
    Wii yourself website,

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Roberto De Prisco
    • 1
  • Paolo Sabatino
    • 1
  • Gianluca Zaccagnino
    • 1
  • Rocco Zaccagnino
    • 1
  1. 1.Laboratorio di Musimatica Dipartimento di InformaticaUniversità di SalernoFiscianoItaly

Personalised recommendations