Skip to main content
SpringerLink
Log in

Personal orchestra: a real-time audio/video system for interactive conducting

  • Published:
Multimedia Systems Aims and scope Submit manuscript

An Erratum to this article was published on 01 June 2004

Abstract.

We present the first multimedia system to conduct a realistic electronic orchestra. Users can control tempo, dynamics, and instrument emphasis of the orchestra through natural conducting gestures with an infrared baton. Using gesture recognition and tempo adjustment algorithms, the system plays back an audio and video recording of an actual orchestra that follows the user’s conducting in real time. A major achievement of this system is its ability to vary playback speed in real time while avoiding audio artifacts such as pitch changes. The system has been deployed as an exhibit and has become a major attraction of a large Vienna-based music exhibition center.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Prosoniq MPEX near-lossless time scaling technology. http://www.prosoniq.net/html/mpex.html

  2. house of music vienna (2000) http://www.hdm.at

  3. Borchers J (1997) Worldbeat: designing a baton-based interface for an interactive music exhibit. In: Proceedings of the CHI ‘97 ACM conference on human factors in computing systems, Atlanta, GA, 22-27 March 1997. ACM Press, New York, pp 131-138.

  4. Borchers J (2001) A pattern approach to interaction design. Wiley, New York. http://hcipatterns.org

  5. Buchla D (2003) Lightning II MIDI controller. http://www.buchla.com/

  6. Ilmonen T (2000) The virtual orchestra performance. “The art of the desktop” invited session participant. In: Extended abstracts of the CHI 2000 ACM conference on human factors in computing systems, The Hague, The Netherlands, 1-6 April 2000. ACM Press, New York, pp 203-204

  7. Laroche J (1998) Time and pitch scale modification of audio signals. In: Kahrs M, Brandenburg K (eds) Applications of DSP to audio and acoustics. Kluwer, Dordrecht, pp 279-309

  8. Laroche J (1999) Improved phase vocoder time-scale modification of audio. IEEE Trans Speech Audio Process 7:323-332

    Article  Google Scholar 

  9. Mathews MV (1991) The conductor program and mechanical baton. In: Current directions in computer music research. MIT Press, Cambridge

  10. Maybeck PS (1979) Stochastic models, estimation, and control, vol 1. Academic, New York. Available at: http://www.cs.unc.edu/~welch/kalman/maybeck.html

  11. Nakra TM (2000) Inside the conductor’s jacket: analysis, interpretation and musical synthesis of expressive gesture. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA

  12. Rudolf M (1995) The grammar of conducting. Wadsworth, London

Download references

Author information

Authors and Affiliations

  1. Media Computing Group, Department of Computer Science, RWTH Aachen, 52056, Aachen, Germany

    Jan Borchers & Eric Lee

  2. Telecooperation Department, University of Linz, 4040, Linz, Austria

    Wolfgang Samminger

  3. Telecooperation Group, Darmstadt University, D-64283, Darmstadt, Germany

    Max Mühlhäuser

Authors
  1. Jan Borchers
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wolfgang Samminger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Max Mühlhäuser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Borchers.

Additional information

An erratum to this article can be found at http://dx.doi.org/10.1007/s00530-004-0139-2

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borchers, J., Lee, E., Samminger, W. et al. Personal orchestra: a real-time audio/video system for interactive conducting. Multimedia Systems 9, 458–465 (2004). https://doi.org/10.1007/s00530-003-0119-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-003-0119-y

Keywords:

Search

Navigation