Advertisement

Using a MyoKinetic Synthesizer to Control of Virtual Instruments

  • Duk Shin
  • Atsushi Katayama
  • Kyoungsik Kim
  • Hiroyuki Kambara
  • Makoto Sato
  • Yasuharu Koike
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4282)

Abstract

We have been developing a new type of human-computer interface, myokinetic synthesizer (MyoKinSynthesizer), using electromyography (EMG) signals. It enables a user to select virtual instruments and to control its properties such as volume and tone without any position or force sensors. The virtual marimba system emulates the basic properties of the real instrument by producing a sound depending on in which of eight zones and how hard the user is hitting. The virtual drum set is composed of 4 different virtual drums controlled by arms and legs. We used a three-layer neural network to estimate position and force of the forearm from EMG signals. After training the neural network and obtaining appropriate weights, the subject was able to control the movement of virtual avatar and to play virtual instruments. The system was destined to be used as a demonstration of VR entertainment and music therapy rehabilitation.

Keywords

Virtual Instrument Human Interface Shoulder Posture Interactive Music Virtual Avatar 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Koike, Y., Kawato, M.: Estimation of arm posture in 3D-space from surface EMG signals using a neural network model. IEICE Transactions Fundamentals D-II (4), 368–375 (1994)Google Scholar
  2. 2.
    Jacobsen, S.C., Wood, J.E., Knutti, D.F., Biggers, K.B.: The Utah/MIT dexterous hand: work in progress. Int. J. Robotics Research 3(4), 21–50 (1984)CrossRefGoogle Scholar
  3. 3.
    Meek, S.G., Jacobsen, S.C., Straight, R.: Development of advanced bodypowered prosthetic arms. Journal of Rehabilitation Research and Development 26 (1989)Google Scholar
  4. 4.
    Bluck, J.: Nasa scientists use hands-off approach to land passenger jet, http://amesnews.arc.nasa.gov/releases/2001/0108AR.html
  5. 5.
    Tanaka, A.: Musical technical issues in using interactive instrument technology with application to the BioMuse. In: EICMC Proceedings, pp. 124–126 (2001)Google Scholar
  6. 6.
    Koike, Y.: Tele-Kinesthetic Interaction: Using Hand Muscles to Interact with A Tangible 3D Object. In: SIGGRAPH 2006 (2006)Google Scholar
  7. 7.
    Koike, Y., Kawato, M.: Estimation of dynamic joint torques and trajectory formation from surface electromyography signals using a neural network model. Biological Cyberbetics 73, 291–300 (1995)CrossRefMATHGoogle Scholar
  8. 8.
    Mathews, V.: The Conductor Program and Mechanical Baton, in Max V. Mathews. In: Pierce, J.R. (ed.) Current Directions in Computer Music Research. MIT Press, Cambridge (1991)Google Scholar
  9. 9.
    Pogue, D.: The dangers of the Digital Orchestra. New York Times Direct, April 5 (2001)Google Scholar
  10. 10.
    Borchers, O.: WorldBeat: Designing a baton-based interface for an interactive music exhibit. In: Proc. CH 1997, pp. 131–138. ACM, New York (1997)Google Scholar
  11. 11.
    Lee, M., Garnett, G., Wessel, D.: An Adaptive Conductor Follower. In: Proc. International Computer Music Conference, ICMA, San Francisco (1992)Google Scholar
  12. 12.
    Kragtwijk, M., Nijholt, A., Zwiers, J.: Implementation of a 3D Virtual Drummer. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Computer Animation and Simulation, pp. 15–26. Springer, Heidelberg (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Duk Shin
    • 1
  • Atsushi Katayama
    • 2
  • Kyoungsik Kim
    • 2
    • 3
  • Hiroyuki Kambara
    • 2
    • 3
  • Makoto Sato
    • 1
  • Yasuharu Koike
    • 1
    • 3
  1. 1.Precision and Intelligence LaborataryTokyo Institute of TechnologyJapan
  2. 2.Department of Intelligence and Systems ScienceTokyo Institute of TechnologyJapan
  3. 3.CRESTJapan Science and Technology AgencyJapan

Personalised recommendations