Abstract
We have been developing a new type of human-computer interface, myokinetic synthesizer (MyoKinSynthesizer), using electromyography (EMG) signals. It enables a user to select virtual instruments and to control its properties such as volume and tone without any position or force sensors. The virtual marimba system emulates the basic properties of the real instrument by producing a sound depending on in which of eight zones and how hard the user is hitting. The virtual drum set is composed of 4 different virtual drums controlled by arms and legs. We used a three-layer neural network to estimate position and force of the forearm from EMG signals. After training the neural network and obtaining appropriate weights, the subject was able to control the movement of virtual avatar and to play virtual instruments. The system was destined to be used as a demonstration of VR entertainment and music therapy rehabilitation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Koike, Y., Kawato, M.: Estimation of arm posture in 3D-space from surface EMG signals using a neural network model. IEICE Transactions Fundamentals D-II (4), 368–375 (1994)
Jacobsen, S.C., Wood, J.E., Knutti, D.F., Biggers, K.B.: The Utah/MIT dexterous hand: work in progress. Int. J. Robotics Research 3(4), 21–50 (1984)
Meek, S.G., Jacobsen, S.C., Straight, R.: Development of advanced bodypowered prosthetic arms. Journal of Rehabilitation Research and Development 26 (1989)
Bluck, J.: Nasa scientists use hands-off approach to land passenger jet, http://amesnews.arc.nasa.gov/releases/2001/0108AR.html
Tanaka, A.: Musical technical issues in using interactive instrument technology with application to the BioMuse. In: EICMC Proceedings, pp. 124–126 (2001)
Koike, Y.: Tele-Kinesthetic Interaction: Using Hand Muscles to Interact with A Tangible 3D Object. In: SIGGRAPH 2006 (2006)
Koike, Y., Kawato, M.: Estimation of dynamic joint torques and trajectory formation from surface electromyography signals using a neural network model. Biological Cyberbetics 73, 291–300 (1995)
Mathews, V.: The Conductor Program and Mechanical Baton, in Max V. Mathews. In: Pierce, J.R. (ed.) Current Directions in Computer Music Research. MIT Press, Cambridge (1991)
Pogue, D.: The dangers of the Digital Orchestra. New York Times Direct, April 5 (2001)
Borchers, O.: WorldBeat: Designing a baton-based interface for an interactive music exhibit. In: Proc. CH 1997, pp. 131–138. ACM, New York (1997)
Lee, M., Garnett, G., Wessel, D.: An Adaptive Conductor Follower. In: Proc. International Computer Music Conference, ICMA, San Francisco (1992)
Kragtwijk, M., Nijholt, A., Zwiers, J.: Implementation of a 3D Virtual Drummer. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Computer Animation and Simulation, pp. 15–26. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shin, D., Katayama, A., Kim, K., Kambara, H., Sato, M., Koike, Y. (2006). Using a MyoKinetic Synthesizer to Control of Virtual Instruments. In: Pan, Z., Cheok, A., Haller, M., Lau, R.W.H., Saito, H., Liang, R. (eds) Advances in Artificial Reality and Tele-Existence. ICAT 2006. Lecture Notes in Computer Science, vol 4282. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11941354_128
Download citation
DOI: https://doi.org/10.1007/11941354_128
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-49776-9
Online ISBN: 978-3-540-49779-0
eBook Packages: Computer ScienceComputer Science (R0)