Using a MyoKinetic Synthesizer to Control of Virtual Instruments
We have been developing a new type of human-computer interface, myokinetic synthesizer (MyoKinSynthesizer), using electromyography (EMG) signals. It enables a user to select virtual instruments and to control its properties such as volume and tone without any position or force sensors. The virtual marimba system emulates the basic properties of the real instrument by producing a sound depending on in which of eight zones and how hard the user is hitting. The virtual drum set is composed of 4 different virtual drums controlled by arms and legs. We used a three-layer neural network to estimate position and force of the forearm from EMG signals. After training the neural network and obtaining appropriate weights, the subject was able to control the movement of virtual avatar and to play virtual instruments. The system was destined to be used as a demonstration of VR entertainment and music therapy rehabilitation.
KeywordsVirtual Instrument Human Interface Shoulder Posture Interactive Music Virtual Avatar
Unable to display preview. Download preview PDF.
- 1.Koike, Y., Kawato, M.: Estimation of arm posture in 3D-space from surface EMG signals using a neural network model. IEICE Transactions Fundamentals D-II (4), 368–375 (1994)Google Scholar
- 3.Meek, S.G., Jacobsen, S.C., Straight, R.: Development of advanced bodypowered prosthetic arms. Journal of Rehabilitation Research and Development 26 (1989)Google Scholar
- 4.Bluck, J.: Nasa scientists use hands-off approach to land passenger jet, http://amesnews.arc.nasa.gov/releases/2001/0108AR.html
- 5.Tanaka, A.: Musical technical issues in using interactive instrument technology with application to the BioMuse. In: EICMC Proceedings, pp. 124–126 (2001)Google Scholar
- 6.Koike, Y.: Tele-Kinesthetic Interaction: Using Hand Muscles to Interact with A Tangible 3D Object. In: SIGGRAPH 2006 (2006)Google Scholar
- 8.Mathews, V.: The Conductor Program and Mechanical Baton, in Max V. Mathews. In: Pierce, J.R. (ed.) Current Directions in Computer Music Research. MIT Press, Cambridge (1991)Google Scholar
- 9.Pogue, D.: The dangers of the Digital Orchestra. New York Times Direct, April 5 (2001)Google Scholar
- 10.Borchers, O.: WorldBeat: Designing a baton-based interface for an interactive music exhibit. In: Proc. CH 1997, pp. 131–138. ACM, New York (1997)Google Scholar
- 11.Lee, M., Garnett, G., Wessel, D.: An Adaptive Conductor Follower. In: Proc. International Computer Music Conference, ICMA, San Francisco (1992)Google Scholar
- 12.Kragtwijk, M., Nijholt, A., Zwiers, J.: Implementation of a 3D Virtual Drummer. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Computer Animation and Simulation, pp. 15–26. Springer, Heidelberg (2001)Google Scholar