Abstract
We present a new digital virtual instrument designed, built and played using 3D interactive technologies, with interest in reducing both learning time and difficulty in playing musical phrases and chords across the musical scale. Design concepts stress on a smart note mapping, rely on bi-manual interaction, and intend an optimized gestural performance on a virtual keyboard to shorten playing trajectories. The proposed keyboard is built stacking note keys in multiple lines, and the note mapping allows performing typical musical intervals and patterns with short gestures. This implementation employed an Oculus Rift head-mounted display and a Razer Hydra for gesture input. The combination of 3D visualization, natural interaction and the proposed mapping is meant to contribute to ease musical performance by allowing fast execution of specific note sequences. The instrument concept encapsulates complexity by creating a fluid way for musicians to perform gesture patterns that would otherwise require non-trivial motor skills.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fels, S.: Designing for intimacy: creating new interfaces for musical expression. Proc. IEEE 92(4), 672–685 (2004)
Fels, S., Lyons, M.: Creating new interfaces for musical expression: introduction to NIME. In: ACM SIGGRAPH 2009 Courses, SIGGRAPH 2009. ACM, New York (2009)
Chadabe, J.: The limitations of mapping as a structural descriptive in electronic instruments. In: Proceedings of the 2002 Conference on New Interfaces for Musical Expression (NIME 2002), Dublin, Ireland (2002)
Wessel, D., Wright, M.: Problems and prospects for intimate musical control of computers. Comput. Music J. 26(3), 11–22 (2002)
Cook, P.R.: Laptop orchestras, robotic drummers, singing machines, and musical kitchenware: learning programming, algorithms, user interface design, and science through the arts. J. Comput. Sci. Coll. 28(1), 157 (2012)
Han, J., Gold, N.: Lessons learned in exploring the leap motion TM sensor for gesture-based instrument design. In: Proceedings of the 2014 Conference on New Interfaces for Musical Expression. (NIME 2014), London, United Kingdom (2014)
Franco, I.: The AirStick: a free-gesture controller using infrared sensing. In Proceedings of the 2005 Conference on New Interfaces for Musical Expression (NIME 2005), Vancouver, BC, Canada (2005)
Hasan, L., et al.: The Termenova: a hybrid free-gesture interface. In: Proceedings of the 2002 Conference on New Interfaces for Musical Expression (NIME 2002), Dublin, Ireland (2002)
Han, Y., Na, J., Lee, K.: Futuregrab: a wearable synthesizer using vowel formants. In: Proceedings of the 2012 Conference on New Interfaces for Musical Expression (NIME 2012), Ann Arbor, Michigan (2012)
Poepel, C., Feitsch, J., Strobel, M., Geiger, C.: Design and evaluation of a gesture controlled singing voice installation. In: Proceedings of the 2014 Conference on New Interfaces for Musical Expression (NIME 2014), London, United Kingdom (2014)
Gillian, N., Paradiso, J.A.: Digito: a fine-grain gesturally controlled virtual musical instrument. In: Proceedings of the 2012 Conference on New Interfaces for Musical Expression (NIME 2012), Ann Arbor, Michigan (2012)
Hunt, A., Wanderley, M.M., Paradis, M.: The importance of parameter mapping in electronic instrument design. In: Proceedings of the 2002 Conference on New Interfaces for Musical Expression (NIME 2002), Dublin, Ireland (2002)
Maupin, S., Gerhard, D., Park, B.: Isomorphic tessellations for musical keyboards. In: Proceedings of Sound and Music Computing Conference, Padova, Italy (2011)
Perez, M.: Gesto musical e o uso de interfaces físicas digitais na performance do live electronics. Master Dissertation in Music, University of São Paulo, São Paulo (2016)
Cabral, M., et al.: Crosscale: a 3D virtual musical instrument interface. In: Proceedings of the 2015 Symposium on 3D User Interfaces (3DUI), Arles, France (2015)
Acknowledgments
This research has been partially supported by CNPq Brazilian National Research Council, DT Productivity Grant process 311681/2012-3. The authors would like to thank Gabriel Roque, Mario Nagamura, Luiz Paulucci, Marcio Cabral and Tainá Saboia for the substantial contribution to this research.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Rodrigues, A.M., Zuffo, M.K., da Rosa Belloc, O., Faria, R.R.A. (2017). A Virtual Musical Instrument for 3D Performance with Short Gestures: Exploring Mapping Strategies with Virtual Reality. In: Aramaki, M., Kronland-Martinet, R., Ystad, S. (eds) Bridging People and Sound. CMMR 2016. Lecture Notes in Computer Science(), vol 10525. Springer, Cham. https://doi.org/10.1007/978-3-319-67738-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-67738-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67737-8
Online ISBN: 978-3-319-67738-5
eBook Packages: Computer ScienceComputer Science (R0)