Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Computer Music Multidisciplinary Research

CMMR 2013: Sound, Music, and Motion pp 41–57Cite as

Vibrotactile Feedback for an Open Air Music Controller

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 8905))

Abstract

In this paper we describe an approach for providing vibrotactile feedback for digital musical instruments (DMIs) that are controlled with open air hand motion. The hand motion was captured with infrared marker based motion capture technology. The marker position data was mapped to the control parameters of both sound and vibrotactile signal synthesis. Vibrotactile feedback was provided to the fingertips of the performer by sending the synthesized signals to voice coils actuators that were embedded in a glove. Vibrotactile strategies were developed for two DMI prototypes that focus on different ways of controlling musical sound. Results of an informal evaluation indicate that the synthesized vibrotactile stimuli can provide useful feedback on how the performer is playing the instrument, as well as enhancing the experience of playing the given DMIs.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Askenfelt, A., Jansson, E.V.: On vibration sensation and finger touch in stringed instrument playing. Music Percept. Interdisc. J. 9(3), 311–349 (1992)

    Article  Google Scholar 

  2. Berdahl, E., Steiner, H.C., Oldham, C.: Practical hardware and algorithms for creating haptic musical instruments. In: Proceedings of the International Conference on New Inferfaces for Musical Expression, Genova, pp. 61–66 (2008)

    Google Scholar 

  3. Birnbaum, D.M.: Musical vibrotactile feedback. Master’s thesis, McGill University, Montréal (2007)

    Google Scholar 

  4. Birnbaum, D.M., Wanderley, M.M.: A systematic approach to musical vibrotactile feedback. In: Proceedings of the International Computer Music Conference, Copenhagen, vol. 2, pp. 397–404 (2007)

    Google Scholar 

  5. Brinkmann, P.: Making Musical Apps: Real-time audio synthesis on Android and iOS. O’Reilly Media Inc, Sebastopol (2012)

    Google Scholar 

  6. Brown, L., Brewster, S., Purchase, H.: A first investigation into the effectiveness of tactons. In: Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 167–176 (2005)

    Google Scholar 

  7. Bryan, N.J., Herrera, J., Oh, J., Wang, G.: MoMu: a mobile music toolkit. In: Proceedings of the International Conference on New Inferfaces for Musical Expression, Sydney (2010)

    Google Scholar 

  8. Chafe, C.: Tactile audio feedback. In: Proceedings of the International Computer Music Conference, Japan, pp. 76–79 (1993)

    Google Scholar 

  9. Chafe, C., OModhrain, S.: Musical muscle memory and the haptic display of performance nuance. In: Proceedings of the International Computer Music Conference, Hong Kong, pp. 429–431 (1996)

    Google Scholar 

  10. Choi, S., Kuchenbecker, K.: Vibrotactile display: Perception, technology, and applications. In: Proceedings of the IEEE, pp. 1–12. IEEE (2012)

    Google Scholar 

  11. Cottle, D.M.: Beginner’s tutorial. In: Wilson, S., Cottle, D., Collins, N. (eds.) The SuperCollider Book, pp. 3–54. MIT Press, London (2011)

    Google Scholar 

  12. Egloff, D.C.: A vibrotactile music system based on sensory substitution. Master’s thesis, Rensselaer Polytechnic Institute, Troy (2011)

    Google Scholar 

  13. Geldard, F.A., Sherrick, C.E.: The cutaneous “rabbit”: a perceptual illusion. Science 178(4057), 178–179 (1972)

    Article  Google Scholar 

  14. Ghamsari, M., Pras, A., Wanderley, M.M.: Combining musical tasks and improvisation in evaluating novel digital musical instruments. In: Proceedings of the 10th International Symposium on Computer Music Multidisciplinary Research, Marseille, pp. 506–515 (2013)

    Google Scholar 

  15. Giordano, M., Wanderley, M.M.: Perceptual and technological issues in the design of vibrotactile-augmented interfaces for music technology and media. In: Oakley, I., Brewster, S. (eds.) HAID 2013. LNCS, vol. 7989, pp. 89–98. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  16. Halata, Z., Baumann, K.I.: Anatomy of receptors. In: Grunwald, M. (ed.) Human Haptic Perception: Basics and Applications, pp. 85–92. Birkhuser, Basel (2008)

    Chapter  Google Scholar 

  17. Hunt, A., Kirk, R.: Mapping strategies for musical performance (reprint). In: Wanderley, M.M., Battier, M. (eds.) Trends in Gestural Control of Music, pp. 231–258. IRCAM, Centre Pompidou, Paris (2000)

    Google Scholar 

  18. Hunt, A., Wanderley, M.M., Kirk, R.: Towards a model for instrumental mapping in expert musical interaction. In: Proceedings of the International Computer Music Conference, Berlin, pp. 209–212 (2000)

    Google Scholar 

  19. Hunt, A., Wanderley, M.M., Paradis, M.: The importance of parameter mapping in electronic instrument design. J. New Music Res. 32, 429–440 (2003)

    Article  Google Scholar 

  20. Jensenius, A.R., Wanderley, M.M., Godøy, R.I., Leman, M.: Musical gestures: concepts and methods in research. In: Godøy, R.I., Leman, M. (eds.) Musical Gestures: Sound, Movement, and Meaning, pp. 12–35. Routledge, New York (2010)

    Google Scholar 

  21. Jordà, S.: Instruments and players: some thoughts on digital lutherie. J. New Music Res. 3, 321–341 (2004)

    Article  Google Scholar 

  22. Kim, Y., Cha, J., Ryu, J., Oakley, I.: A tactile glove design and authoring system for immersive multimedia. IEEE MultiMed. 17(3), 34–45 (2010)

    Article  Google Scholar 

  23. Knutzen, H.: Haptics in the Air - Exploring vibrotactile feedback for digital musical instruments with open air controllers. Master’s thesis, University of Oslo (2013)

    Google Scholar 

  24. Kvifte, T.: On images and representations. In: Instruments and the Electronic Age. Taragot Sounds, Oslo (2007)

    Google Scholar 

  25. Kvifte, T.: On the description of mapping structures. J. New Music Res. 37(4), 353–362 (2008)

    Article  Google Scholar 

  26. Kvifte, T., Jensenius, A.R.: Towards a coherent terminology and model of instrument description and design. In: Proceedings of the International Conference on New Inferfaces for Musical Expression, Paris, pp. 220–225 (2006)

    Google Scholar 

  27. Libmapper. http://libmapper.github.io/. Accessed 4 July 2014

  28. Magee, W.L., Burland, K.: An exploratory study of the use of electronic music technologies in clinical music therapy. Nord. J. Music Ther. 17(2), 124–141 (2008)

    Article  Google Scholar 

  29. Malloch, J., Sinclair, S., Wanderley, M.M.: Libmapper: (a library for connecting things). In: Extended Abstracts on Human Factors in Computing Systems, Paris, pp. 3087–3090 (2013)

    Google Scholar 

  30. Mamedes, C.R., Wanderley, M.M., Manzolli, J., Garcia, D.H.L.: Strategies for mapping control in interactive audiovisual installations. In: 10th International Symposium on Computer Music Multidisciplinary Research, Marseille, pp. 766–778 (2013)

    Google Scholar 

  31. Marshall, M.T., Wanderley, M.M.: Vibrotactile feedback in digital musical instruments. In: Proceedings of the International Conference on New Inferfaces for Musical Expression, Paris, pp. 226–229 (2006)

    Google Scholar 

  32. Mathews, M.V.: The radio baton and conductor program, or: pitch, the most important and least expressive part of music. Comput. Music J. 15(4), 37–46 (1991)

    Article  Google Scholar 

  33. Moss, W., Cunitz, B.: Haptic theremin: developing a haptic musical controller using the sensable phantom omni. In: Proceedings of the International Computer Music Conference, Barcelona, pp. 275–277 (2005)

    Google Scholar 

  34. Nymoen, K., Skogstad, S.A., Jensenius, A.R.: SoundSaber - a motion capture instrument. In: Proceedings of the International Conference on New Inferfaces for Musical Expression, Oslo, pp. 312–315 (2011)

    Google Scholar 

  35. Oakley, I., McGee, M.R., Brewster, S., Gray, P.: Putting the feel in “look and feel”. In: Proceedings of the Conference on Human Factors in Computing Systems, pp. 415–422, New York (2000)

    Google Scholar 

  36. Okazaki, R., Hachisu, T., Sato, M., Fukushima, S., Hayward, V., Kajimoto, H.: Judged consonance of tactile and auditory frequencies. In: Proceedings of the IEEE World Haptics Conference, pp. 663–666, Daejeon (2013)

    Google Scholar 

  37. O’Modhrain, M.S.: Playing by feel: incorporating haptic feedback into computer-based musical instruments. Ph.D. thesis, Stanford University, Stanford (2001)

    Google Scholar 

  38. Paradiso, J.A., Gershenfeld, N.: Musical applications of electric field sensing. Comput. Music J. 21(2), 69–89 (1997)

    Article  Google Scholar 

  39. Park, G., Choi, S.: Perceptual space of amplitude-modulated vibrotactile stimuli. In: IEEE World Haptics Conference, pp. 59–64 (2011)

    Google Scholar 

  40. Partan, S., Marler, P.: Communication goes multimodal. Science 283(5406), 1272–1273 (1999)

    Article  Google Scholar 

  41. Parts Express: Hiwave tactile actuator. http://www.parts-express.com/pe/showdetl.cfm?partnumber=297-228. Accessed 4 July 2014

  42. Picinali, L., Feakes, C., Mauro, D.A., Katz, B.F.G.: Spectral discrimination thresholds comparing audio and haptics for complex stimuli. In: Magnusson, C., Szymczak, D., Brewster, S. (eds.) HAID 2012. LNCS, vol. 7468, pp. 131–140. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  43. de Quay, Y., Skogstad, S., Jensenius, A.: Dance jockey: performing electronic music by dancing. Leonardo Music J. 21, 11–12 (2011)

    Article  Google Scholar 

  44. Rovan, J., Hayward, V.: Typology of tactile sounds and their synthesis in gesture-driven computer music performance. In: Wanderley, M.M., Battier, M. (eds.) Trends in Gestural Control of Music, pp. 355–368. IRCAM, Centre Pompidou, Paris (2000)

    Google Scholar 

  45. Russo, F.A., Ammirante, P., Fels, D.I.: Vibrotactile discrimination of musical timbre. J. Exp. Psychol. Hum. Percept. Perform. 38(4), 822–826 (2012)

    Article  Google Scholar 

  46. Sachs, D.M.: A forearm controller and tactile display. Master’s thesis, Massachusetts Institute of Technology, Cambridge (2005)

    Google Scholar 

  47. Schacher, J.C.: Gesture control of sounds in 3D space. In: Proceedings of the International Conference on New Inferfaces for Musical Expression, New York, pp. 358–362 (2007)

    Google Scholar 

  48. Sparkfun: Class D mono audio amplifier. https://www.sparkfun.com/products/11044. Accessed 4 July 2014

  49. Sziebig, G., Solvang, B., Kiss, C., Korondi, P.: Vibro-tactile feedback for VR systems. In: 2nd Conference on Human System Interactions, pp. 406–410. IEEE, Catania (2009)

    Google Scholar 

  50. Vallbo, Å.B., Johansson, R.S.: Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Hum. Neurobiol. 3(1), 3–14 (1984)

    Google Scholar 

  51. Verrillo, R.T.: Vibration sensation in humans. Music Percept. 2(3), 281–302 (1992)

    Article  Google Scholar 

  52. Wanderley, M.M., Orio, N.: Evaluation of input devices for musical expression: borrowing tools from HCI. Comput. Music J. 26(3), 62–76 (2002)

    Article  Google Scholar 

  53. WebMapper. http://libmapper.github.io/ecosystem/user_interfaces.html. Accessed 4 July 2014

Download references

Acknowledgments

Marcello Giordano, Clayton Mamedes, Mark Zadel, Joseph Malloch, Stephen Sinclair, Aaron Krajeski, Darryl Cameron and Avrum Hollinger at McGill University for helping out with various technical issues. Also, thanks to the IDMIL students that participated in the evaluation.

Author information

Authors and Affiliations

  1. FourMs Lab, Department of Musicology, University of Oslo, Oslo, Norway

    Håkon Knutzen

  2. IDMIL, CIRMMT, McGill University, Montreal, Canada

    Håkon Knutzen & Marcelo M. Wanderley

  3. Telemark University College, Rauland, Norway

    Tellef Kvifte

Authors
  1. Håkon Knutzen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tellef Kvifte
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcelo M. Wanderley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Håkon Knutzen .

Editor information

Editors and Affiliations

  1. CNRS - LMA, Marseille, France

    Mitsuko Aramaki

  2. Toulon-Var University and CNRS - LMA, Marseille, France

    Olivier Derrien

  3. CNRS - LMA, Marseille, France

    Richard Kronland-Martinet

  4. CNRS - LMA, Marseille, France

    Sølvi Ystad

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Knutzen, H., Kvifte, T., Wanderley, M.M. (2014). Vibrotactile Feedback for an Open Air Music Controller. In: Aramaki, M., Derrien, O., Kronland-Martinet, R., Ystad, S. (eds) Sound, Music, and Motion. CMMR 2013. Lecture Notes in Computer Science(), vol 8905. Springer, Cham. https://doi.org/10.1007/978-3-319-12976-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12976-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12975-4

  • Online ISBN: 978-3-319-12976-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation