Skip to main content
SpringerLink
Log in

Scaffolding the Music Listening and Music Making Experience for the Deaf

  • Chapter
  • First Online:
Assistive Augmentation

Part of the book series: Cognitive Science and Technology ((CSAT))

Abstract

Music is an important part of our daily life. We listen to the radio, enjoy concerts or make music.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Birnbaum DM, Wanderley MM (2007) A systematic approach to musical vibrotactile feedback. In: Proceedings of the international computer music conference (ICMC), pp 397–404

    Google Scholar 

  2. Branje C, Fels DI (2014) Playing vibrotactile music: a comparison between the vibrochord and a piano keyboard. Int J Hum Comput Stud 72(4):431–439. http://doi.org/10.1016/j.ijhcs.2014.01.003

  3. Branje C, Maksimouski M, Karam M, Fels DI, Russo F (2010) Vibrotactile display of music on the human back. In: Proceedings of the 2010 third international conference on advances in computer-human interactions. IEEE Computer Society, pp 154–159. http://doi.org/10.1109/ACHI.2010.40

  4. Chasin M (2003) Music and hearing aids. Hear J 56(7):36. http://doi.org/10.1097/01.HJ.0000292553.60032.c2

  5. Cholewiak RW, McGrath C (2006) Vibrotactile targeting in multimodal systems: accuracy and interaction. In: 2006 14th symposium on haptic interfaces for virtual environment and teleoperator systems. IEEE, pp 413–420

    Google Scholar 

  6. Cook AM, Polgar JM (2014) Assistive technologies: principles and practice. Elsevier Health Sciences

    Google Scholar 

  7. Csikszentmihalyi M (2000) Beyond boredom and anxiety. Jossey-Bass, San Francisco, CA, US

    Google Scholar 

  8. Darrow A-A (1993) The role of music in deaf culture: implications for music educators. J Res Music Educ 41(2):93–110. http://doi.org/10.2307/3345402

  9. Drennan WR, Rubinstein JT (2008) Music perception in cochlear implant users and its relationship with psychophysical capabilities. J Rehabil Res Dev 45(5):779–789

    Article  Google Scholar 

  10. Ferguson S, Moere AV, Cabrera D (2005) Seeing sound: real-time sound visualisation in visual feedback loops used for training musicians. In: Ninth international conference on information visualisation, 2005. Proceedings, pp 97–102. http://doi.org/10.1109/IV.2005.114

  11. Fourney D (2012) Can computer representations of music enhance enjoyment for individuals who are hard of hearing? In: Miesenberger K, Karshmer A, Penaz P, Zagler W (eds) Computers helping people with special needs. Springer, Berlin, pp 535–542. http://doi.org/10.1007/978-3-642-31522-0_80. Accessed 9 Feb 2015

  12. Fourney DW, Fels DI (2009) Creating access to music through visualization. In: Science and technology for humanity (TIC-STH), 2009 IEEE Toronto international conference, pp 939–944. http://doi.org/10.1109/TIC-STH.2009.5444364

  13. Galvin JJ, Q-J Fu, Shannon Robert V (2009) Melodic contour identification and music perception by cochlear implant users. Ann NY Acad Sci 1169(1):518–533. http://doi.org/10.1111/j.1749-6632.2009.04551.x

  14. Gault RH (1926) Touch as a substitute for hearing in the interpretation and control of speech. Arch Otolaryngol Head Neck Surg 3(2):121

    Google Scholar 

  15. Glennie E (1993) Hearing essay. http://www.evelyn.co.uk/literature.html. Accessed 12 Oct 2014

  16. Glennie E (2003) How to truly listen. http://www.ted.com/talks/evelyn_glennie_shows_how_to_listen?language=en. Accessed 4 Dec 2014

  17. Goldstein E (2009) Sensation and perception. Cengage Learning

    Google Scholar 

  18. Hagedorn VS (1992) Musical learning for hearing impaired children. Research perspectives in music education. http://eric.ed.gov/?id=ED375031. Accessed 16 Feb 2015

  19. Hash PM (2003) Teaching instrumental music to deaf and hard of hearing students. Res Issues Music Educ 1(1). http://eric.ed.gov/?id=EJ852403. Accessed 16 Feb 2015

  20. Ho-Ching FWL, Mankoff J, Landay JA (2003) Can you see what i hear?: The design and evaluation of a peripheral sound display for the deaf. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 161–168. http://doi.org/10.1145/642611.642641

  21. Huang K, Starner T, Do E, et al (2010) Mobile music touch: mobile tactile stimulation for passive learning. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 791–800. http://doi.org/10.1145/1753326.1753443

  22. Isaacson EJ (2005) What you see is what you get: on visualizing music. In: ISMIR. Citeseer, pp 389–395

    Google Scholar 

  23. Jackson SA, Marsh HW, et al (1996) Development and validation of a scale to measure optimal experience: the flow state scale. J. Sport Exerc Psychol 18:17–35

    Google Scholar 

  24. Jain D, Findlater L, Gilkeson J, et al (2015) Head-mounted display visualizations to support sound awareness for the deaf and hard of hearing. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, pp 241–250. http://doi.org/10.1145/2702123.2702393

  25. Karam M, Branje C, Nespoli G, Thompson N, Russo FA, Fels DI (2010) The emoti-chair: an interactive tactile music exhibit. In: CHI ’10 extended abstracts on human factors in computing systems. ACM, pp 3069–3074. http://doi.org/10.1145/1753846.1753919

  26. Karam M, Nespoli G, Russo F, Fels DI (2009) Modelling perceptual elements of music in a vibrotactile display for deaf users: a field study. In: Second international conferences on advances in computer-human interactions, 2009. ACHI ’09, pp 249–254. http://doi.org/10.1109/ACHI.2009.64

  27. Karam M, Russo F, Branje C, Price E, Fels DI (2008) Towards a model human cochlea: sensory substitution for crossmodal audio-tactile displays. In: Proceedings of graphics interface 2008. Canadian Information Processing Society, pp 267–274. http://dl.acm.org/citation.cfm?id=1375714.1375759. Accessed 5 Feb 2015

  28. Karam M, Russo FA, Fels DI (2009) Designing the model human cochlea: an ambient crossmodal audio-tactile display. IEEE Trans Haptics 2(3):160–169. http://doi.org/10.1109/TOH.2009.32

  29. Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48(2):373–384. http://doi.org/10.1016/j.neuron.2005.09.018

  30. Kim J, Ananthanarayan S, Yeh T (2015) Seen music: ambient music data visualization for children with hearing impairments. In: Proceedings of the 14th international conference on interaction design and children. ACM, pp 426–429. http://doi.org/10.1145/2771839.2771870

  31. La Versa B, Peruzzi I, Diamanti L, Zemolin M (2014) MUVIB: music and vibration. In: Proceedings of the 2014 ACM international symposium on wearable computers: adjunct program. ACM, pp 65–70. http://doi.org/10.1145/2641248.2641267

  32. Levy-Tzedek S, Hanassy S, Abboud S, Maidenbaum S, Amedi A (2012) Fast, accurate reaching movements with a visual-to-auditory sensory substitution device. Restorative Neurol Neurosci 30(4):313–323

    Google Scholar 

  33. Limb C (2011) Building the musical muscle. http://www.ted.com/talks/charles_limb_building_the_musical_muscle. Accessed 22 Nov 2014

  34. Machover T (2008). Tod Machover + Dan Ellsey: Inventing instruments that unlock new music. http://www.ted.com/talks/tod_machover_and_dan_ellsey_play_new_music?language=en. Accessed 22 Nov 2014

  35. Mahns DA, Perkins NM, Sahai V, Robinson L, Rowe MJ (2006) Vibrotactile frequency discrimination in human hairy skin. J Neurophysiol 95(3):1442–1450

    Article  Google Scholar 

  36. Marshall MT, Wanderley MM (2006) Vibrotactile feedback in digital musical instruments. In: Proceedings of the 2006 conference on new interfaces for musical expression. IRCAM—Centre Pompidou, pp 226–229. http://dl.acm.org/citation.cfm?id=1142215.1142272. Accessed 12 Aug 2015

  37. Matthews T, Fong J, Mankoff J (2005) Visualizing non-speech sounds for the deaf. In: Proceedings of the 7th international ACM SIGACCESS conference on computers and accessibility. ACM, pp 52–59. http://doi.org/10.1145/1090785.1090797

  38. May E (1961) Music for deaf children. Music Educ J 47(3):39–42. http://doi.org/10.2307/3389102

  39. McCarthy Music’s Illuminating Piano (2015) McCarthy music’s illuminating piano: learn to play the piano with McCarthy music. https://www.mccarthypiano.com. Accessed 13 Sep 2015

  40. McCord K, Fitzgerald M (2006) Children with disabilities playing musical instruments. Music Educ J 46–52

    Google Scholar 

  41. Mitroo JB, Herman N, Badler NI (1979) Movies from music: visualizing musical compositions. In: Proceedings of the 6th annual conference on computer graphics and interactive techniques. ACM, pp 218–225. http://doi.org/10.1145/800249.807447

  42. Mori J, Fels DI (2009) Seeing the music can animated lyrics provide access to the emotional content in music for people who are deaf or hard of hearing? In: Science and technology for humanity (TIC-STH), 2009 IEEE Toronto international conference, pp 951–956. http://doi.org/10.1109/TIC-STH.2009.5444362

  43. Music animation machine (2016) Music worth watching. http://www.musanim.com/. Accessed 19 March 2016

  44. Music and the deaf (2015) Enriching lives through music. http://matd.org.uk/. Accessed 1 May 2015

  45. Myles K, Binseel MS (2007) The tactile modality: a review of tactile sensitivity and human tactile interfaces. DTIC Document

    Google Scholar 

  46. Nanayakkara S, Taylor E, Wyse L, Ong SH (2009) An enhanced musical experience for the deaf: design and evaluation of a music display and a haptic chair. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 337–346. http://doi.org/10.1145/1518701.1518756

  47. Novich SD, Eagleman DM (2015) Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput. Exp Brain Res 1–12. http://doi.org/10.1007/s00221-015-4346-1

  48. Palmer R (1997). Feeling the music philosophy: a new approach in understanding how people with sensory impairment perceive and interpret music. http://www.russpalmer.com/feeling.html

  49. Palmer R (2015) Music floors. http://www.russpalmer.com/music.html. Accessed 10 May 2015

  50. Palmer R (2016) Tac-tile sound system. http://www.russpalmer.com/tactile.html. Accessed 20 March 2016

  51. Petry B, Illandara T, Nanayakkara S (2016) MuSS-bits:sensor-display blocks for deaf people to explore musical sounds. In proceedings of the 28th Australian conference on computer-human interaction (OzCHI ’16), 72–80. https://doi.org/10.1145/3010915.3010939

  52. Pouris M, Fels DI (2012) Creating an entertaining and informative music visualization. In: Miesenberger K, Karshmer A, Penaz P, Zagler W (eds) Computers helping people with special needs. Springer, Berlin, pp 451–458. http://doi.org/10.1007/978-3-642-31522-0_68. Accessed 26 Oct 2015

  53. Rashid R, Aitken J, Fels DI (2006) Expressing emotions using animated text captions. In: Miesenberger K, Klaus J, Zagler WL, Karshmer AI (eds) Computers helping people with special needs. Springer, Berlin, pp 24–31. http://doi.org/10.1007/11788713_5. Accessed 15 March 2016

  54. Reed CM, Doherty MJ, Braida LD, Durlach NI (1982) Analytic study of the tadoma method further experiments with inexperienced observers. J Speech Lang Hear Res 25(2):216–223

    Article  Google Scholar 

  55. Roebuck J (2007) I am a deaf opera singer. The Guardian. http://www.theguardian.com/theguardian/2007/sep/29/weekend7.weekend2. Accessed 26 Aug 2015

  56. Schlaug G (2015) Musicians and music making as a model for the study of brain plasticity. In: Finger S, Boller F, Altenmüller E (eds) Progress in brain research. Elsevier, pp 37–55. http://www.sciencedirect.com/science/article/pii/S0079612314000211. Accessed 29 July 2015

  57. Shibata D (2001) Brains of deaf people “hear” music. Int Arts-Med Assoc Newslett 16:4

    Google Scholar 

  58. Shinohara K, Tenenberg J (2009) A blind person’s interactions with technology. Commun ACM 52(8):58–66. http://doi.org/10.1145/1536616.1536636

  59. Stumpf C (1883) Tonpsychologie. Leipzig: Hirzel 1

    Google Scholar 

  60. Tan HZ, Robert Gray J, Young J, Traylor Ryan (2003) A haptic back display for attentional and directional cueing. Haptics-e 3(1):1–20

    Google Scholar 

  61. Verillo RT (1991) Vibration sensing in humans. Music Percept 9(3):281–302

    Google Scholar 

  62. Vy QV, Mori JA, Fourney DW, Fels DI (2008) EnACT: a software tool for creating animated text captions. In: Miesenberger K, Klaus J, Zagler W, Karshmer A (eds) Computers helping people with special needs. Springer, Berlin, pp 609–616. http://doi.org/10.1007/978-3-540-70540-6_87. Accessed 15 March 2016

  63. WHO (2015) Deafness and hearing loss. WHO. http://www.who.int/mediacentre/factsheets/fs300/en/. Accessed 6 Aug 2015

  64. Wyse L, Nanayakkara S, Seekings P, Ong SH, Taylor E (2012) Perception of vibrotactile stimuli above 1 kHz by the hearingimpaired. In: NIME’12

    Google Scholar 

  65. Yang H-J, Lay Y-L, Liou Y-C, Tsao W-Y, Lin C-K (2007) Development and evaluation of computer-aided music-learning system for the hearing impaired. J Comput Assist Learn 23(6):466–476. http://doi.org/10.1111/j.1365-2729.2007.00229.x

  66. Zatorre RJ, Chen JL, Penhune VB (2007) When the brain plays music: auditory–motor interactions in music perception and production. Nat Rev Neurosci 8(7):547–558. http://doi.org/10.1038/nrn2152

  67. Zhou Y, Sim KC, Tan P, Wang Y (2012) MOGAT: mobile games with auditory training for children with cochlear implants. In: Proceedings of the 20th ACM international conference on multimedia. ACM, pp 429–438. http://doi.org/10.1145/2393347.2393409

Download references

Author information

Authors and Affiliations

  1. Augmented Human Lab, Singapore University of Technology and Design, Singapore, Singapore

    Benjamin Petry & Suranga Nanayakkara

  2. Synaptics, Zug, Switzerland

    Jochen Huber

Authors
  1. Benjamin Petry
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jochen Huber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Suranga Nanayakkara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Petry .

Editor information

Editors and Affiliations

  1. Synaptics, Zug, Switzerland

    Jochen Huber

  2. Department of Computer Science, Stony Brook University, New York, New York, USA

    Roy Shilkrot

  3. Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts, USA

    Pattie Maes

  4. Augmented Human Lab, Singapore University of Technology and Design, Singapore, Singapore

    Suranga Nanayakkara

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Petry, B., Huber, J., Nanayakkara, S. (2018). Scaffolding the Music Listening and Music Making Experience for the Deaf. In: Huber, J., Shilkrot, R., Maes, P., Nanayakkara, S. (eds) Assistive Augmentation. Cognitive Science and Technology. Springer, Singapore. https://doi.org/10.1007/978-981-10-6404-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-6404-3_3

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-6402-9

  • Online ISBN: 978-981-10-6404-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation