Abstract
The hearing-impaired experience the sound localization problem, defined as the insufficient ability to estimate the direction of sounds. This problem reduces spatial awareness and amplifies social difficulties such as communicating in adverse listening conditions. Current solutions, such as hearing aids and cochlear implants, fail to solve the problem for a multitude of reasons, but recent work in head-computer interfaces can be applied to deliver a robust solution. Thus, this paper develops a smart glass, named EyeHear, that shows the direction of a speech source and its transcription, all in real-time thanks to a modified speech recognition algorithm and speech recognition system. Algorithm testing proved that the device was accurate enough to be feasible in real-world talking situations. User testing affirms that the target audience favors head-mounted displays and indicates that solving the sound localization problem is beneficial. This device paves the way for future work to extend the device to non-speech sounds. EyeHear has the potential to become an assistive technology for the hearing impaired.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Middlebrooks, J., Green, D.: Sound localization by human listeners. Ann. Rev. Psychol. 42, 135–159 (1991)
Jeffress, L.: A place theory on sound localization. J. Comp. Physiol. Psychol. 41(1), 35–39 (1948)
Musicant, A., Butler, R.: The influence of pinnae-based spectral cues on sound localization. J. Acoust. Soc. Am. 75(4), 1195–1200 (1984)
Abel, S., Hay, V.: Sound localization the interaction of aging, hearing loss and hearing protection. Scand. Audiol. 25(1), 3–12 (1996)
Noble, W., Byrne, D., Lepage, B.: Effects on sound localization of configuration and type of hearing impaired. J. Acoust. Soc. Am. 95(2), 992–1005 (1994)
Lorenzi, C., Gatehouse, S., Lever, C.: Sound localization in noise in hearing-impaired listeners. J. Acoust. Soc. Am. 105(6), 3454–3463 (1999)
Kuk, F., Korhonen, P.: Localization 101: hearing aid factors in localization. Hear. Rev. 21(9), 26–33 (2014)
Feinkohl, A.: Psychophysical experiments on sound localization in starlings and humans. Carl von Ossietzky Universität Oldenberg)
Lesica, N.: Why do hearing aids fail to restore normal auditory perception? Trends Neurosci. 41(4), 174–185 (2018)
Cochlear Implants. https://nidcd.nih.gov/health/cochlear-implants. Accessed 1 June 2020
Grieco-Calub, T., Litovsky, R.: Spatial acuity in 2-to-3-year-old children with normal acoustic hearing, unilateral cochlear implants, and bilateral cochlear implants. Ear Hear. 33(5), 561–572 (2012)
Mueller, M., Meisenbacher, K., Lai, W., Dillier, N.: Sound localization with bilateral cochlear implants in noise: how much do head movements contribute to localization? Cochlear Implants Int. 15(1), 36–42 (2014)
Cochlear Implant Cost. www.babyhearing.org/devices/cochlear-implant-cost. Accessed 3 June 2020
McPherson, B.: Innovative technology in hearing instruments: matching needs in the developing world. Trends Amplif. 15(4), 209–214 (2011)
Tucker, B.: Deaf culture, cochlear implants, and elective disability. Hastings Cent. Rep. 28(4), 6–14 (1998)
Darcy, K.: Personal Interview, taken on 2019/1/18
American Sign Language. https://www.nidcd.nih.gov/health/american-sign-language. Accessed 10 June 2020
Mitchell, R., Young, T., Bachleda, B., Karchmer, M.: How many people use ASL in the United States? Why estimates need updating. Sign Lang. Stud. 6(3), 306–335 (2006)
Assistive Devices for People with Hearing, Voice, Speech, or Language Disorders. https://www.nidcd.nih.gov/health/assistive-devices-people-hearing-voice-speech-or-language-disorders. Accessed 4 June 2020
Matthias vom Stein, A., Günthner, W.: Work-by-Inclusion - Inclusive information system for picking using smart glasses. ZWF Magazine for Economical Factory Operation, vol. 112, no. 10, pp. 670–674 (2017)
Ho-Ching, F., Mankoff, J., Landay, A.: Can you see what I hear? The design and evaluation of peripheral sound display for the deaf. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 161–168. Association for Computing Machinery, New York (2003)
Jain, D., et al.: Head-mounted display visualizations to support sound awareness for the deaf and hard of hearing. In: 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI 2015), pp. 241–250. Association for Computing machinery, New York (2015)
Acknowledgements
The authors thank Dr. Sam Sennott, Dr. Martin Siderius, and Darcy Kramer for their support and advice. The authors thank the THINK science research program at the Massachusetts Institute of Technology for their funding and mentorship. The authors thank faculty at the Oregon School for the Deaf, and hearing-impaired residents of the Hillsboro community. The authors thank Benjamin Marquez. Lastly, the authors thank the dear friends and family who are members of the hearing-impaired population, who inspired the authors to pursue this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Sinha, I., Caverly, O. (2020). EyeHear: Smart Glasses for the Hearing Impaired. In: Stephanidis, C., Antona, M., Gao, Q., Zhou, J. (eds) HCI International 2020 – Late Breaking Papers: Universal Access and Inclusive Design. HCII 2020. Lecture Notes in Computer Science(), vol 12426. Springer, Cham. https://doi.org/10.1007/978-3-030-60149-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-60149-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60148-5
Online ISBN: 978-3-030-60149-2
eBook Packages: Computer ScienceComputer Science (R0)