Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 17, pp 22007–22031 | Cite as

Enssat: wearable technology application for the deaf and hard of hearing

  • Shurug Alkhalifa
  • Muna Al-Razgan
Article
  • 302 Downloads

Abstract

A large number of people around the globe experience hearing difficulties of varying degrees. Therefore, recent research has focused on developing applications and systems to aid the Deaf and Hard of Hearing (DHH). Wearable devices have significant potential in assistive applications owing to their low cost and lightweight. Wearable devices help the DHH perform their daily activities easily without requiring assistance from others. We present Enssat, a bilingual (Arabic/English) smartphone-based hearing aid application that uses Google Glass to assist DHH individuals. The application performs important tasks: real-time transcription, real-time translation, and alert management. A user study was performed to evaluate the application. In this study, 10 DHH users evaluated the processes of transcription, translation, and alerting and 15 non-DHH users evaluated the translation function of the system. All of them evaluated the usability and effectiveness of the Enssat application. The results demonstrated the ease of use and utility of the application.

Keywords

Hard of hearing Deaf Assistive technology Wearable computing Head-mounted Speech-to-text Translation Transcription 

Notes

Acknowledgements

This research project was supported by a grant from the Research Center of the Female Scientific and Medical Colleges, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia. We would also like to specially thank the following students: Amal Alfawaz, Amal Alnaqeeb, Ameera Algabr, Ghada Alyousif, and Yosra Aldoeis for their contributions to the implementation and evaluation of the system.

References

  1. 1.
    Barrett K (2013) Is Google glass useful for the deaf?. Keith Barrett Online. http://keithbarrett.com/blog/is-google-glass-useful-for-the-deaf/. Accessed 30 Jan 2016
  2. 2.
    Bianchini CS, Borgia F, Bottoni P, De Marsico M (2012) SWift: a SignWriting improved fast transcriber. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, p 390–393. ACMGoogle Scholar
  3. 3.
    Bragg D, Huynh N, Ladner RE. A personalizable mobile sound detector app design for deaf and hard-of-hearing users. In: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, p 3–13. ACMGoogle Scholar
  4. 4.
    ClearCaptions - Android Apps on Google Play (2015) Rocklin, CA 95765: purple communications. https://play.google.com/store/apps/details?id=us.purple.cc. Accessed 18 Feb 2016
  5. 5.
    Dragon Dictation on the App Store. App Store. https://itunes.apple.com/US/app/id341446764?mt=8. Accessed 17 Feb 2016
  6. 6.
    Emotional Factors of HOH. Hard of Hearing Advocates -- Solutions for Better Hearing. http://www.hohadvocates.org/index.php?contenttype=Feature&target=home&subtarget=feature&selected=19. Accessed 8 June 2016
  7. 7.
    Glass. Google Developers. https://developers.google.com/glass/. Accessed 28 July 2015
  8. 8.
    Google Glass Could Help The Deaf Visualize Nearby Sounds (2013) International business times. http://www.ibtimes.com/google-glass-could-help-deaf-visualize-nearby-sounds-1314639. Accessed 20 Feb 2016
  9. 9.
    Google goggles - Android Apps on Google Play (2014). https://play.google.com/store/apps/details?id=com.google.android.apps.unveil. Accessed: 20 Feb 2016
  10. 10.
    Google Inc. (2015) Google translate. Android Apps on Google Play. Google translate. https://play.google.com/store/apps/details?id=com.google.android.apps.translate&hl=en. Accessed: 18 Feb 2016
  11. 11.
    Google Speech To Text API. Gist. https://gist.github.com/alotaiba/1730160. Accessed 22 Feb 2016
  12. 12.
    Group of researchers from Georgia Institute of Technology (2014) Captioning on Glass “COG.” Georgia Institute of Technology: https://play.google.com/store/apps/details?id=eman.y.deafanddumb. Accessed: 17 Feb 2016
  13. 13.
    Gugenheimer J, Plaumann K, Schaub F, San Vito PD, Duck S, Rabus M, Rukzio E (2017) The impact of assistive technology on communication quality between deaf and hearing individuals. In: CSCW, p 669–682Google Scholar
  14. 14.
    Introducing Computer Speech Technology. https://msdn.microsoft.com/en-us/library/ms870025.aspx. Accessed 22 Feb 2016
  15. 15.
    Jain D, Findlater L, Gilkeson J, Holland B, Duraiswami R, Zotkin D, Vogler C, Froehlich JE (2015) Head-mounted display visualizations to support sound awareness for the deaf and hard of hearing. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, New York, NY, USA, p 241–250Google Scholar
  16. 16.
    Jain D, Findlater L, Gilkeson J, Holland B, Duraiswami R, Zotkin D, Vogler C, Froehlich JE (2015) Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, p 241–250. ACMGoogle Scholar
  17. 17.
    Jones M, Lawler MJ, Hintz E, Bench N, Mangrubang F, Trullender M (2014) Head mounted displays and deaf children: facilitating sign language in challenging learning environments. In: Proceedings of the 2014 conference on interaction design and children, New York, NY, USA, p 317–320Google Scholar
  18. 18.
    Jones MD, Hamilton H, Petmecky J (2015) Mobile phone access to a sign language dictionary. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, New York, NY, USA, p 331–332Google Scholar
  19. 19.
    Lasecki W, Bigham JP (2014) Real-time captioning with the crowd. http://interactions.acm.org/archive/view/may-june-2014/real-time-captioning-with-the-crowd. Accessed 2 Feb 2016
  20. 20.
    LingoCam: Real-Time Translator & Dictionary on the App Store. App Store. https://itunes.apple.com/us/app/lingocam-real-time-translator/id848119884?mt=8. Accessed 20 Feb 2016
  21. 21.
    LiveCaption, Live Caption (2010) Android apps on Google play. Brooklyn, NY, 2010. https://play.google.com/store/apps/details?id=appinventor.ai_ryflydesign.LiveCaption. Accessed 17 Feb 2016
  22. 22.
    Menasy M (2013) Arab deaf sign interpreter. Android Apps on Google Play. https://play.google.com/store/apps/details?id=com.menasy.menasytranslator&hl=en. Accessed 18 Feb 2016
  23. 23.
    Mielke M, Brück R (2015) A pilot study about the smartwatch as assistive device for deaf people. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, p 301–302. ACMGoogle Scholar
  24. 24.
    musicg - Lightweight Java API for audio analysing, Android compatible - Google Project Hosting. https://code.google.com/p/musicg/. Accessed 22 Feb 2016
  25. 25.
    Nguyen V, Gruteser M (2015) First experiences with GOOGLE GLASS in mobile research. Mob Comput Commun 18(4):44–47CrossRefGoogle Scholar
  26. 26.
    OpticText: Text OCR Scanner + Offline Translator on the App Store. App Store. https://itunes.apple.com/tr/app/optictext-text-ocr-scanner/id689401436?mt=8. Accessed 20 Feb 2016
  27. 27.
    Pairing Glass to your Bluetooth phone. https://support.google.com/glass/answer/3064189?hl=en. Accessed 10 Oct 2015
  28. 28.
    Portable Wi-Fi that goes with you anywhere. http://www.wi-fi.org/discover-wi-fi/wi-fi-direct. Accessed 22 Feb 2016
  29. 29.
    Profita H, Albaghli R, Findlater L, Jaeger P, Kane SK (2016) The AT effect: how disability affects the perceived social acceptability of head-mounted display use. In: proceedings of the 2016 CHI conference on human factors in computing systems, p 4884-489. ACMGoogle Scholar
  30. 30.
    Sapargaliyev D (2015) Learning with wearable technologies: a case of Google glass. In: Brown TH, van der Merwe HJ (eds) The mobile learning voyage - from small ripples to massive open waters. Springer international publishing, Cham, pp 343–350CrossRefGoogle Scholar
  31. 31.
    Sicong L, Zimu Z, Junzhao D, Longfei S, Han J, Wang X (2017) UbiEar: bringing location-independent sound awareness to the hard-of-hearing people with smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1(2):17CrossRefGoogle Scholar
  32. 32.
    Sommerville I (2010) Software engineering, 9 Edition. Pearson, BostonGoogle Scholar
  33. 33.
    Stinson MS, Francis P, Elliot LB, Easton D (2014) Real-time Caption Challenge: C-print. In: Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility, New York, NY, USA, p 317–318Google Scholar
  34. 34.
    Tesseract (software) (2015) Wikipedia, the free encyclopediaGoogle Scholar
  35. 35.
    The Deaf And Hearing Impaired-Android Apps on Google Play. MonCapitan (2014) https://play.google.com/store/apps/details?id=kr.ac.kaist.isilab.doyeob.humanityProject2 Accessed 22 Feb 2016
  36. 36.
    Vernon M, Greenberg SF (1999) NCJRS abstract - national criminal justice reference service. Aggress Violent Behav 4(3):259–272CrossRefGoogle Scholar
  37. 37.
    Vondracek D (2016) Tap tap on the app store. App Store. https://itunes.apple.com/us/app/tap-tap/id369747181?mt=8. Accessed 18 Feb 2016
  38. 38.
    WHO | Deafness and hearing loss. WHO. http://www.who.int/mediacentre/factsheets/fs300/en/. Accessed 21 Feb 2016
  39. 39.
    Yassin E (2015) Deaf Application - Android Apps on Google Play. https://play.google.com/store/apps/details?id=eman.y.deafanddumb. Accessed 22 Feb 2016

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Information Technology Department, College of Computer and Information SciencesKing Saud UniversityRiyadhSaudi Arabia

Personalised recommendations