Skip to main content
SpringerLink
Log in

Enssat: wearable technology application for the deaf and hard of hearing

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A large number of people around the globe experience hearing difficulties of varying degrees. Therefore, recent research has focused on developing applications and systems to aid the Deaf and Hard of Hearing (DHH). Wearable devices have significant potential in assistive applications owing to their low cost and lightweight. Wearable devices help the DHH perform their daily activities easily without requiring assistance from others. We present Enssat, a bilingual (Arabic/English) smartphone-based hearing aid application that uses Google Glass to assist DHH individuals. The application performs important tasks: real-time transcription, real-time translation, and alert management. A user study was performed to evaluate the application. In this study, 10 DHH users evaluated the processes of transcription, translation, and alerting and 15 non-DHH users evaluated the translation function of the system. All of them evaluated the usability and effectiveness of the Enssat application. The results demonstrated the ease of use and utility of the application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Barrett K (2013) Is Google glass useful for the deaf?. Keith Barrett Online. http://keithbarrett.com/blog/is-google-glass-useful-for-the-deaf/. Accessed 30 Jan 2016

  2. Bianchini CS, Borgia F, Bottoni P, De Marsico M (2012) SWift: a SignWriting improved fast transcriber. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, p 390–393. ACM

  3. Bragg D, Huynh N, Ladner RE. A personalizable mobile sound detector app design for deaf and hard-of-hearing users. In: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, p 3–13. ACM

  4. ClearCaptions - Android Apps on Google Play (2015) Rocklin, CA 95765: purple communications. https://play.google.com/store/apps/details?id=us.purple.cc. Accessed 18 Feb 2016

  5. Dragon Dictation on the App Store. App Store. https://itunes.apple.com/US/app/id341446764?mt=8. Accessed 17 Feb 2016

  6. Emotional Factors of HOH. Hard of Hearing Advocates -- Solutions for Better Hearing. http://www.hohadvocates.org/index.php?contenttype=Feature&target=home&subtarget=feature&selected=19. Accessed 8 June 2016

  7. Glass. Google Developers. https://developers.google.com/glass/. Accessed 28 July 2015

  8. Google Glass Could Help The Deaf Visualize Nearby Sounds (2013) International business times. http://www.ibtimes.com/google-glass-could-help-deaf-visualize-nearby-sounds-1314639. Accessed 20 Feb 2016

  9. Google goggles - Android Apps on Google Play (2014). https://play.google.com/store/apps/details?id=com.google.android.apps.unveil. Accessed: 20 Feb 2016

  10. Google Inc. (2015) Google translate. Android Apps on Google Play. Google translate. https://play.google.com/store/apps/details?id=com.google.android.apps.translate&hl=en. Accessed: 18 Feb 2016

  11. Google Speech To Text API. Gist. https://gist.github.com/alotaiba/1730160. Accessed 22 Feb 2016

  12. Group of researchers from Georgia Institute of Technology (2014) Captioning on Glass “COG.” Georgia Institute of Technology: https://play.google.com/store/apps/details?id=eman.y.deafanddumb. Accessed: 17 Feb 2016

  13. Gugenheimer J, Plaumann K, Schaub F, San Vito PD, Duck S, Rabus M, Rukzio E (2017) The impact of assistive technology on communication quality between deaf and hearing individuals. In: CSCW, p 669–682

  14. Introducing Computer Speech Technology. https://msdn.microsoft.com/en-us/library/ms870025.aspx. Accessed 22 Feb 2016

  15. Jain D, Findlater L, Gilkeson J, Holland B, Duraiswami R, Zotkin D, Vogler C, Froehlich JE (2015) Head-mounted display visualizations to support sound awareness for the deaf and hard of hearing. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, New York, NY, USA, p 241–250

  16. Jain D, Findlater L, Gilkeson J, Holland B, Duraiswami R, Zotkin D, Vogler C, Froehlich JE (2015) Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, p 241–250. ACM

  17. Jones M, Lawler MJ, Hintz E, Bench N, Mangrubang F, Trullender M (2014) Head mounted displays and deaf children: facilitating sign language in challenging learning environments. In: Proceedings of the 2014 conference on interaction design and children, New York, NY, USA, p 317–320

  18. Jones MD, Hamilton H, Petmecky J (2015) Mobile phone access to a sign language dictionary. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, New York, NY, USA, p 331–332

  19. Lasecki W, Bigham JP (2014) Real-time captioning with the crowd. http://interactions.acm.org/archive/view/may-june-2014/real-time-captioning-with-the-crowd. Accessed 2 Feb 2016

  20. LingoCam: Real-Time Translator & Dictionary on the App Store. App Store. https://itunes.apple.com/us/app/lingocam-real-time-translator/id848119884?mt=8. Accessed 20 Feb 2016

  21. LiveCaption, Live Caption (2010) Android apps on Google play. Brooklyn, NY, 2010. https://play.google.com/store/apps/details?id=appinventor.ai_ryflydesign.LiveCaption. Accessed 17 Feb 2016

  22. Menasy M (2013) Arab deaf sign interpreter. Android Apps on Google Play. https://play.google.com/store/apps/details?id=com.menasy.menasytranslator&hl=en. Accessed 18 Feb 2016

  23. Mielke M, Brück R (2015) A pilot study about the smartwatch as assistive device for deaf people. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, p 301–302. ACM

  24. musicg - Lightweight Java API for audio analysing, Android compatible - Google Project Hosting. https://code.google.com/p/musicg/. Accessed 22 Feb 2016

  25. Nguyen V, Gruteser M (2015) First experiences with GOOGLE GLASS in mobile research. Mob Comput Commun 18(4):44–47

    Article  Google Scholar 

  26. OpticText: Text OCR Scanner + Offline Translator on the App Store. App Store. https://itunes.apple.com/tr/app/optictext-text-ocr-scanner/id689401436?mt=8. Accessed 20 Feb 2016

  27. Pairing Glass to your Bluetooth phone. https://support.google.com/glass/answer/3064189?hl=en. Accessed 10 Oct 2015

  28. Portable Wi-Fi that goes with you anywhere. http://www.wi-fi.org/discover-wi-fi/wi-fi-direct. Accessed 22 Feb 2016

  29. Profita H, Albaghli R, Findlater L, Jaeger P, Kane SK (2016) The AT effect: how disability affects the perceived social acceptability of head-mounted display use. In: proceedings of the 2016 CHI conference on human factors in computing systems, p 4884-489. ACM

  30. Sapargaliyev D (2015) Learning with wearable technologies: a case of Google glass. In: Brown TH, van der Merwe HJ (eds) The mobile learning voyage - from small ripples to massive open waters. Springer international publishing, Cham, pp 343–350

    Chapter  Google Scholar 

  31. Sicong L, Zimu Z, Junzhao D, Longfei S, Han J, Wang X (2017) UbiEar: bringing location-independent sound awareness to the hard-of-hearing people with smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1(2):17

    Article  Google Scholar 

  32. Sommerville I (2010) Software engineering, 9 Edition. Pearson, Boston

  33. Stinson MS, Francis P, Elliot LB, Easton D (2014) Real-time Caption Challenge: C-print. In: Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility, New York, NY, USA, p 317–318

  34. Tesseract (software) (2015) Wikipedia, the free encyclopedia

  35. The Deaf And Hearing Impaired-Android Apps on Google Play. MonCapitan (2014) https://play.google.com/store/apps/details?id=kr.ac.kaist.isilab.doyeob.humanityProject2 Accessed 22 Feb 2016

  36. Vernon M, Greenberg SF (1999) NCJRS abstract - national criminal justice reference service. Aggress Violent Behav 4(3):259–272

    Article  Google Scholar 

  37. Vondracek D (2016) Tap tap on the app store. App Store. https://itunes.apple.com/us/app/tap-tap/id369747181?mt=8. Accessed 18 Feb 2016

  38. WHO | Deafness and hearing loss. WHO. http://www.who.int/mediacentre/factsheets/fs300/en/. Accessed 21 Feb 2016

  39. Yassin E (2015) Deaf Application - Android Apps on Google Play. https://play.google.com/store/apps/details?id=eman.y.deafanddumb. Accessed 22 Feb 2016

Download references

Acknowledgements

This research project was supported by a grant from the Research Center of the Female Scientific and Medical Colleges, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia. We would also like to specially thank the following students: Amal Alfawaz, Amal Alnaqeeb, Ameera Algabr, Ghada Alyousif, and Yosra Aldoeis for their contributions to the implementation and evaluation of the system.

Author information

Authors and Affiliations

  1. Information Technology Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia

    Shurug Alkhalifa & Muna Al-Razgan

Authors
  1. Shurug Alkhalifa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muna Al-Razgan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shurug Alkhalifa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alkhalifa, S., Al-Razgan, M. Enssat: wearable technology application for the deaf and hard of hearing. Multimed Tools Appl 77, 22007–22031 (2018). https://doi.org/10.1007/s11042-018-5860-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5860-5

Keywords

Search

Navigation