Evaluation of an English Word Look-Up Tool for Web-Browsing with Sign Language Video for Deaf Readers

  • Dhananjai Hariharan
  • Sedeeq Al-khazraji
  • Matt HuenerfauthEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10907)


Research has shown that some people who are Deaf or Hard of Hearing (DHH) in the U.S. have lower levels of English language literacy than their hearing peers, which creates a barrier to access web content for these users. We have designed an interface to assist these users in reading English text on web pages; users can click on certain marked words to view an ASL sign video in a pop-up. A user study was conducted to evaluate this tool and compare it with web pages containing only text, as well as pages where users can click on words and see text-definitions using the Google Dictionary plug-in for browsers. The study assessed participants’ subjective preference for these conditions and compared their performance in completing reading comprehension tasks with each of these tools. We found that participants preferred having support tools in their interface as opposed to none, but we did not measure a significant difference in their preferences between the two support tools provided. This paper presents the details of design and development of this proposed tool, design guidelines applied to the prototype, factors influencing the results, and directions for future work.


American Sign Language Reading support tools Deaf users 


  1. 1.
    Adamo-Villani, N., Doublestein, J., Martin, Z.: Sign language for K-8 mathematics by 3D interactive animation. J. Educ. Technol. Syst. 33(3), 241–257 (2005)CrossRefGoogle Scholar
  2. 2.
    Bangham, J.A., Cox, S.J., Lincoln, M., Marshall, I., Tutt, M., Wells, M.: Signing for the deaf using virtual humans. In: IEEE Seminar on Speech and Language Processing for Disabled and Elderly People (Ref. No. 2000/025), pp. 4–1. IET (2000)Google Scholar
  3. 3.
    Beggs, W.D.A., Breslaw, P.I.: Reading retardation or linguistic deficit? III: a further examination of response strategies in a reading test completed by hearing impaired children. J. Res. Reading 6(1), 19–28 (1983)CrossRefGoogle Scholar
  4. 4.
    Bilal, D., Kirby, J.: Differences and similarities in information seeking: children and adults as Web users. Inf. Process. Manage. 38(5), 649–670 (2002)CrossRefGoogle Scholar
  5. 5.
    Bottoni, P., Borgia, F., Buccarella, D., Capuano, D., De Marsico, M., Labella, A.: Stories and signs in an e-learning environment for deaf people. Univ. Access Inf. Soc. 12(4), 369–386 (2013)CrossRefGoogle Scholar
  6. 6.
    Debevc, M., Kosec, P., Holzinger, A.: Improving multimodal web accessibility for deaf people: sign language interpreter module. Multimedia Tools Appl. 54(1), 181–199 (2011)CrossRefGoogle Scholar
  7. 7.
    Debevc, M., Stjepanovič, Z., Povalej, P., Verlič, M., Kokol, P.: Accessible and adaptive e-learning materials: considerations for design and development. In: Stephanidis, C. (ed.) UAHCI 2007. LNCS, vol. 4556, pp. 549–558. Springer, Heidelberg (2007). Scholar
  8. 8.
    Elliott, R., Glauert, J.R., Kennaway, J.R., Marshall, I., Safar, E.: Linguistic modelling and language-processing technologies for avatar-based sign language presentation. Univ. Access Inf. Soc. 6(4), 375–391 (2008)CrossRefGoogle Scholar
  9. 9.
    Fajardo, I., Caas, J.J., Salmern, L., Abascal, J.: Improving deaf users’ accessibility in hypertext information retrieval: are graphical interfaces useful for them? Behav. Inf. Technol. 25(6), 455–467 (2006)CrossRefGoogle Scholar
  10. 10.
    Fajardo, I., Parra, E., Caas, J.J.: Do sign language videos improve web navigation for deaf signer users? J. Deaf Stud. Deaf Educ. 15(3), 242–262 (2010)CrossRefGoogle Scholar
  11. 11.
    Fajardo, I., Vigo, M., Salmern, L.: Technology for supporting web information search and learning in sign language. Interact. Comput. 21(4), 243–256 (2009)CrossRefGoogle Scholar
  12. 12.
    Fels, D.I., Richards, J., Hardman, J., Lee, D.G.: Sign language web pages. Am. Ann. Deaf 151(4), 423–433 (2006)CrossRefGoogle Scholar
  13. 13.
    Gentry, M.M., Chinn, K.M., Moulton, R.D.: Effectiveness of multimedia reading materials when used with children who are deaf. Am. Ann. Deaf 149(5), 394–403 (2004)CrossRefGoogle Scholar
  14. 14.
    Hanson, V.L.: Computing technologies for deaf and hard of hearing users. In: Human-Computer Interaction: Designing for Diverse Users and Domains, p. 125 (2009)Google Scholar
  15. 15.
    Hanson, V.L., Padden, C.: Handson: a multi-media program for bilingual language instruction of deaf children. In: Proceedings of the Johns Hopkins National Search for Computing Applications to Assist Persons with Disabilities, pp. 5–6. IEEE (1992)Google Scholar
  16. 16.
    Hilzensauer, M.: Information technology for deaf people. In: Ichalkaranje, N., Ichalkaranje, A., Jain, L. (eds.) Intelligent Paradigms for Assistive and Preventive Healthcare, pp. 183–206. Springer, Heidelberg (2006). Scholar
  17. 17.
    Huenerfauth, M.: Generating American sign language animation: overcoming misconceptions and technical challenges. Univ. Access Inf. Soc. 6(4), 419–434 (2008). Scholar
  18. 18.
    Huenerfauth, M., Hanson, V.L.: Sign language in the interface: access for deaf signers. In: Universal Access. Erlbaum, Handbook (2009). 38Google Scholar
  19. 19.
    Huenerfauth, M.S.: Planning models of ASL classifier predicates for machine translation. In: Proceedings of the 10th International Conference on Theoretical and Issues in Machine Translation (TMI 2004), Baltimore, MD, USA (2004).
  20. 20.
    Huenerfauth, M., Kacorri, H.: Release of experimental stimuli and questions for evaluating facial expressions in animations of American sign language. In: Proceedings of the 6th Workshop on the Representation and Processing of Sign Languages: Beyond the Manual Channel, The 9th International Conference on Language Resources and Evaluation (LREC 2014), Reykjavik, Iceland (2014)Google Scholar
  21. 21.
    Huenerfauth, M., Lu, P.: Modeling and synthesizing spatially inflected verbs for American sign language animations. In: Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2010), pp. 99–106. ACM, New York (2010).
  22. 22.
    Huenerfauth, M.: Spatial and planning models of ASL classifier predicates for machine translation. In: Proceedings of the 10th International Conference on Theoretical and Issues in Machine Translation (TMI 2004), Baltimore, MD, USA (2004).
  23. 23.
    Jones, M.D., Hamilton, H., Petmecky, J.: Mobile phone access to a sign language dictionary. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, pp. 331–332. ACM (2015)Google Scholar
  24. 24.
    Kelly, L.P.: Considerations for designing practice for deaf readers. J. Deaf Stud. Deaf Educ. 8(2), 171–186 (2003)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Kennaway, J.R., Glauert, J.R., Zwitserlood, I.: Providing signed content on the internet by synthesized animation. ACM Trans. Comput.-Hum. Interact. (TOCHI) 14(3), 15 (2007)CrossRefGoogle Scholar
  26. 26.
    Lin, F.R., Niparko, J.K., Ferrucci, L.: Hearing loss prevalence in the United States. Arch. Intern. Med. 171(20), 1851–1853 (2011)CrossRefGoogle Scholar
  27. 27.
    Luckner, J.L., Handley, C.M.: A summary of the reading comprehension research undertaken with students who are deaf or hard of hearing. Am. Ann. Deaf 153(1), 6–36 (2008)CrossRefGoogle Scholar
  28. 28.
    Marschark, M., Harris, M.: Success and failure in learning to read: the special case (?) of deaf children. In: Reading Comprehension Difficulties: Processes and Intervention, pp. 279–300 (1996)Google Scholar
  29. 29.
    Mayer, R.E.: Principles for reducing extraneous processing in multimedia learning: coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In: The Cambridge Handbook of Multimedia Learning, pp. 183–200 (2005)Google Scholar
  30. 30.
    Mitchell, R.E., Young, T.A., Bachleda, B., Karchmer, M.A.: How many people use ASL in the United States? Why estimates need updating. Sign Lang. Stud. 6(3), 306–335 (2006)CrossRefGoogle Scholar
  31. 31.
    Paul, P.V.: Literacy and deafness: the development of reading, writing, and literate thought. Pearson College Division (1998)Google Scholar
  32. 32.
    Petrie, H., Fisher, W., Weimann, K., Weber, G.: Augmenting icons for deaf computer users. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, pp. 1131–1134. ACM (2004)Google Scholar
  33. 33.
    Reis, J., Solovey, E.T., Henner, J., Johnson, K., Hoffmeister, R.: ASL CLeaR: STEM education tools for deaf students. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, pp. 441–442. ACM (2015)Google Scholar
  34. 34.
    Richards, J., Fels, D., Hardman, J.: The educational potential of the signing web. In: Instructional Technology and Education of the Deaf Symposium (2005)Google Scholar
  35. 35.
    Smith, C.E.: Where is it? How deaf adolescents complete fact- based Internet search tasks. Am. Ann. Deaf 151(5), 519–529 (2006)CrossRefGoogle Scholar
  36. 36.
    Straetz, K., Kaibel, A., Raithel, V., Specht, M., Grote, K., Kramer, F.: An e-learning environment for deaf adults. In: Conference Proceedings 8th ERCIM Workshop User Interfaces for All (2004)Google Scholar
  37. 37.
    Strassman, B.K.: Metacognition and reading in children who are deaf: a review of the research. J. Deaf Stud. Deaf Educ. 2, 140–149 (1997)CrossRefGoogle Scholar
  38. 38.
    Sweller, J.: The redundancy principle in multimedia learning. In: Mayer, R.E. (ed.) The Cambridge Handbook of Multimedia Learning, pp. 183–200. Cambridge University Press, Cambridge (2005)Google Scholar
  39. 39.
    Traxler, C.B.: The Stanford achievement test: national norming and performance standards for deaf and hard-of-hearing students. J. Deaf Stud. Deaf Educ. 5(4), 337–348 (2000)CrossRefGoogle Scholar
  40. 40.
    Webster, A., Wood, D.J., Griffiths, A.J.: Reading retardation or linguistic deficit? I: interpreting reading test performances of hearing impaired adolescents. J. Res. Read. 4(2), 136–147 (1981)CrossRefGoogle Scholar
  41. 41.
    Wood, D.: Teaching and Talking with Deaf Children. Wiley, New York (1986)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Dhananjai Hariharan
    • 1
  • Sedeeq Al-khazraji
    • 1
  • Matt Huenerfauth
    • 1
    Email author
  1. 1.Rochester Institute of TechnologyRochesterUSA

Personalised recommendations