Measuring the Perception of Facial Expressions in American Sign Language Animations with Eye Tracking
Our lab has conducted experimental evaluations of ASL animations, which can increase accessibility of information for signers with lower literacy in written languages. Participants watch animations and answer carefully engineered questions about the information content. Because of the labor-intensive nature of our current evaluation approach, we seek techniques for measuring user’s reactions to animations via eye-tracking technology. In this paper, we analyze the relationship between various metrics of eye movement behavior of native ASL signers as they watch various types of stimuli: videos of human signers, high-quality animations of ASL, and lower-quality animations of ASL. We found significant relationships between the quality of the stimulus and the proportional fixation time on the upper and lower portions of the signers face, the transitions between these portions of the face and the rest of the signer’s body, and the total length of the eye fixation path. Our work provides guidance to researchers who wish to evaluate the quality of sign language animations: to enable more efficient evaluation of animation quality to support the development of technologies to synthesize high-quality ASL animations for deaf users.
KeywordsAmerican Sign Language accessibility technology for people who are deaf eye tracking animation evaluation user study
Unable to display preview. Download preview PDF.
- 3.Ardis, S.: ASL Animations supporting literacy development for learners who are deaf. Closing the Gap 24(5), 1–4 (2006)Google Scholar
- 4.Cavender, A., Rice, E.A., Wilamowska, K.M.: SignWave: Human Perception of Sign Language Video Quality as Constrained by Mobile Phone Technology. Emergency 7(12), 16–20Google Scholar
- 5.Cooke, L., Cuddihy, E.: Using eye tracking to address limitations in think-aloud protocol. In: Proceedings of the International Professional Communication Conference (IPCC 2005), pp. 653–658. IEEE (2005)Google Scholar
- 8.Huenerfauth, M.: Evaluation of a psycholinguistically motivated timing model for animations of American Sign Language. In: Proc. of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 129–136. ACM (2008)Google Scholar
- 9.Huenerfauth, M.: Spatial and planning models of ASL classifier predicates for machine translation. In: 10th International Conference on Theoretical and Methodological Issues in Machine Translation, TMI (2004)Google Scholar
- 10.Huenerfauth, M., Lu, P., Rosenberg, A.: Evaluating importance of facial expression in American Sign Language and pidgin signed English animations. In: Proc. of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 99–106. ACM (2011)Google Scholar
- 11.Huenerfauth, M., Zhao, L., Gu, E., Allbeck, J.: Evaluating American Sign Language generation through the participation of native ASL signers. In: Proc. of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 211–218. ACM (2007)Google Scholar
- 12.Jacob, R.J.K., Karn, K.S.: Eye Tracking in Human- Computer Interaction and Usability Research: Ready to Deliver the Promises. The Mind’s Eye (First Edition). In: Hyönä, J., Radach, R., Deubel, H. (eds.) Amsterdam, pp. 573–605 (2003)Google Scholar
- 13.Kacorri, H., Harper, A., Huenerfauth, M.: Comparing native signers’ perception of American Sign Language animations and videos via eye tracking. In: Proc. of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, p. 9. ACM (2013)Google Scholar
- 16.Lu, P., Huenerfauth, M.: Accessible motion-capture glove calibration protocol for recording sign language data from deaf subjects. In: Proc. of the 11th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 83–90. ACM (2009)Google Scholar
- 19.Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372–422 (1998)Google Scholar
- 21.Wagner, M., Marder, C., Blackorby, J., Cameto, R., Newman, L., Levine, P., et al.: The achievements of 100 youth with disabilities during secondary school: A report from 101 the National Longitudinal Transition Study-2 (NLTS2). SRI International, Menlo 102 Park (2003)Google Scholar