Skip to main content

Improving realism in automated fingerspelling of American sign language

Abstract

Fingerspelling is a process of communicating letters of a spoken language alphabet using a person’s hand or hands. Portraying animations of fingerspelling has proved surprisingly resistant to automation because of the collisions that arise from conventional interpolation of keyframes of individual manual letters. Previous methods have not been able to provide convincingly realistic fingerspelling due to the absence of effective collision avoidance in the underlying animation algorithms. This paper reports on the development and evaluation of a new collision avoidance algorithm that aids fingerspelling. Instead of analyzing letter transitions, the algorithm capitalizes on the transitions of individual fingers. The new strategy is efficient enough to support real-time fingerspelling while still maintaining a high level of predictive accuracy. Utilizing this strategy in signing avatars is expected to improve the current resources for deaf children, hearing teachers, hearing parents, and interpreting students who want to improve their fingerspelling comprehension. Future work will include testing the strategy’s generality when applying it to other one-handed manual alphabets.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

References

  1. Akamatsu CT (1982) The acquisition of fingerspelling in pre-school children. University Microfilms, Ann Arbor

    Google Scholar 

  2. Antona M, Stephanidis C (2015) Universal access in human-computer in- teraction. In: Access to learning, health and well-being: 9th International Conference, UAHCI 2015, Held as Part of HCI International

  3. Baowidan S, Guo N, Johnson S, Moncrief R, Berke L (2017) A New N- gram analytics tool in ELAN and its application to improve automatic fingerspelling generation. DePaul Universityss, Chicago

    Google Scholar 

  4. Calderon R (2000) Parental involvement in deaf children’s education programs as a predictor of child’s language, early reading, and social-emotional development. J Deaf Stud Deaf Educ 5(2):140–155. https://doi.org/10.1093/deafed/5.2.140

    MathSciNet  Article  Google Scholar 

  5. Davidson MJ (2000) Usability testing of computer animation of finger- spelling for American sign language. In: and others (ed) CTI Research Conference

  6. Ebling S, Wolfe R, et al. (2015) Synthesizing the finger alphabet of Swiss German Sign Language and evaluating the comprehensibility of the resulting animations. Proceedings of SLPAT 2015: 6th Workshop on Speech and Language Processing for Assistive Technologies pp 10–16

  7. Foley JD, van Dam A (1982) Fundamentals of interactive computer graphics, vol 2. Addison-Wesley Reading, MA

    Google Scholar 

  8. Guillory LM (1966) Claitors publishing division

  9. Jaklic A, Vodopivec D, Komac V (1995) Learning sign language through multimedia. In: International Conference on Multimedia Computing and Systems, IEEE, pp 282–285

  10. Mcdonald J, Alkoby K, Carter R, Christopher J, Davidson M, Ethridge D, Wolfe R (2001) An improved articulated model of the human hand. Vis Comput 17(3):158–166

    Article  Google Scholar 

  11. Mckee RL (1992) What’s so hard about learning ASL? Students’and teachers’ perceptions. Sign Language Studies 75(1):129–157

    Article  Google Scholar 

  12. Murdock K (2011) 3Ds Max 2012 bible, vol 783. John Wiley & Sons, New york

    Google Scholar 

  13. Padden C, Gunsauls DC (2003) How the alphabet came to be used in a sign language. Sign Language Studies 4(1):10–33. https://doi.org/10.1353/sls.2003.0026

    Article  Google Scholar 

  14. Padden C, Ramsey C (1998) Reading ability in signing deaf children. Top Lang Disord 18:30–46

    Article  Google Scholar 

  15. Patrie C, Johnson R (2011) Fingerspelled word recognition through rapid serial visual presentation. DawnSignPress, San Diego

    Google Scholar 

  16. Schick B (2005) Look who’s being left behind: educational interpreters and access to education for deaf and hard-of-hearing students. J Deaf Stud Deaf Educ 11(1):3–20. https://doi.org/10.1093/deafed/enj007

    Article  Google Scholar 

  17. Shaffer L, Watson W (2004) Peer mentoring: what is that. Proceedings of the 15th National Convention Conference of Interpreter Trainers (CIT), Still shining after 25:77–92

  18. Shipgood LE, Pring TR (1995) The difficulties of learning fingerspelling: an experimental investigation with hearing adult learners. Inter J Nal Lang Commun Disorders 30(4):401–416. https://doi.org/10.3109/13682829509087241

    Article  Google Scholar 

  19. Stoll S, Camgoz NC, Hadfield S, Bowden R (2018) Sign language production using neural machine translation and generative adversarial networks. Proceedings of the 29th British Machine Vision Conference

  20. Toro JA, Mcdonald JC, Wolfe R (2014) Fostering better deaf/hearing com- munication through a novel mobile app for fingerspelling . In: and others (ed) Computers helping people with special needs, Springer International, pp 559–564

  21. Vicars B (2005) URL http://asl.ms

  22. Wilcox S (1992) The phonetics of fingerspelling. John Benjamins Publishing, Amsterdam

    Book  Google Scholar 

  23. Wolfe R, Alba N, Billups S, Davidson MJ, Dwyer C, Jamrozik D, (2006) An improved tool for practicing fingerspelling recognition. In: Conference 2006 international conference on technology and persons with disabilities, pp 17–22

  24. Wolfe R, Mcdonald J, Toro J, Baowidan S, Moncrief R, Schnepp J (2015) Promoting better deaf/hearing communication through an improved interaction design for fingerspelling practice. International conference on universal access in human-computer interaction, pp 495–505

  25. van Zijl L, Raitt L (2004) Implementation experience with collision avoidance in signing avatars. Proceedings of the 3rd international conference on Computer graphics, virtual reality, visualisation and interaction in Africa, pp 55–59

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Souad Baowidan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Baowidan, S. Improving realism in automated fingerspelling of American sign language. Machine Translation 35, 387–404 (2021). https://doi.org/10.1007/s10590-021-09273-1

Download citation

Keywords

  • Sign language
  • Fingerspelling
  • American sign language
  • Collision avoidance
  • Sign language synthesis