A Machine Translation System from English to American Sign Language

  • Liwei Zhao
  • Karin Kipper
  • William Schuler
  • Christian Vogler
  • Norman Badler
  • Martha Palmer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1934)


Research in computational linguistics, computer graphics and autonomous agents has led to the development of increasingly sophisticated communicative agents over the past few years, bringing new perspective to machine translation research. The engineering of language- based smooth, expressive, natural-looking human gestures can give us useful insights into the design principles that have evolved in natural communication between people. In this paper we prototype a machine translation system from English to American Sign Language (ASL), taking into account not only linguistic but also visual and spatial information associated with ASL signs.


Machine Translation American Sign Language Elementary Tree Machine Translation System Imperative Sentence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 2.
    Abeillé, A., Schabes, Y. and Joshi, A.: Using lexicalized tree adjoining grammars for machine translation. In Proc. of the 13th International Conference on Computational Linguistics (COLING’ 90), Helsinki, Finland. (1990)Google Scholar
  2. 3.
    Norman Badler, Martha Palmer, and Bindiganavale, R.: Animation control for real-time virtual humans. Communications of the ACM. (1999) 42(8) 65–73Google Scholar
  3. 4.
    Bartenieff, I. and Lewis, D.: Coping With the Environment. Gordon and Breach Science Publishers, New York. (1980)Google Scholar
  4. 5.
    Battison, R.. Phonological deletion in American Sign Language. Sign Language Studies. 5 (1974) 1–19Google Scholar
  5. 6.
    Chi, D., Costa, M., Zhao, L. and Badler, N.: The EMOTE model for Effort and Shape. In Proceedings of ACM Computer Graphics Annual Conference (SIG-GRAPH), New Orleans, LA. (2000)Google Scholar
  6. 7.
    Chomsky, N. and Halle, M.: The Sound Pattern of English. Harper, New York, NY. (1968)Google Scholar
  7. 9.
    Fisher, S.: Influences in word order change in ASL. In C. Li, (ed.), Word order and word order change. Univ. of Texas Press, Austin, TX. (1975) 1–25.Google Scholar
  8. 10.
    Frishberg, N.: Arbitrariness and iconicity: Historical change in American Sign Language. Language. 51(3) (1975) 696–719CrossRefGoogle Scholar
  9. 11.
    Fromkin, V.: Sign language: Evidence for language universals and the linguistic capacity of the human brain. Sign Language Studies. 59 (1988) 115–127Google Scholar
  10. 12.
    Grieve-Smith, A.: Sign synthesis and sign phonology. In Proceedings of the First High Desert Student Conference in Linguistics, New Mexico. (1998)Google Scholar
  11. 13.
    XTAG Research Group.: A lexicalized tree adjoining grammar for english. Technical report, University of Pennsylvania. (1998)Google Scholar
  12. 14.
    Joshi, A.: An introduction to tree adjoining grammars. In A. Manaster-Ramer, (ed.), Mathematics of Language. John Benjamins, Amsterdam. (1987)Google Scholar
  13. 15.
    Klima, E. and Bellugi, U.: The Signs of Language. Harvard University Press. (1979)Google Scholar
  14. 16.
    Liddell, S.K. and Johnson, R.E.: American Sign Language: The phonological base. Sign Language Studies. 64 (1989) 195–277Google Scholar
  15. 17.
    Magnenat-Thalmann, N., Kalra, P. and Escher, M.: Face to virtual face. Proceedings of the IEEE. 86(5) (1998)Google Scholar
  16. 18.
    Messing, L. and Campbell, R.: Gesture, Speech and Sign. Oxford University Press (1999)Google Scholar
  17. 19.
    Perlin, K.: Real time responsive animation with personality. IEEE Transactions on Visualization and Computer Graphics. 1(1) (1995) 5–15CrossRefGoogle Scholar
  18. 20.
    Shieber, S.: Restricting the weak-generative capability of synchronous tree adjoining grammars. Computational Intelligence. 10(4) (1994)Google Scholar
  19. 21.
    Shieber, S. and Schabes, Y.: Synchronous tree adjoining grammars. In Proceedings of the 13th International Conference on Computational Linguistics (COLING’ 90), Helsinki, Finland. (1990)Google Scholar
  20. 22.
    Siple, P.: Understanding Language Through Sign Language Research. Academic Press. (1976)Google Scholar
  21. 23.
    Sternberg, M.: The American Sign Language Dictionary. Multicom. (1996)Google Scholar
  22. 24.
    Stokoe, W.: Sign language structure. Studies in Linguistics: Occasional Paper 8. Buffalo, New York. (1960)Google Scholar
  23. 25.
    Stokoe, W., Casterline, D. and Croneberg, C.: A Dictionary of American Sign Language on Linguistic Principles. Gallaudet College Press, Washington, DC. (1965, 1976 (2nd ed.))Google Scholar
  24. 26.
    Supalla, T. and Newport, E.: How many seats in a chair? The derivation of nouns and verbs in American Sign Language. In P. Siple, (ed.) Understanding Language through Sign Language Research. Academic Press, New York, NY. (1978)Google Scholar
  25. 27.
    Tolani, D. Inverse Kinematics Methods for Human Modeling and Simulation. PhD thesis, University of Pennsylvania. (1998)Google Scholar
  26. 28.
    Valli, C. and Lucas, C.: Linguistics of ASL: An Introduction. Gallaudet University Press (1995)Google Scholar
  27. 29.
    Vogler, C. and Metaxas, D.: A framework for recognizing the simultaneous aspects of American Sign Language. To appear in Computer Vision and Image Understanding.Google Scholar
  28. 30.
    Wilcox, S. and Wilcox, P.: Learning to See. Gallaudet University Press, Washington, DC. (1996)Google Scholar
  29. 31.
    Zhao, L., Costa, M. and Badler, N.: Interpreting movement manner. In Computer Animation 2000. IEEE Computer Graphics Society. (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Liwei Zhao
    • 1
  • Karin Kipper
    • 1
  • William Schuler
    • 1
  • Christian Vogler
    • 1
  • Norman Badler
    • 1
  • Martha Palmer
    • 1
  1. 1.Department of Computer and Information ScienceUniversity of PennsylvaniaPhiladelphiaUSA

Personalised recommendations