A 3D Conversational Agent for Presenting Digital Information for Deaf People

  • Da Phuc Phan
  • Thi Nhat Thanh Nguyen
  • The Duy Bui
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5044)


Sign languages appear to be the major communication tool for the deaf community. As a minor community, Vietnamese deaf people often have to rely on a small numbers of interpreters who are hearing people knowing sign language in order to communicate with hearing people. It would be very beneficial for deaf as well as hearing people to develop an automated translation system between spoken/written languages and sign language. In this paper, we present our attempt to graphically decompose the gestures in Vietnamese Sign Language so that they can be easily synthesized in a 3D conversational agent. In order to describe the signs in Vietnamese Sign Language, we proposed an animation-level markup language. Based on this, we presented our 3D conversational agent for presenting multi-modal information for deaf people. The 3D agent can assist deaf people easier and in a more natural way.


Sign Language Synthesis Human Computer Interaction 3D Conversational Agents 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bangham, J.A., Cox, S.J., Elliott, R., Glauert, J.R.W., Marshall, I., Rankov, S., Wells, M.: Virtual signing: Capture, animation, storage and transmission - an overview of the visicast project. In: IEE Seminar on Speech and language processing for disabled and elderly people (2000)Google Scholar
  2. 2.
    Bonow Boeira, V., Volz de Oliveira, L.R., Souza Madeira, D., da Rocha Costa, A.C.: Using signwriting as a phonetic notation system. In: Braffort, A., et al. (eds.) Atelier traitement automatique des langues des signes: TALS 2005 (2005)Google Scholar
  3. 3.
    Bui, T.D.: Emotion and Facial Expressions in Creating Embodied Agents. PhD thesis, University of Twente, The Netherlands (2004)Google Scholar
  4. 4.
    Inclusive Education For Hearing Impaired Children in VietNam. Signs of the Deaf in VietNam (1997)Google Scholar
  5. 5.
    Marshall, I., Pezeshkpour, F., Bangham, J.A., Wells, M., Hughes, R.: On the real time elision of text. In: RIFRA 1998 - Proc. Int. Workshop on Extraction, Filtering and Automatic Summarization (1998)Google Scholar
  6. 6.
    Neidle, C., Kegl, J., MacLaughlin, D., Bahan, B., Lee, R.G.: The syntax of American Sign Language: functional categories and hierarchical structure. MIT Press, Cambridge (2000)Google Scholar
  7. 7.
    Pezeshkpour, F., Marshall, I., Elliott, R., Bangham, J.A.: Development of a legible deaf-signing virtual human. In: Proc. IEEE Conf. Multi-Media (1999)Google Scholar
  8. 8.
    Prillwitz, S., Leven, R., Zienert, H., Hanke, T., Henning, J., et al.: Hamburg notation system for sign languages - an introductory guide. In: International Studies on Sign Language and the Communication of the Deaf, vol. 5 (1989)Google Scholar
  9. 9.
    Sacks, O.: Seeing Voices – A Journey into the World of the Deaf. The University of California Press (1989)Google Scholar
  10. 10.
    Stokoe, W., Kuschel, R.: For Sign Language Research. Linstock Press (1978)Google Scholar
  11. 11.
    Thetos: Text into sign language automatic translator for polish,
  12. 12.
  13. 13.
    Wells, M., Pezeshkpour, F., Marshall, I., Tutt, M., Bangham, J.A.: Simon: an innovative approach to signing on television. In: Proc. Int. Broadcasting Convention (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Da Phuc Phan
    • 1
  • Thi Nhat Thanh Nguyen
    • 1
  • The Duy Bui
    • 1
  1. 1.College of TechnologyVietnam National UniversityHanoiVietnam

Personalised recommendations