An Open and Extensible Platform for Machine Translation of Spoken Languages into Sign Languages

  • Rostand CostaEmail author
  • Tiago Maritan
  • Renan Soares
  • Vinicius Veríssimo
  • Suanny Vieira
  • Alexandre Santos
  • Manuella Aschoff
  • Guido Lemos
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 813)


The purpose of this paper is to investigate the feasibility of offering a multilingual platform for text-to-sign translation, i.e., a solution where a machine translates digital contents in several spoken languages to several sign languages in scenarios such as Digital TV, Web and Cinema. This solution called OpenSigns, is an open platform that has several common components for generic functionalities, initially originated from the Suíte VLibras, including the creation and manipulation of 3D animation models, and interchangeable mechanisms specific for each sign language, such as a text-to-gloss machine translation engine, a sign dictionary for each sign language, among others. Our motivation is that the concentration of efforts and resources around a single solution could provide some state-of-the-art improvement, such as a standard solution for the industry and a greater functional flexibility for common components. In addition, we could also share techniques and heuristics between the translation mechanisms, reducing the effort to make a new sign language available on the platform, which may further enhance digital inclusion and accessibility, especially for poorest countries.


Digital video Online video Accessibility Sign language  Machine translation 


  1. 1.
    Deaf sign language on world. Accessed 30 Nov 2016
  2. 2.
    Fundación hetah - herramientas tecnológicas para ayuda humanitaria. Accessed 29 Nov 2016
  3. 3.
    Hand Talkc. Accessed 25 June 2015
  4. 4.
    Learn Sign Language in a Playful Way with SiGame – The App. Accessed 30 Nov 2016
  5. 5.
    ProDeaf. Accessed 25 June 2015
  6. 6.
  7. 7.
    Araújo, T.M.U.: Uma solução para geração automática de trilhas em língua brasileira de sinais em conteúdos multimídia (2012)Google Scholar
  8. 8.
    Araújo, T.M.U., Ferreira, F.L.S., et al.: An approach to generate and embed sign language video tracks into multimedia contents. Inf. Sci. 281, 762–780 (2014)CrossRefGoogle Scholar
  9. 9.
    IBGE: Population census 2010: General characteristics of the population, religion and people with disabilities. Technical report, Brazilian Institute of Geography and Statistics (2010). Accessed 20 Dec 2017
  10. 10.
    Freitas, C., Rocha, P., Bick, E.: Floresta sintá(c)tica: bigger, thicker and easier. In: Proceedings of the 8th International Conference on Computational Processing of the Portuguese Language, PROPOR 2008, Aveiro, Portugal, pp. 216–219 (2008)Google Scholar
  11. 11.
    Lima, M.A., Araújo, T.M., Oliveira, E.S.: Incorporation of syntactic-semantic aspects in a libras machine translation service to multimedia platforms. In: Proceedings of the 21st Brazilian Symposium on Multimedia and the Web, Webmedia 2015, pp. 133–140 (2015)Google Scholar
  12. 12.
    Melo, F.R., Matos, H.C.O., Dias, E.R.B.: Aplicação da métrica bleu para avaliação comparativa dos tradutores automáticos bing tradutor e google tradutor. E-scrita 5(3), 33–45 (2014)Google Scholar
  13. 13.
    World Health Organization (WHO), et al.: Deafness and hearing loss, fact sheet n 300, updated February 2013Google Scholar
  14. 14.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2001)Google Scholar
  15. 15.
    San-segundo, R., Barra, R., et al.: Speech to sign language translation system for Spanish. Speech Commun. 50(11), 1009–1020 (2008)CrossRefGoogle Scholar
  16. 16.
    Stumpf, M.R.: Língua de sinais: escrita dos surdos na internet. In: V Congresso Ibero-Americano de Informática na Educação–RIBIE–Chile (2000)Google Scholar
  17. 17.
    Su, H.Y., Wu, C.H.: Improving structural statistical machine translation for sign language with small corpus using thematic role templates as translation memory. Trans. Audio Speech Lang. Proc. 17(7), 1305–1315 (2009)CrossRefGoogle Scholar
  18. 18.
    Wauters, L.N.: Reading comprehension in deaf children: the impact of the mode of acquisition of word meanings. EAC, Research Centre on Atypical Communication, Radboud University, Nijmegen (2005)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Digital Video Applications Lab - LAVIDFederal University of Paraíba - UFPB João PessoaBrazil

Personalised recommendations