Skip to main content
SpringerLink
Log in

From grammar-based MT to post-processed SL representations

  • Long paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

This paper proposes the implementation of a post-processing stage to a grammar-based machine translation (MT) system from written into sign language for the pair written Greek–Greek sign language (GSL), where translation output is visualized by an avatar. Post-processing is applied on the output of the MT system’s transfer module by activating an editing environment for SL which utilizes a dictionary of signs coded for HamNoSys transcriptions and non-manual elements along with grammar knowledge. This post-processing environment may also be used as a standalone editing environment for expert users allowing for a considerable range of editing actions, as well as an SL editing tool for the general public who are not familiar with the underlying technology. In both cases, this environment provides the opportunity of on-the-fly structuring of new signed utterances going far beyond current limitations in composition with new signed language content.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The term “chunk” is used throughout this paper as defined in Abney (1991).

  2. http://www.sign-lang.uni-hamburg.de/dicta-sign/portal/lang_inform.html.

  3. http://www.ilsp.gr/en/infoprojects/meta?view=project&task=show&id=156.

  4. SignGram COSTS Action IS-1006 “A blueprint for sign language grammars—unravelling the grammars of European sign languages: pathways to full citizenship of deaf signers and to the protection of their linguistic heritage” (www.signgram.eu).

  5. http://www.visicast.co.uk/.

  6. http://speech.ilsp.gr/sl.

  7. http://vh.cmp.uea.ac.uk.

  8. http://www.dictasign.eu/.

  9. DICTA-SIGN deliverable D7.4: Prototype evaluation synthesis (http://www.dictasign.eu/Main/PubliclyAvailableProjectDeliverables).

  10. ENABLE—Network of ICT Supported Learning for Disabled People: Deliverable 3.2: Methodologies for the Categorisation and Evaluation of ICT-Based Lifelong Learning for Disabled People.

  11. ENABLE—Network of ICT Supported Learning for Disabled People: Deliverable 3.3: Categorisation and Evaluation of Existing ICT to Support Lifelong Learning by Disabled People.

References

  1. Stokoe, W.C.: Sign Language Structure, 2nd edn. Linstok Press, Silver Spring (1978)

    Google Scholar 

  2. Prillwitz, S., Leven, R., Zienert, H., Hanke, T. and Henning, J.: HamNoSys. Version 2.0. Hamburg Notation System for Sign Language: An Introductory Guide, Signum Verlag, Hamburg (1989)

  3. Neidle, C.: SignStream™: a database tool for research on visual-gestural language. In: Bergman, B., Boyes-Braem, P., Hanke, T., Pizzuto, E. (eds.) Sign Transcription and Database Storage of Sign Information, Special Issue of Sign Language and Linguistics vol. 4(1), pp. 203–214. John Benjamins Publishing Company (2002)

  4. Pizzuto, E., Pietrandrea, P.: The notation of signed texts: open questions and indications for further research. Sign Lang. Linguist. 4(1/2), 29–45 (2001)

    Article  Google Scholar 

  5. Elliott, R., Glauert, J.R.W., Jennings, V., Kennaway, J.R. An overview of the SiGML notation and SiGML signing software system. In: Proceedings of First Workshop on Representing and Processing of Sign Languages (LREC-2010), pp. 98–104. Lisbon (2004)

  6. Glauert, J., Elliott, R.: Extending the SiGML notation—a progress report. In: Second International Workshop on Sign Language Translation and Avatar Technology (SLTAT), 23 Oct 2011, Dundee (2011)

  7. Schnepp, J., Wolfe, R., McDonald, J. Synthetic corpora: a synergy of linguistics and computer animation. In: Proceedings of Fourth Workshop on Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, (LREC-2010), pp. 217–220 (2010)

  8. Efthimiou, E., Fotinea, S.-E., Hanke, T., Glauert, J., Bowden, R., Braffort, A., Collet, C., Maragos, P., Lefebvre-Albaret, F.: The Dicta-Sign wiki: enabling web communication for the deaf. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) Lecture Notes in Computer Science (LNCS), Book Computers Helping People with Special Needs (Part II), Springer [ISBN: 978-3-642-31533-6 (Print) 978-3-642-31534-3 (Online)], vol. 7383, pp. 205–212 (2012)

  9. Safar, E.E., Marshall, I.: Sign language translation via DRT and HPSG. In: Gelbukh, A. (ed.) Proceedings of the Third International Conference on Intelligent Text Processing and Computational Linguistics, CICLing, Mexico, Lecture Notes in Computer Science 2276, pp. 58–68. Springer (2002)

  10. San-Segundo, R., Montero, J.M., Córdoba, R., Sama, V., Fernández, F., D’Haro, L.F., López-Ludeña, V., Sánchez, D., García, A.: Design, development and field evaluation of a Spanish into sign language translation system. Pattern analysis and applications. Pattern Anal. Appl. 15(2), 203–224 (2011)

    Article  Google Scholar 

  11. Bungeroth, J., Ney, H.: Statistical sign language translation. In: Proceedings of the Workshop on Representation and Processing of Sign Languages, LREC 2004, pp. 105–108 (2004)

  12. Morrissey, S., Way, A.: Bidirectional data-driven machine translation for Irish and German sign languages. Mach. Transl. 27(1), 25–64 (2013)

    Article  Google Scholar 

  13. Morrissey, S., Way, A., Stein, D., Bungeroth, J., Ney, H.: Combining data-driven MT systems for improved sign language translation. In: Machine Translation Summit XI, 10–14 Sept 2007, Copenhagen (2007)

  14. Lee, Yung-Hui, Tsai, Cheng-Yueh: Taiwan sign language (TSL) recognition based on 3D data and neural networks. Expert Syst. Appl. 36(2), 1123–1128 (2009). Part 1

    Article  Google Scholar 

  15. Marshall, I., Sáfár, E.: Grammar development for sign language avatar-based synthesis. In: Proceedings HCII 2005, 11th International Conference on Human Computer Interaction, Las Vegas, July 2005

  16. Kouremenos, D., Fotinea, S.E., Efthimiou, E., Ntalianis, K.: A prototype Greek text to Greek sign language (GSL) conversion system. Behav. Inf. Technol. J. (TBIT) 29(5), 467–481 (2010). doi:10.1080/01449290903420192

    Article  Google Scholar 

  17. Huenerfauth, M.: A multi-path architecture for machine translation of English text into American Sign language animation. In: Proceedings of Student Workshop at Human Language, Technologies conference HLT-NAACL, Boston (2004)

  18. van Zijl, L., Barker, D.: A machine translation system for South African sign language. In: Proceedings of Afrigraph 2003, ACM SIGGRAPH Conference Series, Feb 2003, Cape Town (2003)

  19. Zijl, L.V., Combrink, A. The South African sign language machine translation project: issues on nonmanual sign generation. In: Proceedings of SAICSIT06, Somerset West, pp. 127–134 (2006)

  20. Zhao, L., et al.: A machine translation system from English to american sign language. In: Lecture Notes in Computer Science 1934 (Proceedings of the Association for Machine Translation in the Americas 2000), 54–67 (2000)

  21. Filhol, M., Hadjadj, M.N., Testu, B. A rule triggering system for automatic text-to-sign translation. In: Proceedings of International Workshop on Sign Language Translation and Avatar Technology (SLTAT), Chicago (2013)

  22. Forster, J., Schmidt, C., Koller, O., Bellgardt, M., Ney, H. Extensions of the sign language recognition and translation corpus RWTH-PHOENIX-weather. In: Proceedings of Language Resources and Evaluation Conference (LREC), May 2014, pp. 1911–1916. Reykjavik (2014)

  23. Morrissey, S., Way, A. An example-based approach to translating sign language. In: Proceedings of the Workshop Example-Based Machine Translation (MT X–05), pp. 109–116. Phuket (2005)

  24. Abney, S.: Parsing by chunks. In: Berwick, R., Abney, S., Tenny, C. (eds.) Principle-Based Parsing, p. 1991. Kluwer, Dordrecht (1991)

    Google Scholar 

  25. Boutsis, S., Prokopidis, P., Giouli, V., Piperidis, S.: A robust parser for unrestricted Greek text. In Proceedings of the Second Language Resources and Evaluation Conference, pp. 467–474 (2000)

  26. Papageorgiou, H., Prokopidis, P., Giouli, V., Piperidis, S.: A unified POS tagging architecture and its application to Greek. In: Proceedings of the Second Language Resources and Evaluation Conference, pp. 1455–1462. Athens (2000)

  27. Prokopidis, P., Georgantopoulos, B., Papageorgiou, H.: A suite of NLP tools for Greek. In: The Tenth International Conference of Greek Linguistics, Komotini (2011)

  28. Efthimiou, E., Katsoyannou, M.: Research issues on GSL: a study of vocabulary and lexicon creation. Stud. Greek Linguist. Comput. Linguist. 2, 42–50 (2001). (in Greek)

    Google Scholar 

  29. Matthes, S., Hanke, T., Storz, J., Efthimiou, E., Dimiou, N., Karioris, P., Braffort, A., Choisier, A., Pelhate, J., Safar, E.: Elicitation tasks and materials designed for Dicta-Sign’s multi-lingual corpus. In: Proceedings of the Fourth Workshop on “Representation and Processing of Sign Languages: Corpora and Sign Language Technologies”, Satellite Workshop to LREC-2010, 22–23 May 2010, pp. 158–163. Malta (2010) http://www.lrec-conf.org/proceedings/lrec2010/workshops/W13.pdf

  30. Matthes, S., Hanke, T., Regen, A., Storz, J., Worseck, S., Efthimiou, E., Dimou, A.-L., Braffort, A., Glauert J., Safar, E.: Dicta-Sign—building a multilingual sign language corpus. In: Proceedings of the Fifth Workshop on the Representation and Processing of Sign Languages: Interactions between Corpus and Lexicon. Satellite Workshop to the Eighth International Conference on Language Resources and Evaluation (LREC-2012), 23–27 May 2012, pp. 117–122. Istanbul (2012)

  31. Hanke, T.: HamNoSys—representing sign language data in language resources and language processing contexts. In: Proceedings of First Workshop on Representing and Processing of Sign Languages (LREC-2004) pp. 1–6 (2004)

  32. Dimou, A.-L., Goulas, T., Efthimiou, E., Fotinea, S.-E., Karioris, P., Pissaris, M., Korakakis, D., and Vasilaki, K. Creation of a multipurpose sign language lexical resource: The GSL lexicon database. In: Proceedings of Sixth Workshop on the Representation and Processing of Sign Languages: Beyond the Manual Channel, Language Resources and Evaluation Conference (LREC), May 2014, pp. 37–42, Reykjavik (2014)

  33. Efthimiou, E., Fotinea, S.E., Sapountzaki, G.: Processing linguistic data for GSL structure representation. In: Proceedings of the Workshop on the Representation and Processing of Sign Languages: Lexicographic Matters and Didactic Scenarios, Satellite Workshop to LREC-2006 Conference, pp. 49–54 (2006)

  34. Efthimiou, E., Fotinea, S.E., Hanke, T., Glauert, J., Bowden, R., Braffort, A., Collet, C., Maragos P., Lefebvre-Albaret, F.: Sign language technologies and resources of the Dicta-Sign project. In: Proceedings of the Fifth Workshop on the Representation and Processing of Sign Languages: Interactions Between Corpus and Lexicon (LREC-2012), pp. 37–45 (2012)

  35. Efthimiou, E., Fotinea, S.E.: GSLC: creation and annotation of a Greek sign language corpus for HCI. In: Stephanidis, C. (ed.) Lecture Notes in Computer Science (LNCS), Universal Access in HCI, Part I, HCII 2007, LNCS 4554, pp. 657–666 (2007)

  36. Shieber, S.M.: An Introduction to Unification-Based Approaches to Grammar. Volume 4 of CSLI Lecture Notes. Center for the Study of Language and Information, Stanford (1986)

  37. Sag, I.A., Kaplan, R., Karttunen, L., Kay, M., Pollard, C.J., Shieber, S.M., Zaenen, A.: Unification and grammatical theory. In: Proceedings of the Fifth West Coast Conference on Formal Linguistics, pp. 238–254. Seattle (1986)

  38. Kay, M.: Functional unification grammar: a formalism for machine translation. In: Proceedings of the Tenth International Conference on Computational Linguistics, pp. 75–78. Stanford (1984)

  39. Carpenter, B.: The Logic of Typed Feature Structures. Cambridge Tracts in Theoretical Computer Science 32, Cambridge University Press (1992)

  40. Crasborn, O., Sloetjes, H.: Using ELAN for annotating sign language corpora in a team setting. In: Dreuw, P., et al. (eds.) Proceedings of Fourth Workshop on Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, LREC-2010, pp. 61–64 (2010)

  41. Hanke, T., Storz, J. iLex—a Database tool for integrating sign language corpus linguistics and sign language lexicography. In: Construction and Exploitation of Sign Language Corpora. Proceedings of Third Workshop on the Representation and Processing of Sign Languages, ELRA, pp. 64–67, Paris (2008)

  42. Cox, S.J., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., MandTutt, Abbott, S.: TESSA, a system to aid communication with deaf people. In: Proceedings of the ASSETS 2002, pp. 205–212. Edinburgh (2002)

  43. López-Ludeña, V., San-Segundo, R., González-Morcillo, C., López, J.C., Ferreiro, E., Ferreiros, J. Methodology for developing an advanced communications system for the deaf in a new domain. In: Knowledge-Based Systems, vol. 56, Jan 2014, pp. 240–252 (2014)

  44. Goulas, T., Fotinea, S.E., Efthimiou, E., Pissaris, M.: SiS-Builder: a sign synthesis support tool. In: Proceedings of Fourth Workshop on Representation and Processing of Sign Languages: Corpora and Sign Language Technologies (LREC-2010), pp. 102–105 (2010)

  45. Dimou, A.L., Goulas, T., Efthimiou, E.: Grammar/prosody modelling in Greek sign language: towards the definition of built-in sign synthesis rules. In: Efthimiou, E., Kouroupetroglou, G., Fotinea, S. E. (eds.) Gesture and Sign Language in Human-Computer Interaction and Embodied Communication. Lecture Notes in Computer Science/Lecture Notes in Artificial Intelligence, 7206, pp. 183–193. Springer (2012)

  46. Papineni, K., Roukos, S., Ward, T., Zhu, W.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association of Computational Linguistics (ACL-02), pp. 311–318. Philadelphia (2002)

  47. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J. A study of translation edit rate with targeted human annotation. In: Proceedings of the Seventh Conference of the Association for Machine Translation in the Americas, Aug 2006, pp. 223–231. Cambridge (2006)

  48. Morrissey, S., Way, A.: Lost in translation: the problems of using mainstream MT evaluation metrics for sign language translation. In: Proceedings of the Fifth SALTMIL Workshop on Minority Languages at LREC’06, pp. 91–98. Genoa (2006)

  49. Dicta-Sign Project, Deliverable D7.4: Prototype Evaluation, April 2011. http://www.dictasign.eu/attach/Main/PubliclyAvailableProjectDeliverables/D74d.pdf

Download references

Acknowledgments

The research leading to these results has received funding from POLYTROPON project (KRIPIS-GSRT, MIS: 448306), based on initial basic research carried out in the framework of the Dicta-Sign Project (FP7/2007-2013 Grant Agreement No. 231135). The authors’ thanks go to all Sign Language Technologies Team members at ILSP for their restless efforts towards development of the here-reported modules and GSL resources.

Author information

Authors and Affiliations

  1. Institute for Language and Speech Processing - ILSP/ATHENA RC, Artemidos 6, Maroussi, 15125, Athens, Greece

    Eleni Efthimiou, Stavroula-Evita Fotinea, Athanasia-Lida Dimou & Theodore Goulas

  2. School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece

    Dimitris Kouremenos

Authors
  1. Eleni Efthimiou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stavroula-Evita Fotinea
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Athanasia-Lida Dimou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theodore Goulas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dimitris Kouremenos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eleni Efthimiou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Efthimiou, E., Fotinea, SE., Dimou, AL. et al. From grammar-based MT to post-processed SL representations. Univ Access Inf Soc 15, 499–511 (2016). https://doi.org/10.1007/s10209-015-0414-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-015-0414-3

Keywords

Search

Navigation