Abstract
We describe an experiment in which sign-language output in Swiss French Sign Language (LSF-CH) and Australian Sign Language (Auslan) was added to a limited-domain medical speech translation system using a recorded video method. By constructing a suitable web tool to manage the recording procedure, the overhead involved in creating and manipulating the large set of files involved could be made easily manageable, allowing us to focus on the interesting and non-trivial problems which arise at the translation level. Initial experiences with the system suggest that the recorded videos, despite their unprofessional appearance, are readily comprehensible to Deaf informants, and that the method is promising as a simple short-term solution for this type of application.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Hamburg Notation System for Sign Languages or HamNoSys [11] is the most commonly used formalism for describing the physical forms of signs.
- 2.
The version used in the study reported here contained about 1,600 utterance-types. The current version is considerably larger.
- 3.
- 4.
For presentational reasons, the rules have been simplified and shown as translating English into French. The real rules allow much greater syntactic variation and translate from French into five spoken languages.
- 5.
Since this paper was written, we have added functionality to perform robust matching against the grammar, using input from a large-vocabulary recogniser. This substantially improves speech understanding performance [17].
- 6.
As of late 2017, this has grown to about 5,000 utterance-types and ten subdomains.
- 7.
The app is freely accessible at https://speech2sign.unige.ch/en/applications/babeldr/.
- 8.
- 9.
- 10.
References
Aho, A.V., Ullman, J.D.: Properties of syntax directed translations. J. Comput. Syst. Sci. 3(3), 319–334 (1969)
Bouillon, P., Spechbach, H.: BabelDr: a web platform for rapid construction of phrasebook-style medical speech translation applications. In: Proceedings of EAMT 2016, Vilnius, Latvia (2016)
Cox, S., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., Tutt, M., Abbott, S.: Tessa, a system to aid communication with deaf people. In: Proceedings of the Fifth International ACM Conference on Assistive Technologies, pp. 205–212. ACM (2002)
Ebling, S., Glauert, J.: Exploiting the full potential of JASigning to build an avatar signing train announcements. In: Proceedings of the Third International Symposium on Sign Language Translation and Avatar Technology (SLTAT), Chicago, USA, vol. 18, p. 19, October 2013
Eco, U.: Mouse or rat?: Translation as negotiation. Hachette, UK (2004)
Elliott, R., Glauert, J.R., Kennaway, J., Marshall, I., Safar, E.: Linguistic modelling and language-processing technologies for avatar-based sign language presentation. Univ. Access Inf. Soc. 6(4), 375–391 (2008)
Fuchs, M., Tsourakis, N., Rayner, M.: A scalable architecture for web deployment of spoken dialogue systems. In: Proceedings of LREC 2012, Istanbul, Turkey (2012)
Jennings, V., Elliott, R., Kennaway, R., Glauert, J.: Requirements for a signing avatar. In: Proceedings of Workshop on Corpora and Sign Language Technologies (CSLT), LREC, pp. 33–136 (2010)
Kipp, M., Heloir, A., Nguyen, Q.: Sign Language avatars: animation and comprehensibility. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 113–126. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23974-8_13
Pointurier-Pournin, S.: L’interprètation en Langue des Signes Française: contraintes, tactiques, efforts. Ph.D. thesis, Universitè de la Sorbonne nouvelle-Paris III (2014)
Prillwitz, S., für Deutsche Gebärdensprache und Kommunikation Gehörloser, H.Z.: HamNoSys: version 2.0; Hamburg Notation System for Sign Languages; an introductory guide. Signum-Verlag (1989)
Rayner, M., Baur, C., Chua, C., Bouillon, P., Tsourakis, N.: Helping non-expert users develop online spoken CALL courses. In: Proceedings of the Sixth SLaTE Workshop, Leipzig, Germany (2015)
Rayner, M.: Using the Regulus Lite Speech2Speech Platform, online documentation (2016). http://www.issco.unige.ch/en/research/projects/Speech2SpeechDoc/build/html/index.html
Rayner, M., Bouillon, P., Ebling, S., Strasly, I., Tsourakis, N.: A framework for rapid development of limited-domain speech-to-sign phrasal translators. In: Proceedings of the workshop on Future and Emerging Trends in Language Technology, Sevilla, Spain (2015)
Rayner, M., Bouillon, P., Gerlach, J., Strasly, I., Tsourakis, N.: An open web platform for rule-based speech-to-sign translation. In: Proceedings of ACL 2016, Berlin, Germany (2016)
San-Segundo, R., Montero, J.M., Macías-Guarasa, J., Córdoba, R., Ferreiros, J., Pardo, J.M.: Proposing a speech to gesture translation architecture for spanish deaf people. J. Vis. Lang. Comput. 19(5), 523–538 (2008)
Rayner, M., Tsourakis, N., Gerlach, J.: Lightweight spoken utterance classification with CFG, tf-idf and dynamic programming. In: Camelin, N., Estève, Y., Martín-Vide, C. (eds.) SLSP 2017. LNCS (LNAI), vol. 10583, pp. 143–154. Springer, Le Mans, France (2017). doi:10.1007/978-3-319-68456-7_12
Acknowledgements
The BabelDr project is funded by “La fondation privée des HUG” and carried out in collaboration with HUG. We would like to thank Nuance Inc. for generously allowing us to use their software for research purposes, and Hervé Spechbach and Sarah Ebling for many helpful comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ahmed, F. et al. (2017). Rapid Construction of a Web-Enabled Medical Speech to Sign Language Translator Using Recorded Video. In: Quesada, J., Martín Mateos , FJ., López Soto, T. (eds) Future and Emerging Trends in Language Technology. Machine Learning and Big Data. FETLT 2016. Lecture Notes in Computer Science(), vol 10341. Springer, Cham. https://doi.org/10.1007/978-3-319-69365-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-69365-1_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69364-4
Online ISBN: 978-3-319-69365-1
eBook Packages: Computer ScienceComputer Science (R0)