Advertisement

A machine translation system from Arabic sign language to Arabic

  • Hamzah LuqmanEmail author
  • Sabri A. Mahmoud
Long Paper
  • 36 Downloads

Abstract

Arabic sign language (ArSL) is one of the sign languages that is used in Arab countries. This language has structure and grammar that differ from spoken Arabic. Available ArSL recognition systems perform direct mapping between the recognized sign in the ArSL sentence and its corresponding Arabic word. This results in persevering the structure and grammar of the ArSL sentence. ArSL translation involves converting the recognized ArSL sentence into Arabic sentence that meets the structure and grammar of Arabic. We propose in this work a rule-based machine translation system between ArSL and Arabic. The proposed system performs morphological and syntactic analysis to translate the ArSL sentence lexically and syntactically into Arabic. To evaluate this work, we perform manual and automatic evaluation using a corpus on the health domain. The obtained results show that our translation system provides an accurate translation for more than 80% of the translated sentences.

Keywords

Arabic sign language ArSL ArSL machine translation Sign language translation Sign language recognition 

Notes

Acknowledgements

The authors would like to thank Dr. Nizar Habash for his helpful conversations, resources, and feedback. In addition, we would like to acknowledge the support provided by King Fahd University of Petroleum & Minerals (KFUPM) for funding this work through Project Number IN151008.

Compliance with ethical standards

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

References

  1. 1.
    Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Transform-based Arabic sign language recognition. Proc. Comput. Sci. 117, 2–9 (2017).  https://doi.org/10.1016/j.procs.2017.10.087 CrossRefGoogle Scholar
  2. 2.
    Arab League Educational Cultural, Scientific Organization, First part of the Unified Arabic Sign language Dictionary, League Arab States Arab League Educ. Cult. Sci. Organ, (2001)Google Scholar
  3. 3.
    Arab League Educational Cultural, Scientific Organization, Second part of the Unified Arabic Sign language Dictionary, League Arab States Arab League Educ. Cult. Sci. Organ, (2007)Google Scholar
  4. 4.
    Luqman, H., Mahmoud, S.A.: Automatic translation of Arabic text-to-Arabic sign language. Universal Access in the Information Society 1–13 (2018)Google Scholar
  5. 5.
    Mitra, S., Acharya, T.: Gesture recognition: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 37(3), 311–324 (2007)CrossRefGoogle Scholar
  6. 6.
    Cooper, H., Holt, B., Bowden, R.: Sign language recognition. In: Visual Analysis of Humans, pp. 539–562, Springer, Berlin (2011)CrossRefGoogle Scholar
  7. 7.
    Abdel-Fattah, M.A.: Arabic sign language: a perspective. J. Deaf Stud. Deaf Educ. 10(2), 212–221 (2005)CrossRefGoogle Scholar
  8. 8.
    Semreen, S., Albinali, M.: The rules of Arab Qatari sign standardized language, Supreme Council of Family Affairs, (2010)Google Scholar
  9. 9.
    Abdel-Fattah, M.: Arabic sign language: a perspective. J. Deaf Stud. Deaf Educ. 10(2), 212–221 (2005).  https://doi.org/10.1093/deafed/eni007 CrossRefGoogle Scholar
  10. 10.
    Trujillo, A.: Translation Engines: Techniques for Machine Translation. Springer, Berlin (2012)zbMATHGoogle Scholar
  11. 11.
    Morrissey, S.: Data-driven machine translation for sign languages, Ph.D. thesis, Dublin City University (2008)Google Scholar
  12. 12.
    Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Arabic Sign Language Recognition Using Optical Flow-Based Features and HMM, pp. 297–305, Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-59427-9-32
  13. 13.
    Almohimeed, A., Wald, M., Damper, R.: Arabic text to Arabic sign language translation system for the deaf and hearing-impaired community. In: Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies, Association for Computational Linguistics, pp. 101–109 (2011)Google Scholar
  14. 14.
    Mohandes, M.: Automatic translation of Arabic text to Arabic sign language. AIML J. 6(4), 15–19 (2006)Google Scholar
  15. 15.
    Halawani, S.M.: Arabic sign language translation system on mobile devices. IJCSNS Int. J. Comput. Sci. Netw. Secur. 8(1), 251–256 (2008)Google Scholar
  16. 16.
    Al Ameiri, F., Zemerly, M. J., Al Marzouqi, M.: M-learning and chatting using indexed Arabic sign language, Int. J. Inf. (IJI) 5: 10 (2012)CrossRefGoogle Scholar
  17. 17.
    Al-Nafjan, A., Al-Arifi, B., Al-Wabil, A.: Design and development of an educational Arabic sign language mobile application: collective impact with Tawasol. In: International Conference on Universal Access in Human–Computer Interaction, pp. 319–326, Springer, Berlin (2015)CrossRefGoogle Scholar
  18. 18.
    Al-Khalifa, H. S.: Introducing Arabic sign language for mobile phones. In: International Conference on Computers for Handicapped Persons, pp. 213–220, Springer, Berlin (2010)Google Scholar
  19. 19.
    Al-Rikabi, S., Hafner, AV.: Humanoid robot as a translator from text to sign language. In: 5th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC 2011), pp. 375–379 (2011)Google Scholar
  20. 20.
    Almasoud, A.M., Al-Khalifa, H.S.: Semsignwriting: a proposed semantic system for Arabic text-to-signwriting translation. J. Softw. Eng. Appl. 5, 604–612 (2012)CrossRefGoogle Scholar
  21. 21.
    El Alfi, A.E.E., El Basuony, M.M.R., Atawy, S.M.: Intelligent Arabic text to Arabic sign language translation for easy deaf communication. Int. J. Comput. Appl. 92(8), 22–29 (2014)Google Scholar
  22. 22.
    Ritchings, T., Khadragi, A., Saeb, M.: An intelligent computer-based system for sign language tutoring. Assist. Technol. 24(4), 299–308 (2012).  https://doi.org/10.1080/10400435.2012.680662 CrossRefGoogle Scholar
  23. 23.
    Mohandes, M., Deriche, M.: Arabic sign language recognition by decisions fusion using Dempster–Shafer theory of evidence, 2013 Computing, Commun. IT Appl. Conf. (ComComAp) 2013, 90–94 (2013).  https://doi.org/10.1109/ComComAp.2013.6533615 CrossRefGoogle Scholar
  24. 24.
    Mohandes, M., Quadri, S., Deriche, M.: Arabic sign language recognition an image-based approach. In: 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW’07), pp. 272–276 (2007)  https://doi.org/10.1109/AINAW.2007.98
  25. 25.
    Shanableh, T., Assaleh, K., Al-Rousan, M.: Spatio-temporal feature-extraction techniques for isolated gesture recognition in Arabic sign language. IEEE Trans. Syst. Man Cybern. Part B Cybern. Publ. IEEE Syst. Man Cybern. Soc. 37(3), 641–650 (2007)CrossRefGoogle Scholar
  26. 26.
    AL-Rousan, M., Assaleh, K., Tala’a, a: Video-based signer-independent Arabic sign language recognition using hidden Markov models. Appl. Soft Comput. 9(3), 990–999 (2009).  https://doi.org/10.1016/j.asoc.2009.01.002 CrossRefGoogle Scholar
  27. 27.
    Ahmed, A., Aly, S.: Appearance-based Arabic sign language recognition using hidden Markov models. In: 2014 International Conference on Engineering and Technology (ICET), 2014, pp. 1–6.  https://doi.org/10.1109/ICEngTechnol.2014.7016804
  28. 28.
    Aly, S., Mohammed, S.: Arabic sign language recognition using Spatio-Temporal Local Binary Patterns and Support Vector Machine. In: Hassanien, A., Tolba, M., Taher Azar, A. (eds.) Advanced Machine Learning Technologies and Applications, Vol. 488 of Communications in Computer and Information Science, Springer, Berlin, 2014, pp. 36–45.  https://doi.org/10.1007/978-3-319-13461-1-5
  29. 29.
    Tharwat, A., Gaber, T., Hassanien, A. E., Shahin, M., Refaat, B.: Sift-based Arabic sign language recognition system. In: Afro-European Conference for Industrial Advancement, pp. 359–370, Springer, Berlin (2015)Google Scholar
  30. 30.
    Aujeszky, T., Eid, M.: A gesture recognition architecture for Arabic sign language communication system. Multimed. Tools Appl. 75(14), 8493–8511 (2016)CrossRefGoogle Scholar
  31. 31.
    Aly, S., Osman, B., Aly, W., Saber, M.: Arabic sign language fingerspelling recognition from depth and intensity images. In: 2016 12th International Computer Engineering Conference (ICENCO), pp. 99–104, IEEE (2016)Google Scholar
  32. 32.
    Cooper, H., Ong, E.-J., Pugeault, N., Bowden, R.: Sign language recognition using sub-units. J. Mach. Learn. Res. 13, 2205–2231 (2012)zbMATHGoogle Scholar
  33. 33.
    Paulraj, M. P., Yaacob, S., Zanar Azalan, M. S., Palaniappan, R.: A phoneme based sign language recognition system using 2D moment invariant interleaving feature and neural network. In: Proceedings—2011 IEEE Student Conference on Research and Development, SCOReD 2011, pp. 111–116 (2011)  https://doi.org/10.1109/SCOReD.2011.6148718
  34. 34.
    Elons, A.S., Abull-ela, M., Tolba, M.: A proposed PCNN features quality optimization technique for pose-invariant 3D Arabic sign language recognition. Appl. Soft Comput. 13(4), 1646–1660 (2013).  https://doi.org/10.1016/j.asoc.2012.11.036 CrossRefGoogle Scholar
  35. 35.
    Tolba, M., Samir, A., Abull-ela, M.: 3D Arabic sign language recognition using liner combination of Multible 2D views. In: 2012 8th International Conference on Informatics and Systems (INFOS), IEEE, Cairo (2012)Google Scholar
  36. 36.
    Sutherland, A., Awad, G., Han, J.: Boosted subunits: a framework for recognising sign language from videos, IET Image Processing 7 (September 2012) (2013) 70–80.  https://doi.org/10.1049/iet-ipr.2012.0273 CrossRefGoogle Scholar
  37. 37.
    Awad, G., Han, J., Sutherland, A.: A unified system for segmentation and tracking of face and hands in sign language recognition. In: Proceedings—International Conference on Pattern Recognition, vol. 1, pp. 239–242 (2006).  https://doi.org/10.1109/ICPR.2006.194
  38. 38.
    Pasha, A., Al-Badrashiny, M., Diab, M. T., El Kholy, A., Eskander, R., Habash, N., Pooleery, M.,  Rambow, O., Roth, R.: MADAMIRA: a fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In: LREC, Vol. 14, 2014, pp. 1094–1101Google Scholar
  39. 39.
    Heafield, K., KenLM: Faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, Association for Computational Linguistics, pp. 187–197 (2011)Google Scholar
  40. 40.
    Stolcke, A.: SRILM-an extensible language modeling toolkit. In: Proceedings International Conference on Spoken Language Processing, pp. 257–286 (2002)Google Scholar
  41. 41.
    Federico, M., Bertoldi, N., Cettolo, M.: IRSTLM: an open source toolkit for handling large scale language models. In: Interspeech, pp. 1618–1621 (2008)Google Scholar
  42. 42.
    Neme, A.A., Laporte, E.: Pattern-and-root inflectional morphology: the arabic broken plural. Lang. Sci. 40, 221–250 (2013)CrossRefGoogle Scholar
  43. 43.
    Attia, M., Pecina, P., Tounsi, L., Toral, A., Van Genabith, J.: Lexical profiling for Arabic. In: Proceedings of eLex, pp. 23–33 (2011)Google Scholar
  44. 44.
    Elghamry, K.: A lexical-syntactic solution to the problem of broken plural in Arabic. In: G. U. R. T. G. 2010) (Ed.), Arabic Natural Language Processing Track, Washington, USA, (2010)Google Scholar
  45. 45.
    Yaqoub, E. B.: The detailed dictionary in the plurals. (In Arabic), Scientific Book House, (2004)Google Scholar
  46. 46.
    Chennoufi, A., Mazroui, A.: Morphological, syntactic and diacritics rules for automatic diacritization of Arabic sentences. J. King Saud Univ. Comput. Inf. Sci. 29(2), 156–163 (2017)Google Scholar
  47. 47.
    Gonàlez, M., Giménez, J., Màrquez, L.: A Graphical interface for MT evaluation and error analysis. In: The 50th Annual Meeting of the Association for Computational Linguistics (2012)Google Scholar
  48. 48.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, pp. 311–318 (2002)Google Scholar
  49. 49.
    Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, Vol. 200, (2006)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.King Fahd University of Petroleum and MineralsDhahranSaudi Arabia

Personalised recommendations