Skip to main content
SpringerLink
Log in

ArSign: Toward a Mobile Based Arabic Sign Language Translator Using LMC

  • Conference paper
  • First Online:
Universal Access in Human-Computer Interaction. Applications and Practice (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12189))

Included in the following conference series:

Abstract

Communication connects people by allowing them to exchange messages, to express their feelings either verbally or non-verbally. To communicate with their surroundings, hearing impaired people or deaf use sign language. Unfortunately, practicing sign language is not common among society witch make barriers between peoples. In this work we propose a new system to convert Arabic sign language from gesture to written text. To do that we propose a system based on three main components: 1) The Leap motion controller as gesture recognition device to capture hand and finger movements. 2) A processing module to convert recognized gesture to alphabet using a comparison algorithm working on a predefined dataset of gesture. 3) A user interface to display the text or to convert it to speech through text to speech engine. The proposed system is implemented as a mobile application running into an android based device.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.ultraleap.com/product/leap-motion-controller/.

References

  1. Sadek, M.I., Mikhael, M.N., Mansour, H.A.: A new approach for designing a smart glove for Arabic sign language recognition system based on the statistical analysis of the sign language. In: Proceedings of the National Radio Science Conference (NRSC), pp. 380–388 (2017). https://doi.org/10.1109/NRSC.2017.7893499

  2. Ahmed, M.A., Zaidan, B.B., Zaidan, A.A., Salih, M.M., Lakulu, M.M.B.: A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors (Switzerland) 18 (2018). https://doi.org/10.3390/s18072208

  3. Bui, T.D., Nguyen, L.T.: Recognizing postures in Vietnamese sign language with MEMS accelerometers. IEEE Sens. J. 7, 707–712 (2007). https://doi.org/10.1109/JSEN.2007.894132

    Article  Google Scholar 

  4. López-Noriega, J.E., Fernández-Valladares, M.I., Uc-Cetina, V.: Glove-based sign language recognition solution to assist communication for deaf users. In: 2014 11th International Conference on Electrical Engineering, Computing Science and Automatic Control, CCE 2014 (2014). https://doi.org/10.1109/ICEEE.2014.6978268

  5. Pławiak, P., Sośnicki, T., Niedźwiecki, M., Tabor, Z., Rzecki, K.: Hand body language gesture recognition based on signals from specialized glove and machine learning algorithms. IEEE Trans. Industr. Inf. 12, 1104–1113 (2016). https://doi.org/10.1109/TII.2016.2550528

    Article  Google Scholar 

  6. Preetham, C., Ramakrishnan, G., Kumar, S., Tamse, A., Krishnapura, N.: Hand talk-implementation of a gesture recognizing glove. In: Proceedings - 2013 Texas Instruments India Educators’ Conference, TIIEC 2013, pp. 328–331 (2013). https://doi.org/10.1109/TIIEC.2013.65

  7. Sriram, N., Nithiyanandham, M.: A hand gesture recognition based communication system for silent speakers. In: 2013 International Conference on Human Computer Interactions, ICHCI 2013, pp. 1–5 (2013). https://doi.org/10.1109/ICHCI-IEEE.2013.6887815

  8. Lee, B.G., Lee, S.M.: Smart wearable hand device for sign language interpretation system with sensors fusion. IEEE Sens. J. 18, 1224–1232 (2018). https://doi.org/10.1109/JSEN.2017.2779466

    Article  Google Scholar 

  9. Haq, E.S., Suwardiyanto, D., Huda, M.: Indonesian sign language recognition application for two-way communication deaf-mute people. In: Proceedings - 2018 3rd International Conference on Information Technology, Information System and Electrical Engineering, ICITISEE 2018, pp. 313–318 (2018). https://doi.org/10.1109/ICITISEE.2018.8720982

  10. Fok, K.Y., Ganganath, N., Cheng, C.T., Tse, C.K.: A real-time ASL recognition system using leap motion sensors. In: Proceedings - 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC 2015, pp. 411–414 (2015). https://doi.org/10.1109/CyberC.2015.81

  11. Ibrahim, N.B., Selim, M.M., Zayed, H.H.: An automatic Arabic sign language recognition system (ArSLRS). J. King Saud Univ. Comput. Inf. Sci. 30, 470–477 (2018). https://doi.org/10.1016/j.jksuci.2017.09.007

    Article  Google Scholar 

  12. El-Jaber, M., Assaleh, K.: ISMA10-1 ISMA10-2. In: 7th International Symposium on Mechatronics and its Applications, pp. 1–4 (2010)

    Google Scholar 

  13. Nikam, A.S., Ambekar, A.G.: Bilingual sign recognition using image based hand gesture technique for hearing and speech impaired people. In: Proceedings - 2nd International Conference on Computing Communication Control and automation, ICCUBEA 2016, pp. 1–6 (2017). https://doi.org/10.1109/ICCUBEA.2016.7860057

  14. Lang, S., Block, M., Rojas, R.: Sign language recognition using kinect. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012. LNCS (LNAI), vol. 7267, pp. 394–402. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29347-4_46

    Chapter  Google Scholar 

  15. Ahmed, M., Idrees, M., Ul Abideen, Z., Mumtaz, R., Khalique, S.: Deaf talk using 3D animated sign language: a sign language interpreter using Microsoft’s kinect v2. In: Proceedings of the 2016 SAI Computing Conference, SAI 2016, pp. 330–335 (2016). https://doi.org/10.1109/SAI.2016.7556002

  16. Verma, H.V., Aggarwal, E., Chandra, S.: Gesture recognition using kinect for sign language translation. In: 2013 IEEE 2nd International Conference on Image Information Processing, ICIIP 2013, pp. 96–100. IEEE (2013). https://doi.org/10.1109/ICIIP.2013.6707563

  17. Unutmaz, B., Karaca, A.C., Güllü, M.K.: Kinect İ skelet ve Evri ş imsel Sinir A ğ ları ile Türkçe İş aret Dili Tanıma Turkish Sign Language Recognition Using Kinect Skeleton and Convolutional Neural Network, pp. 2–5 (2019)

    Google Scholar 

  18. Bessa Carneiro, S., De Santos, E.D.F.M., De Barbosa, T.M.A., Ferreira, J.O., Alcalá, S.G.S., Da Rocha, A.F.: Static gestures recognition for Brazilian sign language with kinect sensor. In: Proceedings of the IEEE Sensors (2017). https://doi.org/10.1109/ICSENS.2016.7808522

  19. Roh, M.-C., Lee, S.-W.: Human gesture recognition using a simplified dynamic Bayesian network. Multimedia Syst. 21(6), 557–568 (2014). https://doi.org/10.1007/s00530-014-0414-9

    Article  Google Scholar 

  20. Kang, B., Tripathi, S., Nguyen, T.Q.: Real-time sign language fingerspelling recognition using convolutional neural networks from depth map. In: Proceedings - 3rd IAPR Asian Conference on Pattern Recognition, ACPR 2015, pp. 136–140 (2016). https://doi.org/10.1109/ACPR.2015.7486481

  21. Shangeetha, R.K., Valliammai, V., Padmavathi, S.: Computer vision based approach for Indian sign language character recognition. In: 2012 International Conference on Machine Vision and Image Processing, MVIP 2012, pp. 181–184 (2012). https://doi.org/10.1109/MVIP.2012.6428790

  22. Mohandes, M., Deriche, M., Liu, J.: Image-based and sensor-based approaches to Arabic sign language recognition. IEEE Trans. Hum.-Mach. Syst. 44, 551–557 (2014). https://doi.org/10.1109/THMS.2014.2318280

    Article  Google Scholar 

  23. ZakiAbdo, M., Mahmoud Hamdy, A., Abd El-Rahman Salem, S., Mostafa Saad, E.-S.: Arabic sign language recognition. Int. J. Comput. Appl. 89, 19–26 (2014). https://doi.org/10.5120/15747-4523

    Article  Google Scholar 

  24. Ahmed, A.A., Aly, S.: Appearance-based Arabic sign language recognition using hidden Markov models. In: ICET 2014 – 2nd International Conference on Engineering and Technology (2015). https://doi.org/10.1109/ICEngTechnol.2014.7016804

  25. Hayani, S., Benaddy, M., El Meslouhi, O., Kardouchi, M.: Arab Sign language recognition with convolutional neural networks. In: Proceedings of the 2019 International Conference of Computer Science and Renewable Energies, ICCSRE 2019, pp. 1–4 (2019). https://doi.org/10.1109/ICCSRE.2019.8807586

  26. Latif, G., Mohammad, N., Alghazo, J., AlKhalaf, R., AlKhalaf, R.: ArASL: Arabic alphabets sign language dataset. Data Brief 23, 103777 (2019). https://doi.org/10.1016/j.dib.2019.103777

    Article  Google Scholar 

  27. Shukor, A.Z., et al.: A new data glove approach for Malaysian sign language detection. Procedia Comput. Sci. 76, 60–67 (2015). https://doi.org/10.1016/j.procs.2015.12.276

    Article  Google Scholar 

  28. More, S.P., Sattar, A.: Hand gesture recognition system using image processing. In: 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT), Chennai, pp. 671–675 (2016). https://doi.org/10.1109/ICEEOT.2016.7754766

Download references

Author information

Authors and Affiliations

  1. Information System Department, Taibah University, Medina, Saudi Arabia

    Slim Kammoun, Dawlat Darwish, Hanan Althubeany & Reem Alfull

  2. Research Laboratory of Technologies of Information and Communication and Electrical Engineering, University of Tunis, Tunis, Tunisia

    Slim Kammoun

Authors
  1. Slim Kammoun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dawlat Darwish
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanan Althubeany
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Reem Alfull
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Slim Kammoun .

Editor information

Editors and Affiliations

  1. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Margherita Antona

  2. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kammoun, S., Darwish, D., Althubeany, H., Alfull, R. (2020). ArSign: Toward a Mobile Based Arabic Sign Language Translator Using LMC. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Applications and Practice. HCII 2020. Lecture Notes in Computer Science(), vol 12189. Springer, Cham. https://doi.org/10.1007/978-3-030-49108-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49108-6_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49107-9

  • Online ISBN: 978-3-030-49108-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation