Abstract
Deaf and hearing-impaired individuals who communicate using sign language face several communication difficulties. Because the vast majority of people do not know sign language, the need for a sign language translator is growing significantly, especially for Arabic sign language (ArSL). Today, technology plays a significant role in people’s lives. Leap Motion technology can be used to address the aforementioned issues and improve communication between Saudi Arabia’s deaf and hearing individuals. In this study, we investigated the possibility of using a Leap Motion system to provide continuous ArSL recognition for two-way communication to improve communication between deaf and hearing individuals in terms of speed and independence. The system translates ArSL into spoken words for hearing individuals and transcribes spoken Arabic language into text for deaf individuals.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Deafness and hearing loss. https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss. Accessed 16 May 2020
Disability Survey 2017 | General Authority for Statistics. https://www.stats.gov.sa/en/5669. Accessed 16 May 2020
Population Estimates | General Authority for Statistics. https://www.stats.gov.sa/en/43. Accessed 16 May 2020
Alnafjan, A., Aljumaah, A., Alaskar, H., Alshraihi, R.: Designing ‘Najeeb’: technology-enhanced learning for children with impaired hearing using arabic sign-language ArSL applications. In: International Conference on Computer and Applications (ICCA), pp. 238–243 (2017)
Al-Nafjan, A., Al-Arifi, B., Al-Wabil, A.: Design and development of an educational arabic sign language mobile application: collective impact with tawasol. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2015. LNCS, vol. 9176, pp. 319–326. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20681-3_30
Joy, J., Balakrishnan, K., Sreeraj, M.: SignQuiz: a quiz based tool for learning fingerspelled signs in indian sign language using ASLR. IEEE Access 7, 28363–28371 (2019)
Lee, B.G., Lee, S.M.: Smart wearable hand device for sign language interpretation system with sensors fusion. IEEE Sens. J. 18(3), 1224–1232 (2018)
Mohandes, M., Aliyu, S., Deriche, M.: Prototype arabic sign language recognition using multi-sensor data fusion of two leap motion controllers. In: 12th International Multi-Conference on Systems, Signals and Devices (2015)
Elbadawy, M., Elons, A.S., Shedeed, H.A., Tolba, M.F.: Arabic sign language recognition with 3D convolutional neural networks. In: IEEE 8th International Conference on Intelligent Computing and Information Systems (ICICIS), vol. 2018-Janua, pp. 66–71 (2018)
Mohandes, M., Aliyu, S., Deriche, M.: Arabic sign language recognition using the leap motion controller. In: IEEE International Symposium on Industrial Electronics, pp. 960–965 (2014)
Hisham, B., Hamouda, A.: Arabic static and dynamic gestures recognition using leap motion. J. Comput. Sci. 13(8), 337–354 (2017)
Suarez, J., Murphy, R.R.: Hand gesture recognition with depth images: a review. In: IEEE International Workshop on Robot and Human Interactive Communication, pp. 411–417 (2012)
Guzsvinecz, T., Szucs, V., Sik-Lanyi, C.: Suitability of the kinect sensor and leap motion controller—a literature review. Sensors 19(5), 1072 (2019)
Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with leap motion and kinect devices. In: IEEE International Conference on Image Processing, pp. 1565–1569 (2014)
Almasre, M.A., Al-Nuaim, H.: A real-time letter recognition model for arabic sign language using kinect and leap motion controller v2. Int. J. Adv. Eng. Manag. Sci. (IJAEMS) 2(5), 239469 (2016)
Sun, C., Zhang, T., Xu, C.: Latent support vector machine modeling for sign language recognition with kinect. ACM Trans. Intell. Syst. Technol. (TIST) 6(2), 1–20 (2015)
Naglot, D., Kulkarni, M.: Real time sign language recognition using the leap motion controller. In: International Conference on Inventive Computation Technologies (ICICT), vol. 2016 (2016)
Clark, A., Moodley, D.: A system for a hand gesture-manipulated virtual reality environment. In: Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists, vol. 26-28-Sept, pp. 1–10 (2016)
Khelil, B., Amiri, H.: Hand gesture recognition using leap motion controller for recognition of arabic sign language. In: 3rd International Conference on Automation, Control Engineering and Computer Science (ACECS), pp. 233–238 (2016)
Deriche, M., Aliyu, S., Mohandes, M.: An intelligent arabic sign language recognition system using a pair of LMCs with GMM based classification. IEEE Sens. J. 19(18), 1–12 (2019)
Digital worlds that feel human | Ultraleap. https://www.ultraleap.com/. Accessed: 16 May 2020
Acknowledgment
We thank the Humanistic Co-Design Initiative and the Human-Computer Interaction (HCI) Lab for supporting this work. We also acknowledge the contribution of the Saudi Authority for Intellectual Property (SAIP) and the Saudi Health Council’s National Lab for Emerging Health Technologies in hosting and mentoring this work. This work is part of the authors’ project that is carried out under the CoCreate Fellowship for Humanistic Co-Design of Access Technologies.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Al-Nafjan, A., Al-Abdullatef, L., Al-Ghamdi, M., Al-Khalaf, N., Al-Zahrani, W. (2020). Designing SignSpeak, an Arabic Sign Language Recognition System. In: Stephanidis, C., Antona, M., Gao, Q., Zhou, J. (eds) HCI International 2020 – Late Breaking Papers: Universal Access and Inclusive Design. HCII 2020. Lecture Notes in Computer Science(), vol 12426. Springer, Cham. https://doi.org/10.1007/978-3-030-60149-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-60149-2_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60148-5
Online ISBN: 978-3-030-60149-2
eBook Packages: Computer ScienceComputer Science (R0)