Skip to main content
SpringerLink
Log in

A Comprehensive Study on Relative Distances of Hand Landmarks Approach for American Sign Language Gesture

  • Original Paper
  • Published:
Augmented Human Research Aims and scope Submit manuscript

Abstract

Communication with people with hearing or speaking disabilities is always difficult when there is no knowledge of sign language. The presence of sign language is not enough to communicate smoothly, this process requires another easy medium for communication to make it more efficient, that is, via a digital medium. This paper proposes using Feed-Forward Neural Networks on hand landmarks for real-time sign language identification. The hand landmarks identification was carried out using the MediaPipe Hands library. This approach would make the classification problem efficient by making it faster and requiring less memory. Through this, we aim to bridge the gap between the difficulties that arise during communication between people who do and do not know American Sign Language.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

All relevant data and material are presented in the main paper.

References

  1. Kapur R (2020) The types of communication. ResearchGate 9:1–12

  2. Hess U (2016) Nonverbal communication. Encyclopedia of Mental Health 12.

  3. Stokoe Jr WC (1960) Sign language structure: an outline of the visual communication systems of the American deaf, vol 11, pp 1–78

  4. Klima ES, Bellugi U (1979) The signs of language. Harvard University Press, Cambridge

    Google Scholar 

  5. Morgan G (2014) On language acquisition in speech and sign: development drives combinatorial structure in both modalities. Front Psychol 5:47–57. https://doi.org/10.3389/fpsyg.2014.01217

    Article  Google Scholar 

  6. Garcia J (2020) How many people know sign language? signstation.org/how-many-people-know-sign-language/;

  7. Mustafa E, Dimopoulos K (2014) Sign language recognition using kinect, 0:9

  8. Battison R (1978) Lexical borrowing in American Sign Language. Linguist Am Sign Lang Introduction 9:199–218

    Google Scholar 

  9. Kodandaram SR, Kumar N, Gl S (2021) Sign language recognition 0:7

  10. Bellugi U, Fischer S (1972) A comparison of sign language and spoken language. Cognition 1:173–200. https://doi.org/10.1016/0010-0277(72)90018-2

    Article  Google Scholar 

  11. Corina DP, Hafer S, Welch K (2014) Phonological awareness for American sign language. J Deaf Stud Deaf Educ 19:530–545. https://doi.org/10.1093/deafed/enu023

    Article  PubMed  Google Scholar 

  12. Jachova Z, Kovacheva O, Karovska A (2008) Differences between American Sign Language (ASL) and British Sign Language (BSL). J Spec Educ Rehabilit 1:41–54

    Google Scholar 

  13. Shivashankara S, Srinath S (2018) American Sign language recognition system: an optimal approach. Int J Image Graph Signal Processing 10:18–30. https://doi.org/10.5815/ijigsp.2018.08.03

    Article  Google Scholar 

  14. Rioux-Maldague L, Giguere P (2014) Sign language fingerspelling classification from depth and color images using a deep belief network. In: Proceedings—conference on computer and robot vision, CRV 2014, vol 5, pp 92–7. https://doi.org/10.1109/CRV.2014.20

  15. Jameel HT, Bibi S (2016) Benefits of sign language for the deaf students in classroom learning. International Journal of Advanced and Applied Sciences 3:24–26

    Google Scholar 

  16. Schembri A, Crasborn O (2010) Issues in creating annotation standards for sign language description. In: Corpora and sign language technologies proceedings of the 4th workshop on the representation and processing of sign languages language resources and evaluation conference (LREC), vol 01, pp 212–216

  17. Pathak A, Kumar A, Gupta P, Chu G (2022) Real time sign language detection. Int J Mod Trends Sci Technol 8:32–37

    Google Scholar 

  18. San-Segundo RMDSAGR, López V (2010) Language resources for Spanish–Spanish sign language (LSE) translation, p ADD 2010:01

  19. Jadhav S, Chougula B, Rudrappa G, Vijapur N, Tigadi A (2022) GoogLeNet application towards gesture recognition for ASL character identification. In: IEEE International conference on distributed computing and electrical circuits and electronics, ICDCECE 2022, vol 2022, pp 1–5. https://doi.org/10.1109/ICDCECE53908.2022.9793165.

  20. Xie B, He X, Li Y (2018) RGB-D static gesture recognition based on convolutional neural network. J Eng 2018:1515–1520. https://doi.org/10.1049/joe.2018.8327

    Article  Google Scholar 

  21. Pugeault N, Bowden R (2011) Spelling it out: real-time ASL fingerspelling recognition. In: Proceedings of the IEEE international conference on computer vision, p 1114–9. https://doi.org/10.1109/ICCVW.2011.6130290.

  22. Islam MR, Mitu UK, Bhuiyan RA, Shin J (2018) Hand gesture feature extraction using deep convolutional neural network for recognizing American sign language. In: 2018 4th international conference on frontiers of signal processing, ICFSP 2018, vol 2018, pp 115–9. https://doi.org/10.1109/ICFSP.2018.8552044.

  23. Yan FM, Li SY (2019) Two-stream convolutional neural networks with natural light and depth images for hand gesture recognition. In: 2019 12th Asian control conference, ASCC 2019, vol 2019, pp 1519–24

  24. Jirathampradub H, Nukoolkit C, Suriyathumrongkul K, Watanapa B (2020) A 3D-CNN Siamese network for motion gesture sign language alphabets recognition. In: ACM international conference proceeding series, vol 7, pp 1–6. https://doi.org/10.1145/3406601.3406634.

  25. Izutov E (2020) ASL Recognition with metric-learning based lightweight network 0;4

  26. Joze HR V, Koller O (2018) MS-ASL: A large-scale data set and benchmark for understanding american sign language. CoRR, p 1812

  27. Mohmmad S, Dadi R, Harshavardhan A, Pasha S (2020) Static hand gesture recognition for ASL using MATLAB platform. J Mech Cont Math Sci 15:315–329

    Google Scholar 

  28. Nagarajan S, Subashini TS (2013) Static hand gesture recognition for sign language alphabets using edge oriented histogram and multi class SVM. Int J Comput Appl 82:28–35. https://doi.org/10.5120/14106-2145

    Article  Google Scholar 

  29. Kaslay S, Kesarkar T, Shinde K (2020) ASL gesture recognition using various feature extraction techniques and SVM, vol 07

  30. Ziaie P, Müller T, Foster ME, Knoll A (2009) A naïve Bayes classifier with distance weighting for hand-gesture recognition. In: Sarbazi-Azad BP, Miremadi S-G, Hessabi S (eds) Advances in computer science and engineering. Springer Berlin Heidelberg, Berlin, pp 308–315

    Google Scholar 

  31. Jaiswal P (2018) Automatic ASL gesture recognition system using convolutional neural network. Asian J Converg Technol (AJCT) ISSN 4:1146–2350

    Google Scholar 

  32. Dabwan B (2020) Convolutional neural network-based sign language translation system 9:6

  33. Ma Y, Xu T, Kim K (2022) Two-stream mixed convolutional neural network for American sign language recognition. Sensors 22(16):5959

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  34. Rastgoo R, Kiani K, Escalera S (2022) Real-time isolated hand sign language recognition using deep networks and SVD. J Ambient Intell Humaniz Comput 13(1):591–611

    Article  Google Scholar 

  35. Ravi Kumar R, Mohmmad S, Shabana, Kothandaraman D, Ramesh D (2023) Static hand gesture recognition for ASL using MATLAB platform. Lecture Notes in Networks and Systems 459:379–92. https://doi.org/10.1007/978-981-19-1976-3_47.

  36. Caffe YJ (2020) MediaPipe Hands. mediapipe/solutions/hands.html

  37. Zhang F, Bazarevsky V, Vakunov A, Tkachenka A, Sung G, Chang C et al (2020) Mediapipe hands: on-device real-time hand tracking. CoRR 2006:10214

    Google Scholar 

  38. Simon T, Joo H, Matthews I, Sheikh Y (2017) Hand keypoint detection in single images using multiview bootstrapping. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017 2017, pp 4645–53. https://doi.org/10.1109/CVPR.2017.494.

  39. Arikeri P (2021) {A}merican Sign Language (ASL) dataset

Download references

Acknowledgements

The authors are grateful to Department of Computer Science, North Carolina State University, School of Computing and Augmented Intelligence, Arizona State University, Department of Computer Science Engineering, School of Technology and Department of Chemical Engineering, School of Energy Technology, Pandit Deendayal Energy University for the permission to publish this research.

Funding

Not applicable

Author information

Authors and Affiliations

  1. Department of Computer Engineering, School of Technology, Pandit Deendayal Energy University, Gandhinagar, Gujarat, 382426, India

    Kishan Pipariya

  2. Department of Chemical Engineering, School of Energy Technology, Pandit Deendayal Energy University, Gandhinagar, Gujarat, 382426, India

    Manan Shah

  3. Department of Computer Science, North Carolina State University, North Carolina, Raleigh, USA

    Shail Shah

  4. School of Computing and Augmented Intelligence, Arizona State University, Arizona, Tempe, USA

    Jaynil Vaidya

Authors
  1. Shail Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaynil Vaidya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kishan Pipariya
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manan Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors make a substantial contribution to this manuscript. SS, JV, KP and MS participated in drafting the manuscript. SS, JV and MS wrote the main manuscript. All the authors discussed the results and implication on the manuscript at all stages.

Corresponding author

Correspondence to Manan Shah.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shah, S., Vaidya, J., Pipariya, K. et al. A Comprehensive Study on Relative Distances of Hand Landmarks Approach for American Sign Language Gesture. Augment Hum Res 9, 1 (2024). https://doi.org/10.1007/s41133-024-00064-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41133-024-00064-w

Keywords

Search

Navigation