Skip to main content
SpringerLink
Log in
Book cover

Intelligent Data Communication Technologies and Internet of Things pp 239–249Cite as

Hand Gesture Recognition for Disabled Person with Speech Using CNN

  • 829 Accesses

Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT,volume 101)

Abstract

Because handicapped people account for a large percentage of our community, we should make an effort to interact with them in order to exchange knowledge, perspectives, and ideas. To that aim, we wish to establish a means of contact. Individuals who are deaf or hard of hearing can communicate with one another using sign language. A handicapped person can communicate without using acoustic noises when they use sign language. The objective of this article is to explain the design and development of a hand gesture-based sign language recognition system. To aid handicapped individuals, particularly those who are unable to communicate verbally, sign language is translated into text and subsequently into speech. The solution is based on a web camera as the major component, which is used to record a live stream video using a proprietary MATLAB algorithm. Recognition of hand movements is possible with the technology. Recognizing hand gestures is a straightforward technique of providing a meaningful, highly flexible interaction between robots and their users. There is no physical communication between the user and the devices. A deep learning system that is efficient at picture recognition is used to locate the dynamically recorded hand movements. Convolutional neural networks are used to optimize performance. A static image of a hand gesture is used to train the model. Without relying on a pre-trained model, the CNN is constructed.

Keywords

  • Human–computer interaction
  • Gesture recognition
  • Web camera
  • CNN
  • MATLAB

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Hegde B, Dayananda P, Hegde M, Chetan C (2019) Deep learning technique for detecting NSCLC. Int J Recent Technol Eng (IJRTE) 8(3):7841–7843

    Google Scholar 

  2. Sunitha KA, Anitha Saraswathi P, Aarthi M, Jayapriya K, Sunny L (2016) Deaf mute communication interpreter—a review. Int J Appl Eng Res 11:290–296

    Google Scholar 

  3. Anand MS, Kumar NM, Kumaresan A (2016) An efficient framework for Indian sign language recognition using wavelet transform. Circuits Syst 7:1874–1883

    Google Scholar 

  4. Ahuja MK, Singh A (2015) Hand gesture recognition using PCA. Int J Comput Sci Eng Technol (IJCSET) 5(7):267–27

    Google Scholar 

  5. More SP, Sattar A, Hand gesture recognition system for dumb people. Int J Sci Res (IJSR)

    Google Scholar 

  6. Kaur C, Gill N, An automated system for Indian sign language recognition. Int J Adv Res Comput Sci Software Eng

    Google Scholar 

  7. Pandey P, Jain V (2015) Hand gesture recognition for sign language recognition: a review. Int J Sci Eng Technol Res (IJSETR) 4(3)

    Google Scholar 

  8. Nagpal N, Mitra A, Agrawal P (2019) Design issue and proposed implementation of communication Aid for Deaf & Dumb People. Int J Recent Innov Trends Comput Commun 3(5):147–149

    Google Scholar 

  9. Gilorkar NK, Ingle MM (2015) Real time detection and recognition of Indian and American sign language using sift. Int J Electron Commun Eng Technol (IJECET) 5(5):11–18

    Google Scholar 

  10. Shinde V, Bacchav T, Pawar J, Sanap M (2014) Hand gesture recognition system using camera 03(01)

    Google Scholar 

  11. Gebrera ME (2016) Glove-based gesture recognition system. In: IEEE international conference on robotics and biomimetics 2016

    Google Scholar 

  12. Siam SM, Sakel JA (2016) Human computer interaction using marker based hand gesture recognition

    Google Scholar 

  13. Shneiderman B, Plaisant C, Cohen M, Jacobs S, Elmqvist N, Diakopoulos N (2016) Designing the user interface: strategies for effective human-computer interaction Pearson

    Google Scholar 

  14. Lawrence DO, Ashleigh M (2019) Impact of Human-Computer Interaction (HCI) on users in higher educational system: Southampton University as a case study. Int J Manage Technol 6(3):1–12

    Google Scholar 

  15. Chu JU, Jung DH, Lee YJ (2008) Design and control of a multifunction myoelectric hand with new adaptive grasping and self-locking mechanisms. In: 2008 IEEE international conference on robotics and automation, pp 743–748, May 2008

    Google Scholar 

  16. Marques O (2011) Practical image and video processing using MATLAB. Wiley

    Google Scholar 

  17. McAndrew A (2004) An introduction to digital image processing with Matlab notes for SCM2511 image processing, p 264

    Google Scholar 

  18. Khan TM, Bailey DG, Khan MA, Kong Y (2017) Efficient hardware implementation for fingerprint image enhancement using anisotropic Gaussian filter. IEEE Trans Image Process 26(5):2116–2126

    CrossRef  MathSciNet  Google Scholar 

  19. Nishad PM (2013) Various colour spaces and colour space conversion. J Global Res Comput Sci 4(1):44–48

    Google Scholar 

  20. Abhishek B, Krishi K, Meghana M, Daaniyaal M, Anupama HS (2019) Hand gesture recognition using machine learning algorithms. Int J Recent Technol Eng (IJRTE) 8(1) ISSN: 2277-3878

    Google Scholar 

  21. Ankita W, Parteek K (2020) Deep learning-based sign language recognition system for static signs. Neural Comput Appl

    Google Scholar 

  22. Khan A, Sohail A, Zahoora U, Qureshi AS (2020) A survey of the recent architectures of deep convolutional neural networks. Comput Vis Pattern Recogn

    Google Scholar 

  23. Chuan CH, Regina E, Guardino C (2014)American sign language recognition using leap motion sensor. In: 13th International Conference on Machine Learning and Applications (ICMLA), pp 541–544

    Google Scholar 

  24. Devineau G, Moutarde F, Xi W, Yang J (2018) Deep learning for hand gesture recognition on skeletal data. In: 13th IEEE International conference on automatic face & gesture recognition, pp 106–113

    Google Scholar 

  25. Nair AT, Muthuvel K (2021) Automated screening of diabetic retinopathy with optimized deep convolutional neural network: enhanced moth flame model. J Mech Med Biol 21(1):2150005 (29 p). https://doi.org/10.1142/S0219519421500056

  26. Nair AT, Muthuvel K (2019) Blood vessel segmentation and diabetic retinopathy recognition: an intelligent approach. Comput Methods Biomech Biomed Eng: Imaging Visual. https://doi.org/10.1080/21681163.2019.1647459

  27. Nair AT, Muthuvel K (2020) Research contributions with algorithmic comparison on the diagnosis of diabetic retinopathy. Int J Image Graphics 20(4):2050030(29 p). https://doi.org/10.1142/S0219467820500308

  28. Nair AT, Muthuvel K, Haritha KS (2020) Effectual evaluation on diabetic retinopathy. In: Publication in lecture notes. Springer, Berlin

    Google Scholar 

  29. Nair AT, Muthuvel K, Haritha KS (2021) Blood vessel segmentation for diabetic retinopathy. In: Publication in the IOP: Journal of Physics Conference Series (JPCS)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electronics and Communication Engineering, KMCT College of Engineering, Kozhikode, Kerala, India

    E. P. Shadiya Febin

  2. KMCT College of Engineering, Kozhikode, Kerala, India

    Arun T. Nair

Authors
  1. E. P. Shadiya Febin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arun T. Nair
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electronics and Communication Engineering, Karunya Institute of Technology and Sciences, Coimbatore, India

    Dr. D. Jude Hemanth

  2. Faculty of Communication Sciences, University of Teramo, Teramo, Italy

    Dr. Danilo Pelusi

  3. Department of Computer Engineering, San Jose State University, San Jose, CA, USA

    Dr. Chandrasekar Vuppalapati

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shadiya Febin, E.P., Nair, A.T. (2022). Hand Gesture Recognition for Disabled Person with Speech Using CNN. In: Hemanth, D.J., Pelusi, D., Vuppalapati, C. (eds) Intelligent Data Communication Technologies and Internet of Things. Lecture Notes on Data Engineering and Communications Technologies, vol 101. Springer, Singapore. https://doi.org/10.1007/978-981-16-7610-9_17

Download citation

search

Navigation