Skip to main content

Advertisement

SpringerLink
Log in

Sign language recognition using artificial intelligence

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Sign language is the natural way of communication of speech and hearing-impaired people. Using Indian Sign Language (ISL) interpretation system, hearing impaired people may interact with normal people with the help of Human Computer Interaction (HCI). This paper presents a method for automatic recognition of two-handed signs of Indian Sign language (ISL). The three phases of this work include preprocessing, feature extraction and classification. We trained a BPN with Histogram Oriented Gradient (HOG) features. The trained model is used for testing the real time gestures. The overall accuracy achieved was 89.5% with 5184 input features and 50 hidden neurons. A deep learning approach was also implemented using AlexNet, GoogleNet, VGG-16 and VGG-19 which gave accuracies of 99.11%, 95.84%, 98.42% and 99.11% respectively. MATLAB is used as the simulation platform. The proposed technology is used as a teaching assistant for specially abled persons and has demonstrated an increase in cognitive ability of 60–70% in children. This system demonstrates image processing and machine learning approaches to recognize alphabets from the Indian sign language, which can be used as an ICT (information and communication technology) tool to enhance their cognitive capability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  • Abubakar, F. M. (2012). Image enhancement using histogram equalization and spatial filtering. International Journal of Science and Research (IJSR), 1(3), 105–107.

    Google Scholar 

  • Athira, P. K., Sruthi, C. J., & Lijiya, A. (2019). A signer independent sign language recognition with co-articulation elimination from live videos: an Indian scenario. Journal of King Saud University-Computer and Information Sciences.

  • Balbin, J. R., Padilla, D. A., Caluyo, F. S., Fausto, J. C., Hortinela, C. C., Manlises, C. O., … & Ventura, L. T. (2016). Sign language word translator using neural networks for the aurally impaired as a tool for communication. In 2016 6th IEEE International Conference on Control System, Computing and Engineering (ICCSCE) (pp. 425–429). IEEE.

  • Bhame, V., Sreemathy, R., & Dhumal, H. (2014). Vision based hand gesture recognition using eccentric approach for human computer interaction. In 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (pp. 949–953). IEEE.

  • Diaz, C. A. M., Castaneda, E. E. M., & Vassallo, C. A. M. (2019). Deep learning for plant classification in precision agriculture. In 2019 International Conference on Computer, Control, Informatics and its Applications (IC3INA) (pp. 9–13). IEEE.

  • Dutta, K. K., & GS, A. K. (2015). Double handed Indian Sign Language to speech and text. In 2015 Third International Conference on Image Information Processing (ICIIP) (pp. 374–377). IEEE.

  • Ghani, F. (2004). Improved 2-D median filter for on-line impulse noise suppression. In IEEE Region 10 Conference.

  • Konwar, A. S., Borah, B. S., & Tuithung, C. T. (2014). An American sign language detection system using HSV color model and edge detection. In 2014 International Conference on Communication and Signal Processing (pp. 743–747). IEEE.

  • Kothadiya, D., Bhatt, C., Sapariya, K., Patel, K., Gil-González, A. B., & Corchado, J. M. (2022). Deepsign: Sign language detection and recognition using deep learning. Electronics, 11(11), 1780.

    Article  Google Scholar 

  • Kurhekar, P., Phadtare, J., Sinha, S., & Shirsat, K. P. (2019). Real time sign language estimation system. In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI) (pp. 654–658). IEEE.

  • Nimisha, K. P., & Jacob, A. (2020). A brief review of the recent trends in sign language recognition. In 2020 International Conference on Communication and Signal Processing (ICCSP) (pp. 186–190). IEEE.

  • Papastratis, I., Chatzikonstantinou, C., Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2021). Artificial intelligence technologies for sign language. Sensors, 21(17), 5843.

    Article  Google Scholar 

  • Raheja, J. L., Mishra, A., & Chaudhary, A. (2016). Indian sign language recognition using SVM. Pattern Recognition and Image Analysis, 26(2), 434–441.

    Article  Google Scholar 

  • Rao, G. A., & Kishore, P. V. V. (2018). Selfie video based continuous Indian sign language recognition system. Ain Shams Engineering Journal, 9(4), 1929–1939.

    Article  Google Scholar 

  • Sahoo, A. K., & Ravulakollu, K. K. (2014). Indian sign language recognition using skin color detection. International Journal of Applied Engineering Research (IJAER), 9(20), 7347–7360.

    Google Scholar 

  • Sarkaleh, A. K., Poorahangaryan, F., Zanj, B., & Karami, A. (2009). A Neural Network based system for Persian sign language recognition. In 2009 IEEE International Conference on Signal and Image Processing Applications (pp. 145–149). IEEE.

  • Saxena, A., Jain, D. K., & Singhal, A. (2014). Sign language recognition using principal component analysis. In 2014 Fourth International Conference on Communication Systems and Network Technologies (pp. 810-813). IEEE.

  • Shuhua, L., & Gaizhi, G. (2010). The application of improved HSV color space model in image processing. In 2010 2nd International Conference on Future Computer and Communication (Vol. 2, pp. V2–10). IEEE.

  • Singh, R. P., & Dixit, M. (2015). Histogram equalization: A strong technique for image enhancement. International Journal of Signal Processing, Image Processing and Pattern Recognition, 8(8), 345–352.

    Article  Google Scholar 

  • Sridhar, A., Ganesan, R. G., Kumar, P., & Khapra, M. (2020). Include: A large scale dataset for indian sign language recognition. In Proceedings of the 28th ACM international conference on multimedia (pp. 1366–1375).

  • Stergiopoulou, E., & Papamarkos, N. (2009). Hand gesture recognition using a neural network shape fitting technique. Engineering Applications of Artificial Intelligence, 22(8), 1141–1158.

    Article  Google Scholar 

  • Sural, S., Qian, G., & Pramanik, S. (2002). Segmentation and histogram generation using the HSV color space for image retrieval. In Proceedings. International Conference on Image Processing (Vol. 2, pp. II-II). IEEE.

  • Verma, V. K., Srivastava, S., & Kumar, N. (2015). A comprehensive review on automation of Indian sign language. In 2015 International Conference on Advances in Computer Engineering and Applications (pp. 138-142). IEEE

Download references

Acknowledgements

We thank Rajiv Gandhi science & Technology Commission, Government of Maharashtra, INDIA for funding this work. We also acknowledge the support provided by Pune Institute of Computer Technology, INDIA.

Author information

Authors and Affiliations

  1. Pune Institute of Computer Technology, Pune, India

    R. Sreemathy, Mousami Turuk, Isha Kulkarni & Soumya Khurana

Authors
  1. R. Sreemathy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mousami Turuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Isha Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Soumya Khurana
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mousami Turuk.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sreemathy, R., Turuk, M., Kulkarni, I. et al. Sign language recognition using artificial intelligence. Educ Inf Technol 28, 5259–5278 (2023). https://doi.org/10.1007/s10639-022-11391-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-022-11391-z

Keywords

Search

Navigation