Advertisement

Hybrid SIFT Feature Extraction Approach for Indian Sign Language Recognition System Based on CNN

  • Abhishek DudhalEmail author
  • Heramb Mathkar
  • Abhishek Jain
  • Omkar Kadam
  • Mahesh Shirole
Conference paper
Part of the Lecture Notes in Computational Vision and Biomechanics book series (LNCVB, volume 30)

Abstract

Indian sign language (ISL) is one of the most used sign languages in the Indian subcontinent. This research aims at developing a simple Indian sign language recognition system based on convolutional neural network (CNN). The proposed system needs webcam and laptop and hence can be used anywhere. CNN is used for image classification. Scale invariant feature transformation (SIFT) is hybridized with adaptive thresholding and Gaussian blur image smoothing for feature extraction. Due to unavailability of ISL dataset, a dataset of 5000 images, 100 images each for 50 gestures, has been created. The system is implemented and tested using python-based library Keras. The proposed CNN with hybrid SIFT implementation achieves 92.78% accuracy, whereas the accuracy of 91.84% was achieved for CNN with adaptive thresholding.

Keywords

Indian sign language Convolutional neural network Adaptive thresholding Gaussian blur SIFT 

Notes

Acknowledgements

We would like to thank Principal of Deaf and Dumb School, Mumbai for her help in understanding ISL gestures. We would like to thank teachers and students of Deaf and Dumb School for their help in the creation of ISL dataset.

References

  1. 1.
    Heera SY et al (2017) Talking hands—an Indian sign language to speech translating gloves. In: 2017 International conference on innovative mechanisms for industry applications (ICIMIA)Google Scholar
  2. 2.
     Ekbote J et al (2017) Indian sign language recognition using ANN and SVM classifiers. In: 2017 International conference on innovations in information, embedded and communication systems (ICIIECS)Google Scholar
  3. 3.
    Duch W (2005) Artificial neural network biological inspirations. In: ICANN 2005: 15th international conference, Warsaw, Poland, September 11–15, 2005, proceedings, Pt. 1. Springer, HeidelbergGoogle Scholar
  4. 4.
    Qi X et al (2017) Data classification with support vector machine and generalized support vector machineGoogle Scholar
  5. 5.
    Vo T et al (2015) Tensor decomposition and application in image classification with histogram of oriented gradients. Neurocomputing 165:38–45CrossRefGoogle Scholar
  6. 6.
    Beena MV, Agnisarman Namboodiri MN (2017) Automatic sign language finger spelling using convolution neural network: analysis. Int J Pure Appl Math 117(20)Google Scholar
  7. 7.
    Aghdam HH, Heravi EJ (2017) Convolutional neural networks. In: Guide to convolutional neural networks, pp 85–130Google Scholar
  8. 8.
    Pigou L et al (2015) Sign language recognition using convolutional neural networks. Computer vision—ECCV 2014 workshops lecture notes in computer science, pp 572–578CrossRefGoogle Scholar
  9. 9.
    Gaussian Smoothing (2008) Wolfram Demonstrations ProjectGoogle Scholar
  10. 10.
    Gaussian function. Springer ReferenceGoogle Scholar
  11. 11.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110MathSciNetCrossRefGoogle Scholar
  12. 12.
    Laplacian of Gaussian Filtering (2008) Wolfram Demonstrations ProjectGoogle Scholar
  13. 13.
    Berz M (1997) From Taylor series to Taylor modelsGoogle Scholar
  14. 14.
    Ramachandran P et al (2017) Searching for activation functions. In: ICLR 2018 conferenceGoogle Scholar
  15. 15.
    Zeiler MD (2012) ADADELTA: an adaptive learning rate methodGoogle Scholar
  16. 16.
    Pellegrini T (2015) Comparing SVM, Softmax, and shallow neural networks for eating condition classification. Interspeech 2015Google Scholar
  17. 17.
    Dahal P, Classification and loss evaluation—Softmax and cross entropy loss. https://deepnotes.io/softmax-crossentropy
  18. 18.
    Keras: The Python Deep Learning library. https://keras.io/

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Abhishek Dudhal
    • 1
    Email author
  • Heramb Mathkar
    • 1
  • Abhishek Jain
    • 1
  • Omkar Kadam
    • 1
  • Mahesh Shirole
    • 1
  1. 1.Department of Computer Engineering and Information TechnologyVeermata Jijabai Technological InstituteMumbaiIndia

Personalised recommendations