Abstract
In this paper, we propose a deep learning for sign language recognition using convolutional neural network (CNN) and long short term memory (LSTM). The architecture used CNN as a pretrained model for feature extraction and is passed to the LSTM for capturing spatio-temporal information. One more LSTM is stacked for increasing the accuracy. Deep learning model which captures temporal information is less. There is only less papers which deals with sign language recognition by using the deep learning architectures such as CNN and LSTM. The algorithm was tested in Indian sign language (ISL) dataset. We have presented the performance evaluation after testing with ISL dataset. Literature shows that deep learning models capturing temporal information is still an open research problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Koller, O., et al.: Deep sign: hybrid CNN-HMM for continuous sign language recognition. In: Proceedings of the British Machine Vision Conference (2016)
Cui, R., Liu, H., Zhang, C.: Recurrent convolutional neural networks for continuous sign language recognition by staged optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Geetha, M., et al.: Gesture recognition for American sign language with polygon approximation. In: 2011 IEEE International Conference on Technology for Education. IEEE (2011)
Garcia, B., Viesca, S.A.: Real-time American sign language recognition with convolutional neural networks. In: Convolutional Neural Networks for Visual Recognition, vol. 2 (2016)
Tsironi, E., et al.: An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition. Neurocomputing 268, 76–86 (2017)
Li, C., et al.: Deep fisher discriminant learning for mobile hand gesture recognition. Pattern Recognit. 77, 276–288 (2018)
Nunez, J.C., et al.: Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recognit. 76, 80–94 (2018)
Aloysius, N., Geetha, M.: A review on deep convolutional neural networks. In: 2017 International Conference on Communication and Signal Processing (ICCSP). IEEE (2017)
Bantupalli, K., Xie, Y.: American sign language recognition using deep learning and computer vision. In: 2018 IEEE International Conference on Big Data (Big Data). IEEE (2018)
Taskiran, M., Killioglu, M., Kahraman, N.: A real-time system for recognition of American sign language by using deep learning. In: 2018 41st International Conference on Telecommunications and Signal Processing (TSP). IEEE (2018)
Nguyen, H.B.D., Do, H.N.: Deep learning for American sign language fingerspelling recognition system. In: 2019 26th International Conference on Telecommunications (ICT). IEEE (2019)
Soodtoetong, N., Gedkhaw, E.: The efficiency of sign language recognition using 3D convolutional neural networks. In: 2018 15th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON). IEEE (2018)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Aparna, C., Geetha, M. (2020). CNN and Stacked LSTM Model for Indian Sign Language Recognition. In: Thampi, S., Trajkovic, L., Li, KC., Das, S., Wozniak, M., Berretti, S. (eds) Machine Learning and Metaheuristics Algorithms, and Applications. SoMMA 2019. Communications in Computer and Information Science, vol 1203. Springer, Singapore. https://doi.org/10.1007/978-981-15-4301-2_10
Download citation
DOI: https://doi.org/10.1007/978-981-15-4301-2_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-4300-5
Online ISBN: 978-981-15-4301-2
eBook Packages: Computer ScienceComputer Science (R0)