Skip to main content
SpringerLink
Log in

An Integrated CNN-LSTM Model for Bangla Lexical Sign Language Recognition

  • Conference paper
  • First Online:
Proceedings of International Conference on Trends in Computational and Cognitive Engineering

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1309))

Abstract

The development of Human-Computer Interaction (HCI) will not only empower the disabled with access to technology but also help to build tools for assisting the disable and their caregivers. Some disabled people are deaf, they use signs to communicate. Signs are emblematic forms which convey a specific meaning. These emblems construct the sign language, which is a non-verbal language used by the deaf community. This research attempts to build a Bangla Sign language recognition system that can recognize signs of both hands. Hands vary in shapes and sizes, as well as signs vary in orientations and motions. Accurate feature extraction is necessary for such systems. For such purposes, deep learning approaches can prove to be effective for the classification and feature engineering of images. In the beginning, an integrated CNN-LSTM model is proposed for building a Sign language Recognition System, a Bangla Sign Language (BSL) dataset consisting of Bangla lexical signs is considered. This dataset consists of 13,400 images comprising thirty-six classes of Bangla lexical signs. The model is trained using a CNN-LSTM model. This model produces a training accuracy of 90% and a testing accuracy of 88.5%. The proposed model is compared to CNN and other CNN state of the art models, namely VGG16, VGG9 and MobileNet. The CNN model and other CNN state undergo the problem of overfitting as their training accuracy is greater than the testing accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abedin, M.Z., Nath, A.C., Dhar, P., Deb, K., Hossain, M.S.: License plate recognition system based on contour properties and deep learning model. In: 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC), pp. 590–593 (2017)

    Google Scholar 

  2. Ahmed, T.U., Hossain, S., Hossain, M.S., ul Islam, R., Andersson, K.: Facial expression recognition using convolutional neural network with data augmentation. In: 2019 Joint 8th International Conference on Informatics, Electronics Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision Pattern Recognition (icIVPR), pp. 336–341 (2019)

    Google Scholar 

  3. Al Banna, M.H., Ali Haider, M., Al Nahian, M.J., Islam, M.M., Taher, K.A., Kaiser, M.S.: Camera model identification using deep CNN and transfer learning approach. In: 2019 International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST), pp. 626–630 (2019)

    Google Scholar 

  4. Al Mamun, S., Fukuda, H., Lam, A., Kobayashi, Y., Kuno, Y.: Autonomous bus boarding robotic wheelchair using bidirectional sensing systems. In: Advances in Visual Computing, pp. 737–747. LNCS, Springer International Publishing (2018)

    Google Scholar 

  5. Andersson, K., Hossain, M.S.: Smart risk assessment systems using belief-rule-based DSS and WSN technologies. In: 2014 4th International Conference on Wireless Communications, Vehicular Technology, Information Theory and Aerospace & Electronic Systems (VITAE), pp. 1–5. IEEE (2014)

    Google Scholar 

  6. Bour, A., Castillo-Olea, C., Garcia-Zapirain, B., Zahia, S.: Automatic colon polyp classification using convolutional neural network: a case study at basque country. In: 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), pp. 1–5. IEEE (2019)

    Google Scholar 

  7. Bovik, A., Huang, T., Munson, D.: A generalization of median filtering using linear combinations of order statistics. IEEE Trans. Acoust. Speech Signal Process. 31(6), 1342–1350 (1983)

    Article  Google Scholar 

  8. Deafness: https://www.who.int/news-room/facts-in-pictures/detail/deafness

  9. Gupta, D., Hossain, E., Hossain, M.S., Andersson, K., Hossain, S.: A digital personal assistant using bangla voice command recognition and face detection. In: 2019 IEEE International Conference on Robotics, Automation, Artificial-intelligence and Internet-of-Things (RAAICON), pp. 116–121 (2019)

    Google Scholar 

  10. Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 4, 532–550 (1987)

    Article  Google Scholar 

  11. Hossain, M.S., Khalid, M.S., Akter, S., Dey, S.: A belief rule-based expert system to diagnose influenza. In: 2014 9th International Forum on Strategic Technology (IFOST), pp. 113–116. IEEE (2014)

    Google Scholar 

  12. Hossain, M.S., Rahaman, S., Kor, A.L., Andersson, K., Pattinson, C.: A belief rule based expert system for datacenter PUE prediction under uncertainty. IEEE Trans. Sustain. Comput. 2(2), 140–153 (2017)

    Article  Google Scholar 

  13. Hossen, M., Govindaiah, A., Sultana, S., Bhuiyan, A.: Bengali sign language recognition using deep convolutional neural network. In: 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 369–373. IEEE (2018)

    Google Scholar 

  14. Kaiser, M.S., et al.: Advances in crowd analysis for urban applications through urban event detection. IEEE Trans. Intell. Transp. Syst. 19(10), 3092–3112 (2018)

    Article  Google Scholar 

  15. Karim, R., Andersson, K., Hossain, M.S., Uddin, M.J., Meah, M.P.: A belief rule based expert system to assess clinical bronchopneumonia suspicion. In: 2016 Future Technologies Conference (FTC), pp. 655–660. IEEE (2016)

    Google Scholar 

  16. Karmokar, B.C., Alam, K.M.R., Siddiquee, M.K., et al.: Bangladeshi sign language recognition employing neural network ensemble. Int. J. Comput. Appl. 58(16), 43–46 (2012)

    Google Scholar 

  17. Mahmud, M., Kaiser, M.S., Hussain, A.: Deep learning in mining biological data. arXiv preprint arXiv:2003.00108 (2020)

  18. Mahmud, M., et al.: A brain-inspired trust management model to assure security in a cloud based IoT framework for neuroscience applications. Cogn. Comput. 10(5), 864–873 (2018)

    Article  Google Scholar 

  19. Mahmud, M., Kaiser, M.S., Hussain, A., Vassanelli, S.: Applications of deep learning and reinforcement learning to biological data. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2063–2079 (2018)

    Article  MathSciNet  Google Scholar 

  20. Mamun, S.A., et al.: Terrain recognition for smart wheelchair. In: Intelligent Computing Methodologies, pp. 461–470. LNCS, Springer International Publishing (2016)

    Google Scholar 

  21. Noor, M.B.T., et al.: Application of deep learning in detecting neurological disorders from magnetic resonance images: a survey on the detection of Alzheimer’s disease, Parkinson’s disease and schizophrenia. Brain Inf. 7(1), 1–21 (2020)

    Article  MathSciNet  Google Scholar 

  22. Rahaman, M.A., Jasim, M., Ali, M.H., Hasanuzzaman, M.: Real-time computer vision-based Bengali sign language recognition. In: 2014 17th International Conference on Computer and Information Technology (ICCIT), pp. 192–197. IEEE (2014)

    Google Scholar 

  23. Ruder, S.: An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 (2016)

  24. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  25. Sultana, Z.: Agony of persons with disability—a comparative study of Bangladesh. J. Polit. Law 3, 212 (2010)

    Google Scholar 

  26. Uddin, M.A., Chowdhury, S.A.: Hand sign language recognition for Bangla alphabet using support vector machine. In: 2016 International Conference on Innovations in Science, Engineering and Technology (ICISET), pp. 1–4. IEEE (2016)

    Google Scholar 

  27. Yasir, R., Khan, R.A.: Two-handed hand gesture recognition for Bangla sign language using LDA and ANN. In: The 8th International Conference on Software, Knowledge, Information Management and Applications (SKIMA 2014), pp. 1–5. IEEE (2014)

    Google Scholar 

  28. Zisad, S.N., Hossain, M.S., Andersson, K.: Speech emotion recognition in neurological disorders using convolutional neural network. In: International Conference on Brain Informatics, pp. 287–296. Springer (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Islamic University Chittagong, Chittagong, Bangladesh

    Nanziba Basnin & Lutfun Nahar

  2. University of Chittagong, Chittagong, Bangladesh

    Mohammad Shahadat Hossain

Authors
  1. Nanziba Basnin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lutfun Nahar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Shahadat Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Shahadat Hossain .

Editor information

Editors and Affiliations

  1. Jahangirnagar University, Dhaka, Bangladesh

    M. Shamim Kaiser

  2. National Institute for Materials Science, Tsukuba, Japan

    Anirban Bandyopadhyay

  3. Nottingham Trent University, Nottingham, UK

    Mufti Mahmud

  4. Amity University, Jaipur, Rajasthan, India

    Kanad Ray

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Basnin, N., Nahar, L., Hossain, M.S. (2021). An Integrated CNN-LSTM Model for Bangla Lexical Sign Language Recognition. In: Kaiser, M.S., Bandyopadhyay, A., Mahmud, M., Ray, K. (eds) Proceedings of International Conference on Trends in Computational and Cognitive Engineering. Advances in Intelligent Systems and Computing, vol 1309. Springer, Singapore. https://doi.org/10.1007/978-981-33-4673-4_57

Download citation

Publish with us

Policies and ethics

Search

Navigation