Skip to main content
SpringerLink
Log in

RBI-2RCNN: Residual Block Intensity Feature using a Two-stage Residual Convolutional Neural Network for Static Hand Gesture Recognition

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Hand gesture recognition (HGR) is the most effective and intuitive way for the human–computer interface used in various applications, such as sign language recognition, robotics, and multimedia applications. The performance of the existing handcrafted techniques relies on the less generalized preprocessing and feature extraction steps. The convolutional neural networks (CNNs) can handle the less generalization characteristic of the handcrafted techniques. However, the architecture of CNN is complex and the extracted deep feature from these networks provides the global information only. Therefore, a CNN-based HGR paradigm can be developed with less number of layers and feature fusion with global and local information from different layers. Motivated by the above facts, this work proposes (i) a two-stage residual CNN (2RCNN) architecture for learning of features from the color hand gesture images which overcomes the need of a specific preprocessing step, (ii) a novel residual block intensity (RBI) feature to extract the global and local information from the hand gesture images. After extracting the RBI features, a linear kernel-based multi-class support vector machine classifier is used to recognize the gesture poses. The experimental results are evaluated using a subject-independent cross-validation test on three benchmarked datasets and compared with the earlier reported techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bao, P., Maqueda, A.I., del Blanco, C.R., García, N.: Tiny hand gesture recognition without localization via a deep convolutional network. IEEE Trans. Consum. Electron. 63(3), 251–257 (2017)

    Article  Google Scholar 

  2. Barczak, A., Reyes, N., Abastillas, M., Piccio, A., Susnjak, T.: A new 2D static hand gesture colour image dataset for ASL gestures. Res. Lett. Inf. Math. Sci. 15, 12–20 (2011)

    Google Scholar 

  3. Chakraborty, B.K., Sarma, D., Bhuyan, M.K., MacDorman, K.F.: Review of constraints on vision-based gesture recognition for human-computer interaction. IET Comput. Vis. 12(1), 3–15 (2017)

    Article  Google Scholar 

  4. Chaudhary, A., Raheja, J.: Light invariant real-time robust hand gesture recognition. Optik 159, 283–294 (2018)

    Article  Google Scholar 

  5. Chevtchenko, S.F., Vale, R.F., Macario, V.: Multi-objective optimization for hand posture recognition. Expert Syst. Appl. 92, 170–181 (2018)

    Article  Google Scholar 

  6. Chevtchenko, S.F., Vale, R.F., Macario, V., Cordeiro, F.R.: A convolutional neural network with feature fusion for real-time hand posture recognition. Appl. Soft Comput. 73, 748–766 (2018)

    Article  Google Scholar 

  7. Dadashzadeh, A., Targhi, A.T., Tahmasbi, M., Mirmehdi, M.: HGR-Net: a fusion network for hand gesture segmentation and recognition. IET Comput. Vis. 13(8), 700–707 (2019)

    Article  Google Scholar 

  8. Fang, L., Liang, N., Kang, W., Wang, Z., Feng, D.D.: Real-time hand posture recognition using hand geometric features and fisher vector. Signal Process. Image Commun. 82, 115729 (2020)

    Article  Google Scholar 

  9. Gadekallu, T.R., Alazab, M., Kaluri, R., Maddikunta, P.K.R., Bhattacharya, S., Lakshmanna, K., Parimala, M.: Hand gesture classification using a novel cnn-crow search algorithm. Complex Intell. Syst. pp. 1–14 (2021)

  10. Ghosh, D.K., Ari, S.: On an algorithm for vision-based hand gesture recognition. SIViP 10(4), 655–662 (2016)

    Article  Google Scholar 

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  12. He, M., Horng, S.J., Fan, P., Run, R.S., Chen, R.J., Lai, J.L., Khan, M.K., Sentosa, K.O.: Performance evaluation of score level fusion in multimodal biometric systems. Pattern Recogn. 43(5), 1789–1800 (2010)

    Article  Google Scholar 

  13. Lee, D.L., You, W.S.: Recognition of complex static hand gestures by using the wristband-based contour features. IET Image Proc. 12(1), 80–87 (2018)

    Article  Google Scholar 

  14. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)

    MATH  Google Scholar 

  15. Mahmood, A., Bennamoun, M., An, S., Sohel, F., Boussaid, F.: ResFeats: Residual network based features for underwater image classification. Image Vis. Comput. 93, 103811 (2020)

    Article  Google Scholar 

  16. Oyedotun, O.K., Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl. 28(12), 3941–3951 (2017)

    Article  Google Scholar 

  17. Pisharady, P.K., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition: A review. Comput. Vis. Image Underst. 141, 152–165 (2015)

    Article  Google Scholar 

  18. Pisharady, P.K., Vadakkepat, P., Loh, A.P.: Attention based detection and recognition of hand postures against complex backgrounds. Int. J. Comput. Vis. 101(3), 403–419 (2013)

    Article  Google Scholar 

  19. Plouffe, G., Cretu, A.M.: Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans. Instrum. Meas. 65(2), 305–316 (2016)

    Article  Google Scholar 

  20. Pugeault, N., Bowden, R.: Spelling it out: Real-time asl fingerspelling recognition. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1114–1119 (2011)

  21. Ren, Z., Yuan, J., Meng, J., Zhang, Z.: Robust part-based hand gesture recognition using kinect sensor. IEEE Trans. Multimed. 15(5), 1110–1120 (2013)

    Article  Google Scholar 

  22. Sahoo, J.P., Ari, S., Ghosh, D.K.: Hand gesture recognition using DWT and F-ratio based feature descriptor. IET Image Proc. 12(10), 1780–1787 (2018)

    Article  Google Scholar 

  23. Sahoo, J.P., Ari, S., Patra, S.K.: Hand gesture recognition using pca based deep cnn reduced features and svm classifier. In: IEEE International Symposium on Smart Electronic Systems (iSES)(Formerly iNiS), pp. 221–224 (2019)

  24. Sahoo, J.P., Ari, S., Patra, S.K.: Chapter 13 - A user independent hand gesture recognition system using deep cnn feature fusion and machine learning technique. In: S. Chakraverty (ed.) New Paradigms in Computational Modeling and Its Applications, pp. 189–207. Academic Press (2021)

  25. Sahoo, S.P., Ari, S., Mahapatra, K., Mohanty, S.P.: Har-depth: A novel framework for human action recognition using sequential learning and depth estimated history images. IEEE Trans. Emerg. Top. Comput. Intell. pp. 1–13 (2020)

  26. Tan, Y.S., Lim, K.M., Tee, C., Lee, C.P., Low, C.Y.: Convolutional neural network with spatial pyramid pooling for hand gesture recognition. Neural Comput. Appl. 33(10), 5339–5351 (2021)

    Article  Google Scholar 

  27. Tang, P., Wang, H., Kwong, S.: G-MS2F: Googlenet based multi-stage feature fusion of deep CNN for scene recognition. Neurocomputing 225, 188–197 (2017)

    Article  Google Scholar 

  28. Vapnik, V.: The Nature of Statistical Learning Theory. Springer Science & Business Media, New York (2013)

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology Rourkela, Odisha, 769008, India

    Jaya Prakash Sahoo & Samit Ari

  2. Department of Electronics and Communication Engineering, Chaitanya Bharathi Institute of Technology Gandipet, Hyderabad, Telangana, 500075, India

    Suraj Prakash Sahoo

  3. Indian Institute of Information Technology, Vadodara, Gujarat, 382028, India

    Sarat Kumar Patra

Authors
  1. Jaya Prakash Sahoo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suraj Prakash Sahoo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Samit Ari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sarat Kumar Patra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samit Ari.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sahoo, J.P., Sahoo, S.P., Ari, S. et al. RBI-2RCNN: Residual Block Intensity Feature using a Two-stage Residual Convolutional Neural Network for Static Hand Gesture Recognition. SIViP 16, 2019–2027 (2022). https://doi.org/10.1007/s11760-022-02163-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02163-w

Keywords

Search

Navigation