Skip to main content
SpringerLink
Log in

Automatic and robust hand gesture recognition by SDD features based model matching

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Automatic and robust hand gesture recognition remains challenging after many decades of study. Human beings are able to recognize a variety of hand gestures with 100% accuracy solely based on the contour of the hand. Hence, there must be an automatic method that is able to recognize the same variety of hand gestures solely based on the contour of the hand with 100% accuracy. The key technique lies in how to extract the features of the hand’s contour effectively. In this paper, we propose to recognize the hand gestures with the contour features extracted by slope difference distribution (SDD). Firstly, the hand is segmented, its centroid is computed and its contour is extracted. Secondly, the peak features and valley features of the hand contour are computed by the SDD. Thirdly, the hand gesture is recognized by model matching based on the SDD peak features and the SDD valley features. The proposed hand gesture recognition method was tested on three public datasets and it achieved 100% recognition accuracy for all the 10 gestures in two public datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig.9
Fig. 10

Similar content being viewed by others

References

  1. Du G, Zhang B, Li C, Gao B, Liu PX (2020) Natural Human-Machine Interface With Gesture Tracking and Cartesian Platform for Contactless Electromagnetic Force Feedback. IEEE Trans Industr Inf 16(11):6868–6879. https://doi.org/10.1109/TII.2020.2966756

    Article  Google Scholar 

  2. Ahmed M.A., Zaidan B.B., Zaidan A.A., Salih M.M., Zaydoon Tareq, Alamoodi A.H., Based on Wearable Sensory Device in 3D-Printed Humanoid: A new Real-Time Sign Language Recognition System, Measurement, 108431, 2020.

  3. Cristina Nuzzi, Simone Pasinetti, Matteo Lancini,Franco Docchio, Giovanna Sansoni, “Deep Learning-Based Hand Gesture Recognition for Collaborative Robots”, IEEE Instrumentation & Measurement Magazine, Vol. 22 , Issue: 2 , , 44 – 51, 2019.

  4. N. Pugeault, and R. Bowden, “Spelling it out: real time ASL finger spelling recognition,” IEEE Computer Society Conference on Computer Vision Workshops, pp.1114–1119, 2011.

  5. T. Mantecón, C.R. del Blanco, F. Jaureguizar, N. García, “Hand Gesture Recognition using Infrared Imagery Provided by Leap Motion Controller”, Int. Conf. on Advanced Concepts for Intelligent Vision Systems, ACIVS 2016, Lecce, Italy, pp. 47–57, 24–27 Oct. 2016.

  6. Pisharady PK, Vadakkepat P, Loh AP (2013) Attention based detection and recognition of hand postures against complex backgrounds. Int J Comput Vis 101(3):403–419

    Article  Google Scholar 

  7. Feng B et al (2017) Depth-Projection-Map-Based Bag of Contour Fragments for Robust Hand Gesture Recognition. IEEE Transactions on Human-Machine Systems 47(4):511–523. https://doi.org/10.1109/THMS.2016.2616278

    Article  Google Scholar 

  8. A.I. Maqueda, C. R. del-Blanco, F. Jaureguizar, and N. García, “Human-computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns,” Computer vision and Image understanding, vol. 141, pp. 126–137, 2015.

  9. C Zhang X Yang Y Tian 2013 Histogram of 3D Facets: A characteristic descriptor for hand gesture recognition 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) Shanghai 1 8

  10. Y. Wang, and R. Yang, “Real-time hand gesture recognition based on hand dominant line using kinect,” IEEE International Conference on Multimedia and Expo Workshops, pp.1–4, 2013.

  11. Ren Y, Xie X, Li G, Wang Z (2018) Hand gesture recognition with multiscale weighted histogram of contour direction normalization for wearable applications. IEEE Trans Circ Syst Video Technol 28(2):364–377

    Article  Google Scholar 

  12. Aly W, Aly S, Almotairi S (2019) User-Independent American Sign Language Alphabet Recognition Based on Depth Image and PCANet Features. IEEE Access 7:123138–123150. https://doi.org/10.1109/ACCESS.2019.2938829

    Article  Google Scholar 

  13. Sharma P (2020) Radhey Shyam Anand, “Depth data and fusion of feature descriptors for static gesture recognition.” IET Image Proc 14(5):909–920

    Article  Google Scholar 

  14. Linpu Fang, Ningxin Liang, Wenxiong Kang, Zhiyong Wang, David Dagan Feng, "Real-time hand posture recognition using hand geometric features and Fisher Vector", Signal Processing: Image Communication, pp. 115729, 2019.

  15. Dardas NH, Georganas ND (2011) Real-Time Hand gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques. IEEE Trans Instrum Meas 60(11):3592–3607

    Article  Google Scholar 

  16. C. Keskin, F. Kirac, Y. Kara, and L. Akarun, “Randomized decision forests for static and dynamic hand shape classification,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp.31–36, 2012.

  17. A. Kuznetsova, L. Leal-Taixe, and B. Rosenhahn, “Real-time sign language recognition using a consumer depth camera,” IEEE Computer Society Conference on Computer Vision Workshops, pp.83–90, 2013.

  18. W. Nai, Y. Liu, D. Rempel, and Y. Wang, ‘‘Fast hand posture classification using depth features extracted from random line segments,’’ Pattern Recognit., vol. 65, pp. 1–10, May 2017. Random forest 81.10

  19. Mahdikhanlou K, Ebrahimnezhad H (2020) Multimodal 3D American sign language recognition for static alphabet and numbers using hand joints and shape coding. Multimedia tools and applications 79:22235–22259

    Article  Google Scholar 

  20. Wang C, Liu Z, Chan SC (2015) Superpixel-based hand gesture recognition with Kinect depth camera. IEEE Trans Multimedia 17(1):29–39

    Article  Google Scholar 

  21. Warchoł, Dawid et al. “Recognition of Fingerspelling Sequences in Polish Sign Language Using Point Clouds Obtained from Depth Images.” Sensors (Basel, Switzerland) vol. 19, 5 1078. 3 Mar. 2019, https://doi.org/10.3390/s19051078.

  22. Yao Y, Fu Y (2014) Contour Model-Based Hand-Gesture Recognition Using the Kinect Sensor. IEEE Trans Circuits Syst Video Technol 24(11):1935–1944

    Article  Google Scholar 

  23. Lee DL, You WS (2018) Recognition of complex static hand gesture by using the wristband-based contour features. IET Image Proc 12(1):80–87

    Article  Google Scholar 

  24. Li GF, Wu H, Jiang GZ, Xu S, Liu HH (2019) Dynamic gesture Recognition in the Internet of Things. Access IEEE 7:23713–23724

    Article  Google Scholar 

  25. P. S. Neethu, R. Suguna, Divya Sathish, "An efficient method for human hand gesture detection and recognition using deep learning convolutional neural networks", Soft Computing, 2020.

  26. Rivera-Acosta M, Ortega-Cisneros S, Rivera J (2017) Sandoval-Ibarra, American sign language alphabet recognition using a Neuromorphic sensor and an artificial neural network. Sensors 17(10):2176

    Article  Google Scholar 

  27. S. R. Bose and V. S. Kumar, "Efficient inception V2 based deep convolutional neural network for real-time hand action recognition," in IET Image Processing, vol. 14, no. 4, pp. 688–696, 27 3 2020,  https://doi.org/10.1049/iet-ipr.2019.0985.

  28. Tao W, Leu MC, Yin Z (2018) American sign language alphabet recognition using convolutional neural networks with multiview augmentation and inference fusion. Eng Appl Artif Intell 76:202–213

    Article  Google Scholar 

  29. S. Ameen and S. Vadera, ‘‘A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images,’’ Expert Syst., vol. 34, no. 3, 2017, Art. no. e12197.

  30. Chevtchenko SF, Vale RF, Macario V, Cordeiro FR (2018) A convolutional neural network with feature fusion for real-time hand posture recognition. Appl Soft Comput 73:748–766

    Article  Google Scholar 

  31. Razieh Rastgoo, Kourosh Kiani, Sergio Escalera, Hand sign language recognition using multi-view hand skeleton, Expert Systems with Applications, Vol. 150, pp. 113336, 2020.

  32. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  33. S. Ganapathyraju, "Hand gesture recognition using convexity hull defects to control an industrial robot," 2013 3rd International Conference on Instrumentation Control and Automation (ICA), Ungasan, 2013, pp. 63–67.

  34. Wang Z, Xiong J, Yang Y, Li H (2018) A Flexible and Robust Threshold Selection Method. IEEE Trans Circuits Syst Video Technol 28(9):2220–2232

    Article  Google Scholar 

  35. Wang ZZ (2019) Automatic and Optimal Segmentation of the Left Ventricle in Cardiac Magnetic Resonance Images Independent of the Training Sets. IET Image Proc 13(10):1725–1735

    Article  Google Scholar 

  36. Wang Z (2021) Automatic Localization and Segmentation of the Ventricles in Magnetic Resonance Images. IEEE Trans Circuits Syst Video Technol 31(2):621–631

    Article  Google Scholar 

  37. D’Orazio T, Marani R, Renò V, Cicirelli G (2016) Recent trends in gesture recognition: how depth data has improved classical approaches. Image Vis Comput 52:56

    Article  Google Scholar 

  38. Chakraborty BK, Sarma D, Bhuyan MK, MacDorman KF (2018) Review of constraints on vision-based gesture recognition for human–computer interaction. IET Comput Vision 12(1):3–15

    Article  Google Scholar 

Download references

Acknowledgements

This research work is funded by Natural Science Foundation of Shandong Province with the grant no. ZR2020MF018.

Author information

Authors and Affiliations

  1. College of Electrical and Electronic Engineering, Shandong University of Technology, Zibo, 255000, China

    ZhenZhou Wang

Authors
  1. ZhenZhou Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to ZhenZhou Wang.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z. Automatic and robust hand gesture recognition by SDD features based model matching. Appl Intell 52, 11288–11299 (2022). https://doi.org/10.1007/s10489-021-02933-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02933-y

Keywords

Search

Navigation