Advertisement

Gesture recognition based on skeletonization algorithm and CNN with ASL database

  • Du Jiang
  • Gongfa LiEmail author
  • Ying Sun
  • Jianyi Kong
  • Bo Tao
Article

Abstract

In the field of human-computer interaction, vision-based gesture recognition methods are widely studied. However, its recognition effect depends to a large extent on the performance of the recognition algorithm. The skeletonization algorithm and convolutional neural network (CNN) for the recognition algorithm reduce the impact of shooting angle and environment on recognition effect, and improve the accuracy of gesture recognition in complex environments. According to the influence of the shooting angle on the same gesture recognition, the skeletonization algorithm is optimized based on the layer-by-layer stripping concept, so that the key node information in the hand skeleton diagram is extracted. The gesture direction is determined by the spatial coordinate axis of the hand. Based on this, gesture segmentation is implemented to overcome the influence of the environment on the recognition effect. In order to further improve the accuracy of gesture recognition, the ASK gesture database is used to train the convolutional neural network model. The experimental results show that compared with SVM method, dictionary learning + sparse representation, CNN method and other methods, the recognition rate reaches 96.01%.

Keywords

Layer-by-layer stripping theory Skeletonization algorithm Convolutional neural network Gesture recognition Big data 

Notes

Acknowledgements

This work was supported by grants of National Natural Science Foundation of China (Grant No. 51575407, 51575338, 51575412, 61273106, 51505349) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705). This paper is funded by Wuhan University of Science and Technology graduate students short-term study abroad special funds.

References

  1. 1.
    Błaszczyk Ł (2016) Compressed sensing in MRI – mathematical preliminaries and basic examples. Nukleonika 61(1):41–43CrossRefGoogle Scholar
  2. 2.
    Bu X, Dong H, Han F et al (2018) Event-triggered distributed filtering over sensor networks with deception attacks and partial measurements. Int J Gen Syst 47:395–407MathSciNetCrossRefGoogle Scholar
  3. 3.
    Caselli N, Sehyr Z, Cohen-Goldberg A et al (2017) ASL-LEX: A lexical database of American Sign Language. Behav Res Methods 49(2):784–801CrossRefGoogle Scholar
  4. 4.
    Chang W, Li G, Kong J et al (2018) Thermal Mechanical Stress Analysis of Ladle Lining with Integral Brick Joint. Arch Metall Mater 63(2):659–666Google Scholar
  5. 5.
    Chen D, Li G, Sun Y et al (2017) An interactive image segmentation method in hand gesture recognition. Sensors 17(2):253CrossRefGoogle Scholar
  6. 6.
    Fang Y, Liu H, Li G et al (2015) A multichannel surface emg system for hand motion recognition. International Journal of Humanoid Robotics 12(2).  https://doi.org/10.1142/S0219843615500115 CrossRefGoogle Scholar
  7. 7.
    Han F, Dong H, Wang Z et al (2018) Improved tobit kalman filtering for systems with random parameters via conditional expectation. Signal Process 147.  https://doi.org/10.1016/j.sigpro.2018.01.015 CrossRefGoogle Scholar
  8. 8.
    He Y, Li G, Liao Y et al (2017) Gesture recognition based on an improved local sparse representation classification algorithm. Clust Comput 1.  https://doi.org/10.1007/s10586-017-1237-1
  9. 9.
    He Y, Li G, Zhao Y et al (2018) Numerical simulation-based optimization of contact stress distribution and lubrication conditions in the straight worm drive. Strength of Materials 50(11):1–9Google Scholar
  10. 10.
    Jiang D, Zheng Z, Li G et al (2017) Gesture recognition based on binocular vision. Clust Comput 3.  https://doi.org/10.1007/s10586-018-1844-5
  11. 11.
    Jin KH, Lee D, Ye JC (2017) A general framework for compressed sensing and parallel mri using annihilating filter based low-rank Hankel matrix. IEEE Transactions on Computational Imaging 2(4):480–495MathSciNetCrossRefGoogle Scholar
  12. 12.
    Li G, Gu Y, Kong J et al (2013) Intelligent control of air compressor production process. Applied Mathematics & Information Sciences 7(3):1051–1058CrossRefGoogle Scholar
  13. 13.
    Li G, Kong J, Jiang G et al (2012) Air-fuel ratio intelligent control in coke oven combustion process. International Journal on Information 12(11):4487–4494Google Scholar
  14. 14.
    Li G, Liu J, Jiang G et al (2015) Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material. Advances in Mechanical Engineering 7(4):1687814015575988CrossRefGoogle Scholar
  15. 15.
    Li G, Liu Z, Jiang G et al (2017) Numerical simulation of the influence factors for rotary kiln in temperature field and stress field and the structure optimization. Advances in Mechanical Engineering 7(6):1687814015589667CrossRefGoogle Scholar
  16. 16.
    Li G, Miao W, Jiang G et al (2015) Intelligent control model and its simulation of flue temperature in coke oven. Discrete and Continuous Dynamical Systems - Series S 8(6):1223–1237MathSciNetCrossRefGoogle Scholar
  17. 17.
    Li G, Qu P, Kong J et al (2013) Coke oven intelligent integrated control system. Applied Mathematics & Information Sciences 7(3):1043–1050CrossRefGoogle Scholar
  18. 18.
    Li G, Qu P, Kong J et al (2013) Influence of working lining parameters on temperature and stress field of ladle. Applied Mathematics & Information Sciences 7(2):439–448CrossRefGoogle Scholar
  19. 19.
    Li B, Sun Y, Li G et al (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput 1.  https://doi.org/10.1007/s10586-017-1231-7
  20. 20.
    Li G, Tang H, Zhao Y et al (2017) Hand gesture recognition based on convolution neural network. Clust Comput 3.  https://doi.org/10.1007/s10586-017-1435-x
  21. 21.
    Li G, Zhang L, Sun Y et al (2018) Towards the sEMG hand: internet of things sensors and haptic feedback application. Multimed Tools Applications 1:1–18Google Scholar
  22. 22.
    Liao Y, Sun Y, Li G et al (2017) Simultaneous Calibration: A Joint Optimization Approach for Multiple Kinect and External Cameras. Sensors 17(7):1491.  https://doi.org/10.3390/s17071491 CrossRefGoogle Scholar
  23. 23.
    Luzanin O, Plancak M (2014) Hand gesture recognition using low-budget data glove and cluster-trained probabilistic neural network. Assem Autom 34(1):94–105CrossRefGoogle Scholar
  24. 24.
    Miao W, Li G, Jiang G et al (2015) Optimal Grasp Planning of Multi-Fingered Robotic Hands: A Review. Applied and computational mathematics 14(3):238–247MathSciNetzbMATHGoogle Scholar
  25. 25.
    Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29(9):2352–2449CrossRefGoogle Scholar
  26. 26.
    Saha P, Jin D, Liu Y et al (2018) Fuzzy object skeletonization: theory, algorithms, and applications. IEEE Trans Vis Comput Graph 24(8):2298–2314CrossRefGoogle Scholar
  27. 27.
    Shotton J, Fitzgibbon A, Cook M et al (2011) Real-Time Human Pose Recognition in Parts from Single Depth Images. Computer Vision and Pattern Recognition 56(1):1297–1304Google Scholar
  28. 28.
    Sun Y, Hu J, Li G et al (2018) Gear reducer optimal design based on computer multimedia simulation. J Supercomput 3:1–13.  https://doi.org/10.1007/s11227-018-2255-3 CrossRefGoogle Scholar
  29. 29.
    Sun Y, Li C, Li G et al (2018) Gesture Recognition Based on Kinect and sEMG Signal Fusion. Mobile Networks and Applications 23(4):797–805.  https://doi.org/10.1007/s11036-018-1008-0 CrossRefGoogle Scholar
  30. 30.
    Xiong H, Fan H, Jiang G et al (2017) A simulation-based study of dispatching rules in a dynamic job shop scheduling problem with batch release and extended technical precedence constraints. Eur J Oper Res 257(1):13–24MathSciNetCrossRefGoogle Scholar
  31. 31.
    Xiong H, Fan H, Li G et al (2015) Research on steady-state simulation in dynamic job shop scheduling problem. Advances in Mechanical Engineering 7(9):1–11CrossRefGoogle Scholar
  32. 32.
    Yin Q, Li G, Zhu J (2017) Research on the method of step feature extraction for EOD robot based on 2D laser radar. Discrete and Continuous Dynamical Systems - Series S (DCDS-S) 8(6):1415–1421MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zhong X, Chen Y, Yu H et al (2018) Context-Aware Information Based Ultrasonic Gesture Recognition Method. Journal of Computer-Aided Design & Computer Graphics 30(1):173CrossRefGoogle Scholar
  34. 34.
    Cheng W, Sun Y, Li G et al (2018) Jointly network: a network based on CNN and RBM for gesture recognition. Neural Computing and Applications 1–17.  https://doi.org/10.1007/s00521-018-3775-8

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of EducationWuhan University of Science and TechnologyWuhanChina
  2. 2.Research Center for Biomimetic Robot and Intelligent Measurement and ControlWuhan University of Science and TechnologyWuhanChina
  3. 3.Institute of Precision Manufacturing, Wuhan University of Science and TechnologyWuhan University of Science and TechnologyWuhanChina
  4. 4.Hubei Key Laboratory of Mechanical Transmission and Manufacturing EngineeringWuhan University of Science and TechnologyWuhanChina

Personalised recommendations