A Survey on Vision-Based Hand Gesture Recognition

  • Taiqian Wang
  • Yande Li
  • Junfeng Hu
  • Aamir Khan
  • Li LiuEmail author
  • Caihong Li
  • Ammarah Hashmi
  • Mengyuan Ran
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11010)


Hand gesture recognition is regarded as an important part of artificial intelligence. A great effort was put into human-computer interaction so that hand gesture recognition is gradually becoming a developed technology. In light of the utilization of mouse and keyboard, the increasing needs of human-computer interaction cannot be met; hindrance turns out to be increasingly genuine. In this paper, we reviewed previous investigations of vision-based gesture recognition and summarized their findings. This paper compares the most common human-computer interaction products in recent years, which can be used to capture gesture data. Then we started with the classification of gestures and summarized the research of visual gesture recognition based on static and dynamic gestures. The gesture representations we summarized includes appearance-based and 3D model-based methods. We also introduced the applications of the two kinds of hand gestures recognition in the papers of recent years. A possible classification methods was put forward to improve the performance of gesture recognition. The goal of this paper is to summarize the current technology and research results and compare the differences and the advantage of different hand gesture recognition methods, which will contribute to the following research.


Interaction products Hand gesture recognition Gesture representation Application Classification 



This work was supported by grants from the Fundamental Research Funds for the Key Research Programm of Chongqing Science & Technology Commission (grant no. cstc2017rgzn-zdyf0064), the Chongqing Provincial Human Resource and Social Security Department (grant no. cx2017092), the Central Universities in China (grant nos. 2018CDXYRJ0030, CQU0225001104447), Science and Technology Innovation Project of Foshan City, China (grant no. 2015IT100095), the Fundamental Research Funds for the Central Universities (grant no. lzujbkey-2016-br03), CERNET Innovation Project (grant no. NGII20150603) and Science and Technology Planning Project of Guangdong Province, China (grant no. 2016B010108002).


  1. 1.
    Al-Shamayleh, A.S., Ahmad, R., Abushariah, M.A.M., Alam, K.A., Jomhari, N.: A systematic literature review on vision based gesture recognition techniques. Multimed. Tools Appl. 77(21), 28121–28184 (2018)CrossRefGoogle Scholar
  2. 2.
    Badi, H., Fadhel, M., Sabry, S., Jasem, M.: A survey on humanccomputer interaction technologies and techniques. Int. J. Data Sci. Anal. 3(2), 1–11 (2017)CrossRefGoogle Scholar
  3. 3.
    Chaudhary, A., Raheja, J.L., Das, K., Raheja, S.: A survey on hand gesture recognition in context of soft computing. In: Meghanathan, N., Kaushik, B.K., Nagamalai, D. (eds.) CCSIT 2011. CCIS, vol. 133, pp. 46–55. Springer, Heidelberg (2011). Scholar
  4. 4.
    Chen, Y., Huang, H., Xu, W., Center, T.I.: Gesture recognition system based on wearable accelerometer. Autom. Inf. Eng. (2015)Google Scholar
  5. 5.
    Devineau, G., Xi, W., Moutarde, F., Yang, J.: Deep learning for hand gesture recognition on skeletal data (2018)Google Scholar
  6. 6.
    Fujii, T., Lee, J.H., Okamoto, S.: Gesture recognition system for human-robot interaction and its application to robotic service task. Lect. Notes Eng. Comput. Sci. 2209(1), 63–68 (2014)Google Scholar
  7. 7.
    Gao, Q., Liu, J., Ju, Z., Li, Y., Zhang, T., Zhang, L.: Static hand gesture recognition with parallel CNNs for space human-robot interaction. In: Huang, Y.A., Wu, H., Liu, H., Yin, Z. (eds.) ICIRA 2017. LNCS (LNAI), vol. 10462, pp. 462–473. Springer, Cham (2017). Scholar
  8. 8.
    Hasan, H., Abdul-Kareem, S.: Retracted article: humanccomputer interaction using vision-based hand gesture recognition systems: a survey. Neural Comput. Appl. 25(2), 251–261 (2014)CrossRefGoogle Scholar
  9. 9.
    Huang, J., Zhou, W., Zhang, Q., Li, H., Li, W.: Video-based sign language recognition without temporal segmentation (2018)Google Scholar
  10. 10.
    Huang, Y.A., Wu, H., Liu, H., Yin, Z.: Erratum to: intelligent robotics and applications. In: Huang, Y.A., Wu, H., Liu, H., Yin, Z. (eds.) ICIRA 2017. LNCS (LNAI), vol. 10463, p. E1. Springer, Cham (2017). Scholar
  11. 11.
    Itkarkar, R.R., Nandi, A.V.: A survey of 2D and 3D imaging used in hand gesture recognition for human-computer interaction (HCI). In: IEEE International WIE Conference on Electrical and Computer Engineering, pp. 188–193 (2017)Google Scholar
  12. 12.
    Jing, S., Jiao-Yan, A.I.: The real-time dynamic gesture recognition based on computer vision technology and its application for powerpoint control. Comput. Technol. Autom. (2013)Google Scholar
  13. 13.
    Kalpakas, A.C., Stampoulis, K.N., Zikos, N.A., Zaharos, S.K.: 2D hand gesture recognition methods for interactive board game applications. In: Proceedings of the International Conference on Signal Processing and Multimedia Applications, SIGMAP 2008, Porto, Portugal, 26–29 July 2008, pp. 325–331 (2015)Google Scholar
  14. 14.
    Kishore, P.V.V., Kumar, D.A., Sastry, A.S.C.S., Kumar, E.K.: Motionlets matching with adaptive kernels for 3D Indian sign language recognition. IEEE Sens. J. PP(99), 1–1 (2018)Google Scholar
  15. 15.
    Kumar, P., Gauba, H., Roy, P.P., Dogra, D.P.: Coupled HMM-based multi-sensor data fusion for sign language recognition. Pattern Recogn. Lett. 86(C), 1–8 (2017)CrossRefGoogle Scholar
  16. 16.
    Lad, M.: Soft computing approaches for hand gesture recognition. Int. J. Comput. Sci. Eng. Inf. Technol. Res. 3(4), 55–58 (2013)Google Scholar
  17. 17.
    Lee, K., et al.: Four DoF gesture recognition with an event-based image sensor. In: Consumer Electronics, pp. 293–294 (2012)Google Scholar
  18. 18.
    Li, S., Xu, K., Zhang, H.: Research on virtual Guzheng based on Kinect. In: International Conference on Computer-Aided Design, Manufacturing, Modeling and Simulation, p. 040010 (2018)Google Scholar
  19. 19.
    Liang, B., Zheng, L.: Three dimensional motion trail model for gesture recognition. In: IEEE International Conference on Computer Vision Workshops, pp. 684–691 (2013)Google Scholar
  20. 20.
    Liu, N., Aziz, M.A.A.: A robust deep belief network-based approach for recognizing dynamic hand gestures. In: International Bhurban Conference on Applied Sciences and Technology, pp. 199–205 (2016)Google Scholar
  21. 21.
    Liu, Y., Yin, Y., Zhang, S.: Hand gesture recognition based on HU moments in interaction of virtual reality. In: International Conference on Intelligent Human-Machine Systems and Cybernetics, pp. 145–148 (2012)Google Scholar
  22. 22.
    Naguri, C.R., Bunescu, R.C.: Recognition of dynamic hand gestures from 3D motion data using LSTM and CNN architectures. In: IEEE International Conference on Machine Learning and Applications, pp. 1130–1133 (2017)Google Scholar
  23. 23.
    Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015)CrossRefGoogle Scholar
  24. 24.
    Rokade-Shinde, R., Sonawane, J.: Dynamic hand gesture recognition. In: International Conference on Signal and Information Processing (2017)Google Scholar
  25. 25.
    Ruffieux, S., Lalanne, D., Mugellini, E., Abou Khaled, O.: A survey of datasets for human gesture recognition. In: Kurosu, M. (ed.) HCI 2014. LNCS, vol. 8511, pp. 337–348. Springer, Cham (2014). Scholar
  26. 26.
    Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules (2017)Google Scholar
  27. 27.
    Su, B., Wu, H., Sheng, M.: Human action recognition method based on hierarchical framework via Kinect skeleton data. In: International Conference on Machine Learning and Cybernetics, pp. 83–90 (2017)Google Scholar
  28. 28.
    Suarez, J., Murphy, R.R.: Hand gesture recognition with depth images: a review, pp. 411–417 (2012)Google Scholar
  29. 29.
    Tang, A., Lu, K., Wang, Y., Huang, J., Li, H.: A real-time hand posture recognition system using deep neural networks. ACM Trans. Intell. Syst. Technol. 6(2), 1–23 (2015)CrossRefGoogle Scholar
  30. 30.
    Tewari, A., Taetz, B., Grandidier, F., Stricker, D.: A probabilistic combination of CNN and RNN estimates for hand gesture based interaction in car. In: IEEE International Symposium on Mixed and Augmented Reality (2017)Google Scholar
  31. 31.
    Trigueiros, P., Ribeiro, F., Reis, L.P.: Vision-based Portuguese sign language recognition system. In: Rocha, Á., Correia, A.M., Tan, F.B., Stroetmann, K.A. (eds.) New Perspectives in Information Systems and Technologies, Volume 1. AISC, vol. 275, pp. 605–617. Springer, Cham (2014). Scholar
  32. 32.
    Wu, X., Yang, C., Wang, Y., Li, H., Xu, S.: An intelligent interactive system based on hand gesture recognition algorithm and kinect, vol. 2, no. 4, pp. 294–298 (2012)Google Scholar
  33. 33.
    Wu, X., Yang, C., Wang, Y., Li, H., Xu, S.: An intelligent interactive system based on hand gesture recognition algorithm and kinect. In: Fifth International Symposium on Computational Intelligence and Design, pp. 294–298 (2013)Google Scholar
  34. 34.
    Yang, X.W., Feng, Z.Q., Huang, Z.Z., He, N.N.: A gesture recognition algorithm using hausdorff-like distance template matching based on the main direction of gesture. Appl. Mech. Mater. 713–715(3), 2156–2159 (2015)CrossRefGoogle Scholar
  35. 35.
    Zhang, B., Yun, R., Qiu, H.: Hand gesture recognition in natural state based on rotation invariance and OpenCV realization. In: Zhang, X., Zhong, S., Pan, Z., Wong, K., Yun, R. (eds.) Edutainment 2010. LNCS, vol. 6249, pp. 486–496. Springer, Heidelberg (2010). Scholar
  36. 36.
    Zhang, X., Chen, X., Li, Y., Lantz, V., Wang, K., Yang, J.: A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 41(6), 1064–1076 (2011)CrossRefGoogle Scholar
  37. 37.
    Zhang, Z., He, X., Chen, Z., Wu, K.: Research on dynamic gesture recognition algorithm based on multi-touch human-computer interaction devices in classroom teaching. In: International Conference on Machinery, Materials and Information Technology Applications (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Information Science and EngineeringLanzhou UniversityLanzhouChina
  2. 2.School of Software EngineeringChongqing UniversityChongqingChina
  3. 3.School of Software EngineeringBeijing Institute of TechnologyBeijingChina

Personalised recommendations