Skip to main content
SpringerLink
Log in

Viewpoint-independent hand gesture recognition with Kinect

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The advent and popularity of Kinect provide new choice and opportunity for hand gesture recognition research. Aiming at the effective, accurate and freely used hand gesture recognition with Kinect, this paper presents a viewpoint-independent hand gesture recognition method. Firstly, based on the rules about gesturers posture under optimal viewpoint, the gesturers point clouds are built and transformed to the optimal viewpoint with the exploration of the joint information. Then Laplacian-based contraction is applied to extract representative skeletons from the transformed point clouds. A novel partition-based algorithm is further proposed to recognize the gestures. The promising experiment results show that the proposed method performs satisfyingly on scale and rotation variant in HGR with robustness and high accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Au, O.K.C., Tai, C.L., Chu, H.K., Cohen-Or, D., Lee, T.Y.: Skeleton extraction by mesh contraction. ACM Trans. Graph. 27(3), 44 (2008)

  2. Awad, G., Han, J., Sutherland, A.: A unified system for segmentation and tracking of face and hands in sign language recognition. In: 18th International Conference on Pattern Recognition, vol. 1, pp. 239–242 (2006)

  3. Belongie, S., Mori, G., Malik, J.: Matching with shape contexts. In: Statistics and Analysis of Shapes, pp. 81–105. Springer (2006)

  4. Cao, J., Tagliasacchi, A., Olson, M., Zhang, H., Su, Z.: Point cloud skeletons via laplacian based contraction. In: Shape Modeling International Conference, 2010, pp. 187–197 (2010)

  5. Cooper, H., Holt, B., Bowden, R.: Sign language recognition. In: Visual Analysis of Humans: Looking at people, pp. 539–562 (2011)

  6. Corradini, A.: Real-time gesture recognition by means of hybrid recognizers. In: Gesture and Sign Language in Human-Computer Interaction, Lecture Notes in Computer Science, vol. 2298, pp. 34–47 (2002)

  7. Ershaed, H., Al-Alali, I., Khasawneh, N., Fraiwan, M.: An arabic sign language computer interface using the xbox kinect. In: Third Annual Undergraduate Research Conference on Applied Computing (URC 2011), vol. 1, pp. 22–28 (2011)

  8. Fanello, S.R., Gori, I., Metta, G., Odone, F.: One-shot learning for real-time action recognition. In: Pattern Recognition and Image Analysis, Lecture Notes in Computer Science, vol. 7887, pp. 31–40 (2013)

  9. Fujimura, K., Liu, X.: Sign recognition using depth image streams pp. 381–386 (2006)

  10. Gao,W., Ma, J., Wu, J., Wang, C.: Sign language recognition based on hmm/ann/dp. Int. J. Pattern Recognit. Artif. Intell. 14(05), 587–602 (2000)

  11. Grzeszcuk, R., Bradski, G., Chu, M.H., Bouguet, J.Y.: Stereo based gesture recognition invariant to 3d pose and lighting. In: IEEE Conference on Computer Vision and Pattern Recognition, 2000, vol. 1, pp. 826–833 (2000)

  12. Hadfield, S., Bowden, R.: Generalised pose estimation using depth. In: Trends and Topics in Computer Vision, Lecture Notes in Computer Science, vol. 6553, pp. 312–325 (2012)

  13. Hasanuzzaman, M., Ampornaramveth, V., Zhang, T., Bhuiyan, M., Shirai, Y., Ueno,H.: Real-time vision-based gesture recognition for human robot interaction. In: IEEE International Conference on Robotics and Biomimetics, 2004. ROBIO 2004, pp. 413–418 (2004)

  14. Hernandez-Rebollar, J.L., Lindeman, R.W., Kyriakopoulos, N.: A multi-class pattern recognition system for practical finger spelling translation. In: Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, p. 185 (2002)

  15. Imagawa, K., Lu, S., Igi, S.: Color-based hands tracking system for sign language recognition. In: Third IEEE International Conference on Automatic Face and Gesture Recognition (FG’98), pp. 462–467 (1998)

  16. Ionescu, B., Coquin, D., Lambert, P., Buzuloiu, V.: Dynamic hand gesture recognition using the skeleton of the hand. EURASIP J. Appl. Signal Process. 2005, 2101–2109 (2005)

  17. Kadous, M.W., et al.: Machine recognition of auslan signs using powergloves: towards large-lexicon recognition of sign language. In: Proceedings of the Workshop on the Integration of Gesture in Language and Speech, pp. 165–174 (1996)

  18. Kuhn, H.W.: The hungarian method for the assignment problem. Naval Res. Logist. Q. 2(1–2), 83–97 (1955)

  19. Lange, B., Chang, C.Y., Suma, E., Newman, B., Rizzo, A.S., Bolas, M.: Development and evaluation of low cost game-based balance rehabilitation tool using themicrosoft kinect sensor. In: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1831–1834 (2011)

  20. Liang, R.H., Ouhyoung,M.: A real-time continuous gesture recognition system for sign language. In: Automatic Face and Gesture Recognition, 1998. Proceedings. Third IEEE International Conference on pp. 558–567 (1998)

  21. Liwicki, S., Everingham, M.: Automatic recognition of finger-spelled words in british sign language. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2009. CVPR Workshops, pp. 50–57 (2009)

  22. Lockton, R., Fitzgibbon, A.W.:Real-time gesture recognition using deterministic boosting. In: British Machine Vision Conference, 2002, pp. 1–10 (2002)

  23. Lui, Y.M.: A least squares regression framework on manifolds and its application to gesture recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012, pp. 13–18 (2012)

  24. Mahbub, U., Roy, T., Rahman, M.S., Imtiaz, H., Serikawa, S., Ahad, M.A.R.: One-shot-learning gesture recognition using motion history based gesture silhouettes. In: The Proceedings of the 1st International Conference on Industrial Application Engineering, 2013, pp. 186–193 (2013)

  25. Malgireddy, M.R., Inwogu, I., Govindaraju, V.: A temporal bayesian model for classifying, detecting and localizing activities in video sequences. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012, pp. 43–48 (2012)

  26. Mendhurwar, K., Mudur, S., Desai, B.: Three-dimensional game character creation. In: Encyclopedia of software engineering, vol. 3, pp. 101–109 (2013)

  27. Mitra, S., Acharya, T.: Gesture recognition: a survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 37(3), 311–324 (2007)

  28. Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Efficient model-based 3d tracking of hand articulations using kinect. In: British Machine Vision Conference, 2011. BMVC, pp. 1–11 (2011)

  29. Ong, S.C., Ranganath, S.: Automatic sign language analysis: a survey and the future beyond lexical meaning. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 873–891 (2005)

  30. Pavlovic, V.I., Sharma, R., Huang, T.S.: Visual interpretation of hand gestures for human-computer interaction: a review. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 677–695 (1997)

  31. Rauschert, I., Agrawal, P., Sharma, R., Fuhrmann, S., Brewer, I., MacEachren, A.: Designing a human-centered, multimodal gis interface to support emergency management. In: The 10th ACM international symposium on Advances in geographic information systems, pp. 119–124 (2002)

  32. Sowa, T., Wachsmuth, I.: Interpretation of shape-related iconic gestures in virtual environments. In: Gesture and Sign Language in Human-Computer Interaction, Lecture Notes in Computer Science, vol. 2298, pp. 21–33 (2002)

  33. Standen, P.J., Brown, D.J., Cromby, J.: The effective use of virtual environments in the education and rehabilitation of students with intellectual disabilities. Br. J. Educ. Technol. 32(3), 289–299

  34. Suma, E.A., Lange, B., Rizzo, A., Krum, D.M., Bolas, M.: Faast: The flexible action and articulated skeleton toolkit. In: IEEE Virtual Reality Conference, 2011, pp. 247–248 (2011)

  35. TOKATLI, A.: 3d hand tracking in video sequences. Dissertation, Middle East Technical University (2005)

  36. Vogler, C., Metaxas, D.: Adapting hidden markov models for asl recognition by using three-dimensional computer vision methods. In: IEEE International Conference on Computational Cybernetics and Simulation, vol. 1, pp. 156–161 (1997)

  37. Vogler, C., Metaxas, D.: Asl recognition based on a coupling between hmms and 3d motion analysis. In: Sixth International Conference on Computer Vision, 1998, pp. 363–369 (1998)

  38. Wu, D., Zhu, F., Shao, L.: One shot learning gesture recognition from rgbd images. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012, pp. 7–12 (2012)

  39. Wu, Y., Huang, T.S.: Human hand modeling, analysis and animation in the context of hci. In: Image Processing, 1999. ICIP 99. Proceedings. 1999 International Conference on vol. 3, pp. 6–10 (1999)

  40. Wu, Y., Huang, T.S.: Hand modeling, analysis and recognition. Signal Process. Mag. IEEE 18(3), 51–60 (2001)

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant No. 61100096 and 61272386.

Author information

Authors and Affiliations

  1. School of Computer Science, Harbin Institute of Technology, Harbin , 150001, China

    Feng Jiang, Shen Wu, Gao Yang & Debin Zhao

  2. School of Electrical Engineering, Princeton University, Princeton, NY, 08544, USA

    S. Y. Kung

Authors
  1. Feng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shen Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Debin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. S. Y. Kung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feng Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, F., Wu, S., Yang, G. et al. Viewpoint-independent hand gesture recognition with Kinect. SIViP 8 (Suppl 1), 163–172 (2014). https://doi.org/10.1007/s11760-014-0668-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-014-0668-x

Keywords

Search

Navigation