Hand Gesture Analysis

Chapter

Abstract

This chapter presents a thorough overview of automatic hand gesture analysis. We cover all aspects of hand gesture recognition from detection of the hand to modeling of gestures. Based on a general gesture analysis framework, we present and discuss each necessary building block that is required to design a complete system. The last section presents two example applications: A sign language tutor for isolated signs and a gesture tracking and recognition system for continuous gestures and signs.

This is a preview of subscription content, log in to check access.

References

  1. 1.
    Aran, O., Ari, I., Benoit, A., Campr, P., Carrillo, A.H., Fanard, F.-X., Akarun, L., Caplier, A., Sankur, B.: Signtutor: An interactive system for sign language tutoring. IEEE Multimed. 16(1), 81–93 (2009) CrossRefGoogle Scholar
  2. 2.
    Aran, O., Burger, T., Caplier, A., Akarun, L.: A belief-based sequential fusion approach for fusing manual and non-manual signs. Pattern Recognit. 42(5), 812–822 (2009) MATHCrossRefGoogle Scholar
  3. 3.
    Aubert, G., Barlaud, M., Faugeras, O., Jehan-Besson, S.: Image segmentation using active contours: Calculus of variations or shape gradients. SIAM J. Appl. Math. 63(6), 2128–2154 (2003) MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Bengio, Y., Frasconi, P.: Input-output HMM’s for sequence processing. IEEE Trans. Neural Netw. 7(5), 1231–1249 (1996) CrossRefGoogle Scholar
  5. 5.
    Black, M.J., Jepson, A.D.: Eigentracking: Robust matching and tracking of articulated objects using a view-based representation. In: Proceedings of the 4th European Conference on Computer Vision (ECCV ’96), vol. I, pp. 329–342. Springer, London (1996) Google Scholar
  6. 6.
    Bobick, A., Davis, J.: Real-time recognition of activity using temporal templates. In: Proceedings of the Workshop on Applications of Computer Vision (1996) Google Scholar
  7. 7.
    Clarke, T.A., Fryer, J.G.: The development of camera calibration methods and models. Photogramm. Rec. 16(91), 51–66 (1998) CrossRefGoogle Scholar
  8. 8.
    Cornett, R.O.: Cued speech. Am. Ann. Deaf 112, 3–13 (1967) Google Scholar
  9. 9.
    Cui, Y., Swets, D.L., Weng, J.J.: Learning-based hand sign recognition using shoslif-m. In: Proceedings of the Fifth International Conference on Computer Vision, ICCV ’95, p. 631, Washington, DC, USA. IEEE Comput. Soc., Los Alamitos (1995) Google Scholar
  10. 10.
    Doucet, A., De Freitas, N., Gordon, N. (eds.): Sequential Monte Carlo Methods in Practice. Springer, Berlin (2001) MATHGoogle Scholar
  11. 11.
    Flusser, J., Suk, T.: Rotation moment invariants for recognition of symmetric objects. IEEE Trans. Image Process. 15, 3784–3790 (2006) MathSciNetCrossRefGoogle Scholar
  12. 12.
    Hu, M.K.: Visual pattern recognition by moment invariants. IRE Trans. Inf. Theory IT-8, 179–187 (1962) Google Scholar
  13. 13.
    Hu, W., Tieniu, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern. 34, 334–352 (2004) Google Scholar
  14. 14.
    Isard, M., Blake, A.: Condensation—conditional density propagation for visual tracking. Int. J. Comput. Vis. 26(1), 5–28 (1998) CrossRefGoogle Scholar
  15. 15.
    Just, A., Bernier, O., Marcel, S.: HMM and IOHMM for the recognition of mono- and bi-manual 3D hand gestures. IDIAP-RR 39, IDIAP, 2004 Google Scholar
  16. 16.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82, 35–45 (1960) Google Scholar
  17. 17.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. Int. J. Comput. Vis. 1(4), 321–331 (1988) CrossRefGoogle Scholar
  18. 18.
    Kendon, A.: Current issues in the study of gesture. In: Nespoulous, J.L., Peron, P., Lecours, A.R. (eds.) The Biological Foundations of Gestures: Motor and Semiotic Aspects, pp. 23–47. Erlbaum, Hillsdale (1986) Google Scholar
  19. 19.
    Kendon, A.: Gesture. Cambridge (2004) Google Scholar
  20. 20.
    Keskin, C., Akarun, L.: STARS: Sign tracking and recognition system using input–output HMMs. Pattern Recognit. Lett. 30, 1086–1095 (2009) CrossRefGoogle Scholar
  21. 21.
    Khotanzad, A., Hong, Y.H.: Invariant image recognition by Zernike moments. IEEE Trans. Pattern Anal. Mach. Intell. 12, 489–497 (1990) CrossRefGoogle Scholar
  22. 22.
    Lee, H.-K., Kim, J.H.: An HMM-based threshold model approach for gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 21(10), 961–973 (1999) CrossRefGoogle Scholar
  23. 23.
    Marcel, S., Bernier, O., Viallet, J.-E., Collobert, D.: Hand gesture recognition using input-output hidden Markov models. In: FG ’00: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2008, p. 456, Washington, DC, USA. IEEE Comput. Soc., Los Alamitos (2000) CrossRefGoogle Scholar
  24. 24.
    McNeill, D., Levy, E.: Conceptual representations in language activity and gesture. In: Jarvella, R., Klein, W. (eds.) Speech, Place, and Action. Wiley, New York (1982) Google Scholar
  25. 25.
    Mish, F.C.: The Merriam-Webster Dictionary. Merriam-Webster, Chicago (1997) Google Scholar
  26. 26.
    Mitiche, A., Sekkati, H.: Optical flow 3d segmentation and interpretation: A variational method with active curve evolution and level sets. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1818–1829 (2006) CrossRefGoogle Scholar
  27. 27.
    Morency, L.-P., Quattoni, A., Darrell, T.: Latent-dynamic discriminative models for continuous gesture recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2007) Google Scholar
  28. 28.
    Oka, K., Sato, Y., Koike, H.: Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems. In: Proceedings of Fifth IEEE International Conference on Automatic Face and Gesture Recognition, Washington D.C., US, p. 429 (2002) CrossRefGoogle Scholar
  29. 29.
    Pavlovic, V., Sharma, R., Huang, T.S.: Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 677–695 (1997) CrossRefGoogle Scholar
  30. 30.
    Quek, F.K.H.: Eyes in the interface. Image Vis. Comput. 13(6), 511–525 (1995) CrossRefGoogle Scholar
  31. 31.
    Riviere, J., Guitton, P.: Real time model based tracking using silhouette features. In: Proceedings of RFIA, Toulouse, France (2004) Google Scholar
  32. 32.
    Stokoe, W.C.: Sign language structure: An outline of the visual communication systems of the American deaf. Stud. Linguist., Occas. Pap. 8 (1960) Google Scholar
  33. 33.
    Triesch, J., von der Malsburg, C.: Classification of hand postures against complex backgrounds using elastic graph matching. Image Vis. Comput. 20(13–14), 937–943 (2002) CrossRefGoogle Scholar
  34. 34.
    Ueda, E., Matsumoto, Y., Imai, M., Ogasawara, T.: A hand-pose estimation for vision-based human interfaces. IEEE Trans. Ind. Electron. 50(4), 676–684 (2003) CrossRefGoogle Scholar
  35. 35.
    Wang, S.B., Quattoni, A., Morency, L.-P., Demirdjian, D.: Hidden conditional random fields for gesture recognition. In: CVPR ’06: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp. 1521–1527. IEEE Comput. Soc., Los Alamitos (2006) Google Scholar
  36. 36.
    Wu, Y., Huang, T.S.: Hand modeling, analysis, and recognition for vision based human computer interaction. IEEE Signal Process. Mag. 21(1), 51–60 (2001) Google Scholar
  37. 37.
    Yang, H.-D., Sclaroff, S., Lee, S.-W.: Sign language spotting with a threshold model based on conditional random fields. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1264–1277 (2009) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Computer Engineering DepartmentBoğaziçi UniversityIstanbulTurkey
  2. 2.Idiap Research InstituteMartignySwitzerland

Personalised recommendations