Advertisement

Vision-Based Gesture Recognition: A Review

  • Ying Wu
  • Thomas S. Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1739)

Abstract

The use of gesture as a natural interface serves as a motivating force for research in modeling, analyzing and recognition of gestures. In particular, human computer intelligent interaction needs vision-based gesture recognition, which involves many interdisciplinary studies. A survey on recent vision-based gesture recognition approaches is given in this paper. We shall review methods of static hand posture and temporal gesture recognition. Several application systems of gesture recognition are also described in this paper. We conclude with some thoughts about future research directions.

Keywords

Hide Markov Model Sign Language Hand Posture Gesture Recognition Hand Gesture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Becker, D.: Sensei: A Real-time Recognition, Feedback and Training System for Tai Chi Gestures, MIT Media Lab, MS thesis (1997)Google Scholar
  2. 2.
    Berry, G.: Small-wall: A Multimodal Human Computer Intelligent Interaction Test Bed with Applications, Dept. of ECE, University of Illinois at Urbana-Champaign, MS thesis (1998)Google Scholar
  3. 3.
    Black, M., Jepson, A.: Recognition Temporal Trajectories using the Condensation Algorithm, Int’l Conf. on Automatic Face and Gesture Recognition, Japan, pp.16–21 (1998)Google Scholar
  4. 4.
    Bobick, A., Ivanov, Y.: Action Recognition using Probabilistic Parsing, IEEE Int’l Conf. on Computer Vision and Pattern Recognition (1998)Google Scholar
  5. 5.
    Bobick, A., Wilson, A.: A State-Based Approach to the Representation and Recognition of Gesture, IEEE trans. PAMI, Vol.19, No.12, Dec., pp1325–1337 (1997)Google Scholar
  6. 6.
    Bradski, G., Yeo, B., Yeung, M.: Gesture and Speech for Video Content Navigation, Proc. Workshop on Perceptual User Interfaces (1998)Google Scholar
  7. 7.
    Brand, M., Oliver, N., Pentland, A.: Coupled Hidden Markov Models for Complex Action Recognition, Proc. IEEE Int’l Conf. on Computer Vision and Pattern Recognition (1997)Google Scholar
  8. 8.
    Bregler, C.: Learning and Recognizing Human Dynamics in Video Sequences, Proc. IEEE Int’l Conf. on Computer Vision and Pattern Recognition (1997)Google Scholar
  9. 9.
    Campbell, L., et al.: Invariant Features for 3-D Gesture Recognition, Int’l Conf. on Automatic Face and Gesture Recognition, Killington, pp.157–162. (1996)Google Scholar
  10. 10.
    Cohen, C., Conway, L., Koditschek, D.: Dynamical System Representation, Generation, and Recognition of Basic Oscillatory Motion Gestures, Int’l Conf. on Automatic Face and Gesture Recognition, Killington (1996)Google Scholar
  11. 11.
    Crowley, J., Berard, F., Coutaz, J.: Finger Tracking as An Input Device for Augmented Reality, Int.Workshop on Automatic Face and Gesture Recognition, Zurich, pp.195–200. (1995)Google Scholar
  12. 12.
    Cui, Y, Weng, J.: Hand Sign Recognition from Intensity Image Sequences with Complex Background, Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp.88–93. (1996)Google Scholar
  13. 13.
    Cui, Y., Weng, J.: Hand Segmentation Using Learning-Based Prediction and Verification for Hand Sign Recognition, Int’l Conf. on Automatic Face and Gesture Recognition, Killington (1996)Google Scholar
  14. 14.
    Cui, Y., Swets, D., Weng, J.: Learning-Based Hand Sign Recognition Using SHOSLIF-M, Int. Workshop on Automatic Face and Gesture Recognition, Zurich, pp.201–206. (1995)Google Scholar
  15. 15.
    Cutler, R., Turk, M.: View-based Interpretation of Real-time Optical Flow for Gesture Recognition, IEEE Int. Conf. on Automatic Face and Gesture Recognition, Japan. (1998)Google Scholar
  16. 16.
    Darrell, T., Pentland, A.: Active Gesture Recognition Using Partially Observable Markov Decision Processes, IEEE Int’l Conf. on Pattern Recognition (1996)Google Scholar
  17. 17.
    Davis, J., Bobic k,A.: Virtual PAT: A Virtual Personal Aerobic Trainer, Proc. Workshop on Perceptual User Interfaces, pp.13–18. (1998)Google Scholar
  18. 18.
    Davis, J., Bobick, A.: The Representation and Recognition of Action Using Temporal Templates, IEEE CVPR, pp.928–934. (1997)Google Scholar
  19. 19.
    Davis, J., Shah, M.: Visual Gesture Recognition, Vision, Image and Signal Processing, 141(2), pp.101–106. (1994)CrossRefGoogle Scholar
  20. 20.
    Fernandez, R.: Stochastic Modeling of Physiological Signals with Hidden Markov Models: A Step Toward Frustration Detection in Human-Computer Interfaces, MIT Media Lab, MS thesis. (1997)Google Scholar
  21. 21.
    Gavrila, D.: The Visual Analysis of Human Movement: A Survey, Computer Vision and Image Understanding, Vol.73, No.1, Jan, pp.82–98. (1999)zbMATHCrossRefGoogle Scholar
  22. 22.
    Goncalves, L., Bernardo, E., Perona, P.: Reach Out and Touch Space, IEEE Int. Conf. on Automatic Face and Gesture Recognition, Japan.(1998)Google Scholar
  23. 23.
    Imagawa, K., Lu, S., Igi, S.: Color-Based Hand Tracking System for Sign Language Recognition, IEEE Int. Conf. on Automatic Face and Gesture Recognition, Japan. (1998)Google Scholar
  24. 24.
    Jo, K., Kuno, Y., Shirai, Y.: Manipulative Hand Gestures Recognition Using Task Knowledge for Human Computer Interaction, IEEE Int. Conf. on Automatic Face and Gesture Recognition, Japan. (1998)Google Scholar
  25. 25.
    Ju, S., Black, M., Minneman, S., Kimber, D.: Analysis of Gesture and Action in Technical Talks for Video Indexing, IEEE Conf. on Computer Vision and Pattern Recognition, CVPR97. (1997)Google Scholar
  26. 26.
    Kendon, A.: urrent Issues in the Study of Gesture The Biological Foundation of Gestures: Motor and Semiotic Aspects, pp.23–47, Lawrence Erlbaum Associate, Hillsdale, NJ, (1986)Google Scholar
  27. 27.
    Kjeldsen, R., Kender, J.: Interaction with On-Screen Objects using Visual Gesture Recognition, Proc. IEEE CVPR97, (1997)Google Scholar
  28. 28.
    Kobayashi, T., Haruyama, S.: Partly-Hidden Markov Model and Its Application to Gesture Recognition, IEEE Proceedings of ICASSP97, Vol. VI, pp.3081–84. (1997)Google Scholar
  29. 29.
    Kurita, T., Hayamizu, S.: Gesture Recognition using HLAC Features of PARCOR Images and HMM based Recognizer, IEEE Int. Conf. on Automatic Face and Gesture Recognition, Japan. (1998)Google Scholar
  30. 30.
    Liang, R., Ouhyoung, M.: A Real-time Continuous Gesture Recognition System for Sign Language, IEEE Int. Conf. on Automatic Face and Gesture Recognition, Japan. (1998)Google Scholar
  31. 31.
    McNeil, D.: Hand and Mind, University of Chicago Press, Chicago. (1992)Google Scholar
  32. 32.
    Nam, Y., Wohn, K.: Recognition of Space-time Hand-Gestures using Hidden Markov Mdel, ACM Symposium on Virtual Reality Software and Technology, HongKong, pp. 51–58. (1996)Google Scholar
  33. 33.
    Nolker, C., Ritter, H.: Illumination Independent Recognition of Deictic Arm Postures, Proc. 24 th Annual Conf. of the IEEE Industrial Electronics Society, Germany, pp. 2006–2011. (1998)Google Scholar
  34. 34.
    Pavlovic, V.: Dynamic Bayesian Networks for Information Fusion with Applications to Human-Computer Interfaces, Dept. of ECE, University of Illinois at Urbana-Champaign, Ph.D. Dissertation, (1999)Google Scholar
  35. 35.
    Pavlovic, V., Sharma, R., Huang, T.: Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review, IEEE trans. PAMI, Vol.19, No.7, July, pp677–695, (1997)Google Scholar
  36. 36.
    Pentland, A., Liu, A.: Modeling and Prediction of Human Behavior, IEEE Intelligent Vehicles, (1995)Google Scholar
  37. 37.
    Pinhanez, C. Bobick, A.: Human Action Detection Using PNF Propagation of Temporal Constraints, IEEE ICCV, (1998)Google Scholar
  38. 38.
    Polana, R. Nelson, R.: Low Level Recognition of Human Motion, IEEE Workshop on Motion of Non-Rigid and Articulated Objects, Austin, pp77–82. (1994)Google Scholar
  39. 39.
    Quek, F.: Unencumbered Gestural Interaction, IEEE Multimedia, Vol.3, No.4, pp.36–47, (1997)CrossRefMathSciNetGoogle Scholar
  40. 40.
    Quek, F., Zhao, M.: Inductive Learning in Hand Pose Recognition, IEEE Automatic Face and Gesture Recognition, (1996)Google Scholar
  41. 41.
    Rohr, K.: Towards Model-Based Recognition of Human Movements in Image Sequences, CVGIP:Image Understanding, Vol.59, No.1, Jan, pp.94–115, (1994)CrossRefGoogle Scholar
  42. 42.
    Rittscher, J., Blake, A.: Classification of Human Body Motion, IEEE Int’l Conf. on Computer Vision, (1999)Google Scholar
  43. 43.
    Starner, T., Weaver, J., Pentland, A.: Real-time American Sign Language Recognition Using Desk and Wearable Computer Based Video, IEEE trans. PAMI, (1998)Google Scholar
  44. 44.
    Stokoe, W.: Sign Language Structure, University of Buffalo Press, (1960)Google Scholar
  45. 45.
    Stoll, P., Ohya, J.: Applications of HMM Modeling to Recognizing Human Gestures in Image Sequences for a Man-Machine Interface, IEEE Intl Workshop on Robot and Human Communication, (1995)Google Scholar
  46. 46.
    Triesch, J., Malsburg, C.: Robust Classification of Hand Postures Against Complex Background, Intl Conf. On Automatic Face and Gesture Recognition, (1996)Google Scholar
  47. 47.
    Triesch, J., Malsburg, C.: A Gesture Interface for Human-Robot-Interaction, Intl Conf. On Automatic Face and Gesture Recognition, (1998)Google Scholar
  48. 48.
    Utsumi, A., Miyasato, T., Kishino, F., Nakatsu, R.: Hand Gesture Recognition System Using Multiple Cameras, IEEE ICPR, (1996)Google Scholar
  49. 49.
    Vogler, C., Metaxas, D.: ASL Recognition Based on A Coupling Between HMMs and 3D Motion Analysis, IEEE ICCV, (1998)Google Scholar
  50. 50.
    Vogler, C., Metaxas, D.: Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes, IEEE Gesture Workshop, (1999)Google Scholar
  51. 51.
    Watanabe, T., Yachida, M.: Real Time Gesture Recognition Using Eigenspace from Multi Input Image Sequences, Intl Conf. On Automatic Face and Gesture Recognition, Japan.(1998)Google Scholar
  52. 52.
    Wilson, A., Bobick, A.: Recognition and Interpretation of Parametric Gesture, IEEE Intl Conf. Computer Vision, (1998)Google Scholar
  53. 53.
    Wren, C., Pentland, A.: Dynamic Modeling of Human Motion, IEEE Intl Conf. Automatic Face and Gesture Recognition, (1997)Google Scholar
  54. 54.
    Wu, Y., Huang, T.: Human Hand Modeling, Analysis and Animation in the Context of HCI, IEEE Intl Conf. Image Processing, (1999)Google Scholar
  55. 55.
    Yang, J., Xu, Y., Chen, C.: Gesture Interface: Modeling and Learning, Proc. IEEE Int. Conf. on Robotics and Automation, Vol. 2, pp.1747–1752. (1994)Google Scholar
  56. 56.
    Yang, M., Ahuja, N.: Extraction and Classification of Visual Motion Patterns for Hand Gesture Recognition, IEEE Int’l Conf. on Computer Vision and Pattern Recognition, (1998)Google Scholar
  57. 57.
    Zeller, M., et al.: A Visual Computing Environment for Very Large Scale Biomolecular Modeling, Proc. IEEE Int. Conf. on Application-specific Systems, Architectures and Processors (ASAP), Zurich, pp. 3–12. (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Ying Wu
    • 1
    • 2
  • Thomas S. Huang
    • 1
    • 2
  1. 1.Beckman InstituteUrbana
  2. 2.University of Illinois at Urbana-ChampaignUrbana

Personalised recommendations