Context-based recognition of manipulative hand gestures for human computer interaction

  • Kang-Hyun Jo
  • Yoshinori Kuno
  • Yoshiaki Shirai
Session S2A: Computer Vision & Virtual Reality
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1352)


This paper presents a system recognizing manipulative hand gestures like grasping, moving, holding an object(s) with both hands, and extending or shortening of the object(s) in the virtual world using contextual information. Contextual information is represented by a state transition diagram, each state of which indicates possible gestures at the next moment. Image features obtained from extracted hand regions are used to judge state transition. When we use a gesture recognition system, we sometimes move our hands unintentionally. To solve this problem, our system has a rest state in the state transition diagram. All unintentional actions are considered as taking a rest and ignored. In addition, the system can recognize collaborative gestures with both hands. They are expressed in a single state so that the complexity in combination of gestures of each hand can be avoided. We have realized an experimental human interface system. Operational experiments show promising results.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    T.S. Huang and V.I. Pavlović, “Hand gesture modeling, analysis, and synthesis”, Pooc. Int. Workshop on Automatic Face-and Gesture-Recognition, pp.73–79, 1995.Google Scholar
  2. 2.
    P. Maes, T. Darrell, B. Blumberg, and A. Pentland, “The ALIVE system: wireless, full-body interaction with autonomous agents”, MIT Media Lab. Perceptual Computing Technical Report No. 257, 1995.Google Scholar
  3. 3.
    R. Kjeldsen and J. Kender, “Visual hand gesture recognition for window system control”, Pooc. Int. Workshop on Automatic Face-and Gesture-Recognition, pp.184–188, 1995.Google Scholar
  4. 4.
    Francis K.H. Quek, “Unencumbered gestural interaction”, IEEE Multimedia, Vol.4, No 3, pp.36–47, 1996.Google Scholar
  5. 5.
    M. Fukumoto, K. Mase, and Y. Suenaga, “Real-time detection of pointing actions for a glove-free interface”, Proc. IAPR Workshop on Machine Vision Applications '92, pp.473–476, 1992.Google Scholar
  6. 6.
    R. Cipolla, P.A. Hadfield, and N.J. Hollinghurst, “Uncalibrated stereo vision with pointing for a man-machine interface”, Proc. IAPR Workshop on Machine Vision Applications '94, pp.163–166, 1994.Google Scholar
  7. 7.
    K.-H. Jo, Y. Kuno, and Y. Shirai, “Invariance based human interface system using realtime tracker”, Proc. Second Asian Conf. on Computer Vision '95, Vol.2, pp.22–27, 1995.Google Scholar
  8. 8.
    K.-H. Jo, K. Hayashi, Y. Kuno and Y. Shirai, “Vision-based human interface system with world-fixed and human-centered frames using multiple view invariance”, Trans. IEICE Information and Systems, VoL.E79-D, No.6, pp.219–228, 1996.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Kang-Hyun Jo
    • 1
  • Yoshinori Kuno
    • 1
  • Yoshiaki Shirai
    • 1
  1. 1.Department of Computer-Controlled Mechanical SystemsOsaka UniversityOsaka

Personalised recommendations