Hand Motion from 3D Point Trajectories and a Smooth Surface Model

  • Guillaume Dewaele
  • Frédéric Devernay
  • Radu Horaud
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3021)


A method is proposed to track the full hand motion from 3D points reconstructed using a stereoscopic set of cameras. This approach combines the advantages of methods that use 2D motion (e.g. optical flow), and those that use a 3D reconstruction at each time frame to capture the hand motion. Matching either contours or a 3D reconstruction against a 3D hand model is usually very difficult due to self-occlusions and the locally-cylindrical structure of each phalanx in the model, but our use of 3D point trajectories constrains the motion and overcomes these problems.

Our tracking procedure uses both the 3D point matches between two time frames and a smooth surface model of the hand, build with implicit surface. We used animation techniques to represent faithfully the skin motion, especially near joints. Robustness is obtained by using an EM version of the ICP algorithm for matching points between consecutive frames, and the tracked points are then registered to the surface of the hand model. Results are presented on a stereoscopic sequence of a moving hand, and are evaluated using a side view of the sequence.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Athitsos, V., Sclaroff, S.: Estimating 3D hand pose from a cluttered image. In: Proc. CVPR [10]Google Scholar
  2. 2.
    Blinn, J.: A generalization of algebraic surface drawing. ACM Trans. on Graphics 1(3) (1982)Google Scholar
  3. 3.
    Bloomenthal, J.: Medial based vertex deformation. In: Symposium on Computer Animation. In: SIGGRAPH (2000)Google Scholar
  4. 4.
    Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. CVIU 89(2-3), 114–141 (2003)zbMATHGoogle Scholar
  5. 5.
    Delamarre, Q., Faugeras, O.: Finding pose of hand in video images: a stereobased approach. In: Third International Conference on Automatic Face and Gesture Recognition, pp. 585–590 (April 1998)Google Scholar
  6. 6.
    Delamarre, Q., Faugeras, O.: 3D articulated models and multiview tracking with physical forces. CVIU 81(3), 328–357 (2001)zbMATHGoogle Scholar
  7. 7.
    Feldmar, J., Ayache, N.: Rigid, affine and locally addinne registration of freeform surfaces. IJCV 18(2), 99–119 (1996)CrossRefGoogle Scholar
  8. 8.
    Granger, S., Pennec, X.: Multi-scale EM-ICP: A fast and robust approach for surface registration. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 418–432. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Heap, T., Hogg, D.: Towards 3D hand tracking using a deformable model. In: Proc. Conf. on Automatic Face and Gesture Recognition, pp. 140–145 (1995)Google Scholar
  10. 10.
    IEEE Comp.Soc. Madison, Wisconsin (June 2003)Google Scholar
  11. 11.
    Lu, S., Metaxas, D., Samaras, D., Oliensis, J.: Using multiple cues for hand tracking and model refinement. In: Proc. CVPR [10], pp. 443–450Google Scholar
  12. 12.
    Moeslund, T., Granum, E.: A survey of computer vision-based human motion capture. CVIU 81(3), 231–268 (2001)zbMATHGoogle Scholar
  13. 13.
    Nirei, K., Saito, H., Mochimaru, M., Ozawa, S.: Human hand tracking from binocular image sequences. In: 22th Int’l Conf. on Industrial Electronics, Control, and Instrumentation, Taipei, pp. 297–302 (August 1996)Google Scholar
  14. 14.
    Plänkers, R., Fua, P.: Articulated soft objects for multi-view shape and motion capture. IEEE Trans. Pattern Analysis and Machine Intelligence 25(10) (2003)Google Scholar
  15. 15.
    Rangarajan, A., Chui, H., Bookstein, F.L.: The softassign procrustes matching algorithm. In: IPMI, pp. 29–42 (1997)Google Scholar
  16. 16.
    Rehg, J.M., Kanade, T.: Model-based tracking of self-occluding articulated objects. In: Proc. 5th International Conference on Computer Vision, Boston, MA, June 1995, pp. 612–617. IEEE Comp.Soc. Press, Los Alamitos (1995)CrossRefGoogle Scholar
  17. 17.
    Shi, J., Tomasi, C.: Good features to track. In: Proc. CVPR, Seattle, WA, June 1994, pp. 593–600. IEEE, Los Alamitos (1994)Google Scholar
  18. 18.
    Shimada, N., Kimura, K., Shirai, Y.: Real-time 3-D hand posture estimation based on 2-D appearance retrieval using monocular camera. In: Proc. 2nd Int’l Workshop Recognition, Analysis, and Tracking of Faces and Gestures in Realtime Systems (RATFFG-RTS), Vancouver, Canada (July 2001)Google Scholar
  19. 19.
    Stenger, B., Mendonça, P.R.S., Cipolla, R.: Model based 3D tracking of an articulated hand. In: Proc. CVPR, pp. 310–315. IEEE Comp. Soc., Los Alamitos (2001)Google Scholar
  20. 20.
    Utsumi, A., Ohya, J.: Multiple-hand-gesture tracking using multiple cameras. In: Proc. CVPR, Fort Collins, Colorado, June 1999, pp. 1473–1478. IEEE Comp. Soc., Los Alamitos (1999)Google Scholar
  21. 21.
    Wu, Y., Lin, J.Y., Huang, T.S.: Capturing natural hand articulation. In: Proc. 8th International Conference on Computer Vision, Vancouver, Canada, pp. 426–432. IEEE Comp.Soc. Press, Los Alamitos (2001)Google Scholar
  22. 22.
    Wyvill, B., McPheeters, C., Wyvill, G.: Data structure for soft objects. The Visual Computer 2(4), 227–234 (1986)CrossRefGoogle Scholar
  23. 23.
    Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. IJCV 13(2), 119–152 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Guillaume Dewaele
    • 1
  • Frédéric Devernay
    • 1
  • Radu Horaud
    • 1
  1. 1.INRIA Rhône-AlpesSaint Ismier CedexFrance

Personalised recommendations