Abstract
A method is proposed to track the full hand motion from 3D points reconstructed using a stereoscopic set of cameras. This approach combines the advantages of methods that use 2D motion (e.g. optical flow), and those that use a 3D reconstruction at each time frame to capture the hand motion. Matching either contours or a 3D reconstruction against a 3D hand model is usually very difficult due to self-occlusions and the locally-cylindrical structure of each phalanx in the model, but our use of 3D point trajectories constrains the motion and overcomes these problems.
Our tracking procedure uses both the 3D point matches between two time frames and a smooth surface model of the hand, build with implicit surface. We used animation techniques to represent faithfully the skin motion, especially near joints. Robustness is obtained by using an EM version of the ICP algorithm for matching points between consecutive frames, and the tracked points are then registered to the surface of the hand model. Results are presented on a stereoscopic sequence of a moving hand, and are evaluated using a side view of the sequence.
Chapter PDF
Similar content being viewed by others
References
Athitsos, V., Sclaroff, S.: Estimating 3D hand pose from a cluttered image. In: Proc. CVPR [10]
Blinn, J.: A generalization of algebraic surface drawing. ACM Trans. on Graphics 1(3) (1982)
Bloomenthal, J.: Medial based vertex deformation. In: Symposium on Computer Animation. In: SIGGRAPH (2000)
Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. CVIU 89(2-3), 114–141 (2003)
Delamarre, Q., Faugeras, O.: Finding pose of hand in video images: a stereobased approach. In: Third International Conference on Automatic Face and Gesture Recognition, pp. 585–590 (April 1998)
Delamarre, Q., Faugeras, O.: 3D articulated models and multiview tracking with physical forces. CVIU 81(3), 328–357 (2001)
Feldmar, J., Ayache, N.: Rigid, affine and locally addinne registration of freeform surfaces. IJCV 18(2), 99–119 (1996)
Granger, S., Pennec, X.: Multi-scale EM-ICP: A fast and robust approach for surface registration. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 418–432. Springer, Heidelberg (2002)
Heap, T., Hogg, D.: Towards 3D hand tracking using a deformable model. In: Proc. Conf. on Automatic Face and Gesture Recognition, pp. 140–145 (1995)
IEEE Comp.Soc. Madison, Wisconsin (June 2003)
Lu, S., Metaxas, D., Samaras, D., Oliensis, J.: Using multiple cues for hand tracking and model refinement. In: Proc. CVPR [10], pp. 443–450
Moeslund, T., Granum, E.: A survey of computer vision-based human motion capture. CVIU 81(3), 231–268 (2001)
Nirei, K., Saito, H., Mochimaru, M., Ozawa, S.: Human hand tracking from binocular image sequences. In: 22th Int’l Conf. on Industrial Electronics, Control, and Instrumentation, Taipei, pp. 297–302 (August 1996)
Plänkers, R., Fua, P.: Articulated soft objects for multi-view shape and motion capture. IEEE Trans. Pattern Analysis and Machine Intelligence 25(10) (2003)
Rangarajan, A., Chui, H., Bookstein, F.L.: The softassign procrustes matching algorithm. In: IPMI, pp. 29–42 (1997)
Rehg, J.M., Kanade, T.: Model-based tracking of self-occluding articulated objects. In: Proc. 5th International Conference on Computer Vision, Boston, MA, June 1995, pp. 612–617. IEEE Comp.Soc. Press, Los Alamitos (1995)
Shi, J., Tomasi, C.: Good features to track. In: Proc. CVPR, Seattle, WA, June 1994, pp. 593–600. IEEE, Los Alamitos (1994)
Shimada, N., Kimura, K., Shirai, Y.: Real-time 3-D hand posture estimation based on 2-D appearance retrieval using monocular camera. In: Proc. 2nd Int’l Workshop Recognition, Analysis, and Tracking of Faces and Gestures in Realtime Systems (RATFFG-RTS), Vancouver, Canada (July 2001)
Stenger, B., Mendonça, P.R.S., Cipolla, R.: Model based 3D tracking of an articulated hand. In: Proc. CVPR, pp. 310–315. IEEE Comp. Soc., Los Alamitos (2001)
Utsumi, A., Ohya, J.: Multiple-hand-gesture tracking using multiple cameras. In: Proc. CVPR, Fort Collins, Colorado, June 1999, pp. 1473–1478. IEEE Comp. Soc., Los Alamitos (1999)
Wu, Y., Lin, J.Y., Huang, T.S.: Capturing natural hand articulation. In: Proc. 8th International Conference on Computer Vision, Vancouver, Canada, pp. 426–432. IEEE Comp.Soc. Press, Los Alamitos (2001)
Wyvill, B., McPheeters, C., Wyvill, G.: Data structure for soft objects. The Visual Computer 2(4), 227–234 (1986)
Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. IJCV 13(2), 119–152 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dewaele, G., Devernay, F., Horaud, R. (2004). Hand Motion from 3D Point Trajectories and a Smooth Surface Model. In: Pajdla, T., Matas, J. (eds) Computer Vision - ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, vol 3021. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24670-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-540-24670-1_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21984-2
Online ISBN: 978-3-540-24670-1
eBook Packages: Springer Book Archive