Global Hand Pose Estimation by Multiple Camera Ellipse Tracking

  • Jorge Usabiaga
  • Ali Erol
  • George Bebis
  • Richard Boyle
  • Xander Twombly
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4291)


Immersive virtual environments with life-like interaction capabilities have very demanding requirements including high precision and processing speed. These issues raise many challenges for computer vision-based motion estimation algorithms. In this study, we consider the problem of hand tracking using multiple cameras and estimating its 3D global pose (i.e., position and orientation of the palm). Our interest is in developing an accurate and robust algorithm to be employed in an immersive virtual training environment, called ”Virtual GloveboX” (VGX) [1], which is currently under development at NASA Ames. In this context, we present a marker-based, hand tracking and 3D global pose estimation algorithm that operates in a controlled, multi-camera, environment built to track the user’s hand inside VGX. The key idea of the proposed algorithm is tracking the 3D position and orientation of an elliptical marker placed on the dorsal part of the hand using model-based tracking approaches and active camera selection. It should be noted that, the use of markers is well justified in the context of our application since VGX naturally allows for the use of gloves without disrupting the fidelity of the interaction. Our experimental results and comparisons illustrate that the proposed approach is more accurate and robust than related approaches. A byproduct of our multi-camera ellipse tracking algorithm is that, with only minor modifications, the same algorithm can be used to automatically re-calibrate (i.e., fine-tune) the extrinsic parameters of a multi-camera system leading to more accurate pose estimates.


Multiple Camera Extrinsic Parameter Matching Error Reprojection Error Hand Tracking 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Twombly, X., Smith, J., Montgomery, K., Boyle, R.: The virtual glovebox (vgx): a semi-immersive virtual environment for training astronauts in life science experiments. Journal of Systemics and Informatics 6 (2006)Google Scholar
  2. 2.
    Pavlovic, V.I., Sharma, R.S., Huang, T.S.: Visual interpretation of hand gestures for human-computer interaction: A review. PAMI (1997)Google Scholar
  3. 3.
    Erol, A., Bebis, G., Nicolescu, M., Boyle, R., Twombly, X.: A review on vision based full dof hand motion estimation. In: IEEE Workshop on Vision for Human Computer Interaction (2005)Google Scholar
  4. 4.
    Chua, C.S., Guan, H.Y., Ho, Y.K.: Model-based finger posture estimation. In: ACCV 2000 (2000)Google Scholar
  5. 5.
    Chua, C., Guan, H., Ho, Y.: Model-based 3d hand posture estimation from a single 2d image. Image and Vision Computing 20, 191–202 (2002)CrossRefGoogle Scholar
  6. 6.
    Holden, E.: Visual Recognition of Hand Motion. PhD thesis, Department of Computer Science, University of Western Australia (1997)Google Scholar
  7. 7.
    Lien, C.C., Huang, C.L.: Model based articulated hand motion tracking for gesture recognition. Image and Vision Computing 16, 121–134 (1998)CrossRefGoogle Scholar
  8. 8.
    Lee, J., Kunii, T.: Constraint-based hand animation. In: Models and Techniques in Computer Animation, pp. 110–127. Springer, Tokyo (1993)Google Scholar
  9. 9.
    Kim, H., Fellner, D.W.: Interaction with hand gesture for a back-projection wall. In: CGI 2004: Proceedings of the Computer Graphics International (CGI 2004), pp. 395–402. IEEE Computer Society Press, Washington (2004)Google Scholar
  10. 10.
    Ma, S.D.: Conics-based stereo, motion estimation and pose determination. IJCV 10, 7–25 (1993)CrossRefGoogle Scholar
  11. 11.
    Quan, L.: Conic reconstruction and correspondence from two views. IEEE Transactions on Pattern Analysis and Machine Intelligence 18, 151–160 (1996)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Maggioni, C., B.K.: Gesturecomputer - history, design and applications. In: Cipolla, R., Pentland, A. (eds.) Computer Vision for Human-Machine Interaction, Cambridge, pp. 312–325 (1998)Google Scholar
  13. 13.
    Bouguet, J.Y.: Camera calibration toolbox for matlabGoogle Scholar
  14. 14.
    Svoboda, T., Martinec, D., Pajdla, T.: A convenient multi-camera self-calibration for virtual environmentsGoogle Scholar
  15. 15.
    Martin, F., Horaud, R.: Multiple-camera tracking of rigid objects. In: INRIA (2001)Google Scholar
  16. 16.
    Lowe, D.: Fitting parameterized three-dimensional models to images. PAMI 13, 441–450 (1991)Google Scholar
  17. 17.
    (Gnu scientific library),

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jorge Usabiaga
    • 1
  • Ali Erol
    • 1
  • George Bebis
    • 1
  • Richard Boyle
    • 2
  • Xander Twombly
    • 2
  1. 1.Computer Vision LaboratoryUniversity of NevadaRenoUSA
  2. 2.BioVis LaboratoryNASA Ames Research CenterUSA

Personalised recommendations