Integrating Multiple Uncalibrated Views for Human 3D Pose Estimation

  • Zibin Wang
  • Ronald Chung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6455)

Abstract

We address the problem of how human pose in 3D can be estimated from video data. The use of multiple views has the potential of tackling self-occlusion of the human subject in any particular view, as well as of estimating the human pose more precisely. We propose a scheme of allowing multiple views to be put together naturally for determining human pose, allowing hypotheses of the body parts in each view to be pruned away efficiently through consistency check over all the views. The scheme relates the different views through a linear combination-like expression of all the image data, which captures the rigidity of the human subject in 3D. The scheme does not require thorough calibration of the cameras themselves nor the camera inter-geometry. A formulation is also introduced that expresses the multi-view scheme, as well as other constraints, in the pose estimation problem. A belief propagation approach is used to reach a final human pose under the formulation. Experimental results on in-house captured image data as well as publicly available benchmark datasets are shown to illustrate the performance of the system.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Moeslund, T., Granum, E.: A Survey of Computer Vision-Based Human Motion Capture. Computer vision and Image Understanding 81(3), 231–268 (2001)CrossRefMATHGoogle Scholar
  2. 2.
    Moeslund, T., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Computer vision and Image Understanding 103, 90–126 (2006)CrossRefGoogle Scholar
  3. 3.
    Lee, M.W., Cohen, I.: Proposal maps driven mcmc for estimating human body pose in static images. In: CVPR, vol. 2, pp. 334–341 (2004)Google Scholar
  4. 4.
    Hua, G., Yang, M.H., Wu, Y.: Learning to estimate human pose with data driven belief propagation. In: CVPR, vol. 2, pp. 747–754 (2005)Google Scholar
  5. 5.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Pictorial structures for object recognition. IJCV 61(1), 55–79 (2005)CrossRefGoogle Scholar
  6. 6.
    Peursum, P., Venkatesh, S., West, G.: A Study on Smoothing for Particle-Filtered 3D Human Body Tracking. IJCV 87(1-2) (2010)Google Scholar
  7. 7.
    Ramanan, D., Forsyth, D., Zisserman, A.: Tracking People by Learning Their Appearance. TPAMI 29(1), 65–81 (2007)CrossRefGoogle Scholar
  8. 8.
    Andriluka, M., Roth, S., Schiele, B.: Pictorial Structures Revisited: People Detection and Articulated Pose Estimation. In: CVPR (2009)Google Scholar
  9. 9.
    Ferrari, V., Marin, M., Zisserman, A.: Progressive search space reduction for human pose estimation. In: CVPR (2008)Google Scholar
  10. 10.
    Andriluka, M., Roth, S., Schiele, B.: Monocular 3D Pose Estimation and Tracking by Detection. In: CVPR (2010)Google Scholar
  11. 11.
    Wang, Z., Chung, R.: Articulated Human Body: 3D Pose Estimation using a Single Camera. In: ICPR (2010)Google Scholar
  12. 12.
    Sminchisescu, C., Triggs, B.: Estimating articulated human motion with covariance scaled sampling. International Journal of Robotics Research 22(6), 371–391 (2003)CrossRefGoogle Scholar
  13. 13.
    Agarwal, A., Triggs, B.: 3D human pose from silhouettes by relevance vector regression. In: CVPR, pp. 882–888 (2004)Google Scholar
  14. 14.
    Elgammal, A., Lee, C.: Inferring 3d body pose from silhouettes using activity manifold learning. In: CVPR, pp. 681–688 (2004)Google Scholar
  15. 15.
    Dalal, N., Triggs, B.: Histogram of oriented gradients for human detection. In: CVPR (2005)Google Scholar
  16. 16.
    Bergtholdt, M., Kappes, J., Schmidt, S., Schnörr, C.: A Study of Parts-Based Object Class Detection Using Complete Graphs complete graph. IJCV 87(1-2) (2010)Google Scholar
  17. 17.
    Bernier, O., Cheung-Mon-Chan, P., Bouguet, A.: Fast nonparametric belief propagation for real-time stereo articulated body tracking. Computer Vision and Image Understanding 113, 29–47 (2009)CrossRefGoogle Scholar
  18. 18.
    Gupta, A., Mittal, A., Davis, L.S.: Constraint Integration for Efficient Multiview Pose Estimation with Self-Occlusions. TPAMI 30(3) (2008)Google Scholar
  19. 19.
    Cheung, K.M., Baker, S., Kanade, T.: Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture. In: CVPR, vol. 1, pp. 77–84 (2003)Google Scholar
  20. 20.
    Corazza, S., Mundermann, L., Gambaretto, E., Ferrigno, G., Andriacchi, T.: Markerless motion capture through visual hull, articulated ICP and subject specific model generation. IJCV 87(1-2) (2010)Google Scholar
  21. 21.
    Sigal, L., Black, M.J.: Guest Editorial: State of Art in Image- and Video-Based Human Pose and Motion Estimation. IJCV 87(1-3) (2010)Google Scholar
  22. 22.
    Sigal, L., Black, M.J.: Humaneva: Synchronized video and motion capture dataset for evaluation of articulated human motion. In: Techniacl Report CS-06-08, Brown University (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Zibin Wang
    • 1
  • Ronald Chung
    • 1
  1. 1.Department of Mechanical and Automation EngineeringThe Chinese University of Hong KongHong Kong

Personalised recommendations