2D and 3D Pose Recovery from a Single Uncalibrated Video

A View and Activity Independent Framework
  • Jean-Christophe Nebel
  • Paul Kuo
  • Dimitrios Makris
Part of the Studies in Computational Intelligence book series (SCI, volume 346)

Abstract

Human pose recovery from video sequences is an important task in computer vision since a set of reconstructed body postures provides essential information for the analysis of human behaviour and activity. Although systems have been proposed, they all rely on either controlled environments involving several and, generally, calibrated cameras or motion models learned for specific scenarios. Unfortunately, these constrains are not suitable for most real-life applications such as the study of athletes’ performances during competition, human computer interfaces for nomadic devices, video retrieval or the detection of antisocial behaviours from images captured from a closed-circuit television (CCTV) camera. Therefore, pose recovery remains a major challenge for the computer vision community.

Keywords

Foreground Pixel Human Body Model Nonlinear Dimensionality Reduction Mocap Data British Machine Vision Conf 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Armstrong, M., Zisserman, A., et al.: Euclidean Reconstructing from Image Triplets. In: Buxton, B.F., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1064, pp. 3–16. Springer, Heidelberg (1996)Google Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation 15(6), 1373–1396 (2003)CrossRefMATHGoogle Scholar
  3. 3.
    Bhatia, S., Sigal, L., et al.: 3D Human Limb Detection using Space Carving and Multi-View Eigen Models. In: Proc. Conf. on Computer Vision and Pattern Recognition, pp. 17–24 (2004)Google Scholar
  4. 4.
    Brubaker, M., Fleet, D., et al.: Physics-Based Human Pose Tracking. In: Proc. NIPS Workshop on Evaluation of Articulated Human Motion and Pose Estimation (2006)Google Scholar
  5. 5.
    Caillette, F., Howard, T.: Real-Time Markerless Human Body Tracking with Multi-View 3-D Voxel Reconstruction. In: Proc. British Machine Vision Conf. (2004)Google Scholar
  6. 6.
    Cheng, S., Trivedi, M.: Articulated Body Pose Estimation from Voxel Reconstructions using Kinematically Constrained Gaussian Mixture Models: Algorithm and Evaluation. In: Proc. Workshop on Evaluation of Articulated Human Motion and Pose Estimation (2007)Google Scholar
  7. 7.
    Carnegie Mellon University, Motion Capture Library, http://mocap.cs.cmu.edu (last accessed January 2010)
  8. 8.
    Da Vinci, L.: Description of Vitruvian Man (1492)Google Scholar
  9. 9.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. Int. J. Computer Vision 61(1), 55–79 (2005)CrossRefGoogle Scholar
  10. 10.
    Fritsch, J., Kleinehagenbrock, M., et al.: Audiovisual person tracking with a mobile robot. In: Proc. Int. Conf. on Intelligent Autonomous Systems, pp. 898–906 (2004)Google Scholar
  11. 11.
    Fryer, C.: Biomechanics of the lower extremity. Instruct Course Lect. 20, 124–130 (1971)Google Scholar
  12. 12.
    Gavrila, D.: The visual analysis of human movement: A survey. Computer Vision and Image Understanding 73(1), 82–98 (1999)CrossRefMATHGoogle Scholar
  13. 13.
    Hartigan, J., Wong, M.: A K-means clustering algorithm. Applied Statistics 28(1), 100–108 (1979)CrossRefMATHGoogle Scholar
  14. 14.
    Hou, S., Galata, A., et al.: Real-time Body Tracking Using a Gaussian Process Latent Variable Model. In: Proc. Int. Conf. on Computer Vision (2007)Google Scholar
  15. 15.
    Howe, N.: Silhouette lookup for monocular 3D pose tracking. Image and Vision Computing 25(3), 331–341 (2007)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Howe, N.: Recognition-Based Motion Capture and the HumanEva II Test Data. In: Proc. Workshop on Evaluation of Articulated Human Motion and Pose Estimation (2007)Google Scholar
  17. 17.
    Hua, G., Yang, M., et al.: Learning to estimate human poses with data driven belief propagation. In: Proc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 747–754 (2005)Google Scholar
  18. 18.
    Husz, Z., Wallace, A., Green, P.: Evaluation of a Hierarchical Partitioned Particle Filter with Action Primitives. In: Proc. Workshop on Evaluation of Articulated Human Motion and Pose Estimation (2007)Google Scholar
  19. 19.
    Izo, T., Grimson, W.: Simultaneous pose recovery and camera registration from multiple views of a walking person. J. Image and Vision Computing 25(3), 342–351 (2007)CrossRefGoogle Scholar
  20. 20.
    Krahnstoever, N., Mendonca, P.: Bayesian autocalibration for surveillance. In: Proc. Int. Conf. on Computer Vision (2005)Google Scholar
  21. 21.
    Kuo, P., Nebel, J.-C., et al.: Camera Auto-Calibration from Articulated Motion. In: Proc. Advanced Video and Signal Based Surveillance, pp. 135–140 (2007)Google Scholar
  22. 22.
    Kuo, P., Makris, D., et al.: Integration of Local Image Cues for Probabilistic 2D Pose Recovery. In: Proc. Int. Symp. on Visual Computing (2008)Google Scholar
  23. 23.
    Kuo, P., Thibault, A.: Exploiting Human Bipedal Motion Constraints for 3D Pose Recovery from a Single Uncalibrated Camera. In: Proc. Int. Conf. on Computer Vision theory and Applications (2009)Google Scholar
  24. 24.
    Lawrence, N., Carl Henrik, E., et al.: Gaussian Process Latent Variable Models for Human Pose Estimation. In: Popescu-Belis, A., Renals, S., Bourlard, H. (eds.) MLMI 2007. LNCS, vol. 4892, pp. 132–143. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Lee, C.-S., Elgammal, A.: Body Pose Tracking From Uncalibrated Camera Using Supervised Manifold Learning. In: Proc. NIPS Workshop on Evaluation of Articulated Human Motion and Pose Estimation (2006)Google Scholar
  26. 26.
    Lee, C.-S., Elgammal, A.: Nonlinear manifold learning for dynamic shape and dynamic appearance. Computer Vision and Image Understanding 106(1), 31–46 (2007)CrossRefGoogle Scholar
  27. 27.
    Lee, M., Cohen, I.: A Model-Based Approach for Estimating Human 3D Poses in Static Images. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(6), 905–916 (2006)CrossRefGoogle Scholar
  28. 28.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. Imaging Understanding Workshop, pp. 121–130 (1981)Google Scholar
  29. 29.
    Luong, Q., Faugeras, O.: Self-Calibration of a Moving Camera from Point correspondences and Fundamental Matrices. Int. J. Computer Vision 22(3), 261–289 (1997)CrossRefGoogle Scholar
  30. 30.
    Lv, F., Zhao, T., et al.: Self-Calibration of a Camera from Video of a Walking Human. In: Proc. Int. Conf. on Pattern Recognition (2002)Google Scholar
  31. 31.
    Mariano, V., Min, J., et al.: Performance Evaluation of Object Detection Algorithms. In: Proc. Int. Conf. on Pattern Recognition, pp. 965–969 (2002)Google Scholar
  32. 32.
    Martinez-del-Rincon, J., Nebel, J.-C., et al.: Tracking Human Body Parts Using Particle Filters Constrained by Human Biomechanics. In: Proc. British Machine Vision Conf.Google Scholar
  33. 33.
    Mckenna, S., Raja, Y., et al.: Tracking colour objects using adaptive mixture models. Image and Vision Computing 17, 225–231 (1999)CrossRefGoogle Scholar
  34. 34.
    Menier, C., Boyer, E., et al.: 3D Skeleton-Based Body Pose Recovery. In: Proc. Int. Symp. on 3D Data Processing, Visualization and Transmission (2004)Google Scholar
  35. 35.
    OpenCV 2.0 C Reference, http://opencv.willowgarage.com/documentation/index.html (last accessed January 2010)
  36. 36.
    Mori, G., Malik, J.: Estimating Human Body Configurations Using Shape Context Matching. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 666–680. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  37. 37.
    Mori, G., Ren, X., et al.: Recovering human body configurations: Combing segmentation and recognition. In: Proc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 326–333 (2004)Google Scholar
  38. 38.
    Mori, G., Malki, J.: Recovering 3D Human Body Configurations Using Shape Contexts. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(7), 1052–1062 (2006)CrossRefGoogle Scholar
  39. 39.
    MuHAVi: Multicamera Human Action Video Data, http://dipersec.king.ac.uk/MuHAVi-MAS (last accessed January 2010)
  40. 40.
    Okada, R., Soatto, S.: Relevant Feature Selection for Human Pose Estimation and Localization in Cluttered Images. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 434–445. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  41. 41.
    Pollefeys, M., Van Gool, L.: Stratified Self-Calibration with the Modulus Constraint. IEEE Trans. on Pattern Analysis and Machine Intelligence 21(8), 707–724 (1999)CrossRefGoogle Scholar
  42. 42.
    Poppe, R., Poel, M.: Comparison of silhouette shape descriptors for example-based human pose recovery. In: Proc. Int. Conf. on Automatic Face and Gesture Recognition, pp. 541–546 (2006)Google Scholar
  43. 43.
    Poppe, R.: Evaluating Example-based Pose Estimation: Experiments on the HumanEva sets. In: Proc. Workshop on Evaluation of Articulated Human Motion and Pose Estimation (2007)Google Scholar
  44. 44.
    Poppe, R.: Vision-based human motion analysis: An overview. Computer Vision and Image Understanding 108(1-2), 4–18 (2007)CrossRefGoogle Scholar
  45. 45.
    Ramanan, D., Forsyth, D.: Finding and traking people from the bottom up. In: Proc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 467–474 (2003)Google Scholar
  46. 46.
    Ramanan, D., Forsyth, D., et al.: Strike a pose: Tracking people by finding stylized poses. In: Proc. Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 271–278 (2005)Google Scholar
  47. 47.
    Ramanan, D.: Learning to parse images of articulated bodies. In: Proc. Advanced in Neural Information Processing Systems, pp. 1129–1136 (2007)Google Scholar
  48. 48.
    Rasmussen, C., Williams, G.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)MATHGoogle Scholar
  49. 49.
    Remondino, F., Roditakis, A.: 3D Reconstruction of Human Skeleton from Single Images or Monocular Video Sequences. In: Noltemeier, H. (ed.) WG 1980. LNCS, vol. 100, pp. 100–107. Springer, Heidelberg (1981)Google Scholar
  50. 50.
    Ren, X., Berg, A., et al.: Recovering human body configurations using pairwise constraints. In: Proc. Int. Conf. on Computer Vision, pp. 824–831 (2005)Google Scholar
  51. 51.
    Renno, J., Remagnino, P., et al.: Learning Surveillance Tracking Models from the Self-Calibrated Ground Plane. Acta Automatica Sinica 29(3), 381–392 (2003)Google Scholar
  52. 52.
    Saul, L., Roweis, S.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  53. 53.
    Sidenbladh, H.: Image sequences provided, http://www.csc.kth.se/~hedvig/data.html (last accessed January 2010)
  54. 54.
    Sigal, L., Black, M.: HumanEva: Synchronized Video and Motion Capture Dataset for Evaluation of Articulated Human Motion, Technical Report CS-06-08, Brown University (2006)Google Scholar
  55. 55.
    Sigal, L., Black, M.J.: Predicting 3D People from 2D Pictures. In: Perales, F.J., Fisher, R.B. (eds.) AMDO 2006. LNCS, vol. 4069, pp. 185–195. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  56. 56.
    Sigal, L., Black, M.: Measure locally, reason globally: Occlusion-sensitive articulated pose estimation. In: Proc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 2041–2048 (2006)Google Scholar
  57. 57.
    Spencer, N., Carter, J.: Towards pose invariant gait reconstruction. In: Proc. Int. Conf. on Image Processing, vol. 2, pp. 261–264 (2005)Google Scholar
  58. 58.
    Srinivasan, P., Shi, J.: Bottom-up recognition and parsing of the human body. In: Proc. Conf. on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  59. 59.
    Taylor, C.: Reconstruction of Articulated Objects from Point Correspondences in a Single Image. In: Proc. Conf. on Computer Vision and Pattern Recognition, pp. 677–684 (2000)Google Scholar
  60. 60.
    Tenenbaum, J.: Global Geometric Framework for Nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)CrossRefGoogle Scholar
  61. 61.
    Tsai, R.: A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV cameras and lensesp. IEEE J. Robotics and Automation RA 3, 323–343 (1987)CrossRefGoogle Scholar
  62. 62.
    Urtasun, R., Darrell, T.: Sparse Probabilistic Regression for Activity-independent Human Pose Inference. In: Proc. Conf. on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  63. 63.
    Motion Capture Systems from Vicon, www.vicon.com (last accessed January 2010)
  64. 64.
    Vondrak, M., Sigal, L., Jenkins, O.: Physical Simulation for Probabilistic Motion Tracking. In: Proc. Conf. on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  65. 65.
    Wang, Y., Mori, G.: Boosted multiple deformable trees for parsing human poses. In: Proc. Workshop on Human Motion Understanding, Modeling, Capture and Animation, pp. 16–27 (2007)Google Scholar
  66. 66.
    Yang, H., Lee, S.: Reconstructing 3D human body pose from stereo image sequences using hierarchical human body model learning. In: Proc. Int. Conf. on Pattern Recognition, vol. 3, pp. 1004–1007 (2006)Google Scholar
  67. 67.
    Zhao, T., Nevatia, R.: Bayesian human segmentation in crowded situations. In: Proc. Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 459–466 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jean-Christophe Nebel
    • 1
  • Paul Kuo
    • 1
  • Dimitrios Makris
    • 1
  1. 1.Digital Imaging Research CentreKingston UniversityUK

Personalised recommendations