PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts

  • Matthieu Bray
  • Pushmeet Kohli
  • Philip H. S. Torr
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3952)


We present a novel algorithm for performing integrated segmentation and 3D pose estimation of a human body from multiple views. Unlike other related state of the art techniques which focus on either segmentation or pose estimation individually, our approach tackles these two tasks together. Normally, when optimizing for pose, it is traditional to use some fixed set of features, e.g. edges or chamfer maps. In contrast, our novel approach consists of optimizing a cost function based on a Markov Random Field (MRF). This has the advantage that we can use all the information in the image: edges, background and foreground appearances, as well as the prior information on the shape and pose of the subject and combine them in a Bayesian framework. Previously, optimizing such a cost function would have been computationally infeasible. However, our recent research in dynamic graph cuts allows this to be done much more efficiently than before. We demonstrate the efficacy of our approach on challenging motion sequences. Note that although we target the human pose inference problem in the paper, our method is completely generic and can be used to segment and infer the pose of any specified rigid, deformable or articulated object.


Image Segmentation Gaussian Mixture Model Segmentation Result Markov Random Field Appearance Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agarwal, A., Triggs, B.: 3d human pose from silhouettes by relevance vector regression. In: CVPR, vol. II, pp. 882–888 (2004)Google Scholar
  2. 2.
    Kehl, R., Bray, M., Van Gool, L.: Full body tracking from multiple views using stochastic sampling. In: CVPR, vol. II, pp. 129–136 (2005)Google Scholar
  3. 3.
    Shakhnarovich, G., Viola, P., Darrell, T.: Fast pose estimation with parameter-sensitive hashing. In: ICCV, pp. 750–757 (2003)Google Scholar
  4. 4.
    Gavrila, D., Davis, L.: 3D model-based tracking of humans in action: a multi-view approach. In: CVPR, pp. 73–80 (1996)Google Scholar
  5. 5.
    Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3D human figures using 2D image motion. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 702–718. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  6. 6.
    Sminchisescu, C., Triggs, B.: Covariance scaled sampling for monocular 3D body tracking. In: CVPR, pp. 447–454 (2001)Google Scholar
  7. 7.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: CVPR, pp. 246–252 (1999)Google Scholar
  8. 8.
    Kumar, M., Torr, P., Zisserman, A.: Obj cut. In: CVPR, vol. I, pp. 18–25 (2005)Google Scholar
  9. 9.
    Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: ICCV, pp. 105–112 (2001)Google Scholar
  10. 10.
    Huang, R., Pavlovic, V., Metaxas, D.: A graphical model framework for coupling mrfs and deformable models. In: CVPR, vol. II, pp. 739–746 (2004)Google Scholar
  11. 11.
    Freedman, D., Zhang, T.: Interactive graph cut based segmentation with shape priors. In: CVPR, vol. I, pp. 755–762 (2005)Google Scholar
  12. 12.
    Kohli, P., Torr, P.: Efficiently solving dynamic markov random fields using graph cuts. In: ICCV (2005)Google Scholar
  13. 13.
    Press, W., Flannery, B., Teukolsky, S., Vetterling, W.: Numerical recipes in C. Cambridge Uni. Press, Cambridge (1988)MATHGoogle Scholar
  14. 14.
    Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive image segmentation using an adaptive GMMRF model. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 428–441. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  15. 15.
    Felzenszwalb, P., Huttenlocher, D.: Distance transforms of sampled functions. Technical Report TR2004-1963, Cornell University (2004)Google Scholar
  16. 16.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 65–81. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  17. 17.
    Stenger, B., Thayananthan, A., Torr, P., Cipolla, R.: Filtering using a tree-based estimator. In: ICCV, pp. 1063–1070 (2003)Google Scholar
  18. 18.
    Bhatia, S., Sigal, L., Isard, M., Black, M.: 3d human limb detection using space carving and multi-view eigen models. In: ANM Workshop, vol. I, p. 17 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Matthieu Bray
    • 1
  • Pushmeet Kohli
    • 1
  • Philip H. S. Torr
    • 1
  1. 1.Dept. of ComputingOxford Brookes UniversityUK

Personalised recommendations