Segmenting Highly Articulated Video Objects with Weak-Prior Random Forests

  • Hwann-Tzong Chen
  • Tyng-Luh Liu
  • Chiou-Shann Fuh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3954)


We address the problem of segmenting highly articulated video objects in a wide variety of poses. The main idea of our approach is to model the prior information of object appearance via random forests. To automatically extract an object from a video sequence, we first build a random forest based on image patches sampled from the initial template. Owing to the nature of using a randomized technique and simple features, the modeled prior information is considered weak, but on the other hand appropriate for our application. Furthermore, the random forest can be dynamically updated to generate prior probabilities about the configurations of the object in subsequent image frames. The algorithm then combines the prior probabilities with low-level region information to produce a sequence of figure-ground segmentations. Overall, the proposed segmentation technique is useful and flexible in that one can easily integrate different cues and efficiently select discriminating features to model object appearance and handle various articulations.


Random Forest Prior Probability Image Patch Prior Model Video Object 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9, 1545–1588 (1997)CrossRefGoogle Scholar
  2. 2.
    Blake, A., Isard, M.: Active Contours: The Application of Techniques from Graphics, Vision, Control Theory and Statistics to Visual Tracking of Shapes in Motion. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  3. 3.
    Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive image segmentation using an adaptive GMMRF model. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 428–441. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Borenstein, E., Ullman, S.: Class-specific, top-down segmentation. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 109–122. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Markov random fields with efficient approximations. In: CVPR, pp. 648–655 (1998)Google Scholar
  6. 6.
    Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: ICCV, vol. 1, pp. 105–112 (2001)Google Scholar
  7. 7.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  8. 8.
    Chan, T., Zhu, W.: Level set based shape prior segmentation. In: CVPR, vol. 2, pp. 1164–1170 (2005)Google Scholar
  9. 9.
    Cremers, D., Tischhauser, F., Weickert, J., Schnorr, C.: Diffusion snakes: Introducing statistical shape knowledge into the Mumford-Shah functional. IJCV 50(3), 295–313 (2002)CrossRefzbMATHGoogle Scholar
  10. 10.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, Chichester (2001)zbMATHGoogle Scholar
  11. 11.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Pictorial structures for object recognition. IJCV 61(1), 55–79 (2005)CrossRefGoogle Scholar
  12. 12.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: CVPR, vol. 2, pp. 264–271 (2003)Google Scholar
  13. 13.
    Fowlkes, C.C., Martin, D.R., Malik, J.: Learning affinity functions for image segmentation: Combining patch-based and gradient-based approaches. In: CVPR, vol. 2, pp. 54–61 (2003)Google Scholar
  14. 14.
    Gavrila, D.M.: Pedestrian detection from a moving vehicle. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 37–49. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  15. 15.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. PAMI 6(6), 721–741 (1984)CrossRefzbMATHGoogle Scholar
  16. 16.
    The Berkeley Segmentation Dataset and Benchmark,
  17. 17.
    The Berkeley Segmentation Engine,
  18. 18.
    Kumar, M.P., Torr, P.H.S., Zisserman, A.: OBJ CUT. In: CVPR, vol. 1, pp. 18–25 (2005)Google Scholar
  19. 19.
    Lepetit, V., Lagger, P., Fua, P.: Randomized trees for real-time keypoint recognition. In: CVPR, vol. 2, pp. 775–781 (2005)Google Scholar
  20. 20.
    Malik, J., Belongie, S., Leung, T., Shi, J.: Contour and texture analysis for image segmentation. IJCV 43(1), 7–27 (2001)CrossRefzbMATHGoogle Scholar
  21. 21.
    Martin, D.R., Fowlkes, C.C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001)Google Scholar
  22. 22.
    McInerney, T., Terzopoulos, D.: Deformable models in medical image analysis: A survey. MIA 1(2), 91–108 (1996)Google Scholar
  23. 23.
    Mori, G., Ren, X., Efros, A.A., Malik, J.: Recovering human body configurations: Combining segmentation and recognition. In: CVPR, vol. 2, pp. 326–333 (2004)Google Scholar
  24. 24.
    Mumford, D., Shah, J.: Boundary detection by minimizing functionals. In: CVPR, pp. 22–26 (1985)Google Scholar
  25. 25.
    Paragios, N., Deriche, R.: Coupled geodesic active regions for image segmentation: A level set approach. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 224–240. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  26. 26.
    Ren, X., Fowlkes, C., Malik, J.: Cue integration for figure/ground labeling. In: NIPS 18 (2005)Google Scholar
  27. 27.
    Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)CrossRefGoogle Scholar
  28. 28.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. PAMI 22(8), 888–905 (2000)CrossRefGoogle Scholar
  29. 29.
    Toyama, K., Blake, A.: Probabilistic tracking in a metric space. In: ICCV, vol. 2, pp. 50–57 (2001)Google Scholar
  30. 30.
    Tu, Z., Chen, X., Yuille, A.L., Zhu, S.C.: Image parsing: Unifying segmentation, detection, and recognition. In: ICCV, pp. 18–25 (2003)Google Scholar
  31. 31.
    Viola, P., Jones, M.J.: Rapid object detection using a boosted cascade of simple features. In: CVPR, vol. 1, pp. 511–518 (2001)Google Scholar
  32. 32.
    Wu, Y., Zhang, A.: Adaptive pattern discovery for interactive multimedia retrieval. In: CVPR, vol. 2, pp. 649–655 (2003)Google Scholar
  33. 33.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Understanding belief propagation and its generalizations. In: Exploring Artificial Intelligence in the New Millennium, pp. 239–269 (2003)Google Scholar
  34. 34.
    Yu, S.X., Shi, J.: Object-specific figure-ground segregation. In: CVPR, vol. 2, pp. 39–45 (2003)Google Scholar
  35. 35.
    Yuille, A.L., Cohen, D.S., Hallinan, P.W.: Feature extraction from faces using deformable templates. IJCV 8(2), 99–111 (1992)CrossRefGoogle Scholar
  36. 36.
    Zhu, S.C., Yuille, A.: Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. PAMI 18(9), 884–900 (1996)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hwann-Tzong Chen
    • 1
  • Tyng-Luh Liu
    • 1
  • Chiou-Shann Fuh
    • 2
  1. 1.Institute of Information ScienceAcademia SinicaTaipeiTaiwan
  2. 2.Department of CSIENational Taiwan UniversityTaipeiTaiwan

Personalised recommendations