Joint Classification-Regression Forests for Spatially Structured Multi-object Segmentation

  • Ben Glocker
  • Olivier Pauly
  • Ender Konukoglu
  • Antonio Criminisi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7575)


In many segmentation scenarios, labeled images contain rich structural information about spatial arrangement and shapes of the objects. Integrating this rich information into supervised learning techniques is promising as it generates models which go beyond learning class association, only. This paper proposes a new supervised forest model for joint classification-regression which exploits both class and structural information. Training our model is achieved by optimizing a joint objective function of pixel classification and shape regression. Shapes are represented implicitly via signed distance maps obtained directly from ground truth label maps. Thus, we can associate each image point not only with its class label, but also with its distances to object boundaries, and this at no additional cost regarding annotations. The regression component acts as spatial regularization learned from data and yields a predictor with both class and spatial consistency. In the challenging context of simultaneous multi-organ segmentation, we demonstrate the potential of our approach through experimental validation on a large dataset of 80 three-dimensional CT scans.


Leaf Node Class Label Tree Depth Split Node Spatial Regularization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)zbMATHCrossRefGoogle Scholar
  2. 2.
    Criminisi, A., Shotton, J., Konukoglu, E.: Decision Forests: A Unified Framework. Foundations and Trends in Computer Graphics and Vision 7(2-3) (2011)Google Scholar
  3. 3.
    Ho, T.K.: Random Decision Forests. In: ICDAR, vol. 1, pp. 278–282 (1995)Google Scholar
  4. 4.
    Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. PAMI 20(8), 832–844 (1998)CrossRefGoogle Scholar
  5. 5.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-Time Human Pose Recognition in Parts from a Single Depth Image. In: CVPR, pp. 1297–1304 (2011)Google Scholar
  6. 6.
    Amit, Y., Geman, D.: Shape Quantization and Recognition with Randomized Trees. Neural Computation 9, 1545–1588 (1997)CrossRefGoogle Scholar
  7. 7.
    Criminisi, A., Shotton, J., Robertson, D., Konukoglu, E.: Regression Forests for Efficient Anatomy Detection and Localization in CT Studies. In: Menze, B., Langs, G., Tu, Z., Criminisi, A. (eds.) MICCAI 2010. LNCS, vol. 6533, pp. 106–117. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Bosch, A., Zisserman, A., Munoz, X.: Image Classification Using Random Forests and Ferns. In: ICCV (2007)Google Scholar
  9. 9.
    Maree, R., Geurts, P., Piater, J., Wehenkel, L.: Random Subwindows for Robust Image Classification. In: CVPR (2005)Google Scholar
  10. 10.
    Caruana, R., Karampatziakis, N., Yessenalina, A.: An Empirical Evaluation of Supervised Learning in High Dimensions. In: ICML, pp. 96–103 (2008)Google Scholar
  11. 11.
    Yin, P., Criminisi, A., Essa, I., Winn, J.: Tree-based Classifiers for Bilayer Video Segmentation. In: CVPR, pp. 1–8 (2007)Google Scholar
  12. 12.
    Payet, N., Todorovic, S.: (RF)2 Random Forest Random Field. In: NIPS (2010)Google Scholar
  13. 13.
    Kontschieder, P., Rota Buló, S., Bischof, H., Pelillo, M.: Structured class-labels in random forests for semantic image labelling. In: ICCV (2011)Google Scholar
  14. 14.
    Montillo, A., Shotton, J., Winn, J., Iglesias, J.E., Metaxas, D., Criminisi, A.: Entangled Decision Forests and Their Application for Semantic Segmentation of CT Images. In: Székely, G., Hahn, H.K. (eds.) IPMI 2011. LNCS, vol. 6801, pp. 184–196. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Nowozin, S., Rother, C., Bagon, S., Sharp, T., Yao, B., Kohli, P.: Decision Tree Fields. In: ICCV (2011)Google Scholar
  16. 16.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Pictorial Structures for Object Recognition. IJCV 61(1), 55–79 (2005)CrossRefGoogle Scholar
  17. 17.
    Girshick, R., Shotton, J., Kohli, P., Criminisi, A., Fitzgibbon, A.: Efficient Regression of General-Activity Human Poses from Depth Images. In: ICCV, pp. 415–422 (2011)Google Scholar
  18. 18.
    Gall, J., Yao, A., Razavi, N., Van Gool, L., Lempitsky, V.: Hough Forests for Object Detection, Tracking, and Action Recognition. PAMI 33(11), 2188–2202 (2011)CrossRefGoogle Scholar
  19. 19.
    Cootes, T., Edwards, G., Taylor, C.: Active Appearance Models. PAMI 23(6), 681–685 (2001)CrossRefGoogle Scholar
  20. 20.
    Boykov, Y., Funka-Lea, G.: Graph Cuts and Efficient N-D Image Segmentation. IJCV 70(2), 109–131 (2006)CrossRefGoogle Scholar
  21. 21.
    Viola, P., Jones, M.J.: Robust Real-Time Face Detection. IJCV 57(2), 137–154 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ben Glocker
    • 1
  • Olivier Pauly
    • 2
    • 3
  • Ender Konukoglu
    • 1
  • Antonio Criminisi
    • 1
  1. 1.Microsoft ResearchCambridgeUK
  2. 2.Institute of Biomathematics and BiometryHelmholtz Zentrum MünchenGermany
  3. 3.Computer Aided Medical ProceduresTechnische Universität MünchenGermany

Personalised recommendations