Entangled Decision Forests and Their Application for Semantic Segmentation of CT Images

  • Albert Montillo
  • Jamie Shotton
  • John Winn
  • Juan Eugenio Iglesias
  • Dimitri Metaxas
  • Antonio Criminisi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6801)


This work addresses the challenging problem of simultaneously segmenting multiple anatomical structures in highly varied CT scans. We propose the entangled decision forest (EDF) as a new discriminative classifier which augments the state of the art decision forest, resulting in higher prediction accuracy and shortened decision time. Our main contribution is two-fold. First, we propose entangling the binary tests applied at each tree node in the forest, such that the test result can depend on the result of tests applied earlier in the same tree and at image points offset from the voxel to be classified. This is demonstrated to improve accuracy and capture long-range semantic context. Second, during training, we propose injecting randomness in a guided way, in which node feature types and parameters are randomly drawn from a learned (non-uniform) distribution. This further improves classification accuracy. We assess our probabilistic anatomy segmentation technique using a labeled database of CT image volumes of 250 different patients from various scan protocols and scanner vendors. In each volume, 12 anatomical structures have been manually segmented. The database comprises highly varied body shapes and sizes, a wide array of pathologies, scan resolutions, and diverse contrast agents. Quantitative comparisons with state of the art algorithms demonstrate both superior test accuracy and computational efficiency.


Entanglement auto-context decision forests CT segmentation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9(7), 1545–1588 (1997)CrossRefGoogle Scholar
  2. 2.
    Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  3. 3.
    Menze, B.H., Kelm, B.M., Masuch, R., Himmelreich, U., Petrich, W., Hamprecht, F.A.: A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC Bioinformatics 10, 213 (2009)CrossRefGoogle Scholar
  4. 4.
    Andres, B., Köthe, U., Helmstaedter, M., Denk, W., Hamprecht, F.A.: Segmentation of SBFSEM volume data of neural tissue by hierarchical classification. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 142–152. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: Proc. of CVPR, pp. 1–8 (2008)Google Scholar
  6. 6.
    Yi, Z., Criminisi, A., Shotton, J., Blake, A.: Discriminative, semantic segmentation of brain tissue in MR images. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5762, pp. 558–565. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Lempitsky, V.S., Verhoek, M., Noble, J.A., Blake, A.: Random forest classification for automatic delineation of myocardium in real-time 3D echocardiography. In: Functional Imaging and Modeling of the Heart, pp. 447–456 (2009)Google Scholar
  8. 8.
    Geremia, E., Menze, B.H., Clatz, O., Konukoglu, E., Criminisi, A., Ayache, N.: Spatial Decision Forests for MS Lesion Segmentation in Multi-Channel MR Images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6361, pp. 111–118. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning. Springer, Heidelberg (2009)CrossRefzbMATHGoogle Scholar
  10. 10.
    Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comp. Vision 81(1), 2–23 (2009)CrossRefGoogle Scholar
  11. 11.
    Tu, Z., Bai, X.: Auto-context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1744–1757 (2010)CrossRefGoogle Scholar
  12. 12.
    Tu, Z.: Probabilistic boosting tree: Learning discriminative models for classification, recognition, and clustering. In: Proc. of ICCV, pp. 1589–1596 (2005)Google Scholar
  13. 13.
    Zheng, Y., Georgescu, B., Comaniciu, D.: Marginal space learning for efficient detection of 2D/3D anatomical structures in medical images. In: Prince, J.L., Pham, D.L., Myers, K.J. (eds.) IPMI 2009. LNCS, vol. 5636, pp. 411–422. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Machine Learning 36(1), 3–42 (2006)CrossRefzbMATHGoogle Scholar
  15. 15.
    Viola, P., Jones, M.J.: Robust Real-Time Face Detection. Int. J. Comp. Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  16. 16.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes (VOC) Challenge. Int. J. Comp. Vision 88(2), 303–338 (2010)CrossRefGoogle Scholar
  17. 17.
    Rother, C., Kolmogorov, V., Blake, A.: GrabCut -Interactive Foreground Extraction using Iterated Graph Cuts. In: SIGGRAPH, vol. 23(3), pp. 309–314 (2004)Google Scholar
  18. 18.
    Criminisi, A., Sharp, T., Blake, A.: GeoS: Geodesic Image Segmentation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 99–112. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Konukoglu, E., Criminisi, A., Pathak, S., Robertson, D., White, S., Siddiqui, K.: Robust Linear Registration of CT Images using Random Regression Forests. In: SPIE Medical Imaging, vol. 7962, p. 79621X (2011)Google Scholar
  20. 20.
    Criminisi, A., Shotton, J., Bucciarelli, S.: Decision forests with long-range spatial context for organ localization in CT volumes. In: Proc. of MICCAI-PMMIA (2009)Google Scholar
  21. 21.
    Iglesias, J., Konukoglu, E., Montillo, A., Tu, Z., Criminisi, A.: Combining Generative & Discriminative Models for Semantic Segmentation of CT Scans via Active Learning. In: Proc. of Info. Proc. In: Medical Imaging (2011)Google Scholar
  22. 22.
    Criminisi, A., Shotton, J., Robertson, D., Konukoglu, E.: Regression Forests for Efficient Anatomy Detection and Localization in CT Scans, In: MICCAI-MCV Workshop (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Albert Montillo
    • 1
    • 2
  • Jamie Shotton
    • 2
  • John Winn
    • 2
  • Juan Eugenio Iglesias
    • 2
    • 3
  • Dimitri Metaxas
    • 4
  • Antonio Criminisi
    • 2
  1. 1.GE Global Research CenterNiskayunaUSA
  2. 2.Microsoft ResearchCambridgeUK
  3. 3.University of CaliforniaLos AngelesUSA
  4. 4.Rutgers UniversityPiscatawayUSA

Personalised recommendations