Joint Supervoxel Classification Forest for Weakly-Supervised Organ Segmentation

  • Fahdi Kanavati
  • Kazunari Misawa
  • Michitaka Fujiwara
  • Kensaku Mori
  • Daniel Rueckert
  • Ben Glocker
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10541)


This article presents an efficient method for weakly-supervised organ segmentation. It consists in over-segmenting the images into object-like supervoxels. A single joint forest classifier is then trained on all the images, where (a) the supervoxel indices are used as labels for the voxels, (b) a joint node optimisation is done using training samples from all the images, and (c) in each leaf node, a distinct posterior distribution is stored per image. The result is a forest with a shared structure that efficiently encodes all the images in the dataset. The forest can be applied once on a given source image to obtain supervoxel label predictions for its voxels from all the other target images in the dataset by simply looking up the target’s distribution in the leaf nodes. The output is then regularised using majority voting within the boundaries of the source’s supervoxels. This yields sparse correspondences on an over-segmentation-based level in an unsupervised, efficient, and robust manner. Weak annotations can then be propagated to other images, extending the labelled set and allowing an organ label classification forest to be trained. We demonstrate the effectiveness of our approach on a dataset of 150 abdominal CT images where, starting from a small set of 10 images with scribbles, we perform weakly-supervised image segmentation of the kidneys, liver and spleen. Promising results are obtained.

Supplementary material

454062_1_En_10_MOESM1_ESM.pdf (1.9 mb)
Supplementary material 1 (pdf 1896 KB)


  1. 1.
    Boykov, Y.Y., Jolly, M.M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. In: ICCV 2001, vol. 1, pp. 105–112 (2001)Google Scholar
  2. 2.
    Chen, X., Shrivastava, A., Gupta, A.: Enriching visual knowledge bases via object discovery and segmentation. In: Proceedings of the IEEE conference on CVPR, pp. 2027–2034 (2014)Google Scholar
  3. 3.
    Conze, P.H., Tilquin, F., Noblet, V., Rousseau, F., Heitz, F., Pessaux, P.: Hierarchical multi-scale supervoxel matching using random forests for automatic semi-dense abdominal image registration. In: IEEE ISBI (2017)Google Scholar
  4. 4.
    Criminisi, A., Shotton, J.: Decision Forests for Computer Vision and Medical Image Analysis. Springer, London (2013). doi: 10.1007/978-1-4471-4929-3 CrossRefGoogle Scholar
  5. 5.
    Criminisi, A., Shotton, J., Robertson, D., Konukoglu, E.: Regression forests for efficient anatomy detection and localization in CT studies. In: Menze, B., Langs, G., Tu, Z., Criminisi, A. (eds.) MCV 2010. LNCS, vol. 6533, pp. 106–117. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-18421-5_11 CrossRefGoogle Scholar
  6. 6.
    Deselaers, T., Alexe, B., Ferrari, V.: Weakly supervised localization and learning with generic knowledge. IJCV 100(3), 275–293 (2012)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Dutt Jain, S., Grauman, K.: Active image segmentation propagation. In: Proceedings of the IEEE Conference on CVPR, pp. 2864–2873 (2016)Google Scholar
  8. 8.
    Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. IJCV 59, 167–181 (2004)CrossRefGoogle Scholar
  9. 9.
    Glocker, B., Zikic, D., Haynor, D.R.: Robust registration of longitudinal spine CT. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 251–258. Springer, Cham (2014). doi: 10.1007/978-3-319-10404-1_32 Google Scholar
  10. 10.
    Grauman, K., Darrell, T.: Unsupervised learning of categories from sets of partially matching image features. In: 2006 IEEE Computer Society Conference on CVPR, vol. 1, pp. 19–25. IEEE (2006)Google Scholar
  11. 11.
    Kanavati, F., Tong, T., Misawa, K., Fujiwara, M., Mori, K., Rueckert, D., Glocker, B.: Supervoxel classification forests for estimating pairwise image correspondences. Pattern Recogn. 63, 561–569 (2017)CrossRefGoogle Scholar
  12. 12.
    Montillo, A., Shotton, J., Winn, J., Iglesias, J.E., Metaxas, D., Criminisi, A.: Entangled decision forests and their application for semantic segmentation of CT images. In: Székely, G., Hahn, H.K. (eds.) IPMI 2011. LNCS, vol. 6801, pp. 184–196. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-22092-0_16 CrossRefGoogle Scholar
  13. 13.
    Rubinstein, M., Liu, C., Freeman, W.T.: Joint inference in weakly-annotated image datasets via dense correspondence. IJCV 119(1), 23–45 (2016)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Russell, B.C., Efros, A.A., Sivic, J., Freeman, W.T., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: Proceedings of the IEEE Computer Society Conference on CVPR, vol. 2, pp. 1605–1612 (2006)Google Scholar
  15. 15.
    Tong, T., Wolz, R., Wang, Z., Gao, Q., Misawa, K., Fujiwara, M., Mori, K., Hajnal, J.V., Rueckert, D.: Discriminative dictionary learning for abdominal multi-organ segmentation. Med. Image Anal. 23(1), 92–104 (2015)CrossRefGoogle Scholar
  16. 16.
    Uijlings, J.R., Van De Sande, K.E., Gevers, T., Smeulders, A.W.: Selective search for object recognition. IJCV 104(2), 154–171 (2013)CrossRefGoogle Scholar
  17. 17.
    Vicente, S., Rother, C., Kolmogorov, V.: Object cosegmentation. In: 2011 IEEE Conference on CVPR, pp. 2217–2224. IEEE (2011)Google Scholar
  18. 18.
    Xu, J., Schwing, A.G., Urtasun, R.: Learning to segment under various forms of weak supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3781–3790 (2015)Google Scholar
  19. 19.
    Zhou, S.: Introduction to medical image recognition, segmentation, and parsing, Chap. 1. In: Zhou, S.K. (ed.) Medical Image Recognition, Segmentation and Parsing, pp. 1–21. Academic Press, New York (2016)Google Scholar
  20. 20.
    Zikic, D., Glocker, B., Criminisi, A.: Encoding atlases by randomized classification forests for efficient multi-atlas label propagation. Med. Image Anal. 18, 1262–1273 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Fahdi Kanavati
    • 1
  • Kazunari Misawa
    • 2
  • Michitaka Fujiwara
    • 3
  • Kensaku Mori
    • 4
  • Daniel Rueckert
    • 1
  • Ben Glocker
    • 1
  1. 1.Biomedical Image Analysis Group, Department of ComputingImperial College LondonLondonUK
  2. 2.Aichi Cancer CenterNagoyaJapan
  3. 3.Nagoya University HospitalNagoyaJapan
  4. 4.Information and CommunicationsNagoya UniversityNagoyaJapan

Personalised recommendations