Flexible and Latent Structured Output Learning

Application to Histology
  • Gustavo CarneiroEmail author
  • Tingying Peng
  • Christine Bayer
  • Nassir Navab
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9352)


Malignant tumors that contain a high proportion of regions deprived of adequate oxygen supply (hypoxia) in areas supplied by a microvessel (i.e., a microcirculatory supply unit - MCSU) have been shown to present resistance to common cancer treatments. Given the importance of the estimation of this proportion for improving the clinical prognosis of such treatments, a manual annotation has been proposed, which uses two image modalities of the same histological specimen and produces the number and proportion of MCSUs classified as normoxia (normal oxygenation level), chronic hypoxia (limited diffusion), and acute hypoxia (transient disruptions in perfusion), but this manual annotation requires an expertise that is generally not available in clinical settings. Therefore, in this paper, we propose a new methodology that automates this annotation. The major challenge is that the training set comprises weakly labeled samples that only contains the number of MCSU types per sample, which means that we do not have the underlying structure of MCSU locations and classifications. Hence, we formulate this problem as a latent structured output learning that minimizes a high order loss function based on the number of MCSU types, where the underlying MCSU structure is flexible in terms of number of nodes and connections. Using a database of 89 pairs of weakly annotated images (from eight tumors), we show that our methodology produces highly correlated number and proportion of MCSU types compared to the manual annotations.


Weakly supervised training Latent structured output learning High order loss function 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bayer, C., Vaupel, P.: Acute versus chronic hypoxia in tumors. Strahlentherapie und Onkologie 188(7), 616–627 (2012)CrossRefGoogle Scholar
  2. 2.
    Maftei, C.A., et al.: Changes in the fraction of total hypoxia and hypoxia subtypes in human squamous cell carcinomas upon fractionated irradiation: evaluation using pattern recognition in microcirculatory supply units. Radiotherapy and Oncology 101(1), 209–216 (2011)CrossRefGoogle Scholar
  3. 3.
    Yu, C.N.J., Joachims, T.: Learning structural svms with latent variables. In: ICML, pp. 1169–1176 (2009)Google Scholar
  4. 4.
    Kumar, M.P.: Weakly Supervised Learning for Structured Output Prediction. PhD thesis, Ecole Normale Supérieure de Cachan (2014)Google Scholar
  5. 5.
    Lou, X., Hamprecht, F.: Structured learning from partial annotations (2012). arXiv preprint arXiv:1206.6421
  6. 6.
    Pletscher, P., Kohli, P.: Learning low-order models for enforcing high-order statistics. In: AISTATS, pp. 886–894 (2012)Google Scholar
  7. 7.
    Carneiro, G., et al.: Semantic-based indexing of fetal anatomies from 3-d ultrasound data using global/semi-local context and sequential sampling. In: CVPR (2008)Google Scholar
  8. 8.
    Patenaude, B., et al.: A bayesian model of shape and appearance for subcortical brain segmentation. Neuroimage 56(3), 907–922 (2011)CrossRefGoogle Scholar
  9. 9.
    Tu, Z., et al.: Brain anatomical structure segmentation by hybrid discriminative/generative models. TMI 27(4), 495–508 (2008)Google Scholar
  10. 10.
    Barbu, A., et al.: Automatic detection and segmentation of lymph nodes from ct data. TMI 31(2), 240–250 (2012)Google Scholar
  11. 11.
    Pauly, O., Glocker, B., Criminisi, A., Mateus, D., Möller, A.M., Nekolla, S., Navab, N.: Fast multiple organ detection and localization in whole-body MR dixon sequences. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011, Part III. LNCS, vol. 6893, pp. 239–247. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  12. 12.
    Zhou, S.K.: Discriminative anatomy detection: Classification vs regression. Pattern Recognition Letters 43, 25–38 (2014)CrossRefGoogle Scholar
  13. 13.
    Fiaschi, L., et al.: Tracking indistinguishable translucent objects over time using weakly supervised structured learning. In: CVPR (2014)Google Scholar
  14. 14.
    Mahapatra, D., et al.: Weakly supervised semantic segmentation of crohn’s disease tissues from abdominal mri. In: ISBI (2013)Google Scholar
  15. 15.
    Quellec, G., et al.: Weakly supervised classification of medical images. In: ISBI (2012)Google Scholar
  16. 16.
    Yuille, A.L., Rangarajan, A.: The concave-convex procedure. Neural Computation 15(4), 915–936 (2003)CrossRefGoogle Scholar
  17. 17.
    Joachims, T., et al.: Cutting-plane training of structural svms. Machine Learning 77(1), 27–59 (2009)CrossRefGoogle Scholar
  18. 18.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  19. 19.
    Zhu, J., et al.: Multi-class adaboost. Statistics and Its (2009)Google Scholar
  20. 20.
    Tsochantaridis, I., et al.: Support vector machine learning for interdependent and structured output spaces. In: ICML (2004)Google Scholar
  21. 21.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)CrossRefGoogle Scholar
  22. 22.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar
  23. 23.
    Grygorash, O., Zhou, Y., Jorgensen, Z.: Minimum spanning tree based clustering algorithms. In: ICTAI (2006)Google Scholar
  24. 24.
    Peng, T., Yigitsoy, M., Eslami, A., Bayer, C., Navab, N.: Deformable registration of multi-modal microscopic images using a pyramidal interactive registration-learning methodology. In: Ourselin, S., Modat, M. (eds.) WBIR 2014. LNCS, vol. 8545, pp. 144–153. Springer, Heidelberg (2014) Google Scholar
  25. 25.
    Altman, D.G., Bland, J.M.: Measurement in medicine: the analysis of method comparison studies. The statistician, 307–317 (1983)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Gustavo Carneiro
    • 1
    Email author
  • Tingying Peng
    • 2
  • Christine Bayer
    • 3
  • Nassir Navab
    • 2
  1. 1.ACVTUniversity of AdelaideAdelaideAustralia
  2. 2.CAMPTechnical University of MunichMunichGermany
  3. 3.Department of Radiation OncologyTechnical University of MunichMunichGermany

Personalised recommendations