Detection and Localization of Landmarks in the Lower Extremities Using an Automatically Learned Conditional Random Field

  • Alexander Oliver MaderEmail author
  • Cristian Lorenz
  • Martin Bergtholdt
  • Jens  von  Berg
  • Hauke Schramm
  • Jan Modersitzki
  • Carsten Meyer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10551)


The detection and localization of single or multiple landmarks is a crucial task in medical imaging. It is often required as initialization for other tasks like segmentation or registration. A common approach to localize multiple landmarks is to exploit their spatial correlations, e.g., by using a conditional random field (CRF) to incorporate geometric information between landmark pairs. This CRF is usually applied to resolve ambiguities of a localizer, e.g., a random forest or a deep neural network. In this paper, we apply a random forest/CRF combination to the task of jointly detecting and localizing 6 landmarks in the lower extremities, taken from a dataset of 660 X-ray images. The dataset is challenging since a significant number of images does not show all the landmarks. Furthermore, 11.3% of the target landmarks are altered by prostheses or pathologies.

To account for this, we introduce a “missing” label for each landmark (represented by a node in the CRF). Moreover, instead of manually specifying the CRF model by selecting suitable potential functions and the graph topology, we suggest to automatically optimize both in a learning framework. Specifically, we define a pool of potential functions and learn their CRF weights (relative contributions), in addition to the potential values in case of missing landmarks. Potentials with a low weight are removed, thus optimizing the graph topology. Detailed evaluations on our database show the feasibility of our approach. Our algorithm removed on average 23 of the initial 51 CRF potentials, and correctly detected and localized (within 10 mm tolerance) on average 92.8% of the landmarks, with individual rates ranging from 90.0% to 97.4%.



The authors thank the Diagnosezentrum Urania, Vienna and the Dartmouth Hitchcock Medical Center, Lebanon for providing the radiographs that served as training and test sets; Gooßen [8] for the annotations. This work has been financially supported by the Federal Ministry of Education and Research under the grant 03FH013IX5. The liability for the content of this work lies with the authors.


  1. 1.
    Bergtholdt, M., Kappes, J.H., Schnörr, C.: Learning of graphical models and efficient inference for object class recognition. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 273–283. Springer, Heidelberg (2006). doi: 10.1007/11861898_28 CrossRefGoogle Scholar
  2. 2.
    Bergtholdt, M., et al.: A study of parts-based object class detection using complete graphs. IJCV 87(1), 93–117 (2010)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15561-1_56 CrossRefGoogle Scholar
  4. 4.
    Criminisi, A., et al.: Regression forests for efficient anatomy detection and localization in computed tomography scans. Med. Image Anal. 17(8), 1293–1303 (2013)CrossRefGoogle Scholar
  5. 5.
    Donner, R., et al.: Sparse MRF appearance models for fast anatomical structure localisation. In: BMVC (2007)Google Scholar
  6. 6.
    Donner, R., et al.: Global localization of 3d anatomical structures by pre-filtered hough forests and discrete optimization. Med. Image Anal. 17(8), 1304–1314 (2013)CrossRefGoogle Scholar
  7. 7.
    Glocker, B., Zikic, D., Konukoglu, E., Haynor, D.R., Criminisi, A.: Vertebrae localization in pathological spine CT via dense classification from sparse annotations. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 262–270. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40763-5_33 CrossRefGoogle Scholar
  8. 8.
    Gooßen, A.: Computational Imaging in Orthopaedic Radiography. BoD (2012)Google Scholar
  9. 9.
    Hahmann, F., et al.: Model interpolation for eye localization using the discriminative generalized hough transform. In: BIOSIG (2012)Google Scholar
  10. 10.
    Ishikawa, H.: Higher-order clique reduction in binary graph cut. In: CVPR, pp. 2993–3000. IEEE (2009)Google Scholar
  11. 11.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2014)Google Scholar
  12. 12.
    Komodakis, N., Xiang, B., Paragios, N.: A framework for efficient structured max-margin learning of high-order mrf models. IEEE TPAMI 37(7), 1425–1441 (2015)CrossRefGoogle Scholar
  13. 13.
    LeCun, Y., Chopra, S., Hadsell, R.: A tutorial on energy-based learning. In: Predicting Structured Data (2006)Google Scholar
  14. 14.
    Mader, A.O., Schramm, H., Meyer, C.: Efficient epiphyses localization using regression tree ensembles and a conditional random field. Bildverarbeitung für die Medizin 2017. Informatik aktuell, pp. 179–184. Springer, Heidelberg (2017). doi: 10.1007/978-3-662-54345-0_42 CrossRefGoogle Scholar
  15. 15.
    Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_27 CrossRefGoogle Scholar
  16. 16.
    Ruppertshofen, H., et al.: Discriminative generalized hough transform for localization of joints in the lower extremities. CSRD 26(1), 97–105 (2011)Google Scholar
  17. 17.
    Ruppertshofen, H., et al.: Shape model training for concurrent localization of the left and right knee. In: SPIE Medical Imaging (2011)Google Scholar
  18. 18.
    Štern, D., Ebner, T., Urschler, M.: From local to global random regression forests: exploring anatomical landmark localization. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 221–229. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_26 CrossRefGoogle Scholar
  19. 19.
    Wang, C., Komodakis, N., Paragios, N.: Markov random field modeling, inference & learning in computer vision & image understanding: a survey. CVIU 117, 1610–1627 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Alexander Oliver Mader
    • 1
    • 2
    • 3
    Email author
  • Cristian Lorenz
    • 3
  • Martin Bergtholdt
    • 3
  • Jens  von  Berg
    • 3
  • Hauke Schramm
    • 1
    • 2
  • Jan Modersitzki
    • 4
  • Carsten Meyer
    • 1
    • 2
    • 3
  1. 1.Institute of Computer ScienceKiel University of Applied SciencesKielGermany
  2. 2.Department of Computer Science, Faculty of EngineeringKiel UniversityKielGermany
  3. 3.Department of Digital ImagingPhilips ResearchHamburgGermany
  4. 4.Institute of Mathematics and Image ComputingLübeck UniversityLübeckGermany

Personalised recommendations