Detection and Localization of Landmarks in the Lower Extremities Using an Automatically Learned Conditional Random Field
The detection and localization of single or multiple landmarks is a crucial task in medical imaging. It is often required as initialization for other tasks like segmentation or registration. A common approach to localize multiple landmarks is to exploit their spatial correlations, e.g., by using a conditional random field (CRF) to incorporate geometric information between landmark pairs. This CRF is usually applied to resolve ambiguities of a localizer, e.g., a random forest or a deep neural network. In this paper, we apply a random forest/CRF combination to the task of jointly detecting and localizing 6 landmarks in the lower extremities, taken from a dataset of 660 X-ray images. The dataset is challenging since a significant number of images does not show all the landmarks. Furthermore, 11.3% of the target landmarks are altered by prostheses or pathologies.
To account for this, we introduce a “missing” label for each landmark (represented by a node in the CRF). Moreover, instead of manually specifying the CRF model by selecting suitable potential functions and the graph topology, we suggest to automatically optimize both in a learning framework. Specifically, we define a pool of potential functions and learn their CRF weights (relative contributions), in addition to the potential values in case of missing landmarks. Potentials with a low weight are removed, thus optimizing the graph topology. Detailed evaluations on our database show the feasibility of our approach. Our algorithm removed on average 23 of the initial 51 CRF potentials, and correctly detected and localized (within 10 mm tolerance) on average 92.8% of the landmarks, with individual rates ranging from 90.0% to 97.4%.
The authors thank the Diagnosezentrum Urania, Vienna and the Dartmouth Hitchcock Medical Center, Lebanon for providing the radiographs that served as training and test sets; Gooßen  for the annotations. This work has been financially supported by the Federal Ministry of Education and Research under the grant 03FH013IX5. The liability for the content of this work lies with the authors.
- 1.Bergtholdt, M., Kappes, J.H., Schnörr, C.: Learning of graphical models and efficient inference for object class recognition. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 273–283. Springer, Heidelberg (2006). doi: 10.1007/11861898_28 CrossRefGoogle Scholar
- 5.Donner, R., et al.: Sparse MRF appearance models for fast anatomical structure localisation. In: BMVC (2007)Google Scholar
- 7.Glocker, B., Zikic, D., Konukoglu, E., Haynor, D.R., Criminisi, A.: Vertebrae localization in pathological spine CT via dense classification from sparse annotations. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 262–270. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40763-5_33 CrossRefGoogle Scholar
- 8.Gooßen, A.: Computational Imaging in Orthopaedic Radiography. BoD (2012)Google Scholar
- 9.Hahmann, F., et al.: Model interpolation for eye localization using the discriminative generalized hough transform. In: BIOSIG (2012)Google Scholar
- 10.Ishikawa, H.: Higher-order clique reduction in binary graph cut. In: CVPR, pp. 2993–3000. IEEE (2009)Google Scholar
- 11.Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2014)Google Scholar
- 13.LeCun, Y., Chopra, S., Hadsell, R.: A tutorial on energy-based learning. In: Predicting Structured Data (2006)Google Scholar
- 15.Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_27 CrossRefGoogle Scholar
- 16.Ruppertshofen, H., et al.: Discriminative generalized hough transform for localization of joints in the lower extremities. CSRD 26(1), 97–105 (2011)Google Scholar
- 17.Ruppertshofen, H., et al.: Shape model training for concurrent localization of the left and right knee. In: SPIE Medical Imaging (2011)Google Scholar
- 18.Štern, D., Ebner, T., Urschler, M.: From local to global random regression forests: exploring anatomical landmark localization. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 221–229. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_26 CrossRefGoogle Scholar
- 19.Wang, C., Komodakis, N., Paragios, N.: Markov random field modeling, inference & learning in computer vision & image understanding: a survey. CVIU 117, 1610–1627 (2013)Google Scholar