Landmark Localisation in Radiographs Using Weighted Heatmap Displacement Voting

  • Adrian K. DavisonEmail author
  • Claudia Lindner
  • Daniel C. Perry
  • Weisang Luo
  • Medical Student Annotation Collaborative
  • Timothy F. Cootes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11404)


We propose a new method for fully automatic landmark localisation using Convolutional Neural Networks (CNNs). Training a CNN to estimate a Gaussian response (“heatmap”) around each target point is known to be effective for this task. We show that better results can be obtained by training a CNN to predict the offset to the target point at every location, then using these predictions to vote for the point position. We show the advantages of the approach, including those of using a novel loss function and weighting scheme. We evaluate on a dataset of radiographs of child hips, including both normal and severely diseased cases. We show the effect of varying the training set size. Our results show significant improvements in accuracy and robustness for the proposed method compared to a standard heatmap prediction approach and comparable results with a traditional Random Forest method.


Perthes disease X-rays Paediatrics Convolutional neural network (CNN) Fully convolutional network (FCN) Deep learning Voting 



A. K. Davison was funded by Arthritis Research UK as part of the ORCHiD project. C. Lindner was funded by the Engineering and Physical Sciences Research Council, UK (EP/M012611/1) and by the Medical Research Council, UK (MR/S00405X/1). Manual landmark annotations were provided by the Medical Student Annotation Collaborative (Grace Airey, Evan Araia, Aishwarya Avula, Emily Gargan, Mihika Joshi, Muhammad Khan, Kantida Koysombat, Jason Lee, Sophie Munday and Allen Roby).


  1. 1.
    Lindner, C., Thiagarajah, S., Wilkinson, J., The arcOGEN Consortium, Wallis, G., Cootes, T.: Fully automatic segmentation of the proximal femur using random forest regression voting. IEEE Trans. Med. Imag. 32(8), 1462–1472 (2013). Scholar
  2. 2.
    Lindner, C., Bromiley, P., Ionita, M., Cootes, T.: Robust and accurate shape model matching using random forest regression-voting. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1862–1874 (2015). Scholar
  3. 3.
    Chen, C., Xie, W., Franke, J., Grutzner, P., Nolte, L., Zheng, G.: Automatic X-ray landmark detection and shape segmentation via data-driven joint estimation of image displacements. Med. Image Anal. 18(3), 487–499 (2014). Scholar
  4. 4.
    Pfister, T., Charles, J., Zisserman, A.: Flowing ConvNets for human pose estimation in videos. In: International Conference on Computer Vision, ICCV 2015, pp. 1913–1921. IEEE (2015).
  5. 5.
    Aubert, B., Vidal, P.A., Parent, S., Cresson, T., Vazquez, C., De Guise, J.: Convolutional neural network and in-painting techniques for the automatic assessment of scoliotic spine surgery from biplanar radiographs. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 691–699. Springer, Cham (2017). Scholar
  6. 6.
    Belagiannis, V., Zisserman, A.: Recurrent human pose estimation. In: Proceedings of 12th International Conference on Automatic Face & Gesture Recognition, FG 2017, pp. 468–475. IEEE (2017).
  7. 7.
    Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). Scholar
  8. 8.
    Zhang, J., Liu, M., Shen, D.: Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks. IEEE Trans. Image Process. 26(10), 4753–4764 (2017). Scholar
  9. 9.
    Bulat, A., Tzimiropoulos, G.: Human pose estimation via convolutional part heatmap regression. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 717–732. Springer, Cham (2016). Scholar
  10. 10.
    Tompson, J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. In: Ghahramani, Z., et al. (eds.) Advances in Neural Information Processing Systems, NIPS Proceedings, vol. 27, pp. 1799–1807 (2014)Google Scholar
  11. 11.
    Papandreou, G., et al.: Towards accurate multi-person pose estimation in the wild. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 3711–3719. IEEE (2017).
  12. 12.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, pp. 770–778. IEEE (2016).
  13. 13.
    Sofka, M., Milletari, F., Jia, J., Rothberg, A.: Fully convolutional regression network for accurate detection of measurement points. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS 2017. LNCS, vol. 10553, pp. 258–266. Springer, Cham (2017). Scholar
  14. 14.
    Zhang, J., et al.: Joint craniomaxillofacial bone segmentation and landmark digitization by context-guided fully convolutional networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 720–728. Springer, Cham (2017). Scholar
  15. 15.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  16. 16.
    Perry, D., Hall, A.: The epidemiology and etiology of Perthes disease. Orthop. Clin. North Am. 42(3), 279–283 (2011). Scholar
  17. 17.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
  18. 18.
    Keras: deep learning for humans (2015).
  19. 19.
    TensorFlow: large-scale machine learning on heterogeneous systems (2015).
  20. 20.
    Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Adrian K. Davison
    • 1
    Email author
  • Claudia Lindner
    • 1
  • Daniel C. Perry
    • 2
    • 3
  • Weisang Luo
    • 3
  • Medical Student Annotation Collaborative
    • 2
    • 3
  • Timothy F. Cootes
    • 1
  1. 1.Centre for Imaging SciencesThe University of ManchesterManchesterUK
  2. 2.University of LiverpoolLiverpoolUK
  3. 3.Alder Hey Children’s HospitalLiverpoolUK

Personalised recommendations