Advertisement

Landmark Localisation in Radiographs Using Weighted Heatmap Displacement Voting

  • Adrian K. DavisonEmail author
  • Claudia Lindner
  • Daniel C. Perry
  • Weisang Luo
  • Medical Student Annotation Collaborative
  • Timothy F. Cootes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11404)

Abstract

We propose a new method for fully automatic landmark localisation using Convolutional Neural Networks (CNNs). Training a CNN to estimate a Gaussian response (“heatmap”) around each target point is known to be effective for this task. We show that better results can be obtained by training a CNN to predict the offset to the target point at every location, then using these predictions to vote for the point position. We show the advantages of the approach, including those of using a novel loss function and weighting scheme. We evaluate on a dataset of radiographs of child hips, including both normal and severely diseased cases. We show the effect of varying the training set size. Our results show significant improvements in accuracy and robustness for the proposed method compared to a standard heatmap prediction approach and comparable results with a traditional Random Forest method.

Keywords

Perthes disease X-rays Paediatrics Convolutional neural network (CNN) Fully convolutional network (FCN) Deep learning Voting 

Notes

Acknowledgements

A. K. Davison was funded by Arthritis Research UK as part of the ORCHiD project. C. Lindner was funded by the Engineering and Physical Sciences Research Council, UK (EP/M012611/1) and by the Medical Research Council, UK (MR/S00405X/1). Manual landmark annotations were provided by the Medical Student Annotation Collaborative (Grace Airey, Evan Araia, Aishwarya Avula, Emily Gargan, Mihika Joshi, Muhammad Khan, Kantida Koysombat, Jason Lee, Sophie Munday and Allen Roby).

References

  1. 1.
    Lindner, C., Thiagarajah, S., Wilkinson, J., The arcOGEN Consortium, Wallis, G., Cootes, T.: Fully automatic segmentation of the proximal femur using random forest regression voting. IEEE Trans. Med. Imag. 32(8), 1462–1472 (2013).  https://doi.org/10.1109/TMI.2013.2258030CrossRefGoogle Scholar
  2. 2.
    Lindner, C., Bromiley, P., Ionita, M., Cootes, T.: Robust and accurate shape model matching using random forest regression-voting. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1862–1874 (2015).  https://doi.org/10.1109/TPAMI.2014.2382106CrossRefGoogle Scholar
  3. 3.
    Chen, C., Xie, W., Franke, J., Grutzner, P., Nolte, L., Zheng, G.: Automatic X-ray landmark detection and shape segmentation via data-driven joint estimation of image displacements. Med. Image Anal. 18(3), 487–499 (2014).  https://doi.org/10.1016/j.media.2014.01.002CrossRefGoogle Scholar
  4. 4.
    Pfister, T., Charles, J., Zisserman, A.: Flowing ConvNets for human pose estimation in videos. In: International Conference on Computer Vision, ICCV 2015, pp. 1913–1921. IEEE (2015).  https://doi.org/10.1109/ICCV.2015.222
  5. 5.
    Aubert, B., Vidal, P.A., Parent, S., Cresson, T., Vazquez, C., De Guise, J.: Convolutional neural network and in-painting techniques for the automatic assessment of scoliotic spine surgery from biplanar radiographs. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 691–699. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66185-8_78CrossRefGoogle Scholar
  6. 6.
    Belagiannis, V., Zisserman, A.: Recurrent human pose estimation. In: Proceedings of 12th International Conference on Automatic Face & Gesture Recognition, FG 2017, pp. 468–475. IEEE (2017).  https://doi.org/10.1109/FG.2017.64
  7. 7.
    Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46723-8_27CrossRefGoogle Scholar
  8. 8.
    Zhang, J., Liu, M., Shen, D.: Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks. IEEE Trans. Image Process. 26(10), 4753–4764 (2017).  https://doi.org/10.1109/TIP.2017.2721106MathSciNetCrossRefGoogle Scholar
  9. 9.
    Bulat, A., Tzimiropoulos, G.: Human pose estimation via convolutional part heatmap regression. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 717–732. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46478-7_44CrossRefGoogle Scholar
  10. 10.
    Tompson, J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. In: Ghahramani, Z., et al. (eds.) Advances in Neural Information Processing Systems, NIPS Proceedings, vol. 27, pp. 1799–1807 (2014)Google Scholar
  11. 11.
    Papandreou, G., et al.: Towards accurate multi-person pose estimation in the wild. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 3711–3719. IEEE (2017).  https://doi.org/10.1109/CVPR.2017.395
  12. 12.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, pp. 770–778. IEEE (2016).  https://doi.org/10.1109/CVPR.2016.90
  13. 13.
    Sofka, M., Milletari, F., Jia, J., Rothberg, A.: Fully convolutional regression network for accurate detection of measurement points. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS 2017. LNCS, vol. 10553, pp. 258–266. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-67558-9_30CrossRefGoogle Scholar
  14. 14.
    Zhang, J., et al.: Joint craniomaxillofacial bone segmentation and landmark digitization by context-guided fully convolutional networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 720–728. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66185-8_81CrossRefGoogle Scholar
  15. 15.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  16. 16.
    Perry, D., Hall, A.: The epidemiology and etiology of Perthes disease. Orthop. Clin. North Am. 42(3), 279–283 (2011).  https://doi.org/10.1016/j.ocl.2011.03.002CrossRefGoogle Scholar
  17. 17.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
  18. 18.
    Keras: deep learning for humans (2015). https://github.com/keras-team/keras
  19. 19.
    TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org
  20. 20.
    Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46484-8_29CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Adrian K. Davison
    • 1
    Email author
  • Claudia Lindner
    • 1
  • Daniel C. Perry
    • 2
    • 3
  • Weisang Luo
    • 3
  • Medical Student Annotation Collaborative
    • 2
    • 3
  • Timothy F. Cootes
    • 1
  1. 1.Centre for Imaging SciencesThe University of ManchesterManchesterUK
  2. 2.University of LiverpoolLiverpoolUK
  3. 3.Alder Hey Children’s HospitalLiverpoolUK

Personalised recommendations