Skip to main content

Improved Heatmap-Based Landmark Detection

Part of the Lecture Notes in Computer Science book series (LNIP,volume 13003)


Mitral valve repair is a very difficult operation, often requiring experienced surgeons. The doctor will insert a prosthetic ring to aid in the restoration of heart function. The location of the prosthesis’ sutures is critical. Obtaining and studying them during the procedure is a valuable learning experience for new surgeons. This paper proposes a landmark detection network for detecting sutures in endoscopic pictures, which solves the problem of a variable number of suture points in the images. Because there are two datasets, one from the simulated domain and the other from real intraoperative data, this work uses cycleGAN to interconvert the images from the two domains to obtain a larger dataset and a better score on real intraoperative data. This paper performed the tests using a simulated dataset of 2708 photos and a real dataset of 2376 images. The mean sensitivity on the simulated dataset is about 75.64 ± 4.48% and the precision is about 73.62 ± 9.99%. The mean sensitivity on the real dataset is about 50.23 ± 3.76% and the precision is about 62.76 ± 4.93%. The data is from the AdaptOR MICCAI Challenge 2021, which can be found at


  • Heatmap
  • Landmark detection
  • CycleGAN

Huifeng Yao and Ziyu Guo—contributed equally to this work and should be considered joint first-authors.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-88210-5_11
  • Chapter length: 9 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-88210-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   69.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.


  1. Carpentier, A., Adams, D.H., Filsoufi, F.: Carpentier’s Reconstructive Valve Surgery E-Book. Elsevier Health Sciences, Amsterdam (2011)

    Google Scholar 

  2. Engelhardt, S., De Simone, R., Full, P.M., Karck, M., Wolf, I.: Improving surgical training phantoms by hyperrealism: deep unpaired image-to-image translation from real surgeries. In: Bildverarbeitung für die Medizin 2019. I, pp. 282–282. Springer, Wiesbaden (2019).

    CrossRef  Google Scholar 

  3. Engelhardt, S., Sauerzapf, S., Brčić, A., Karck, M., Wolf, I., De Simone, R.: Replicated mitral valve models from real patients offer training opportunities for minimally invasive mitral valve repair. Interact. Cardiovasc. Thorac. Surg. 29(1), 43–50 (2019)

    CrossRef  Google Scholar 

  4. Engelhardt, S., Sauerzapf, S., Preim, B., Karck, M., Wolf, I., De Simone, R.: Flexible and comprehensive patient-specific mitral valve silicone models with chordae tendineae made from 3d-printable molds. Int. J. Comput. Assist. Radiol. Surg. 14(7), 1177–1186 (2019)

    CrossRef  Google Scholar 

  5. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    MathSciNet  CrossRef  Google Scholar 

  6. Payer, C., Štern, D., Bischof, H., Urschler, M.: Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Med. Image Anal. 54, 207–219 (2019)

    CrossRef  Google Scholar 

  7. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).

    CrossRef  Google Scholar 

  8. Sharan, L., et al.: Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation. arXiv preprint arXiv:2107.06941 (2021)

  9. Stern, A., et al.: Heatmap-based 2d landmark detection with a varying number of landmarks. Bildverarbeitung für die Medizin 2021. Informatik aktuell. Springer Vieweg, Wiesbaden (2021)

    Google Scholar 

  10. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)

    Google Scholar 

  11. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Yao, H., Guo, Z., Zhang, Y., Li, X. (2021). Improved Heatmap-Based Landmark Detection. In: , et al. Deep Generative Models, and Data Augmentation, Labelling, and Imperfections. DGM4MICCAI DALI 2021 2021. Lecture Notes in Computer Science(), vol 13003. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88209-9

  • Online ISBN: 978-3-030-88210-5

  • eBook Packages: Computer ScienceComputer Science (R0)