Skip to main content

Deep Placental Vessel Segmentation for Fetoscopic Mosaicking

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12263)

Abstract

During fetoscopic laser photocoagulation, a treatment for twin-to-twin transfusion syndrome (TTTS), the clinician first identifies abnormal placental vascular connections and laser ablates them to regulate blood flow in both fetuses. The procedure is challenging due to the mobility of the environment, poor visibility in amniotic fluid, occasional bleeding, and limitations in the fetoscopic field-of-view and image quality. Ideally, anastomotic placental vessels would be automatically identified, segmented and registered to create expanded vessel maps to guide laser ablation, however, such methods have yet to be clinically adopted. We propose a solution utilising the U-Net architecture for performing placental vessel segmentation in fetoscopic videos. The obtained vessel probability maps provide sufficient cues for mosaicking alignment by registering consecutive vessel maps using the direct intensity-based technique. Experiments on 6 different in vivo fetoscopic videos demonstrate that the vessel intensity-based registration outperformed image intensity-based registration approaches showing better robustness in qualitative and quantitative comparison. We additionally reduce drift accumulation to negligible even for sequences with up to 400 frames and we incorporate a scheme for quantifying drift error in the absence of the ground-truth. Our paper provides a benchmark for fetoscopy placental vessel segmentation and registration by contributing the first in vivo vessel segmentation and fetoscopic videos dataset.

Keywords

  • Fetoscopy
  • Deep learning
  • Vessel segmentation
  • Vessel registration
  • Mosaicking
  • Twin-to-twin transfusion syndrome

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-59716-0_73
  • Chapter length: 11 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-59716-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   149.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

Notes

  1. 1.

    Pixel annotation tool: https://github.com/abreheret/PixelAnnotationTool.

References

  1. Almoussa, N., et al.: Automated vasculature extraction from placenta images. In: Medical Imaging 2011: Image Processing, vol. 7962, p. 79621L. International Society for Optics and Photonics (2011)

    Google Scholar 

  2. Aughwane, R., Ingram, E., Johnstone, E.D., Salomon, L.J., David, A.L., Melbourne, A.: Placental MRI and its application to fetal intervention. Prenatal diagnosis (2019)

    Google Scholar 

  3. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    CrossRef  Google Scholar 

  4. Bano, S., et al.: Deep sequential mosaicking of fetoscopic videos. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 311–319. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_35

    CrossRef  Google Scholar 

  5. Bano, S., et al.: FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos. Int. J. Comput. Assist. Radiol. Surg. 15(5), 791–801 (2020)

    CrossRef  Google Scholar 

  6. Baschat, A., et al.: Twin-to-twin transfusion syndrome (TTTS). J. Perinat. Med. 39(2), 107–112 (2011)

    Google Scholar 

  7. Bouguet, J.Y., et al.: Pyramidal implementation of the affine Lucas Kanade feature tracker description of the algorithm. Intel Corporation 5(1–10), 4 (2001)

    Google Scholar 

  8. Brunet, F., Bartoli, A., Navab, N., Malgouyres, R.: Direct image registration without region of interest. In: Vision, Modeling, and Visualization, pp. 323–330 (2010)

    Google Scholar 

  9. Chadebecq, F., et al.: Refractive structure-from-motion through a flat refractive interface. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5315–5323 (2017)

    Google Scholar 

  10. Chadebecq, F., et al.: Refractive two-view reconstruction for underwater 3D vision. Int. J. Comput. Vis. 128, 1–17 (2019)

    Google Scholar 

  11. Chang, J.M., Huynh, N., Vazquez, M., Salafia, C.: Vessel enhancement with multiscale and curvilinear filter matching for placenta images. In: International Conference on Systems, Signals and Image Processing, pp. 125–128. IEEE (2013)

    Google Scholar 

  12. Daga, P., et al.: Real-time mosaicing of fetoscopic videos using SIFT. In: Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9786, p. 97861R. International Society for Optics and Photonics (2016)

    Google Scholar 

  13. Deprest, J., Van Schoubroeck, D., Van Ballaer, P., Flageole, H., Van Assche, F.A., Vandenberghe, K.: Alternative technique for nd: YAG laser coagulation in twin-to-twin transfusion syndrome with anterior placenta. Ultrasound Obstet. Gynecol. Official J. Int. Soc. Ultrasound Obstet. Gynecol. 11(5), 347–352 (1998)

    CrossRef  Google Scholar 

  14. Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: Wells, W.M., Colchester, A., Delp, S. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 130–137. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056195

    CrossRef  Google Scholar 

  15. Gaisser, F., Peeters, S.H., Lenseigne, B.A., Jonker, P.P., Oepkes, D.: Stable image registration for in-vivo fetoscopic panorama reconstruction. J. Imaging 4(1), 24 (2018)

    CrossRef  Google Scholar 

  16. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  17. Lewi, L., Deprest, J., Hecher, K.: The vascular anastomoses in monochorionic twin pregnancies and their clinical consequences. Am. J. Obstet. Gynecol. 208(1), 19–30 (2013)

    CrossRef  Google Scholar 

  18. Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    CrossRef  Google Scholar 

  19. Peter, L., et al.: Retrieval and registration of long-range overlapping frames for scalable mosaicking of in vivo fetoscopy. Int. J. Comput. Assist. Radiol. Surg. 13(5), 713–720 (2018). https://doi.org/10.1007/s11548-018-1728-4

    CrossRef  Google Scholar 

  20. Reeff, M., Gerhard, F., Cattin, P., Gábor, S.: Mosaicing of endoscopic placenta images. INFORMATIK 2006-Informatik für Menschen, Band 1 (2006)

    Google Scholar 

  21. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    CrossRef  Google Scholar 

  22. Sadda, P., Imamoglu, M., Dombrowski, M., Papademetris, X., Bahtiyar, M.O., Onofrey, J.: Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery. Int. J. Comput. Assist. Radiol. Surg. 14(2), 227–235 (2018). https://doi.org/10.1007/s11548-018-1886-4

    CrossRef  Google Scholar 

  23. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  24. Tella-Amo, M., et al.: Pruning strategies for efficient online globally consistent mosaicking in fetoscopy. J. Med. Imaging 6(3), 035001 (2019)

    CrossRef  Google Scholar 

  25. Bano, S., et al.: Deep learning-based fetoscopic mosaicking for field-of-view expansion. Int. J. Comput. Assist. Radiol. Surg., 1–10 (2020)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) at UCL (203145Z/16/Z), EPSRC (EP/P027938/1, EP/R004080/1, NS/A000027/1), the H2020 FET (GA 863146) and Wellcome [WT101957]. Danail Stoyanov is supported by a Royal Academy of Engineering Chair in Emerging Technologies (CiET1819/2/36) and an EPSRC Early Career Research Fellowship (EP/P012841/1). Tom Vercauteren is supported by a Medtronic/Royal Academy of Engineering Research Chair [RCSRF1819/7/34].

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sophia Bano .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Bano, S. et al. (2020). Deep Placental Vessel Segmentation for Fetoscopic Mosaicking. In: , et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12263. Springer, Cham. https://doi.org/10.1007/978-3-030-59716-0_73

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59716-0_73

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59715-3

  • Online ISBN: 978-3-030-59716-0

  • eBook Packages: Computer ScienceComputer Science (R0)