Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery

Abstract

Introduction

Twin-to-twin transfusion syndrome (TTTS) is a potentially lethal condition that affects pregnancies in which twins share a single placenta. The definitive treatment for TTTS is fetoscopic laser photocoagulation, a procedure in which placental blood vessels are selectively cauterized. Challenges in this procedure include difficulty in quickly identifying placental blood vessels due to the many artifacts in the endoscopic video that the surgeon uses for navigation. We propose using deep-learned segmentations of blood vessels to create masks that can be recombined with the original fetoscopic video frame in such a way that the location of placental blood vessels is discernable at a glance.

Methods

In a process approved by an institutional review board, intraoperative videos were acquired from ten fetoscopic laser photocoagulation surgeries performed at Yale New Haven Hospital. A total of 345 video frames were selected from these videos at regularly spaced time intervals. The video frames were segmented once by an expert human rater (a clinician) and once by a novice, but trained human rater (an undergraduate student). The segmentations were used to train a fully convolutional neural network of 25 layers.

Results

The neural network was able to produce segmentations with a high similarity to ground truth segmentations produced by an expert human rater (sensitivity = 92.15% ± 10.69%) and produced segmentations that were significantly more accurate than those produced by a novice human rater (sensitivity = 56.87% ± 21.64%; p < 0.01).

Conclusion

A convolutional neural network can be trained to segment placental blood vessels with near-human accuracy and can exceed the accuracy of novice human raters. Recombining these segmentations with the original fetoscopic video frames can produced enhanced frames in which blood vessels are easily detectable. This has significant implications for aiding fetoscopic surgeons—especially trainees who are not yet at an expert level.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Cordero L, Franco A, Joy D, O’Shaughnessy W (2005) Monochorionic diamniotic infants without twin-to-twin transfusion syndrome. J Perinatol 25:753–758. https://doi.org/10.1038/sj.jp.7211405

    Article  PubMed  Google Scholar 

  2. 2.

    Bahtiyar O, Emery P, Dashe S, Wilkins-Haug E, Johnson A, Paek W, Moon-Grady J, Skupski W, OʼBrien M, Harman R, Simpson L (2015) The North American Fetal Therapy Network consensus statement: prenatal surveillance of uncomplicated monochorionic gestations. Obstet Gynecol 125:118–123. https://doi.org/10.1097/AOG.0000000000000599

    Article  PubMed  Google Scholar 

  3. 3.

    Faye-Petersen M, Crombleholme M (2008) Twin-to-twin transfusion syndrome. NeoReviews 9:370–379

    Google Scholar 

  4. 4.

    Emery P, Bahtiyar O, Moise J (2015) The North American Fetal Therapy Network consensus statement: management of complicated monochorionic gestations. Obstet Gynecol 126:575–584. https://doi.org/10.1097/AOG.0000000000000994

    Article  PubMed  Google Scholar 

  5. 5.

    Luks F (2009) Schematic illustration of endoscopic fetal surgery for twin-to-twin trans-fusion syndrome

  6. 6.

    Pratt R, Deprest J, Vercauteren T, Ourselin S, David L (2015) Computer-assisted surgical planning and intraoperative guidance in fetal surgery: a systematic review. Prenat Diagn 35:1159–1166. https://doi.org/10.1002/pd.4660

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Miller R, Novotny J, Laidlaw H, Luks F, Merck D, Collins S (2016) Virtually visualizing vessels: a study of the annotation of placental vasculature from MRI in large-scale virtual reality for surgical planning. Brown University, Providence

    Google Scholar 

  8. 8.

    Tella-Amo M, Daga P, Chadebecq F, Thompson S, Shakir I, Dwyer G, Wimalasundera R, Deprest J, Stoyanov D, Vercauteren T, Ourselin S (2016) A combined EM and visual tracking probabilistic model for robust mosaicking: application to fetoscopy. In: Proceedings of IEEE CVPR workshops, vol 31, pp 84–92. https://doi.org/10.1515/10.1109/cvprw.2016.72

  9. 9.

    Graves E, Harrison R, Padilla E (2017) Minimally invasive fetal surgery. Clin Perinatol 44:729–751. https://doi.org/10.1016/j.clp.2017.08.001

    Article  PubMed  Google Scholar 

  10. 10.

    Tchirikov M, Oshovskyy V, Steetskamp J, Falkert A, Huber G, Entezami M (2011) Neonatal outcome using ultrathin fetoscope for laser coagulation in twin-to-twin-transfusion syndrome. J Perinat Med. https://doi.org/10.1515/jpm.2011.091

    Article  PubMed  Google Scholar 

  11. 11.

    Olguner M, Akgür M, Özdemir T, Aktuğ T, Özer E (2000) Amniotic fluid exchange for the prevention of neural tissue damage in myelomeningocele: an alternative minimally invasive method to open in utero surgery. Pediatr Neurosurg 33:252–256. https://doi.org/10.1159/000055964

    CAS  Article  PubMed  Google Scholar 

  12. 12.

    Yang L, Wang J, Ando T, Kubota A, Yamashita H, Sakuma I, Chiba T, Kobayashi E (2016) Towards scene adaptive image correspondence for placental vasculature mosaic in computer assisted fetoscopic procedures. Int J Med Robot Comput Assist Surg 12:375–386. https://doi.org/10.1002/rcs.1700

    Article  Google Scholar 

  13. 13.

    Gaisser F, Peeters S, Lenseigne B, Jonker P, Oepkes D (2018) Stable image registration for in vivo fetoscopic panorama reconstruction. J Imaging 4:24. https://doi.org/10.3390/jimaging4010024

    Article  Google Scholar 

  14. 14.

    Almoussa N, Dutra B, Lampe B, Getreuer P, Wittman T, Salafia C, Vese L (2011) Automated vasculature extraction from placenta images. In: Medical imaging 2011: image processing, vol 7962. International Society for Optics and Photonics

  15. 15.

    Park M, Yampolsky M, Shlakhter O, VanHorn S, Dygulska B, Kiryankova N, Salafia C (2013) Vessel enhancement with multiscale and curvilinear filter matching for placenta images. Placenta 34:A12

    Google Scholar 

  16. 16.

    Chang JM, Huynh N, Vazquez M, Salafia C. (2013) Vessel enhancement with multiscale and curvilinear filter matching for placenta images. In: 2013 20th international conference on systems, signals and image processing (IWSSIP), pp 125–128

  17. 17.

    Perera Bel E (2017) Ultrasound segmentation for vascular network reconstruction in twin-to-twin transfusion syndrome. M.S. Thesis, Pompeu Fabra University, Barcelona, Spain. https://repositori.upf.edu/handle/10230/33180. Accessed 13 Nov 2018

  18. 18.

    Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. MICCAI 18:234–241. https://doi.org/10.1007/978-3-319-24574-4_28

    Article  Google Scholar 

  19. 19.

    Panchapagesan S, Sun M, Khare A, Matsoukas S, Mandal A, Hoffmeister B, Vitaladevuni S (2016) Multi-task learning and weighted cross-entropy for DNN-based keyword spotting. In: Interspeech, pp. 760–764. https://doi.org/10.21437/Interspeech.2016-1485

  20. 20.

    Dice R (1945) Measures of the amount of ecologic association between species. Ecology 26:297–302. https://doi.org/10.2307/1932409

    Article  Google Scholar 

  21. 21.

    Frangi F, Niessen J, Vincken L, Viergever A (1998) Multiscale vessel enhancement filtering. MICCAI 1496:130–137

    Google Scholar 

  22. 22.

    Srivastava R, Wong K, Duan L, Liu J, Wong TY (2015) Red lesion detection in retinal fundus images using Frangi-based filters. IEEE EMBC 37:5663–5666. https://doi.org/10.1109/EMBC.2015.7319677

    Article  Google Scholar 

  23. 23.

    Jiang Y, Zhuang W, Sinusas J, Staib H, Papademetris X (2011) Vessel connectivity using Murray’s hypothesis. MICCAI 14:528–536

    PubMed  Google Scholar 

Download references

Funding

This work was supported by the National Institutes of Health Grant Number T35DK104689 (NIDDK Medical Student Research Fellowship).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Praneeth Sadda.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PPTX 34 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sadda, P., Imamoglu, M., Dombrowski, M. et al. Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery. Int J CARS 14, 227–235 (2019). https://doi.org/10.1007/s11548-018-1886-4

Download citation

Keywords

  • Segmentation
  • Vessels
  • Deep learning
  • Convolutional neural network
  • Fetoscopy
  • Twin-to-twin transfusion syndrome