Abstract
During fetoscopic laser photocoagulation, a treatment for twin-to-twin transfusion syndrome (TTTS), the clinician first identifies abnormal placental vascular connections and laser ablates them to regulate blood flow in both fetuses. The procedure is challenging due to the mobility of the environment, poor visibility in amniotic fluid, occasional bleeding, and limitations in the fetoscopic field-of-view and image quality. Ideally, anastomotic placental vessels would be automatically identified, segmented and registered to create expanded vessel maps to guide laser ablation, however, such methods have yet to be clinically adopted. We propose a solution utilising the U-Net architecture for performing placental vessel segmentation in fetoscopic videos. The obtained vessel probability maps provide sufficient cues for mosaicking alignment by registering consecutive vessel maps using the direct intensity-based technique. Experiments on 6 different in vivo fetoscopic videos demonstrate that the vessel intensity-based registration outperformed image intensity-based registration approaches showing better robustness in qualitative and quantitative comparison. We additionally reduce drift accumulation to negligible even for sequences with up to 400 frames and we incorporate a scheme for quantifying drift error in the absence of the ground-truth. Our paper provides a benchmark for fetoscopy placental vessel segmentation and registration by contributing the first in vivo vessel segmentation and fetoscopic videos dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Pixel annotation tool: https://github.com/abreheret/PixelAnnotationTool.
References
Almoussa, N., et al.: Automated vasculature extraction from placenta images. In: Medical Imaging 2011: Image Processing, vol. 7962, p. 79621L. International Society for Optics and Photonics (2011)
Aughwane, R., Ingram, E., Johnstone, E.D., Salomon, L.J., David, A.L., Melbourne, A.: Placental MRI and its application to fetal intervention. Prenatal diagnosis (2019)
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Bano, S., et al.: Deep sequential mosaicking of fetoscopic videos. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 311–319. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_35
Bano, S., et al.: FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos. Int. J. Comput. Assist. Radiol. Surg. 15(5), 791–801 (2020)
Baschat, A., et al.: Twin-to-twin transfusion syndrome (TTTS). J. Perinat. Med. 39(2), 107–112 (2011)
Bouguet, J.Y., et al.: Pyramidal implementation of the affine Lucas Kanade feature tracker description of the algorithm. Intel Corporation 5(1–10), 4 (2001)
Brunet, F., Bartoli, A., Navab, N., Malgouyres, R.: Direct image registration without region of interest. In: Vision, Modeling, and Visualization, pp. 323–330 (2010)
Chadebecq, F., et al.: Refractive structure-from-motion through a flat refractive interface. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5315–5323 (2017)
Chadebecq, F., et al.: Refractive two-view reconstruction for underwater 3D vision. Int. J. Comput. Vis. 128, 1–17 (2019)
Chang, J.M., Huynh, N., Vazquez, M., Salafia, C.: Vessel enhancement with multiscale and curvilinear filter matching for placenta images. In: International Conference on Systems, Signals and Image Processing, pp. 125–128. IEEE (2013)
Daga, P., et al.: Real-time mosaicing of fetoscopic videos using SIFT. In: Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9786, p. 97861R. International Society for Optics and Photonics (2016)
Deprest, J., Van Schoubroeck, D., Van Ballaer, P., Flageole, H., Van Assche, F.A., Vandenberghe, K.: Alternative technique for nd: YAG laser coagulation in twin-to-twin transfusion syndrome with anterior placenta. Ultrasound Obstet. Gynecol. Official J. Int. Soc. Ultrasound Obstet. Gynecol. 11(5), 347–352 (1998)
Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: Wells, W.M., Colchester, A., Delp, S. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 130–137. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056195
Gaisser, F., Peeters, S.H., Lenseigne, B.A., Jonker, P.P., Oepkes, D.: Stable image registration for in-vivo fetoscopic panorama reconstruction. J. Imaging 4(1), 24 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Lewi, L., Deprest, J., Hecher, K.: The vascular anastomoses in monochorionic twin pregnancies and their clinical consequences. Am. J. Obstet. Gynecol. 208(1), 19–30 (2013)
Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Peter, L., et al.: Retrieval and registration of long-range overlapping frames for scalable mosaicking of in vivo fetoscopy. Int. J. Comput. Assist. Radiol. Surg. 13(5), 713–720 (2018). https://doi.org/10.1007/s11548-018-1728-4
Reeff, M., Gerhard, F., Cattin, P., Gábor, S.: Mosaicing of endoscopic placenta images. INFORMATIK 2006-Informatik für Menschen, Band 1 (2006)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sadda, P., Imamoglu, M., Dombrowski, M., Papademetris, X., Bahtiyar, M.O., Onofrey, J.: Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery. Int. J. Comput. Assist. Radiol. Surg. 14(2), 227–235 (2018). https://doi.org/10.1007/s11548-018-1886-4
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)
Tella-Amo, M., et al.: Pruning strategies for efficient online globally consistent mosaicking in fetoscopy. J. Med. Imaging 6(3), 035001 (2019)
Bano, S., et al.: Deep learning-based fetoscopic mosaicking for field-of-view expansion. Int. J. Comput. Assist. Radiol. Surg., 1–10 (2020)
Acknowledgments
This work was supported by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) at UCL (203145Z/16/Z), EPSRC (EP/P027938/1, EP/R004080/1, NS/A000027/1), the H2020 FET (GA 863146) and Wellcome [WT101957]. Danail Stoyanov is supported by a Royal Academy of Engineering Chair in Emerging Technologies (CiET1819/2/36) and an EPSRC Early Career Research Fellowship (EP/P012841/1). Tom Vercauteren is supported by a Medtronic/Royal Academy of Engineering Research Chair [RCSRF1819/7/34].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bano, S. et al. (2020). Deep Placental Vessel Segmentation for Fetoscopic Mosaicking. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12263. Springer, Cham. https://doi.org/10.1007/978-3-030-59716-0_73
Download citation
DOI: https://doi.org/10.1007/978-3-030-59716-0_73
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59715-3
Online ISBN: 978-3-030-59716-0
eBook Packages: Computer ScienceComputer Science (R0)
