Skip to main content
Log in

Self-contained image mapping of placental vasculature in 3D ultrasound-guided fetoscopy

  • Dynamic Manuscript
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Surgical navigation technology directed at fetoscopic procedures is relatively underdeveloped compared with other forms of endoscopy. The narrow fetoscopic field of views and the vast vascular network on the placenta make examination and photocoagulation treatment of twin-to-twin transfusion syndrome challenging. Though ultrasonography is used for intraoperative guidance, its navigational ability is not fully exploited. This work aims to integrate 3D ultrasound imaging and endoscopic vision seamlessly for placental vasculature mapping through a self-contained framework without external navigational devices.

Methods

This is achieved through development, integration, and experimentation of novel navigational modules. Firstly, a framework design that addresses the current limitations based on identified gaps is conceptualized. Secondly, integration of navigational modules including (1) ultrasound-based localization, (2) image alignment, and (3) vision-based tracking to update the scene texture map is implemented. This updated texture map is projected to an ultrasound-constructed 3D model for photorealistic texturing of the 3D scene creating a panoramic view of the moving fetoscope. In addition, a collaborative scheme for the integration of the modular workflow system is proposed to schedule updates in a systematic fashion. Finally, experiments are carried out to evaluate each modular variation and an integrated collaborative scheme of the framework.

Results

The modules and the collaborative scheme are evaluated through a series of phantom experiments with controlled trajectories for repeatability. The collaborative framework demonstrated the best accuracy (5.2 % RMS error) compared with all the three single-module variations during the experiment. Validation on an ex vivo monkey placenta shows visual continuity of the freehand fetoscopic panorama.

Conclusions

The proposed developed collaborative framework and the evaluation study of the framework variations provide analytical insights for effective integration of ultrasonography and endoscopy. This contributes to the development of navigation techniques in fetoscopic procedures and can potentially be extended to other applications in intraoperative imaging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Fisk NM, Duncombe GJ, Sullivan MHF (2009) The basic and clinical science of twin–twin transfusion syndrome. Placenta 30(5):379–390

    Article  CAS  PubMed  Google Scholar 

  2. Robyr R, Lewi L, Salomon LJ et al (2006) Prevalence and management of late fetal complications following successful selective laser coagulation of chorionic plate anastomoses in twin-to-twin transfusion syndrome. Am J Obstet Gynecol 194(3):796–803

    Article  PubMed  Google Scholar 

  3. Lopriore E, Oepkes D, Walther FJ (2011) Neonatal morbidity in twin–twin transfusion syndrome. Early Hum Dev 87(9):595–599

    Article  PubMed  Google Scholar 

  4. Baud D, Windrim R, Keunen J et al (2013) Fetoscopic laser therapy for twin-twin transfusion syndrome before 17 and after 26 weeks’ gestation. Am J Obstet Gynecol 208(3):197-e1

    Article  Google Scholar 

  5. Rossi AC, D’Addario V (2008) Laser therapy and serial amnioreduction as treatment for twin-twin transfusion syndrome: a metaanalysis and review of literature. Am J Obstet Gynecol 198(2):147–152

    Article  PubMed  Google Scholar 

  6. Senat M-V, Deprest J, Boulvain M et al (2004) Endoscopic laser surgery versus serial amnioreduction for severe twin-to-twin transfusion syndrome. N Engl J Med 351(2):136–144

    Article  CAS  PubMed  Google Scholar 

  7. Hu Y, Yamanaka N, Masamune K (2014) Automatic tracking algorithm in coaxial near-infrared laser ablation endoscope for fetus surgery. Int J Optomechatron 8(3):159–178

    Article  Google Scholar 

  8. Yamanaka N, Yamashita H, Masamune K et al (2010) An endoscope with 2 DOFs steering of coaxial Nd: YAG laser beam for fetal surgery. Trans Mechatron IEEE/ASME 15(6):898–905

    Google Scholar 

  9. Reeff M, Gerhard F, Cattin PC et al (2006) Mosaicing of endoscopic placenta images. GI Jahrestag 1(2006):467–474

    Google Scholar 

  10. Liao H, Tsuzuki M, Mochizuki T et al (2009) Fast image mapping of endoscopic image mosaics with three-dimensional ultrasound image for intrauterine fetal surgery. Minim Invasive Ther Allied Technol 18(6):332–340

    Article  PubMed  Google Scholar 

  11. Yang L, Wang J, Kobayashi E et al (2013) Ultrasound image-guided mapping of endoscopic views on a 3D placenta model: a tracker-less approach. In: Liao H, Linte CA, Masamune K, Peters TM, Zheng G (eds) Augmented reality environments for medical imaging and computer-assisted interventions, Springer, Heidelberg, pp 107–116

    Chapter  Google Scholar 

  12. Yang L, Wang J, Kobayashi E et al (2015) Image mapping of untracked free-hand endoscopic views to an ultrasound image-constructed 3D placenta model. Int J Med Robot Comput Assist Surg 11(22):223–234

    Article  Google Scholar 

  13. Yang L, Wang J, Ando T et al (2014) Vision-based endoscope tracking for 3D ultrasound image-guided surgical navigation. Comput Med Imaging Graph 40:205–216

    Article  PubMed  Google Scholar 

  14. Yang L, Wang J, Kobayashi E et al (2013) Ultrasound image-based endoscope localization for minimally invasive fetoscopic surgery. In: IEEE international conference on engineering in medicine and biology conference. IEEE, pp 1411–1413. doi:10.1109/EMBC.2013.6609774

  15. Liao H, Tsuzuki M, Kobayashi et al E (2009) GPU-based fast 3D ultrasound-endoscope image fusion for complex-shaped objects. In: World Congress on Medical Physics and Biomedical Engineering. Springer, Berlin Heidelberg, pp 206–209. doi:10.1007/978-3-642-03904-1_58

    Google Scholar 

  16. Cleary K, Peters TM (2010) Image-guided interventions: technology review and clinical applications. Annu Rev Biomed Eng 12:119–142

    Article  CAS  PubMed  Google Scholar 

  17. Peters T, Cleary K (2008) Image-guided interventions: technology and applications. Springer, New York

    Book  Google Scholar 

  18. Elfring R, de la Fuente M, Radermacher K (2010) Assessment of optical localizer accuracy for computer aided surgery systems. Comput Aided Surg 15(1–3):1–12

    Article  PubMed  Google Scholar 

  19. Harada K, Miwa M, Fukuyo T et al (2009) ICG fluorescence endoscope for visualization of the placental vascular network: ORIGINAL ARTICLE. Minim Invasive Ther Allied Technol 18(1):3–7

    Article  Google Scholar 

  20. Ishiyama A, Kim K, Yamashita H et al (2011) New fluorescence endoscope for use in twin–twin transfusion syndrome: in vivo visualization of placental blood vessels. Med Eng Phys 33(3):381–385

    Article  PubMed  Google Scholar 

  21. Kim K, Kubota M, Ohkawa Y et al (2011) A novel ultralow-illumination endoscope system. Surg Endosc 25(6):2029–2033

    Article  PubMed  Google Scholar 

  22. Kubota A, Yang L, Wang J et al (2014) Contrast enhancement between vasculature and placenta using narrow band images for TTTS surgery. Int J Comput Assist Radiol Surg 9(Suppl 1):S98–S99

    Google Scholar 

  23. Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. Computer vision—ECCV 2006, Springer, pp 404–417

  24. Fryer JG, Brown DC (1986) Lens distortion for close-range photogrammetry. Photogramm Eng Remote Sen 52(1):51–58

    Google Scholar 

  25. Lepetit V, Moreno-Noguer F, Fua P (2009) Epnp: an accurate o (n) solution to the pnp problem. Int J Comput Vis 81(2):155–166

    Article  Google Scholar 

  26. Kobayashi E, Ando T, Yamashita H et al (2009) A high-resolution, three-dimensional thin endoscope for fetal surgery. Surg Endosc 23(11):2450–2453

    Article  CAS  PubMed  Google Scholar 

  27. Mashiach R, Mezhybovsky V, Nevler A et al (2014) Three-dimensional imaging improves surgical skill performance in a laparoscopic test model for both experienced and novice laparoscopic surgeons. Surg Endosc 28(12):3489–3493

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This work was supported by JSPS KAKENHI Grant Number 26108008, JSPS KAKENHI Grant number 20345268, and Grant for Translational Systems Biology and Medicine Initiative (TSBMI) from the Ministry of Education, Culture, Sports, Science and Technology of Japan.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Liangjing Yang.

Ethics declarations

Disclosures

Dr. Ichiro Sakuma receives grants from the Japan Science and Technology Agency. Dr. Toshio Chiba and Dr. Etsuko Kobayashi receive grants from the Japan Society for the Promotion of Science. Dr. Liangjing Yang, Dr. Junchen Wang, Dr. Takehiro Ando, Dr. Hiromasa Yamashita, and Mr. Akihiro Kubota have no conflicts of interest or financial ties to disclose.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Video 1: 3D Placental Vasculature Image Mapping

Video 2: Ultrasound-Based Image Mapping

Video 3: Vision-Based Image Mapping

Video 4: 3D Placental Vasculature Image Mapping on Monkey Placenta

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Wang, J., Ando, T. et al. Self-contained image mapping of placental vasculature in 3D ultrasound-guided fetoscopy. Surg Endosc 30, 4136–4149 (2016). https://doi.org/10.1007/s00464-015-4690-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-015-4690-z

Keywords

Navigation