Skip to main content
SpringerLink
Log in

Using multiple images and contours for deformable 3D–2D registration of a preoperative CT in laparoscopic liver surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Laparoscopic liver resection is a challenging procedure because of the difficulty to localise inner structures such as tumours and vessels. Augmented reality overcomes this problem by overlaying preoperative 3D models on the laparoscopic views. It requires deformable registration of the preoperative 3D models to the laparoscopic views, which is a challenging task due to the liver flexibility and partial visibility.

Methods

We propose several multi-view registration methods exploiting information from multiple views simultaneously in order to improve registration accuracy. They are designed to work on two scenarios: on rigidly related views and on non-rigidly related views. These methods exploit the liver’s anatomical landmarks and texture information available in all the views to constrain registration.

Results

We evaluated the registration accuracy of our methods quantitatively on synthetic and phantom data, and qualitatively on patient data. We measured 3D target registration errors in mm for the whole liver for the quantitative case, and 2D reprojection errors in pixels for the qualitative case.

Conclusion

The proposed rigidly related multi-view methods improve registration accuracy compared to the baseline single-view method. They comply with the 1 cm oncologic resection margin advised for hepatocellular carcinoma interventions, depending on the available registration constraints. The non-rigidly related multi-view method does not provide a noticeable improvement. This means that using multiple views with the rigidity assumption achieves the best overall registration error.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Koo B, Ozgur E, Le Roy B, Buc E, Bartoli A (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. In: MICCAI

  2. Adagolodjo Y, Trivisonne R, Haouchine N, Cotin S, Courtecuisse H (2017) Silhouette-based pose estimation for deformable organs application to surgical augmented reality. In: IROS

  3. Modrzejewski R, Collins T, Seeliger B, Bartoli A, Hostettler A, Marescaux J (2019) An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions. In: IPCAI

  4. Chen L, Tang W, John NW, Wuan TR, Zhang JJ (2018) SLAM-based dense surface reconstruction in monocular minimally invasive surgery and its application to augmented reality. Comput Methods Programs Biomed 158:135–146

    Article  PubMed  Google Scholar 

  5. Haouchine N, Roy F, Untereiner L, Cotin S (2016) Using contours as boundary conditions for elastic registration during minimally invasive hepatic surgery. In: IROS

  6. Robu MR, Ramalhinho J, Thompson S, Gurusamy K, Davidson B, Hawkes D, Stoyanov D, Clarkson M (2018) Global rigid registration of CT to video in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 13:947–956

    Article  PubMed  PubMed Central  Google Scholar 

  7. Thompson S, Totz J, Song Y, Johnsen S, Stoyanov D, Ourselin S, Gurusamy K, Schneider C, Davidson B, Hawkes D, Clarkson M (2015) Accuracy validation of an image guided laparoscopy system for liver resection. Proceedings of SPIE - The International Society for Optical Engineering 9415(09):1–12

    Google Scholar 

  8. Plantefeve R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44:139–153

    Article  PubMed  Google Scholar 

  9. Clements L, Collins J, Weis J, Simpson A, Kingham T, Jarnagin W, Miga M (2017) Deformation correction for image guided liver surgery: an intraoperative fidelity assessment. Surgery 162(3):537–547

    Article  PubMed  Google Scholar 

  10. Bernhardt S, Nicolau S, Bartoli A, Agnus V, Soler L, Doignon C (2015) Using shading to register an intraoperative CT scan to a laparoscopic image. In: Computer-assisted and robotic endoscopy. CARE

  11. Plantefeve R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44:139–153

    Article  PubMed  Google Scholar 

  12. Thompson S, Hu M, Johnsen S, Gurusamy K, Davidson B, Hawkes D (2011) Towards image guided laparoscopic liver surgery, defining the system requirement. In: LIVIM

  13. Bender J, Koschier D, Charrier P, Weber D (2014) Position-based simulation of continuous materials. Comput Graph 44:1–10

    Article  Google Scholar 

  14. Zhong FP, Zhang YJ, Liu Y, Zou SB (2017) Prognostic impact of surgical margin in patients with hepatocellular carcinoma: a meta-analysis. Medicine 96(37):e8043

    Article  PubMed  PubMed Central  Google Scholar 

  15. Totz J, Thompson S, Stoyanov D, Gurusamy K, Davidson B, Hawkes D, Clarkson M (2014) Fast semi-dense surface reconstruction from stereoscopic video in laparoscopic surgery. In: IPCAI

  16. Pfeiffer M, Riediger C, Leger S, Kühn J, Seppelt D, Hoffmann R, Weitz J, Speidel S (2020) Non-rigid volume to surface registration using a data-driven biomechanical model. In: MICCAI

  17. Chui C, Kobayashi E, Chen X, Hisada T, Sakuma I (2004) Combined compression and elongation experiments and non-linear modelling of liver tissue for surgical simulation. Med Biol Eng Comput 44:787–798

    Article  Google Scholar 

  18. Shi H, Farag A, Fahmi R, Chen D (2008) Validation of finite element models of liver tissue using micro-CT. IEEE Trans Bio-med Eng 55:978–984

    Article  Google Scholar 

  19. Pizarro D, Bartoli A (2010) Feature-based deformable surface detection with self-occlusion reasoning. Int J Comput Vis 97:54–70

    Article  Google Scholar 

  20. Bartoli A, Perriollat M, Chambon S (2010) Generalized thin-plate spline warps. Int J Comput Vis 88:85–110

    Article  Google Scholar 

  21. 3DS Abaqus. http://edu.3ds.com/en/software/abaqus-student-edition. Accessed 2 Mar 2021

  22. Ali S, Espinel Y, Jin Y, Maier-Hein L, Bartoli A (2022) Preoperative to intraoperative laparoscopy fusion (P2ILF). In: MICCAI, 2022. https://p2ilf.grand-challenge.org/

Download references

Author information

Authors and Affiliations

  1. EnCoV, Institut Pascal, UMR 6602 CNRS/UCA, Clermont-Ferrand, France

    Yamid Espinel, Lilian Calvet, Karim Botros, Emmanuel Buc, Christophe Tilmant & Adrien Bartoli

  2. University Hospital of Clermont-Ferrand, Clermont-Ferrand, France

    Lilian Calvet, Emmanuel Buc & Adrien Bartoli

Authors
  1. Yamid Espinel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lilian Calvet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karim Botros
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emmanuel Buc
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christophe Tilmant
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Adrien Bartoli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yamid Espinel.

Ethics declarations

Conflict of interest

Yamid Espinel declares to have no potential conflicts of interest. Lilian Calvet declares to have no potential conflicts of interest. Karim Botros declares to have no potential conflicts of interest. Emmanuel Buc declares to have no potential conflicts of interest. Christophe Tilmant declares to have no potential conflicts of interest. Adrien Bartoli declares to have no potential conflicts of interest. All procedures involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study is also supported by an ethical approval with ID IRB00008526-2019-CE58 issued by CPP Sud-Est VI in Clermont-Ferrand, France. Informed consent was obtained from the patients included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 8944 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Espinel, Y., Calvet, L., Botros, K. et al. Using multiple images and contours for deformable 3D–2D registration of a preoperative CT in laparoscopic liver surgery. Int J CARS 17, 2211–2219 (2022). https://doi.org/10.1007/s11548-022-02774-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02774-1

Keywords

Search

Navigation