A key component of computer- assisted surgery systems is the accurate and robust registration of preoperative planning data with intraoperative sensor data. In laparoscopic surgery, this image-based registration remains challenging due to soft tissue deformations. This paper presents a novel approach for biomechanical soft tissue registration of preoperative CT data with stereo endoscopic image data.
The proposed method consists of two registrations steps. First, we use a 3D surface mosaic from partial surfaces reconstructed from stereo endoscopic images to initially align the biomechanical model with the intraoperative position and shape of the organ. After this initialization, the biomechanical model is projected onto newly captured surfaces, resulting in displacement boundary conditions, which in turn are used to update the biomechanical model.
The method is evaluated in silico, using a human liver model, and in vivo, using porcine data. The quantitative in silico data shows a stable behaviour of the biomechanical model and root-mean-square deviation of volume vertices of under 3 mm with adjusted biomechanical parameters.
This work contributes a fully automatic featureless non-rigid registration approach. The results of the in silico and in vivo experiments suggest that our method is able to handle dynamic deformations during surgery. Additional experiments, especially regarding human tissue behaviour, are an important next step towards clinical applications.
This is a preview of subscription content,to check access.
Access this article
Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201
Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 17(8):974–996
Maier-Hein L, Groch A, Bartoli A, Bodenstedt S, Boissonnat G, Chang PL, Clancy NT, Elson DS, Haase S, Heim E, Hornegger J (2014) Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction. IEEE Trans Med Imaging 33(10):1913–1930
Allan M, Kapoor A, Mewes P, Mountney P (2015) Non rigid registration of 3D images to laparoscopic video for image guided surgery. International workshop on computer-assisted and robotic endoscopy. Springer International Publishing, Berlin
Stefansic JD, Herline AJ, Shyr Y, Chapman WC, Fitzpatrick JM, Dawant BM, Galloway RL (2002) Registration of physical space to laparoscopic image space for use in minimally invasive hepatic surgery. In: 5th IEEE EMBS international summer school on biomedical imaging. IEEE, p 12
Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4):896–900
Herline AJ, Stefansic JD, Debelak JP, Hartmann SL, Pinson CW, Galloway RL, Chapman WC (1999) Image-guided surgery: preliminary feasibility studies of frameless stereotactic liver surgery. Arch Surg 134(6):644–650
Gupta T, Shin D, Sivagnanadasan N, Hoiem D (2016) 3DFS: deformable dense depth fusion and segmentation for object reconstruction from a handheld camera. arXiv:1606.05002
Newcombe RA, Fox D, Seitz SM (2015) Dynamicfusion: reconstruction and tracking of non-rigid scenes in real-time. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 343–352
Bregler C, Hertzmann A, Biermann H (2000) Recovering non-rigid 3D shape from image streams. In: Proceedings of IEEE conference on computer vision and pattern recognition, 2000, vol 2. IEEE, pp 690–696
Rucker DC, Wu Y, Clements L, Ondrake J, Pheiffer T, Simpson A, Jarnagin W, Miga M (2014) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imaging 33(1):147–158
Plantefve R, Peterlik I, Haouchine N, Cotin S (2015) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann. Biomed. Eng. 44(1):139–153
Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, Thiago S, Maier-Hein L, Wagner M, Wnscher J, Kenngott H, Müller BP, Speidel S (2014) Physics-based shape matching for intraoperative image guidance. Med Phys 41(11):111901
Pratt P, Stoyanov D, Visentini-Scarzanella M, Yang G (2010) Dynamic guidance for robotic surgery using image-constrained biomechanical models. In: Medical image computing and computer-assisted intervention-MICCAI, pp 77–85
Oktay O, Zhang L, Mansi T, Mountney P, Mewes P, Nicolau S, Soler L, Chefd’hotel C (2013) Biomechanically driven registration of pre-to intra-operative 3D images for laparoscopic surgery. In: Medical image computing and computer-assisted intervention-MICCAI, pp 1–9
Nosrati MS, Abugharbieh R, Peyrat JM, Abinahed J, Al-Alao O, Al-Ansari A, Hamarneh G (2016) Simultaneous multi-structure segmentation and 3D nonrigid pose estimation in image-guided robotic surgery. IEEE Trans Med Imaging 35(1):1–9
Collins T, Bartoli A, Bourdel N, Canis M (2016) Robust, real-time, dense and deformable 3D organ tracking in laparoscopic videos. In: International conference on medical image computing and computer-assisted intervention. Springer International Publishing (2016)
dos Santos TR, Seitel A, Kilgus T, Suwelack S, Wekerle AL, Kenngott H, Speidel S, Schlemmer H, Meinzer H, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. Med Image Anal 18(7):1101–1114
Rusu RB, Blodow N, Beetz M (2009) Fast point feature histograms (FPFH) for 3D registration. In: IEEE international conference on robotics and automation, 2009. ICRA’09. IEEE
Bodenstedt S, Goertler J,Wagner M, Kenngott H, Mïller-Stich B, Dillmann R, Speidel S (2016) Superpixel-based structure classification for laparoscopic surgery. In: Webster RJ, Yaniv ZR (eds) Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, vol 9786. Bellingham. doi:10.1117/12.2216750
Roehl S, Bodenstedt S, Suwelack S, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2012) Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. Med Phys 39(3):1632–1645
Nolden M, Zelzer S, Seitel A, Wald D, Mller M, Franz AM, Maleike D, Fangerau M, Baumhauer M, Maier-Hein L, Maier-Hein KH, Meinzer HP, Wolf I (2013) The medical imaging interaction toolkit: challenges and advances. Int J Comput Assist Radiol Surg 8:607–620
Yeh WC, Li PC, Jeng YM, Hsu HC, Kuo PL, Li ML, Lee PH (2002) Elastic modulus measurements of human liver and correlation with pathology. Ultrasound Med Biol 28(4):467–474
Fung YC (1981) Biomechanics: mechanical properties of living tissues. Springer, New York
Reichard D, Bodenstedt S, Suwelack S, Mayer B, Preukschas A, Wagner M, Kenngott H, Müller-Stich B, Dillmann R, Speidel S (2015) Intraoperative on-the-fly organ-mosaicking for laparoscopic surgery. J Med Imaging 2(4):045001
Shi H (2007) Finite element modeling of soft tissue deformation. ProQuest Dissertations Publishing, University of Louisville
The present research was supported by the Klaus Tschira Foundation.
Conflict of interest
The authors declare that they have no conflict of interest.
All applicable international, national and/or institutional guidelines for the care and use of animals were followed. All procedures performed in studies involving animals were in accordance with the ethical standards of the institution or practice at which the studies were conducted.
This article does not contain patient data.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Reichard, D., Häntsch, D., Bodenstedt, S. et al. Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery. Int J CARS 12, 1101–1110 (2017). https://doi.org/10.1007/s11548-017-1613-6