Surgical Endoscopy

, Volume 32, Issue 3, pp 1192–1201 | Cite as

Augmented reality in a tumor resection model

  • Pauline Chauvet
  • Toby Collins
  • Clement Debize
  • Lorraine Novais-Gameiro
  • Bruno Pereira
  • Adrien Bartoli
  • Michel Canis
  • Nicolas BourdelEmail author



Augmented Reality (AR) guidance is a technology that allows a surgeon to see sub-surface structures, by overlaying pre-operative imaging data on a live laparoscopic video. Our objectives were to evaluate a state-of-the-art AR guidance system in a tumor surgical resection model, comparing the accuracy of the resection with and without the system. Our system has three phases. Phase 1: using the MRI images, the kidney’s and pseudotumor’s surfaces are segmented to construct a 3D model. Phase 2: the intra-operative 3D model of the kidney is computed. Phase 3: the pre-operative and intra-operative models are registered, and the laparoscopic view is augmented with the pre-operative data.


We performed a prospective experimental study on ex vivo porcine kidneys. Alginate was injected into the parenchyma to create pseudotumors measuring 4–10 mm. The kidneys were then analyzed by MRI. Next, the kidneys were placed into pelvictrainers, and the pseudotumors were laparoscopically resected. The AR guidance system allows the surgeon to see tumors and margins using classical laparoscopic instruments, and a classical screen. The resection margins were measured microscopically to evaluate the accuracy of resection.


Ninety tumors were segmented: 28 were used to optimize the AR software, and 62 were used to randomly compare surgical resection: 29 tumors were resected using AR and 33 without AR. The analysis of our pathological results showed 4 failures (tumor with positive margins) (13.8%) in the AR group, and 10 (30.3%) in the Non-AR group. There was no complete miss in the AR group, while there were 4 complete misses in the non-AR group. In total, 14 (42.4%) tumors were completely missed or had a positive margin in the non-AR group.


Our AR system enhances the accuracy of surgical resection, particularly for small tumors. Crucial information such as resection margins and vascularization could also be displayed.


Augmented reality Laparoscopic surgery Partial nephrectomy Resection margins 



This research has received funding from the EU’s FP7 through the ERC research grant 307483 FLEXABLE.

Compliance with ethical standards


Dr Pauline Chauvet, Toby Collins, Clement Debize, Lorraine Novais-Gameiro, Bruno Pereira, Prs Adrien Bartoli and Michel Canis, and Dr Nicolas Bourdel have no conflicts of interest or financial ties to disclose.

Supplementary material

Supplementary material 1 (MP4 56711 kb)


  1. 1.
    Pessaux P, Diana M, Soler L, Piardi T, Mutter D, Marescaux J (2014) Robotic duodenopancreatectomy assisted with augmented reality and real-time fluorescence guidance. Surg Endosc 28(8):2493–2498CrossRefPubMedGoogle Scholar
  2. 2.
    Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented-reality-assisted laparoscopic adrenalectomy. JAMA 292(18):2214–2215Google Scholar
  3. 3.
    Pessaux P, Diana M, Soler L, Piardi T, Mutter D, Marescaux J (2015) Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch Surg Dtsch Ges Für Chir 400(3):381–385Google Scholar
  4. 4.
    Simpfendörfer T, Baumhauer M, Müller M, Gutt CN, Meinzer H-P, Rassweiler JJ et al (2011) Augmented reality visualization during laparoscopic radical prostatectomy. J Endourol Endourol Soc 25(12):1841–1845CrossRefGoogle Scholar
  5. 5.
    Grimson WL, Ettinger GJ, White SJ, Lozano-Perez T, Wells WM, Kikinis R (1996) An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization. IEEE Trans Med Imaging 15(2):129–140CrossRefPubMedGoogle Scholar
  6. 6.
    Bourdel N, Collins T, Pizarro D, Bartoli A, Da Ines D, Perreira B et al (2016) Augmented reality in gynecologic surgery: evaluation of potential benefits for myomectomy in an experimental uterine model. Surg Endosc 2016Google Scholar
  7. 7.
    Bourdel N, Collins T, Pizarro D, Debize C, Grémeau A, Bartoli A et al (2017) Use of augmented reality in laparoscopic gynecology to visualize myomas. Fertil Steril 107(3):737–739Google Scholar
  8. 8.
    Wolf I, Vetter M, Wegner I, Böttger T, Nolden M, Schöbinger M et al (2005) The medical imaging interaction toolkit. Med Image Anal 9(6):594–604CrossRefPubMedGoogle Scholar
  9. 9.
    Collins T, Pizarro D, Bartoli A, Canis M, Bourdel N (2013) Realtime wide-baseline registration of the uterus in laparoscopic videos using multiple texture maps. In: Liao H, Linte CA, Masamune K, Peters TM, Zheng G (eds) Augmented reality environments for medical imaging and computer-assisted interventions [Internet]. Springer, Berlin; [cité 10 oct 2015], pp 162–171 (Lecture Notes in Computer Science). Disponible sur:
  10. 10.
    Teber D, Guven S, Simpfendörfer T, Baumhauer M, Güven EO, Yencilek F et al (2009) Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol 56(2):332–338CrossRefPubMedGoogle Scholar
  11. 11.
    Hafez KS, Fergany AF, Novick AC (1999) Nephron sparing surgery for localized renal cell carcinoma: impact of tumor size on patient survival, tumor recurrence and TNM staging. J Urol 162(6):1930–1933CrossRefPubMedGoogle Scholar
  12. 12.
    Herr HW (1999) Partial nephrectomy for unilateral renal carcinoma and a normal controlateral kidney: 10-year followup. J Urol 161(1):33–35CrossRefPubMedGoogle Scholar
  13. 13.
    Lee CT, Katz J, Shi W, Thaler HT, Reuter VE, Russo P (2000) Surgical management of renal tumors 4 cm or less in a contemporary cohort. J Urol 163(3):730–736CrossRefPubMedGoogle Scholar
  14. 14.
    McKiernan J, Simmons R, Katz J, Russo P (2002) Natural history of chronic renal insufficiency after partial and radical nephrectomy. Urology 59(6):816–820CrossRefPubMedGoogle Scholar
  15. 15.
    Clark PE, Schover LR, Uzzo RG, Hafez KS, Rybicki LA, Novick AC (2001) Quality of life and psychological adaptation after surgical treatment for localized renal cell carcinoma: impact of the amount of remaining renal tissue. Urology 57(2):252–256CrossRefPubMedGoogle Scholar
  16. 16.
    Hafez KS, Novick AC, Campbell SC (1997) Patterns of tumor recurrence and guidelines for followup after nephron sparing surgery for sporadic renal cell carcinoma. J Urol 157(6):2067–2070CrossRefPubMedGoogle Scholar
  17. 17.
    Breda A, Stepanian SV, Liao J, Lam JS, Guazzoni G, Stifelman M et al (2007) Positive margins in laparoscopic partial nephrectomy in 855 cases: a multi-institutional survey from the United States and Europe. J Urol 178(1):47–50CrossRefPubMedGoogle Scholar
  18. 18.
    Gill IS, Kavoussi LR, Lane BR, Blute ML, Babineau D, Colombo JR et al (2007) Comparison of 1,800 laparoscopic and open partial nephrectomies for single renal tumors. J Urol 178(1):41–46CrossRefPubMedGoogle Scholar
  19. 19.
    Yossepowitch O, Thompson RH, Leibovich BC, Eggener SE, Pettus JA, Kwon ED et al (2008) Positive surgical margins at partial nephrectomy: predictors and oncological outcomes. J Urol 179(6):2158–2163CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Ani I, Finelli A, Alibhai SMH, Timilshina N, Fleshner N, Abouassaly R (2013) Prevalence and impact on survival of positive surgical margins in partial nephrectomy for renal cell carcinoma: a population-based study: positive surgical margins after partial nephrectomy for RCC. BJU Int 111(8):E300–E305CrossRefPubMedGoogle Scholar
  21. 21.
    Tabayoyong W, Abouassaly R, Kiechle JE, Cherullo EE, Meropol NJ, Shah ND et al (2015) Variation in surgical margin status by surgical approach among patients undergoing partial nephrectomy for small renal masses. J Urol 194(6):1548–1553CrossRefPubMedGoogle Scholar
  22. 22.
    Vermooten V (1950) Indications for conservative surgery in certain renal tumors: a study based on the growth pattern of the cell carcinoma. J Urol 64(2):200–208CrossRefPubMedGoogle Scholar
  23. 23.
    Russo P (2000) Renal cell carcinoma: presentation, staging, and surgical treatment. Semin Oncol 27(2):160–176PubMedGoogle Scholar
  24. 24.
    Sutherland SE, Resnick MI, Maclennan GT, Goldman HB (2002) Does the size of the surgical margin in partial nephrectomy for renal cell cancer really matter? J Urol 167(1):61–64CrossRefPubMedGoogle Scholar
  25. 25.
    Castilla EA, Liou LS, Abrahams NA, Fergany A, Rybicki LA, Myles J et al (2002) Prognostic importance of resection margin width after nephron-sparing surgery for renal cell carcinoma. Urology 60(6):993–997CrossRefPubMedGoogle Scholar
  26. 26.
    Rizzo S, Calareso G, De Maria F, Zanagnolo V, Lazzari R, Cecconi A et al (2013) Gynecologic tumors: how to communicate imaging results to the surgeon. Cancer Imaging Off Publ Int Cancer Imaging Soc 13(4):611–625Google Scholar
  27. 27.
    Nakamoto M, Nakada K, Sato Y, Konishi K, Hashizume M, Tamura S (2008) Intraoperative magnetic tracker calibration using a magneto-optic hybrid tracker for 3-D ultrasound-based navigation in laparoscopic surgery. IEEE Trans Med Imaging 27(2):255–270CrossRefPubMedGoogle Scholar
  28. 28.
    Hughes-Hallett A, Mayer EK, Marcus HJ, Cundy TP, Pratt PJ, Darzi AW et al (2014) Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 83(2):266–273CrossRefPubMedGoogle Scholar
  29. 29.
    Simpfendörfer T, Gasch C, Hatiboglu G, Müller M, Maier-Hein L, Hohenfellner M et al (2016) Intraoperative computed tomography imaging for navigated laparoscopic renal surgery: first clinical experience. J Endourol 30(10):1105–1111CrossRefPubMedGoogle Scholar
  30. 30.
    Allen BC, Tirman P, Jennings Clingan M, Manny J, Del Gaizo AJ, Leyendecker JR (2014) Characterizing solid renal neoplasms with MRI in adults. Abdom Imaging 39(2):358–387CrossRefPubMedGoogle Scholar
  31. 31.
    Fananapazir G, Lamba R, Lewis B, Corwin MT, Naderi S, Troppmann C (2015) Utility of MRI in the characterization of indeterminate small renal lesions previously seen on screening CT scans of potential renal donor patients. AJR Am J Roentgenol 205(2):325–330CrossRefPubMedGoogle Scholar
  32. 32.
    Paulus CJ, Haouchine N, Kong S-H, Soares RV, Cazier D, Cotin S (2017) Handling topological changes during elastic registration: application to augmented reality in laparoscopic surgery. Int J Comput Assist Radiol Surg 12(3):461–470Google Scholar
  33. 33.
    Nakamura K, Naya Y, Zenbutsu S, Araki K, Cho S, Ohta S et al (2010) Surgical navigation using three-dimensional computed tomography images fused intraoperatively with live video. J Endourol 24(4):521–524CrossRefPubMedGoogle Scholar
  34. 34.
    Ukimura O, Gill IS (2008) Imaging-assisted endoscopic surgery: cleveland clinic experience. J Endourol 22(4):803–810CrossRefPubMedGoogle Scholar
  35. 35.
    Altamar HO, Ong RE, Glisson CL, Viprakasit DP, Miga MI, Herrell SD et al (2011) Kidney deformation and intraprocedural registration: a study of elements of image-guided kidney surgery. J Endourol 25(3):511–517CrossRefPubMedGoogle Scholar
  36. 36.
    Herrell SD, Kwartowitz DM, Milhoua PM, Galloway RL (2009) Toward image guided robotic surgery: system validation. J Urol 181(2):783–790CrossRefPubMedGoogle Scholar
  37. 37.
    Benincasa AB, Clements LW, Herrell SD, Galloway RL (2008) Feasibility study for image‐guided kidney surgery: Assessment of required intraoperative surface for accurate physical to image space registrations. Med Phys 35(9):4251–4261CrossRefPubMedPubMedCentralGoogle Scholar
  38. 38.
    Nakamoto M, Ukimura O, Gill IS, Mahadevan A, Miki T, Hashizume M et al (2008) Realtime organ tracking for endoscopic augmented reality visualization using miniature wireless magnetic tracker. In: Medical imaging and augmented reality. Springer-Verlag, Heidelberg, German, pp 359–366Google Scholar
  39. 39.
    Baumhauer M, Simpfendörfer T, Müller-Stich BP, Teber D, Gutt CN, Rassweiler J et al (2008) Soft tissue navigation for laparoscopic partial nephrectomy. Int J Comput Assist Radiol Surg 3(3-4):307–314CrossRefGoogle Scholar
  40. 40.
    Pratt P, Mayer E, Vale J, Cohen D, Edwards E, Darzi A et al (2012) An effective visualisation and registration system for image-guided robotic partial nephrectomy. J Robot Surg 6(1):23–31CrossRefPubMedGoogle Scholar
  41. 41.
    Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4):896–900CrossRefPubMedGoogle Scholar
  42. 42.
    Nosrati MS, Amir-Khalili A, Peyrat JM, Abinahed J, Al-Alao O, Al-Ansari A et al (2016) Endoscopic scene labelling and augmentation using intraoperative pulsatile motion and colour appearance cues with preoperative anatomical priors. Int J Comput Assist Radiol Surg 11(8):1409–1418CrossRefPubMedGoogle Scholar
  43. 43.
    Puerto-Souza GA, Cadeddu JA, Mariottini G-L (2014) Toward long-term and accurate augmented-reality for monocular endoscopic videos. IEEE Trans Biomed Eng 61(10):2609–2620Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of Gynecologic SurgeryClermont-Ferrand University Hospital EstaingClermont-FerrandFrance
  2. 2.Faculté de MédecineALCoV UMR6284, CNRS/Université d’Auvergne, ISITClermont-FerrandFrance
  3. 3.Faculté de MédecineCICS Centre Imagerie Cellulaire SantéClermont-FerrandFrance
  4. 4.Biostatistics Unit (DRCI)University Hospital Clermont-FerrandClermont-Ferrand CedexFrance

Personalised recommendations