Skip to main content
SpringerLink
Log in

A survey of augmented reality methods to guide minimally invasive partial nephrectomy

  • Topic Paper
  • Published:
World Journal of Urology Aims and scope Submit manuscript

Abstract

Introduction

Minimally invasive partial nephrectomy (MIPN) has become the standard of care for localized kidney tumors over the past decade. The characteristics of each tumor, in particular its size and relationship with the excretory tract and vessels, allow one to judge its complexity and to attempt predicting the risk of complications. The recent development of virtual 3D model reconstruction and computer vision has opened the way to image-guided surgery and augmented reality (AR).

Objective

Our objective was to perform a systematic review to list and describe the different AR techniques proposed to support PN.

Materials and methods

The systematic review of the literature was performed on 12/04/22, using the keywords "nephrectomy" and "augmented reality" on Embase and Medline. Articles were considered if they reported surgical outcomes when using AR with virtual image overlay on real vision, during ex vivo or in vivo MIPN. We classified them according to the registration technique they use.

Results

We found 16 articles describing an AR technique during MIPN procedures that met the eligibility criteria. A moderate to high risk of bias was recorded for all the studies. We classified registration methods into three main families, of which the most promising one seems to be surface-based registration.

Conclusion

Despite promising results, there do not exist studies showing an improvement in clinical outcomes using AR. The ideal AR technique is probably yet to be established, as several designs are still being actively explored. More clinical data will be required to establish the potential contribution of this technology to MIPN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. F. Kunath et al., Partial nephrectomy versus radical nephrectomy for clinical localised renal masses », Cochrane Database Syst Rev, vol. 5, p. CD012045, mai 2017, https://doi.org/10.1002/14651858.CD012045.pub2.

  2. P. Zeuschner et al., « Open versus robot-assisted partial nephrectomy: A longitudinal comparison of 880 patients over 10 years », Int J Med Robot, vol. 17, no 1, p. 1‑8, févr. 2021, doi: https://doi.org/10.1002/rcs.2167.

  3. S.-H. Tsai et al., « Open versus robotic partial nephrectomy: Systematic review and meta-analysis of contemporary studies », Int J Med Robotics Comput Assist Surg, vol. 15, no 1, p. e1963, févr. 2019, doi: https://doi.org/10.1002/rcs.1963.

  4. N. M. Buffi et al., « Robot-assisted Partial Nephrectomy for Complex (PADUA Score ≥10) Tumors: Techniques and Results from a Multicenter Experience at Four High-volume Centers », European Urology, vol. 77, no 1, p. 95‑100, janv. 2020, doi: https://doi.org/10.1016/j.eururo.2019.03.006.

  5. A. J. Hung, J. Cai, M. N. Simmons, et I. S. Gill, « “Trifecta” in Partial Nephrectomy », Journal of Urology, vol. 189, no 1, p. 36‑42, janv. 2013, doi: https://doi.org/10.1016/j.juro.2012.09.042.

  6. A. Larcher et al., « The Learning Curve for Robot-assisted Partial Nephrectomy: Impact of Surgical Experience on Perioperative Outcomes », European Urology, vol. 75, no 2, p. 253‑256, févr. 2019, doi: https://doi.org/10.1016/j.eururo.2018.08.042.

  7. F. Porpiglia, C. Fiori, E. Checcucci, D. Amparore, et R. Bertolo, « Hyperaccuracy Three-dimensional Reconstruction Is Able to Maximize the Efficacy of Selective Clamping During Robot-assisted Partial Nephrectomy for Complex Renal Masses », European Urology, vol. 74, no 5, p. 651‑660, nov. 2018, doi: https://doi.org/10.1016/j.eururo.2017.12.027.

  8. C. Michiels et al., « 3D-Image guided robotic-assisted partial nephrectomy: a multi-institutional propensity score-matched analysis (UroCCR study 51) », World Journal of Urology, avr. 2021, doi: https://doi.org/10.1007/s00345-021-03645-1.

  9. Y. Sun, W. Wang, Q. Zhang, X. Zhao, L. Xu, et H. Guo, « Intraoperative ultrasound: technique and clinical experience in robotic-assisted renal partial nephrectomy for endophytic renal tumors », Int Urol Nephrol, oct. 2020, doi: https://doi.org/10.1007/s11255-020-02664-y.

  10. G. Di Cosmo et al., « Intraoperative ultrasound in robot-assisted partial nephrectomy: State of the art », Arch Ital Urol Androl, vol. 90, no 3, p. 195‑198, sept. 2018, doi: https://doi.org/10.4081/aiua.2018.3.195.

  11. A. Hughes-Hallett et al., « Augmented Reality Partial Nephrectomy: Examining the Current Status and Future Perspectives », Urology, vol. 83, no 2, p. 266‑273, févr. 2014, doi: https://doi.org/10.1016/j.urology.2013.08.049.

  12. G. Teluob et al., « Preliminary Trial of Augmented Reality Performed on a Regular and a Robot-Assisted Laparoscopic Partial Nephrectomies », Videourology, vol. 33, no 3, p. vid.2019.0004, juin 2019, doi: https://doi.org/10.1089/vid.2019.0004.

  13. M. S. Nosrati et al., « Endoscopic scene labelling and augmentation using intraoperative pulsatile motion and colour appearance cues with preoperative anatomical priors », Int J CARS, vol. 11, no 8, p. 1409‑1418, août 2016, doi: https://doi.org/10.1007/s11548-015-1331-x.

  14. F. Esperto et al., « New Technologies for Kidney Surgery Planning 3D, Impression, Augmented Reality 3D, Reconstruction: Current Realities and Expectations », Curr Urol Rep, vol. 22, no 7, p. 35, mai 2021, doi: https://doi.org/10.1007/s11934-021-01052-y.

  15. S. Roberts et al., « “Augmented reality” applications in urology: a systematic review », Minerva Urol Nephrol, avr. 2022, doi: https://doi.org/10.23736/S2724-6051.22.04726-7.

  16. M. Cumpston et al., « Updated guidance for trusted systematic reviews: a new edition of the Cochrane Handbook for Systematic Reviews of Interventions », Cochrane Database Syst Rev, vol. 10, p. ED000142, oct. 2019, doi: https://doi.org/10.1002/14651858.ED000142.

  17. PRISMA-P Group et al., « Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement », Syst Rev, vol. 4, no 1, p. 1, déc. 2015, doi: https://doi.org/10.1186/2046-4053-4-1.

  18. J. A. Sterne et al., « ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions », BMJ, vol. 355, p. i4919, oct. 2016, doi: https://doi.org/10.1136/bmj.i4919.

  19. S. D. Herrell, D. M. Kwartowitz, P. M. Milhoua, et R. L. Galloway, « Toward Image Guided Robotic Surgery: System Validation », Journal of Urology, vol. 181, no 2, p. 783‑790, févr. 2009, doi: https://doi.org/10.1016/j.juro.2008.10.022.

  20. D. Hutchison et al., « Fused Video and Ultrasound Images for Minimally Invasive Partial Nephrectomy: A Phantom Study », in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010, vol. 6363, T. Jiang, N. Navab, J. P. W. Pluim, et M. A. Viergever, Éd. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, p. 408‑415. doi: https://doi.org/10.1007/978-3-642-15711-0_51.

  21. P. Chauvet et al., « Augmented reality in a tumor resection model », Surg Endosc, vol. 32, no 3, p. 1192‑1201, mars 2018, doi: https://doi.org/10.1007/s00464-017-5791-7.

  22. R. Singla, P. Edgcumbe, P. Pratt, C. Nguan, et R. Rohling, « Intra‐operative ultrasound‐based augmented reality guidance for laparoscopic surgery », Healthc. technol. lett., vol. 4, no 5, p. 204‑209, oct. 2017, doi: https://doi.org/10.1049/htl.2017.0063.

  23. P. Edgcumbe, R. Singla, P. Pratt, C. Schneider, C. Nguan, et R. Rohling, « Follow the light: projector-based augmented reality intracorporeal system for laparoscopic surgery », J. Med. Imag., vol. 5, no 02, p. 1, févr. 2018, doi: https://doi.org/10.1117/1.JMI.5.2.021216.

  24. O. Ukimura et I. S. Gill, « Imaging-Assisted Endoscopic Surgery: Cleveland Clinic Experience », Journal of Endourology, vol. 22, no 4, p. 803‑810, avr. 2008, doi: https://doi.org/10.1089/end.2007.9823.

  25. D. Teber et al., « Augmented Reality: A New Tool To Improve Surgical Accuracy during Laparoscopic Partial Nephrectomy? Preliminary In Vitro and In Vivo Results », European Urology, vol. 56, no 2, p. 332‑338, août 2009, doi: https://doi.org/10.1016/j.eururo.2009.05.017.

  26. K. Nakamura et al., « Surgical Navigation Using Three-Dimensional Computed Tomography Images Fused Intraoperatively with Live Video <sup/> », Journal of Endourology, vol. 24, no 4, p. 521‑524, avr. 2010, doi: https://doi.org/10.1089/end.2009.0365.

  27. Y. Chen, H. Li, D. Wu, K. Bi, et C. Liu, « Surgical planning and manual image fusion based on 3D model facilitate laparoscopic partial nephrectomy for intrarenal tumors », World J Urol, vol. 32, no 6, p. 1493‑1499, déc. 2014, doi: https://doi.org/10.1007/s00345-013-1222-0.

  28. T. Simpfendörfer et al., « Augmented Reality Visualization During Laparoscopic Radical Prostatectomy », Journal of Endourology, vol. 25, no 12, p. 1841‑1845, déc. 2011, doi: https://doi.org/10.1089/end.2010.0724.

  29. R. Schiavina et al., « Augmented Reality to Guide Selective Clamping and Tumor Dissection During Robot-assisted Partial Nephrectomy: A Preliminary Experience », Clinical Genitourinary Cancer, vol. 19, no 3, p. e149‑e155, juin 2021, doi: https://doi.org/10.1016/j.clgc.2020.09.005.

  30. D. Amparore et al., « Indocyanine Green Drives Computer Vision Based 3D Augmented Reality Robot Assisted Partial Nephrectomy: The Beginning of “Automatic” Overlapping Era », Urology, p. S0090429522000292, janv. 2022, doi: https://doi.org/10.1016/j.urology.2021.10.053.

  31. D. Amparore et al., « Identification of Recurrent Anatomical Clusters Using Three-dimensional Virtual Models for Complex Renal Tumors with an Imperative Indication for Nephron-sparing Surgery: New Technological Tools for Driving Decision-making », European Urology Open Science, vol. 38, p. 60‑66, avr. 2022, doi: https://doi.org/10.1016/j.euros.2022.02.006.

  32. F. Piramide et al., « Augmented reality 3D robot-assisted partial nephrectomy: Tips and tricks to improve surgical strategies and outcomes », Urology Video Journal, vol. 13, p. 100137, mars 2022, doi: https://doi.org/10.1016/j.urolvj.2022.100137.

  33. F. Porpiglia et al., « Three-dimensional Augmented Reality Robot-assisted Partial Nephrectomy in Case of Complex Tumours (PADUA ≥10): A New Intraoperative Tool Overcoming the Ultrasound Guidance », European Urology, vol. 78, no 2, p. 229‑238, août 2020, doi: https://doi.org/10.1016/j.eururo.2019.11.024.

  34. C. Michiels, E. Jambon, et J. C. Bernhard, « Measurement of the Accuracy of 3D-Printed Medical Models to Be Used for Robot-Assisted Partial Nephrectomy », American Journal of Roentgenology, vol. 213, no 3, p. 626‑631, sept. 2019, doi: https://doi.org/10.2214/AJR.18.21048.

  35. C. M. Andrews, A. B. Henry, I. M. Soriano, M. K. Southworth, et J. R. Silva, « Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices », IEEE J Transl Eng Health Med, vol. 9, p. 4900214, 2021, doi: https://doi.org/10.1109/JTEHM.2020.3045642.

  36. J. S. Lam, J. Bergman, A. Breda, et P. G. Schulam, « Importance of surgical margins in the management of renal cell carcinoma », Nat Rev Urol, vol. 5, no 6, p. 308‑317, juin 2008, doi: https://doi.org/10.1038/ncpuro1121.

  37. J. Connor et al., « Postoperative Complications After Robotic Partial Nephrectomy », J Endourol, vol. 34, no 1, p. 42‑47, janv. 2020, doi: https://doi.org/10.1089/end.2019.0434.

  38. M. B. Patil, D. J. Lee, et I. S. Gill, « Eliminating global renal ischemia during partial nephrectomy: an anatomical approach », Curr Opin Urol, vol. 22, no 2, p. 83‑87, mars 2012, doi: https://doi.org/10.1097/MOU.0b013e32834ef70c.

  39. C. Xu, C. Lin, Z. Xu, S. Feng, et Y. Zheng, « Tumor Enucleation vs. Partial Nephrectomy for T1 Renal Cell Carcinoma: A Systematic Review and Meta-Analysis », Front Oncol, vol. 9, p. 473, 2019, doi: https://doi.org/10.3389/fonc.2019.00473.

  40. M. Baumhauer et al., « Soft tissue navigation for laparoscopic partial nephrectomy », Int J CARS, vol. 3, no 3‑4, p. 307‑314, sept. 2008, doi: https://doi.org/10.1007/s11548-008-0216-7.

  41. F. Joeres, T. Mielke, et C. Hansen, « Laparoscopic augmented reality registration for oncological resection site repair », Int J CARS, vol. 16, no 9, p. 1577‑1586, sept. 2021, doi: https://doi.org/10.1007/s11548-021-02336-x.

  42. H. O. Altamar et al., « Kidney Deformation and Intraprocedural Registration: A Study of Elements of Image-Guided Kidney Surgery », Journal of Endourology, vol. 25, no 3, p. 511‑517, mars 2011, doi: https://doi.org/10.1089/end.2010.0249.

  43. S. Madad Zadeh et al., « SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology », Surg Endosc, vol. 34, no 12, p. 5377‑5383, déc. 2020, doi: https://doi.org/10.1007/s00464-019-07330-8.

  44. T. Jia, Z. A. Taylor, et X. Chen, « Long term and robust 6DoF motion tracking for highly dynamic stereo endoscopy videos », Computerized Medical Imaging and Graphics, vol. 94, p. 101995, déc. 2021, doi: https://doi.org/10.1016/j.compmedimag.2021.101995.

  45. L.-M. Su, B. P. Vagvolgyi, R. Agarwal, C. E. Reiley, R. H. Taylor, et G. D. Hager, « Augmented Reality During Robot-assisted Laparoscopic Partial Nephrectomy: Toward Real-Time 3D-CT to Stereoscopic Video Registration », Urology, vol. 73, no 4, p. 896‑900, avr. 2009, doi: https://doi.org/10.1016/j.urology.2008.11.040.

  46. X. Zhang et al., « A markerless automatic deformable registration framework for augmented reality navigation of laparoscopy partial nephrectomy », Int J CARS, vol. 14, no 8, p. 1285‑1294, août 2019, doi: https://doi.org/10.1007/s11548-019-01974-6.

  47. D. Stoyanov, M. V. Scarzanella, P. Pratt, et G.-Z. Yang, « Real-time stereo reconstruction in robotically assisted minimally invasive surgery », Med Image Comput Comput Assist Interv, vol. 13, no Pt 1, p. 275‑282, 2010, doi: https://doi.org/10.1007/978-3-642-15705-9_34.

  48. P. Pratt et al., « An effective visualisation and registration system for image-guided robotic partial nephrectomy », J Robotic Surg, vol. 6, no 1, p. 23‑31, mars 2012, doi: https://doi.org/10.1007/s11701-011-0334-z.

  49. R. Anteby et al., « Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis », Surg Endosc, vol. 35, no 4, p. 1521‑1533, avr. 2021, doi: https://doi.org/10.1007/s00464-020-08168-1.

  50. T. François et al., « Detecting the occluding contours of the uterus to automatise augmented laparoscopy: score, loss, dataset, evaluation and user study », Int J CARS, vol. 15, no 7, p. 1177‑1186, juill. 2020, doi: https://doi.org/10.1007/s11548-020-02151-w.

  51. T. Collins et al., « Augmented Reality Guided Laparoscopic Surgery of the Uterus », IEEE Trans Med Imaging, vol. 40, no 1, p. 371‑380, janv. 2021, doi: https://doi.org/10.1109/TMI.2020.3027442.

  52. N. Bourdel et al., (2017) Augmented reality in gynecologic surgery: evaluation of potential benefits for myomectomy in an experimental uterine model. Surg Endosc. 31(1): 456‑461. doi: https://doi.org/10.1007/s00464-016-4932-8.

  53. Nosrati MS et al (2016) Simultaneous multi-structure segmentation and 3D nonrigid pose estimation in image-guided robotic surgery. IEEE Trans Med Imaging 35(1):1–12. https://doi.org/10.1109/TMI.2015.2452907

    Article  PubMed  Google Scholar 

  54. Zhang X, Wang T, Zhang X, Zhang Y, Wang J (2020) Assessment and application of the coherent point drift algorithm to augmented reality surgical navigation for laparoscopic partial nephrectomy. Int J CARS 15(6):989–999. https://doi.org/10.1007/s11548-020-02163-6

    Article  Google Scholar 

Download references

Acknowledgements

No acknowledgements.

Funding

This research received no external funding.

Author information

Authors and Affiliations

  1. Department of Urology, Hôpital Pellegrin, Bordeaux University Hospital, Bordeaux, France

    Abderrahmane Khaddad, Jean-Christophe Bernhard, Gaëlle Margue, Clément Michiels & Franck Bladou

  2. French Research Network On Kidney Cancer UroCCR, Bordeaux, France

    Jean-Christophe Bernhard & Solène Ricard

  3. Institut Pascal, UMR6602 CNRS, UCA, Clermont-Ferrand University Hospital, Clermont-Ferrand, France

    Kilian Chandelon, Nicolas Bourdel & Adrien Bartoli

  4. Department of Obstetrics and Gynecology, Clermont-Ferrand University Hospital, Clermont-Ferrand, France

    Nicolas Bourdel

  5. Department of Clinical Research and Innovation, Clermont-Ferrand University Hospital, Clermont-Ferrand, France

    Adrien Bartoli

Authors
  1. Abderrahmane Khaddad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Christophe Bernhard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gaëlle Margue
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Clément Michiels
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Solène Ricard
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kilian Chandelon
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Franck Bladou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Nicolas Bourdel
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Adrien Bartoli
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AK: project development, methodology, data collection, data analysis, manuscript writing. JCB: project development, methodology, manuscript editing, supervision. GM: methodology, data collection, data analysis. CM: project development. SR: project development. KC: project development, data analysis. FB: manuscript editing. NB: project development, methodology, supervision. AB: project development, methodology, manuscript editing, supervision. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Abderrahmane Khaddad.

Ethics declarations

Conflicts of interest

Nicolas Bourdel is the CEO of the SurgAR company. Kilian Chandelon is an R&D engineer at the SurgAR company. Adrien Bartoli is the CSO of the SurgAR company.

Informed consent statement

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 13 KB)

Supplementary file2 (DOCX 14 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khaddad, A., Bernhard, JC., Margue, G. et al. A survey of augmented reality methods to guide minimally invasive partial nephrectomy. World J Urol 41, 335–343 (2023). https://doi.org/10.1007/s00345-022-04078-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00345-022-04078-0

Keywords

Search

Navigation