Skip to main content
SpringerLink
Log in

Minimally invasive and invasive liver surgery based on augmented reality training: a review of the literature

  • Review Article
  • Published:
Journal of Robotic Surgery Aims and scope Submit manuscript

Abstract

A liver surgeon's knowledge of anatomy is critical. Due to the patient's small field of vision, patient specific, complex nerve system, and other factors, even a minor loss can result in irreversible damage. Surgeons could benefit from the use of augmented reality (AR) technology, which would bring three-dimensional image data into the operating room. AR visualization can improve surgical procedures, facilitate intraoperative planning, and enhance surgical guidance for the anatomy of interest, all of which contribute to the application's minimal invasiveness. This literature review on image guidance in liver surgery provides the reader with information about AR techniques. To ascertain the current state of Augmented reality technology's application in liver surgery, a PubMed and Embase search were conducted using the following keywords: < (Augmented reality) AND (liver surgery) > and < ‘Augmented reality’ AND ‘liver surgery’ > (publication date from January 1991 until Jun 2022). The query yielded a total of 205 publications—excluded papers in other languages, virtual reality (VR), and reviews leaving 135 studies for review. After removing duplication, the titles and abstracts of those studies were manually reviewed. Finally, 31 pertinent studies were determined to be pertinent to the subject. Generally, augmented reality technology includes preoperative planning and three-dimensional reconstruction, intraoperative three-dimensional navigation, and registration. Visualization may be aided by virtual three-dimensional reconstruction models of the liver from Computed Tomography/Magnetic Resonance Imaging scans. The results demonstrate that by utilizing augmented reality technology, blood vessels and tumor structures in the liver can be visualized during surgery, allowing for precise navigation during complicated surgical procedures. Augmented reality has been demonstrated to be safe and effective in both minimally invasive and invasive liver surgery. With recent advancements and significant effort by liver surgeons, augmented reality technologies have the potential to increase hepatobiliary surgical procedures dramatically. However, further clinical trials will be necessary to evaluate augmented reality as a tool for reducing post-operative morbidity and mortality. The impact of these cutting-edge computerized image guidance techniques on clinically relevant outcome parameters should be assessed in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Beller S, Hünerbein M, Lange T et al (2007) Image-guided surgery of liver metastases by three-dimensional ultrasound-based optoelectronic navigation. Br J Surg 94:866–875. https://doi.org/10.1002/bjs.5712

    Article  CAS  PubMed  Google Scholar 

  2. Santambrogio R, Vertemati M, Picardi E, Zappa M (2022) Planning the treatment: preoperative 3D reconstruction. Laparoscopic Surgery 6. https://doi.org/10.21037/ls-22-6

  3. Douglas D, Wilke C, Gibson D et al (2017) Virtual reality and augmented reality: Advances in surgery. Biol Eng Med 3:1–8. https://doi.org/10.15761/BEM.1000131

  4. Quero G, Lapergola A, Soler L et al (2019) Virtual and augmented reality in oncologic liver surgery. Surg Oncol Clin N Am 28:31–44. https://doi.org/10.1016/j.soc.2018.08.002

    Article  PubMed  Google Scholar 

  5. Tinguely P, Fusaglia M, Freedman J et al (2017) Laparoscopic image-based navigation for microwave ablation of liver tumors-A multi-center study. Surg Endosc 31:4315–4324. https://doi.org/10.1007/s00464-017-5458-4

    Article  PubMed  Google Scholar 

  6. Hashimoto D, Dohi T, Tsuzuki M et al (1991) Development of a computer-aided surgery system: three-dimensional graphic reconstruction for treatment of liver cancer. Surgery 109:589–596. https://doi.org/10.5555/uri:pii:0039606091903894

    Article  CAS  PubMed  Google Scholar 

  7. Yasuda J, Okamoto T, Onda S et al (2018) Novel navigation system by augmented reality technology using a tablet PC for hepatobiliary and pancreatic surgery. Int J Med Robot 14:e1921. https://doi.org/10.1002/rcs.1921

    Article  PubMed  Google Scholar 

  8. Tang R, Ma L, Xiang C et al (2017) Augmented reality navigation in open surgery for hilar cholangiocarcinoma resection with hemihepatectomy using video-based in situ three-dimensional anatomical modeling: a case report. Medicine 96:e8083. https://doi.org/10.1097/MD.0000000000008083

    Article  PubMed  PubMed Central  Google Scholar 

  9. Hagopian EJ (2020) Liver ultrasound: A key procedure in the surgeon’s toolbox. J Surg Oncol 122:61–69. https://doi.org/10.1002/jso.25908

    Article  PubMed  Google Scholar 

  10. Shin J, Lee S, Yoon JK et al (2021) LI-RADS major features on MRI for diagnosing hepatocellular carcinoma: a systematic review and meta-analysis. J Magn Reson Imaging 54:518–525. https://doi.org/10.1002/jmri.27570

    Article  PubMed  Google Scholar 

  11. Willatt J, Ruma JA, Azar SF, Dasika NL, Syed F (2017) Imaging of hepatocellular carcinoma and image guided therapies—how we do it. Cancer Imaging 17:9. https://doi.org/10.1186/s40644-017-0110-z

    Article  PubMed  PubMed Central  Google Scholar 

  12. Aoki T, Mansour DA, Koizumi T et al (2021) Laparoscopic liver surgery guided by virtual real-time CT-guided volume navigation. J Gastrointest Surg 25:1779–1786. https://doi.org/10.1007/s11605-020-04784-3

    Article  PubMed  Google Scholar 

  13. Mutter D, Soler L, Marescaux J (2010) Recent advances in liver imaging. Expert Rev Gastroenterol Hepatol 4:613–621. https://doi.org/10.1586/egh.10.57

    Article  CAS  PubMed  Google Scholar 

  14. Volonté F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P (2011) Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 18:506–509. https://doi.org/10.1007/s00534-011-0385-6

    Article  PubMed  Google Scholar 

  15. Onda S, Okamoto T, Kanehira M et al (2013) Short rigid scope and stereo-scope designed specifically for open abdominal navigation surgery: clinical application for hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci 20:448–453. https://doi.org/10.1007/s00534-012-0582-y

    Article  PubMed  Google Scholar 

  16. Okamoto T, Onda S, Matsumoto M et al (2013) Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci 20:249–253. https://doi.org/10.1007/s00534-012-0504-z

    Article  PubMed  Google Scholar 

  17. Hallet J, Soler L, Diana M et al (2015) Trans-thoracic minimally invasive liver resection guided by augmented reality. J Am Coll Surg 220:55–60. https://doi.org/10.1016/j.jamcollsurg.2014.12.053

    Article  Google Scholar 

  18. Ntourakis D, Memeo R, Soler L, Marescaux J, Mutter D, Pessaux P (2016) Augmented reality guidance for the resection of missing colorectal liver metastases: an initial experience. World J Surg 40:419–426. https://doi.org/10.1007/s00268-015-3229-8

    Article  PubMed  Google Scholar 

  19. Banz VM, Müller PC, Tinguely P et al (2016) Intraoperative image-guided navigation system: development and applicability in 65 patients undergoing liver surgery. Langenbecks Arch Surg 401:495–502. https://doi.org/10.1007/s00423-016-1417-0

    Article  PubMed  Google Scholar 

  20. Luo H, Yin D, Zhang S et al (2019) Augmented reality navigation for liver resection with a stereoscopic laparoscope. Comput Methods Programs Biomed 187:105099. https://doi.org/10.1016/j.cmpb.2019.105099

    Article  PubMed  Google Scholar 

  21. Schneider C, Thompson S, Totz J et al (2020) Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study. Surg Endosc 34:4702–4711. https://doi.org/10.1007/s00464-020-07807-x

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Prevost GA, Eigl B, Paolucci I et al (2020) Efficiency, accuracy and clinical applicability of a new image-guided surgery system in 3d laparoscopic liver surgery. J Gastrointest Surg 24:2251–2258. https://doi.org/10.1007/s11605-019-04395-7

    Article  PubMed  Google Scholar 

  23. Pelanis E, Teatini A, Eigl B et al (2021) Evaluation of a novel navigation platform for laparoscopic liver surgery with organ deformation compensation using injected fiducials. Med Image Anal 69:101946. https://doi.org/10.1016/j.media.2020.101946

    Article  PubMed  Google Scholar 

  24. Golse N, Petit A, Lewin M et al (2021) Augmented reality during open liver surgery using a markerless non-rigid registration system. J Gastrointest Surg 25:662–671. https://doi.org/10.1007/s11605-020-04519-4

    Article  PubMed  Google Scholar 

  25. Solbiati M, Ierace T, Muglia R et al (2022) Thermal ablation of liver tumors guided by augmented reality: an initial clinical experience. Cancers 14:1312. https://doi.org/10.3390/cancers14051312

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Koo B, Robu MR, Allam M (2022) Global registration in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 17:167–176. https://doi.org/10.1007/s11548-021-02518-7

    Article  PubMed  Google Scholar 

  27. Adballah M, Espinel Y, Calvet L et al (2022) Augmented reality in laparoscopic liver resection evaluated on an ex-vivo animal model with pseudo-tumours. Surg Endosc 36:833–843. https://doi.org/10.1007/s00464-021-08798-z

    Article  PubMed  Google Scholar 

  28. Peterhans M, vom Berg A, Dagon B et al (2011) A navigation system for open liver surgery: design, workflow and first clinical applications. Int J Med Robot 7:7–16. https://doi.org/10.1002/rcs.360

    Article  CAS  PubMed  Google Scholar 

  29. Conrad C, Fusaglia M, Peterhans M et al (2016) Augmented reality navigation surgery facilitates laparoscopic rescue of failed portal vein embolization. J Am Coll Surg 223:31–34. https://doi.org/10.1016/j.jamcollsurg.2016.06.392

    Article  Google Scholar 

  30. Koo B, Özgür E, Le Roy B, Buc E, Bartoli A (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. International Conference on Medical Image Computing and Computer-Assisted Intervention pp 326–334. https://doi.org/10.1007/978-3-319-66182-7_38

  31. Pfeiffer M, Funke I, Robu MR et al (2019) Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation. International Conference on Medical Image Computing and Computer-Assisted Intervention pp 1–8. https://doi.org/10.1007/978-3-030-32254-0_14

  32. Espinel Y, Özgür E, Calvet L et al (2020) Combining visual cues with interactions for 3d–2d registration in liver laparoscopy. Ann Biomed Eng 48:1712–1727. https://doi.org/10.1007/s10439-020-02479-z

    Article  PubMed  Google Scholar 

  33. López-Mir F, Naranjo V, Fuertes JJ, Alcañiz M, Bueno J, Pareja E (2013) Design and validation of an augmented reality system for laparoscopic surgery in a real environment. Biomed Res Int 2013:758491. https://doi.org/10.1155/2013/758491

    Article  PubMed  PubMed Central  Google Scholar 

  34. Kang X, Azizian M, Wilson E et al (2014) Stereoscopic augmented reality for laparoscopic surgery. Surg Endosc 28:2227–2235. https://doi.org/10.1007/s00464-014-3433-x

    Article  PubMed  Google Scholar 

  35. Labrunie M, Ribeiro M, Mourthadhoi F et al (2022) Automatic preoperative 3d model registration in laparoscopic liver resection. Int J Comput Assist Radiol Surg 17:1429–1436. https://doi.org/10.1007/s11548-022-02641-z

    Article  CAS  PubMed  Google Scholar 

  36. Pelanis E, Kumar RP, Aghayan DL et al (2020) Use of mixed reality for improved spatial understanding of liver anatomy. Minim Invasive Ther Allied Technol 29:154–160. https://doi.org/10.1080/13645706.2019.1616558

    Article  PubMed  Google Scholar 

  37. Condino S, Carbone M, Piazza R, Ferrari M, Ferrari V (2020) Perceptual limits of optical see-through visors for augmented reality guidance of manual tasks. IEEE Trans Biomed Eng 67:411–419. https://doi.org/10.1109/TBME.2019.2914517

    Article  PubMed  Google Scholar 

  38. Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20:189–201. https://doi.org/10.1016/j.suronc.2011.07.002

    Article  PubMed  Google Scholar 

  39. Vosburgh KG, San José Estépar R (2007) Natural orifice transluminal endoscopic surgery (NOTES): an opportunity for augmented reality guidance. Stud Health Technol Inform 125:485–490

    PubMed  Google Scholar 

  40. Soler L, Nicolau S, Hostettler A et al (2009) Virtual reality and augmented reality applied to endoscopic and notes procedures. World Congress Med Phys Biomed Eng Munich Germany 25:6. https://doi.org/10.1007/978-3-642-03906-5_101

    Article  Google Scholar 

Download references

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. University Medicine Greifswald, Greifswald, Germany

    Maryam Gholizadeh

  2. Mashhad University of Medical Sciences, Mashhad, Islamic Republic of Iran

    Mohamad Amin Bakhshali, Mohsen Aliakbarian, Farzaneh Gholizadeh & Saeid Eslami

  3. Islamic Azad University, Tehran, Islamic Republic of Iran

    Seyed Reza Mazlooman

  4. Pomeranian Medical University, Szczecin, Poland

    Andrzej Modrzejewski

Authors
  1. Maryam Gholizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamad Amin Bakhshali
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seyed Reza Mazlooman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohsen Aliakbarian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Farzaneh Gholizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Saeid Eslami
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andrzej Modrzejewski
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by M.A.Bakhshali, F.Gholizadeh and S.R.Mazlooman. The first draft of the manuscript was written by M.Gholizadeh, S.Eslami, A.Modrzejewski, M.Aliakbarian commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mohamad Amin Bakhshali.

Ethics declarations

Conflict of interests

The authors have no conflicts of interest to declare. All co-authors have seen and agree with the contents of the manuscript and there is no financial interest to report. We certify that the submission is original work and is not under review at any other publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gholizadeh, M., Bakhshali, M.A., Mazlooman, S.R. et al. Minimally invasive and invasive liver surgery based on augmented reality training: a review of the literature. J Robotic Surg 17, 753–763 (2023). https://doi.org/10.1007/s11701-022-01499-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11701-022-01499-2

Keywords

Search

Navigation