Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy



Augmented reality (AR) in surgery consists in the fusion of synthetic computer-generated images (3D virtual model) obtained from medical imaging preoperative workup and real-time patient images in order to visualize unapparent anatomical details. The 3D model could be used for a preoperative planning of the procedure. The potential of AR navigation as a tool to improve safety of the surgical dissection is outlined for robotic hepatectomy.

Materials and methods

Three patients underwent a fully robotic and AR-assisted hepatic segmentectomy. The 3D virtual anatomical model was obtained using a thoracoabdominal CT scan with a customary software (VR-RENDER®, IRCAD). The model was then processed using a VR-RENDER® plug-in application, the Virtual Surgical Planning (VSP®, IRCAD), to delineate surgical resection planes including the elective ligature of vascular structures. Deformations associated with pneumoperitoneum were also simulated. The virtual model was superimposed to the operative field. A computer scientist manually registered virtual and real images using a video mixer (MX 70; Panasonic, Secaucus, NJ) in real time.


Two totally robotic AR segmentectomy V and one segmentectomy VI were performed. AR allowed for the precise and safe recognition of all major vascular structures during the procedure. Total time required to obtain AR was 8 min (range 6–10 min). Each registration (alignment of the vascular anatomy) required a few seconds. Hepatic pedicle clamping was never performed. At the end of the procedure, the remnant liver was correctly vascularized. Resection margins were negative in all cases. The postoperative period was uneventful without perioperative transfusion.


AR is a valuable navigation tool which may enhance the ability to achieve safe surgical resection during robotic hepatectomy.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. 1.

    Haidegger T, Sandor J, Benyo Z (2011) Surgery in space: the future of robotic telesurgery. Surg Endosc 25:681–690

  2. 2.

    Marescaux J, Leroy J, Gagner M, Rubino F, Mutter D, Vix M, Butner SE, Smith MK (2001) Transatlantic robot-assisted telesurgery. Nature 413:379–380

  3. 3.

    Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20:189–201

  4. 4.

    D’Agostino J, Diana M, Soler L, Vix M, Marescaux J (2012) 3D virtual neck exploration prior to parathyroidectomy. N Engl J Med

  5. 5.

    Marescaux J, Clement JM, Tassetti V, Koehl C, Cotin S, Russier Y, Mutter D, Delingette H, Ayache N (1998) Virtual reality applied to hepatic surgery simulation: the next revolution. Ann Surg 228:627–634

  6. 6.

    Gauss T, Merckx P, Brasher C, Kavafyan J, Le Bihan E, Aussilhou B, Belghiti J, Mantz J. (2013) Deviation from a preoperative surgical and anesthetic care plan is associated with increased risk of adverse intraoperative events in major abdominal surgery. Langenbecks Arch Surg 398:277–85

  7. 7.

    Volonte F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P (2011) Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreatol Sci 18:506–509

  8. 8.

    Diana M, Pessaux P, Marescaux J (2014) New technologies for single-site robotic surgery in hepato-biliary-pancreatic surgery. J Hepatobiliary Pancreatol Sci 21:34–42

  9. 9.

    Bano J, Hostettler A, Nicolau SA, Cotin S, Doignon C, Wu HS, Huang MH, Soler L, Marescaux J (2012) Simulation of pneumoperitoneum for laparoscopic surgery planning. Med Image Comput Comput Assist Interv 15:91–98

  10. 10.

    D’Agostino J, Diana M, Vix M, Soler L, Marescaux J (2012) Three-dimensional virtual neck exploration before parathyroidectomy. N Engl J Med 367:1072–1073

  11. 11.

    D’Agostino J, Wall J, Soler L, Vix M, Duh QY, Marescaux J (2013) Virtual neck exploration for parathyroid adenomas: a first step toward minimally invasive image-guided surgery. JAMA Surg 148:232–238, discussion 238

  12. 12.

    Nam WH, Kang DG, Lee D, Lee JY, Ra JB (2012) Automatic registration between 3D intra-operative ultrasound and pre-operative CT images of the liver based on robust edge matching. Phys Med Biol 57:69–91

  13. 13.

    Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C, Sutton E, George I, Kavic S, Mezrich R, Park A (2010) Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24:1976–1985

  14. 14.

    Hostettler A, Nicolau SA, Remond Y, Marescaux J, Soler L (2010) A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol 103:169–184

Download references


The authors are grateful to Anne Blandine Mackowski and Mourad Bouhadjar for the 3D reconstruction and to the IRCAD audiovisual team for their technical support. In addition, the authors would like to thank Christopher Burel and Guy Temporal for their valuable assistance in proofreading the manuscript.


This work is part of the eHealth project PASSPORT funded by the ICT program of the European Community as part of the 7th Framework Program.

Conflicts of interest

The authors have no conflicts of interest or financial ties to disclose.

Author information

Correspondence to Patrick Pessaux.

Electronic supplementary material

Below is the link to the electronic supplementary material.


(MOV 65388 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pessaux, P., Diana, M., Soler, L. et al. Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch Surg 400, 381–385 (2015). https://doi.org/10.1007/s00423-014-1256-9

Download citation


  • Augmented reality
  • Computer-assisted surgery
  • Surgical margin
  • Hepatectomy
  • Robotic surgery
  • Surgical planning