Skip to main content

Advertisement

SpringerLink
Log in

Clinical application of navigation surgery using augmented reality in the abdominal field

  • Review Article
  • Published:
Surgery Today Aims and scope Submit manuscript

Abstract

This article presents general principles and recent advancements in the clinical application of augmented reality-based navigation surgery (AR based NS) for abdominal procedures and includes a description of our clinical trial and subsequent outcomes. Moreover, current problems and future aspects are discussed. The development of AR-based NS in the abdomen is delayed compared with another field because of the problem of intraoperative organ deformations or the existence of established modalities. Although there are a few reports on the clinical use of AR-based NS for digestive surgery, sophisticated technologies in urology have often been reported. However, the rapid widespread use of video- or robot assisted surgeries requires this technology. We have worked to develop a system of AR-based NS for hepatobiliary and pancreatic surgery. Then we developed a short rigid scope that enables surgeons to obtain 3D view. We recently focused on pancreatic surgery, because intraoperative organ shifting is minimal. The position of each organ in overlaid image almost corresponded with that of the actual organ with about 5 mm of mean registration errors. Intraoperative information generated from this system provided us with useful navigation. However, AR-based NS has several problems to overcome such as organ deformity, evaluation of utility, portability or cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Marescaux J, Clement JM, Tassetti, Koehl C, Cotin S, Russier Y, et al. Virtual reality applied to hepatic surgery simulation: the next revolution. Ann Surg. 1998;228:627–34.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  2. Lamada W, Glombitza G, Ficher L, Chiu P, Cárdenas CE Sr, Thorn M, et al. The impact of 3D reconstruction on operation planning in liver surgery. Arch Surg. 2000;135:1256–61.

    Article  Google Scholar 

  3. Satou S, Yamanaka J, Miura K, Nakao N, Nagao T, Sugimoto T, et al. A novel 3D hepatectomy simulation based on liver circulation: application to liver resection and transplantation. Hepatology. 2005;41:1297–304.

    Article  Google Scholar 

  4. Yamanaka J, Saito S, Fujimoto J. Impact of preoperative planning using virtual segmental volumetry on liver resection for hepatocellular carcinoma. World J Surg. 2007;31:1249–55.

    Article  PubMed  Google Scholar 

  5. Endo I, Shimada H, Sugita M, Fujii Y, Morioka D, Takeda K, et al. Role of 3D imaging in operative planning for hilar cholangiocarcinoma. Surgery. 2007;142:666–75.

    Article  PubMed  Google Scholar 

  6. Aggarwal R, Crochet P, Dias A, Misra A, Ziprin P, Darzi A. Development of a virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg. 2009;96:1086–93.

    Article  CAS  PubMed  Google Scholar 

  7. Aoki T, Mise Y, Kokudo N. Hepatic resections assisted by operation planning using a 3D navigation/simulation software (in Japanese). Tan to sui. 2013;34:27–34.

    Google Scholar 

  8. Lamade W, Vetter M, Hassenphlug P, Thorn M, Meinzer HP, Herfarth C. Navigation and image-guided HBP surgery: a review and preview. J Hepatobiliary Pancreat Sci. 2002;9:592–9.

    Article  Google Scholar 

  9. Shuhaiber JH. Augmented reality in surgery. Arch Surg. 2004;139:170–4.

    Article  PubMed  Google Scholar 

  10. Sugimoto M. Recent advances in visualization, imaging, and navigation in hepatobiliary and pancreatic sciences. J Hepatobiliary Pancreat Sci. 2010;17:574–6.

    Article  PubMed  Google Scholar 

  11. Nicolau S, Soler L, Mutter D, Marescaux J, et al. Augmented reality in laparoscopic surgical oncology. Surg Oncol. 2011;20:189–201.

    Article  PubMed  Google Scholar 

  12. Das M, Sauer F, Schoepf UJ, Khamene A, Vogt SK, Scaller S, et al. Augmented reality visualization for CT-guided interventions: system description, feasibility, and initial evaluation in an abdominal phantom. Radiology. 2006;240:230–5.

    Article  PubMed  Google Scholar 

  13. Robb RA. Vrsp: virtual reality assisted surgery program. In: 1st International symposium on computer aided surgery 1994; 18–19.

  14. Mezger U, Jendrewski C, Bartels M. Navigation in surgery. Langenbecks Arch Surg. 2013;398:501–14.

    Article  PubMed Central  PubMed  Google Scholar 

  15. Grunert P, Darabi K, Espinosa J, Filippi R. Computer-aided navigation in neurosurgery. Neurosurg Rev. 2003;26:73–99.

    Article  CAS  PubMed  Google Scholar 

  16. MarkiewiczMR BellRB. Traditional and contemporary surgical approaches to the orbit. Oral Maxillofac Surg Clin North Am. 2012;24:573–607. doi:10.1016/j.coms.2012.08.004.

    Article  Google Scholar 

  17. Hohlweg-Majert B, Schon R, Schmelzeisen R, Gellrich NC, Schramm A. Navigational maxillofacial surgery using virtual models. World J Surg. 2005;29:1530–8.

    Article  CAS  PubMed  Google Scholar 

  18. Hamada H, Hayashi N, Asahi T, Kurimoto M, Hirashima Y, Endo S. Efficacy of a navigation system in neuro-endoscopic surgery. Minim Invasive Neurosurg. 2005;48:197–201.

    Article  CAS  PubMed  Google Scholar 

  19. Zhou F, Shao JH, Zou SB, Huang MW, Yin XB, Yu X. Laparoscopic hepatectomy is associated with a higher incident frequency in hepatolithiasis patients. Surg Today. 2013;12:1371–81.

    Article  Google Scholar 

  20. Satava RM. Emerging technology for surgery in the 21st century. Arch Surg. 1999;134:1197–2002.

    Article  CAS  PubMed  Google Scholar 

  21. Mårvik R, Langø T, Tangen GA, Andersen JO, Kaspersen JH, Ystgaard B, et al. Laparoscopic navigation pointer for 3D image-guided surgery. Surg Endosc. 2004;18:1242–8.

    Article  PubMed  Google Scholar 

  22. Okamoto T, Onda S, Matsumoto M, Gocho T, Futagawa Y, Fujioka S, et al. Utility of augmented reality system in hepatobiliary–pancreatic surgery with laparotomy. J Jpn Soc Comput Aided Surg (in Japanese with English abstract). 2010;12:312–3.

    Google Scholar 

  23. Okamoto T, Onda S, Matsumoto M, Gocho T, Futagawa Y, Fujioka S, et al. Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci. 2013;20:249–53.

    Article  PubMed  Google Scholar 

  24. Onda S, Okamoto T, Kanehira M, Fujioka S, Suzuki N, Hattori A, et al. Short rigid scope and stereo-scope designed specifically for open abdominal navigation surgery: clinical application for hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci. 2013;20:448–53.

    Article  PubMed  Google Scholar 

  25. Onda S, Okamoto T, Matsumoto M, Son K, Gocho T, Futagawa Y, et al. Utility of augmented reality system in pancreatectomy. J Jpn Soc Comput Aided Surg (in Japanese with English abstract). 2011;13:258–9.

    Google Scholar 

  26. Onda S, Okamoto T, Kanehira M, Ito R, Fujioka S, Yanaga K, et al. Clinical application of a stereoscopic image display system for surgical navigation in hepatobiliary and pancreatic surgery. J Jpn Soc Comput Aided Surg (in Japanese with English abstract). 2012;14:222–3.

    Google Scholar 

  27. Onda S, Okamoto T, Kanehira M, Suzuki F, Ito R, Fujioka S, et al. Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality-based navigation system. J Hepatobiliary Pancreat Sci. 2013. doi:10.1002/jhbp.25.

    Google Scholar 

  28. Ukimura O, Magi-Galluzzi C, Gill IS. Real-time transrectal ultrasound guidance during laparoscopic radical prostatectomy: impact on surgical margins. J Urol. 2006;175:1304–10.

    Article  PubMed  Google Scholar 

  29. Simpfendörfer T, Baumhauer M, Müller M, Gutt CN, Meinzer HP, Rassweiler JJ, et al. Augmented reality visualization during laparoscopic radical prostatectomy. J Endourol. 2011;25:1841–5.

    Article  PubMed  Google Scholar 

  30. Ukimura O, Gill IS. Image-fusion, augmented reality, and predictive surgical navigation. Urol Clin North Am. 2009;36:115–23.

    Article  PubMed  Google Scholar 

  31. Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C, et al. Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc. 2010;24:1976–85.

    Article  PubMed  Google Scholar 

  32. Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T, et al. Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci. 2010;17:629–36.

    Article  PubMed  Google Scholar 

  33. Liao H, Inomata T, Sakuma I, Dohi T. 3D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng. 2010;57:1476–86.

    Article  PubMed  Google Scholar 

  34. Hansen C, Wieferich J, Ritter F, Rieder C, Peitgen HO. Illustrative visualization of 3D planning models for augmented reality in liver surgery. Int J Comput Assist Radiol Surg. 2010;5:133–41.

    Article  PubMed  Google Scholar 

  35. Marescaux J, Rubino F, Arenas M, Mutter D, Soler L. Augmented-reality-assisted laparoscopic adrenalectomy. JAMA. 2004;292:2214–5.

    CAS  PubMed  Google Scholar 

  36. Ieiri S, Uemura M, Konishi K, Souzaki R, Nagao Y, Tsutsumi N, et al. Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device. Pediatr Surg Int. 2012;28:341–6.

    Article  PubMed  Google Scholar 

  37. Herline AJ, Stefansic JD, Debelak JP, Hartmann SL, Pinson CW, Galloway RL, et al. Image-guided surgery: preliminary feasibility studies of frameless stereotactic liver surgery. Arch Surg. 1999;134:644–9.

    Article  CAS  PubMed  Google Scholar 

  38. Hostettler A, Nicolau SA, Rémond Y, Marescaux J, Soler L. A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol. 2010;103:169–84.

    Article  CAS  PubMed  Google Scholar 

  39. Beller S, Eulenstein S, Lange T, Hünerbein M, Schlag PM. Upgrade of an optical navigation system with a permanent electromagnetic position control: a first step towards “navigated control” for liver surgery. J Hepatobiliary Pancreat Surg. 2009;16:165–70.

    Article  PubMed  Google Scholar 

  40. Cash DM, Miga MI, Sinha TK, Galloway RL, Chapman WC. Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements. IEEE Trans Med Imaging. 2005;24:1479–91.

    Article  PubMed  Google Scholar 

  41. Cash DM, Miga MI, Glasgow SC, Dawant BM, Clements LW, Cao Z, et al. Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg. 2007;11:844–59.

    Article  PubMed  Google Scholar 

  42. Konishi K, Hashizume M, Nakamoto M, Kakeji Y, Yoshino I, Taketomi A, et al. Augmented reality navigation system for endoscopic surgery based on three-dimensional ultrasound and computed tomography: application to 20 clinical cases. Inter Congr Ser. 2005;1281:537–42.

    Article  Google Scholar 

  43. Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD. Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology. 2009;73:896–900.

    Article  PubMed  Google Scholar 

  44. Nakamoto M, Ukimura O, Faber K, Gill IS. Current progress on augmented reality visualization in endoscopic surgery. Curr Opin Urol. 2012;22:121–6.

    Article  PubMed  Google Scholar 

  45. Nicolau SA, Vemuri A, Wu HS, Huang MH, Ho Y, Charnoz A, et al. A cost effective simulator for education of ultrasound image interpretation and probe manipulation. Stud Health Technol Inform. 2011;163:403–7.

    CAS  PubMed  Google Scholar 

  46. Harders M, Bianchi G, Knoerlein B, Székely G. Calibration, registration, and synchronization for high precision augmented reality haptics. IEEE Trans Vis Comput Graph. 2009;15:138–49.

    Article  PubMed  Google Scholar 

  47. Teber D, Guven S, Simpfendörfer T, Baumhauer M, Güven EO, Yencilek F, et al. Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol. 2009;56:332–8.

    Article  PubMed  Google Scholar 

  48. Gavaghan KA, Peterhans M, Oliveira-Santos T, Weber S. A portable image overlay projection device for computer-aided open liver surgery. IEEE Trans Biomed Eng. 2011;58:1855–64.

    Article  PubMed  Google Scholar 

  49. Baumhauer M, Feuerstein M, Meinzer HP, Rassweiler J. Navigation in endoscopic soft tissue surgery: perspectives and limitations. J Endourol. 2008;22:751–66.

    Article  PubMed  Google Scholar 

  50. Gavaghan K, Oliveira-Santos T, Peterhans M, Reyes M, Kim H, et al. Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies. Int J Comput Assist Radiol Surg. 2012;7:547–56.

    Article  CAS  PubMed  Google Scholar 

  51. Masamune K, Fichtinger G, Deguet A, Matsuka D, Taylor R. An image overlay system with enhanced reality for percutaneous therapy performed inside CT scanner. LNCS. 2002;2489:77–84.

    Google Scholar 

  52. Argotti Y, Davis L, Outters V, Rolland JP. Dynamic superimposition of synthetic objects on rigid and simple deformable real objects. Comput Graph. 2002;26:919–30.

    Article  Google Scholar 

  53. Ferrari V, Megali G, Troia E, Pietrabissa A, Mosca F. A 3D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng. 2009;56:2627–33.

    Article  PubMed  Google Scholar 

  54. Hattori A, Suzuki N, Hashizume M, Akahoshi T, Konishi K, Yamaguchi S, et al. A robotic surgery system (da Vinci) with image guided function system architecture and cholecystectomy. Stud Health Technol Inform. 2003;94:110–6.

    PubMed  Google Scholar 

  55. Liao H, Ishihara H, Tran HH, Masamune K, Sakuma I, Dohi T. Precision-guided surgical navigation system using laser guidance and 3D autostereoscopic image overlay. Comput Med Imaging Graph. 2010;34:46–54.

    Article  PubMed  Google Scholar 

  56. Wong DY, Leveque JC, Brumblay H, Krebsbach PH, Hollister SJ, Lamarca F. Macro-architectures in spinal cord scaffold implants influence regeneration. J Neurotrauma. 2008;25:1027–37.

    Article  PubMed Central  PubMed  Google Scholar 

  57. Aoki T, Murakami Y, Enami Y, Fujimori T, Koizumi T, Yamada K, et al. Minimally invasive liver surgery using combination of virtual endoscopy and new ultrasound techniques with image fusion or global positioning system (GPS) tracking. J Jpn Soc Comput Aided Surg (in Japanese with English abstract). 2012;14:164–5.

    Google Scholar 

  58. Matsuki M, Okuda J, Kanazawa S, Kanamoto T, Inada Y, Tatsugami F, et al. Virtual CT colectomy by three-dimensional imaging using multidetector-row CT for laparoscopic colorectal surgery. Abdom Imaging. 2005;30:698–708.

    Article  CAS  PubMed  Google Scholar 

  59. Kowalczuk J, Meyer A, Carlson J, Psota ET, Buettner S, Pérez LC, et al. Real-time 3D soft tissue reconstruction for laparoscopic surgery. Surg Endosc. 2012;26:3413–7.

    Article  PubMed  Google Scholar 

  60. Peterhans M, vom Berg A, Dagon B, Inderbitzin D, Baur C, Candinas D, et al. A navigation system for open liver surgery: design, workflow and first clinical applications. Int J Med Robot. 2011;7:7–16.

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Surgery, The Jikei University Daisan Hospital, 4-11-1 Izumihoncho, Komae-shi, Tokyo, Japan

    Tomoyoshi Okamoto

  2. Division of Digestive Surgery, The Jikei University School of Medicine, 3-25-8 Nishishinbashi, Minato-ku, Tokyo, Japan

    Shinji Onda & Katsuhiko Yanaga

  3. Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, 4-11-1 Izumihoncho, Komae-shi, Tokyo, Japan

    Naoki Suzuki & Asaki Hattori

Authors
  1. Tomoyoshi Okamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shinji Onda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Katsuhiko Yanaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naoki Suzuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Asaki Hattori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomoyoshi Okamoto.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Okamoto, T., Onda, S., Yanaga, K. et al. Clinical application of navigation surgery using augmented reality in the abdominal field. Surg Today 45, 397–406 (2015). https://doi.org/10.1007/s00595-014-0946-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00595-014-0946-9

Keywords

Search

Navigation