Skip to main content
SpringerLink
Log in

Utility of augmented reality system in hepatobiliary surgery

  • Technical view
  • Published:
Journal of Hepato-Biliary-Pancreatic Sciences

Abstract

Background/purpose

The aim of this study was to evaluate the utility of an image display system for augmented reality in hepatobiliary surgery under laparotomy.

Methods

An overlay display of organs, vessels, or tumor was obtained using a video see-through system as a display system developed at our institute. Registration between visceral organs and the surface-rendering image reconstructed by preoperative computed tomography (CT) was carried out with an optical location sensor. Using this system, we performed laparotomy for a patient with benign biliary stricture, a patient with gallbladder carcinoma, and a patient with hepatocellular carcinoma.

Results

The operative procedures performed consisted of choledochojejunostomy, right hepatectomy, and microwave coagulation therapy. All the operations were carried out safely using images of the site of tumor, preserved organs, and resection aspect overlaid onto the operation field images observed on the monitors. The position of each organ in the overlaid image closely corresponded with that of the actual organ. Intraoperative information generated from this system provided us with useful navigation. However, several problems such as registration error and lack of depth knowledge were noted.

Conclusion

The image display system appeared to be useful in performing hepatobiliary surgery under laparotomy. Further improvement of the system with individualized function for each operation will be essential, with feedback from clinical trials in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Marescaux J, Clement JM, Tassetti V, Koehl C, Cotin S, Russier Y, et al. Virtual reality applied to hepatic surgery simulation: the next revolution. Ann Surg. 1998;228:627–34.

    Article  PubMed  CAS  Google Scholar 

  2. Lamada W, Glombitza G, Ficher L, Chiu P, Cárdenas CE Sr, Thorn M, et al. The impact of 3-dimensional reconstruction on operation planning in liver surgery. Arch Surg. 2000;135:1256–61.

    Article  Google Scholar 

  3. Endo I, Shimada H, Sugita M, Fujii Y, Morioka D, Takeda K, et al. Role of three-dimensional imaging in operative planning for hilar cholangiocarcinoma. Surgery. 2007;142:666–75.

    Article  PubMed  Google Scholar 

  4. Aggarwal R, Crochet P, Dias A, Misra A, Ziprin P, Darzi A. Development of a virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg. 2009;96:1086–93.

    Article  PubMed  CAS  Google Scholar 

  5. Saito S, Yamanaka J, Miura K, Nakao N, Nagao T, Sugimoto T, et al. A novel 3-D hepatectomy simulation based on liver circulation: application to liver resection and transplantation. Hepatology. 2005;41:1297–304.

    Article  PubMed  Google Scholar 

  6. Yonemura Y, Taketomi A, Soejima Y, Yoshizumi T, Uchiyama H, Gion T, et al. Validity of preoperative volumetric analysis of congestion volume in living donor liver transplantation using three-dimensional computed tomography. Liver Transpl. 2005;11:1556–62.

    Article  PubMed  Google Scholar 

  7. Yamanaka J, Saito S, Fujimoto J. Impact of preoperative planning using virtual segmental volumetry on liver resection for hepatocellular carcinoma. World J Surg. 2007;31:1249–55.

    Article  PubMed  Google Scholar 

  8. Satou S, Sugawara Y, Kishi Y, Kaneko J, Matsui Y, et al. Preoperative estimation of right sector graft by three-dimensional computed tomography. Transplant Proc. 2007;39:145–9.

    Article  PubMed  CAS  Google Scholar 

  9. Hattori A, Suzuki N, Hashizume M, Akahoshi T, Konishi K, Yamaguchi S, et al. A robotic surgery system (da Vinci) with image guided function system architecture and cholecystectomy. Stud Health Technol Inform. 2003;94:110–6.

    PubMed  Google Scholar 

  10. Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T, et al. Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci. 2010;17:629–36.

    Article  PubMed  Google Scholar 

  11. Herline AJ, Stefansic JD, Debelak JP, Hartmann SL, Pinson CW, Galloway RL, et al. Image-guided surgery. Arch Surg. 1999;134:644–50.

    Article  PubMed  CAS  Google Scholar 

  12. Cash DM, Miga MI, Glascow SC, Dawant BM, Clements LW, Cao Z, et al. Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg. 2007;11:844–59.

    Article  PubMed  Google Scholar 

  13. Robb RA. The biomedical imaging resource at Mayo clinic. Guest editorial. IEEE Trans Med Imaging. 2001;20:854–67.

    Article  PubMed  CAS  Google Scholar 

  14. Robb RA, Barillot C. Interactive display and analysis of 3-D medical images. IEEE Trans Med Imaging. 1989;8:217–26.

    Article  PubMed  CAS  Google Scholar 

  15. Robb RA, Hanson DP, Karwoski RA, Larson AG, Workman EL, Stacy MC. Analyze: a comprehensive, operator-interactive software package for multidimensional medical image display and analysis. Comput Med Imaging Graph. 1989;13:433–54.

    Article  PubMed  CAS  Google Scholar 

  16. Lamade W, Vetter M, Hassenphlug P, Thorn M, Meinzer HP, Herfarth C. Navigation and image-guided HBP surgery: a review and preview. J Hepatobiliary Pancreat Sci. 2002;9:592–9.

    Article  Google Scholar 

  17. Sugimoto M. Recent advances in visualization, imaging, and navigation in hepatobiliary and pancreatic sciences. J Hepatobiliary Pancreat Sci. 2010;17:574–6.

    Article  PubMed  Google Scholar 

  18. Shuhaiber JH. Augmented reality in surgery. Arch Surg. 2004;139:170–4.

    Article  PubMed  Google Scholar 

  19. Morikawa S, Inubushi T, Kurumi Y, Kurumi Y, Naka S, Haque HA, et al. MR-guided microwave thermocoagulation therapy of liver tumors: initial clinical experience using a 0.5T MR system. Magn Reson Med Sci. 2005;4:89–94.

    Article  PubMed  Google Scholar 

  20. Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C, et al. Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc. 2010;24:1976–85.

    Article  PubMed  Google Scholar 

  21. Nakamoto M, Nakada K, Sato Y, Konishi K, Hashizume M, Tamura S. Intraoperative magnetic tracker calibration using a magneto-optic hybrid tracker for 3-D ultrasound-based navigation in laparoscopic surgery. IEEE Trans Med Imaging. 2008;27:255–70.

    Article  PubMed  Google Scholar 

  22. Sato Y, Nakamoto M, Tamaki Y, Sasama T, Sakita I, Nakajima Y, et al. Image guidance of breast cancer surgery using 3-D ultrasound images and augmented reality visualization. IEEE Trans Med Imaging. 1998;17:681–93.

    Article  PubMed  CAS  Google Scholar 

Download references

Conflict of interest

None.

Author information

Authors and Affiliations

  1. Department of Surgery, The Jikei University Daisan Hospital, 4-11-1 Izumihoncho, Komae, Tokyo, Japan

    Tomoyoshi Okamoto, Shinji Onda, Michinori Matsumoto, Takeshi Gocho, Yasuro Futagawa & Shuichi Fujioka

  2. Department of Surgery, The Jikei University School of Medicine, Tokyo, Japan

    Katsuhiko Yanaga

  3. Institute for High Dimensional Medical Imaging, The Jikei University School of Medicine, Tokyo, Japan

    Naoki Suzuki & Asaki Hattori

Authors
  1. Tomoyoshi Okamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shinji Onda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michinori Matsumoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Takeshi Gocho
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yasuro Futagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuichi Fujioka
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Katsuhiko Yanaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Naoki Suzuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Asaki Hattori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomoyoshi Okamoto.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (M1V 21396 kb)

About this article

Cite this article

Okamoto, T., Onda, S., Matsumoto, M. et al. Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci 20, 249–253 (2013). https://doi.org/10.1007/s00534-012-0504-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00534-012-0504-z

Keywords

Search

Navigation