Augmented reality in open surgery
Augmented reality (AR) has been successfully providing surgeons an extensive visual information of surgical anatomy to assist them throughout the procedure. AR allows surgeons to view surgical field through the superimposed 3D virtual model of anatomical details. However, open surgery presents new challenges. This study provides a comprehensive overview of the available literature regarding the use of AR in open surgery, both in clinical and simulated settings. In this way, we aim to analyze the current trends and solutions to help developers and end/users discuss and understand benefits and shortcomings of these systems in open surgery. We performed a PubMed search of the available literature updated to January 2018 using the terms (1) “augmented reality” AND “open surgery”, (2) “augmented reality” AND “surgery” NOT “laparoscopic” NOT “laparoscope” NOT “robotic”, (3) “mixed reality” AND “open surgery”, (4) “mixed reality” AND “surgery” NOT “laparoscopic” NOT “laparoscope” NOT “robotic”. The aspects evaluated were the following: real data source, virtual data source, visualization processing modality, tracking modality, registration technique, and AR display type. The initial search yielded 502 studies. After removing the duplicates and by reading abstracts, a total of 13 relevant studies were chosen. In 1 out of 13 studies, in vitro experiments were performed, while the rest of the studies were carried out in a clinical setting including pancreatic, hepatobiliary, and urogenital surgeries. AR system in open surgery appears as a versatile and reliable tool in the operating room. However, some technological limitations need to be addressed before implementing it into the routine practice.
KeywordsAugmented reality Mixed reality Open surgery Image-guided surgery Surgical navigation
This work was funded by the HORIZON2020 Project VOSTARS, Project ID: 731974. Call: ICT-29-2016—Photonics KET 2016.
Compliance with ethical standards
Conflicts of interest
The authors declare that they have no conflict of interest.
Research involving human participants and/or animals
The authors confirm that no animal nor human testing were performed in this study.
No informed consent is required.
- 4.Cutolo F, Parchi PD, Ferrari V (2014) Video see through AR head-mounted display for medical procedures. In: 2014 IEEE international symposium on mixed and augmented reality (ISMAR), 10–12 Sept 2014, pp 393–396. https://doi.org/10.1109/ismar.2014.6948504
- 9.Bimber O, Raskar R (2006) Modern approaches to augmented reality. Paper presented at the ACM SIGGRAPH 2006 Courses, Boston, MassachusettsGoogle Scholar
- 12.Sauer IM, Queisner M, Tang P, Moosburner S, Hoepfner O, Horner R, Lohmann R, Pratschke J (2017) Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg 266(5):706–712. https://doi.org/10.1097/SLA.0000000000002448 CrossRefPubMedGoogle Scholar
- 16.Onda S, Okamoto T, Kanehira M, Fujioka S, Suzuki N, Hattori A, Yanaga K (2013) Short rigid scope and stereo-scope designed specifically for open abdominal navigation surgery: clinical application for hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci 20(4):448–453. https://doi.org/10.1007/s00534-012-0582-y CrossRefPubMedGoogle Scholar
- 17.Onda S, Okamoto T, Kanehira M, Suzuki F, Ito R, Fujioka S, Suzuki N, Hattori A, Yanaga K (2014) Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality-based navigation system. J Hepatobiliary Pancreat Sci 21(4):281–287. https://doi.org/10.1002/jhbp.25 CrossRefPubMedGoogle Scholar
- 18.KleinJan GH, van den Berg NS, van Oosterom MN, Wendler T, Miwa M, Bex A, Hendricksen K, Horenblas S, van Leeuwen FW (2016) Toward (hybrid) navigation of a fluorescence camera in an open surgery setting. J Nucl Med 57(10):1650–1653. https://doi.org/10.2967/jnumed.115.171645 CrossRefPubMedGoogle Scholar
- 19.van Oosterom MN, Meershoek P, KleinJan GH, Hendricksen K, Navab N, van de Velde CJH, van der Poel HG, van Leeuwen FWB (2018) Navigation of fluorescence cameras during soft tissue surgery—is it possible to use a single navigation setup for various open and laparoscopic urological surgery applications? J Urol 199(4):1061–1068. https://doi.org/10.1016/j.juro.2017.09.160 CrossRefPubMedGoogle Scholar
- 21.Tang R, Ma L, Xiang C, Wang X, Li A, Liao H, Dong J (2017) Augmented reality navigation in open surgery for hilar cholangiocarcinoma resection with hemihepatectomy using video-based in situ three-dimensional anatomical modeling: a case report. Medicine (Baltimore) 96(37):e8083. https://doi.org/10.1097/MD.0000000000008083 CrossRefGoogle Scholar
- 28.Haouchine N, Dequidt J, Berger MO, Cotin S (2013) Deformation-based augmented reality for hepatic surgery. Stud Health Technol Inf 184:182–188Google Scholar
- 30.Rolland JP, Holloway RL, Fuchs H (1994) A comparison of optical and video see-through head-mounted displays. P Soc Photo Opt Ins 2351:293–307Google Scholar
- 36.Kilgus T, Heim E, Haase S, Prufer S, Muller M, Seitel A, Fangerau M, Wiebe T, Iszatt J, Schlemmer HP, Hornegger J, Yen K, Maier-Hein L (2015) Mobile markerless augmented reality and its application in forensic medicine. Int J Comput Ass Rad 10(5):573–586. https://doi.org/10.1007/s11548-014-1106-9 CrossRefGoogle Scholar