Skip to main content
SpringerLink
Log in

Image Overlay Surgery Based on Augmented Reality: A Systematic Review

  • Chapter
  • First Online:
Biomedical Visualisation

Part of the book series: Advances in Experimental Medicine and Biology ((AEMB,volume 1320))

Abstract

Augmented Reality (AR) applied to surgical guidance is gaining relevance in clinical practice. AR-based image overlay surgery (i.e. the accurate overlay of patient-specific virtual images onto the body surface) helps surgeons to transfer image data produced during the planning of the surgery (e.g. the correct resection margins of tissue flaps) to the operating room, thus increasing accuracy and reducing surgery times. We systematically reviewed 76 studies published between 2004 and August 2018 to explore which existing tracking and registration methods and technologies allow healthcare professionals and researchers to develop and implement these systems in-house. Most studies used non-invasive markers to automatically track a patient’s position, as well as customised algorithms, tracking libraries or software development kits (SDKs) to compute the registration between patient-specific 3D models and the patient’s body surface. Few studies combined the use of holographic headsets, SDKs and user-friendly game engines, and described portable and wearable systems that combine tracking, registration, hands-free navigation and direct visibility of the surgical site. Most accuracy tests included a low number of subjects and/or measurements and did not normally explore how these systems affect surgery times and success rates. We highlight the need for more procedure-specific experiments with a sufficient number of subjects and measurements and including data about surgical outcomes and patients’ recovery. Validation of systems combining the use of holographic headsets, SDKs and game engines is especially interesting as this approach facilitates an easy development of mobile AR applications and thus the implementation of AR-based image overlay surgery in clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Andress S, Johnson A, Unberath M, Winkler AF, Yu K, Fotouhi J, Weidert S, Osgood G, Navab N (2018) On-the-fly augmented reality for orthopedic surgery using a multimodal fiducial. J Med Imaging 5(2):021209

    Article  Google Scholar 

  • Azuma RT (1997) A survey of augmented reality. Presence Teleop Virt 6(4):355–385

    Article  Google Scholar 

  • Badiali G, Ferrari V, Cutolo F, Freschi C, Caramella D, Bianchi A, Marchetti C (2014) Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Cranio-Maxillo-Facial Surg 42(8):1970–1976

    Article  Google Scholar 

  • Bertolo R, Hung A, Porpiglia F, Bove P, Schleicher M, Dasgupta P (2019) Systematic review of augmented reality in urological interventions: the evidences of an impact on surgical outcomes are yet to come. World J Urol Available from: https://doi.org/10.1007/s00345-019-02711-z.

  • Besharati Tabrizi L, Mahvash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123(1):206–211

    Article  PubMed  Google Scholar 

  • Bosc R, Fitoussi A, Hersant B, Dao T, Meningaud J (2019) Intraoperative augmented reality with heads-up displays in maxillofacial surgery: a systematic review of the literature and a classification of relevant technologies. Int J Oral Maxillofac Surg 48(1):132–139

    Article  CAS  PubMed  Google Scholar 

  • Cabrilo I, Schaller K, Bijlenga P (2015) Augmented reality-assisted bypass surgery: embracing minimal invasiveness. World Neurosurg 83(4):596–602

    Article  PubMed  Google Scholar 

  • Contreras López WO, Navarro PA, Crispin S (2019) Intraoperative clinical application of augmented reality in neurosurgery: a systematic review. Clin Neurol Neurosurg 177:6–11

    Article  PubMed  Google Scholar 

  • Cutolo F, Carbone M, Parchi PD, Ferrari V, Lisanti M, Ferrari M (2016) Application of a new wearable augmented reality video see-through display to aid percutaneous procedures in spine surgery. Augment Reality Virtual Reality Comput Graphics 9769(Pt II):43–54

    Article  Google Scholar 

  • Deng W, Li F, Wang M, Song Z (2014) Easy-to-use augmented reality neuronavigation using a wireless tablet PC. Stereotact Funct Neurosurg 92(1):17–24

    Article  PubMed  Google Scholar 

  • Drouin S, Kochanowska A, Kersten-Oertel M, Gerard IJ, Zelmann R, De ND, Beriault S, Arbel T, Sirhan D, Sadikot AF, Hall JA, Sinclair DS, Petrecca K, DelMaestro RF, Collins DL (2017) IBIS: an OR ready open-source platform for image-guided neurosurgery. Int J Comput Assist Radiol Surg 12(3):363–378

    Article  PubMed  Google Scholar 

  • Eckert M, Volmerg JS, Friedrich CM (2019) Augmented reality in medicine: systematic and bibliographic review. JMIR Mhealth Uhealth 7(4):e10967

    Article  PubMed  PubMed Central  Google Scholar 

  • Eftekhar B (2016) App-assisted external ventricular drain insertion. J Neurosurg 125(3):754–758

    Article  PubMed  Google Scholar 

  • Fichtinger G, Deguet A, Masamune K, Balogh E, Fischer G, Mathieu H, Taylor R, Zinreich S, Fayad L (2005) Image overlay guidance for needle insertion in CT scanner. IEEE Trans Biomed Eng 52(8):1415–1424

    Article  PubMed  Google Scholar 

  • Fida B, Cutolo F, di Franco G, Ferrari M, Ferrari V (2018) Augmented reality in open surgery. Updates Surg 70(3):389–400

    Article  PubMed  Google Scholar 

  • Fitzpatrick JM, West JB (2001) The distribution of target registration error in rigid-body point-based registration. IEEE Trans Med Imaging 20(9):917–927

    Article  CAS  PubMed  Google Scholar 

  • Fritz J, U-Thainual P, Ungi T, Flammang AJ, Fichtinger G, Iordachita II, Carrino JA (2013) Augmented reality visualisation using an image overlay system for MR-guided interventions: technical performance of spine injection procedures in human cadavers at 1.5 Tesla. Eur Radiol 23(1):235–245

    Article  PubMed  Google Scholar 

  • Gavaghan K, Oliveira-Santos T, Peterhans M, Reyes M, Kim H, Anderegg S, Weber S (2012) Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies. Int J Comput Assist Radiol Surg 7(4):547–556

    Article  CAS  PubMed  Google Scholar 

  • Gibby JT, Swenson SA, Cvetko S, Rao R, Javan R (2019) Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography. Int J Comput Assist Radiol Surg 14(3):525–535

    Article  PubMed  Google Scholar 

  • Giraldez JG, Caversaccio M, Pappas I, Kowal J, Rohrer U, Marti G, Baur C, Nolte L-P, Gonzalez BM (2007) Design and clinical evaluation of an image-guided surgical microscope with an integrated tracking system. Int J Comput Assist Radiol Surg 1(5):253–264

    Article  Google Scholar 

  • Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schanemann HJ (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336(7650):924

    Article  PubMed  PubMed Central  Google Scholar 

  • Han S, Lee C, Kim S, Jeon M, Kim J, Kim C (2013) In vivo virtual intraoperative surgical photoacoustic microscopy. Appl Phys Lett 103(20):203702

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • He C, Liu Y, Wang Y (2016) Sensor-fusion based augmented-reality surgical navigation system. In: International Instrumentation and Measurement Technology Conference; 2016 May 23–26; Taipei, Taiwan. IEEE; 2016. Available from: https://doi.org/10.1109/I2MTC.2016.7520404

  • Hou Y, Ma L, Zhu R, Chen X (2016) iPhone-assisted augmented reality localization of basal ganglia hypertensive hematoma. World Neurosurg 94:480–492

    Article  PubMed  Google Scholar 

  • Hu L, Wang M, Song Z (2013) A convenient method of video see-through augmented reality based on image-guided surgery system. In: Seventh International Conference on Internet Computing for Engineering and Science; 2013 September 20–22; Shanghai, China. IEEE; 2013. Available from: https://doi.org/10.1109/ICICSE.2013.27

  • Huang CH, Hsieh CH, Lee JD, Huang WC, Lee ST, Wu CT, Sun YN, Wu YT (2012) A CT-ultrasound-coregistered augmented reality enhanced image-guided surgery system and its preliminary study on brain-shift estimation. J Instrum 7:P08016

    Google Scholar 

  • Hummelink S, Hameeteman M, Hoogeveen Y, Slump CH, Ulrich DJO, Schultze Kool LJ (2015) Preliminary results using a newly developed projection method to visualize vascular anatomy prior to DIEP flap breast reconstruction. J Plast Reconstr Aesthet Surg 68(3):390–394

    Article  CAS  PubMed  Google Scholar 

  • Jiang T, Zhu M, Zan T, Gu B, Li Q (2017) A novel augmented reality-based navigation system in perforator flap transplantation – a feasibility study. Ann Plast Surg 79(2):192–196

    Article  CAS  PubMed  Google Scholar 

  • Jiang W, Ma L, Boyu Z, Yingwei F, Qu X, Zhang X, Liao H (2018) Evaluation of the 3D augmented reality-guided intraoperative positioning of dental implants in edentulous mandibular models. Int J Oral Maxillofac Implants 33:1219–1228

    Article  PubMed  Google Scholar 

  • Joda T, Gallucci GO, Wismeijer D, Zitzmann NU (2019) Augmented and virtual reality in dental medicine: a systematic review. Comput Biol Med 108:93–100

    Article  CAS  PubMed  Google Scholar 

  • Keller K, State A, Fuchs H (2008) Head mounted displays for medical use. J Disp Technol 4:468–472

    Article  Google Scholar 

  • Kersten-Oertel M, Chen SS, Drouin S, Sinclair DS, Collins DL (2012) Augmented reality visualization for guidance in neurovascular surgery. Stud Health Technol Inform 173:225–229

    PubMed  Google Scholar 

  • Khan MF, Dogan S, Maataoui A, Wesarg S, Gurung J, Ackermann H, Schiemann M, Wimmer-Greinecker G, Vogl TJ (2006) Navigation-based needle puncture of a cadaver using a hybrid tracking navigational system. Investig Radiol 41(10):713–720

    Article  Google Scholar 

  • Khor WS, Baker B, Amin K, Chan A, Patel K, Wong J (2016) Augmented and virtual reality in surgery-the digital surgical environment: applications, limitations and legal pitfalls. Ann Transl Med 4(23):454

    Article  PubMed  PubMed Central  Google Scholar 

  • Kim Y, Kim H, Kim YO (2017) Virtual reality and augmented reality in plastic surgery: A review. Arch Plast Surg 44(3):179–187

    Article  PubMed  PubMed Central  Google Scholar 

  • Kolodzey L, Grantcharov PD, Rivas H, Schijven MP, Grantcharov TP (2017) Wearable technology in the operating room: a systematic review. BMJ Innov 3(1):55–63

    Article  Google Scholar 

  • Kosterhon M, Gutenberg A, Kantelhardt SR, Archavlis E, Giese A (2017) Navigation and image injection for control of bone removal and osteotomy planes in spine surgery. Operative Neurosurg 13(2):297–304

    Article  Google Scholar 

  • Kramers M, Armstrong R, Bakhshmand SM, Fenster A, de Ribaupierre S, Eagleson R (2014) Evaluation of a mobile augmented reality application for image guidance of neurosurgical interventions. Stud Health Technol Inform 196:204–208

    PubMed  Google Scholar 

  • Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Minter MW, Debus J, Harms W (2008) Projector-basted augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiat Oncol Biol Phys 70(3):944–952

    Article  PubMed  Google Scholar 

  • Lee J-D, Huang C-H, Wang S-T, Lin C-W, Lee S-T (2010) Fast-MICP for frameless image-guided surgery. Med Phys 37(9):4551–4559

    Article  PubMed  Google Scholar 

  • Li L, Yang J, Chu Y, Wu W, Xue J, Liang P, Chen L (2016) A novel augmented reality navigation system for endoscopic sinus and skull base surgery: a feasibility study. PLOS ONE 11(1):e0146996

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Liang JT, Doke T, Onogi S, Ohashi S, Ohnishi I, Sakuma I, Nakajima Y (2012) A fluorolaser navigation system to guide linear surgical tool insertion. Int J Comput Assist Radiol Surg 7(6):931–939

    Article  PubMed  Google Scholar 

  • Liao H, Inomata T, Sakuma I, Dohi T (2010) 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng 57(6):1476–1486

    Article  PubMed  Google Scholar 

  • Liberati A, FAU AD, Tetzlaff JF, Mulrow C, Peter C, Clarke M, Kleijnen JF, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 62(10):e1–e34

    Article  PubMed  Google Scholar 

  • Lin Y, Yau H, Wang I, Zheng C, Chung K (2015) A novel dental implant guided surgery based on integration of surgical template and augmented reality. Clin Implant Dent Relat Res 17(3):543–553

    Article  PubMed  Google Scholar 

  • Lin L, Shi Y, Tan A, Bogari M, Zhu M, Xin Y, Xu H, Zhang Y, Xie L, Chai G (2016) Mandibular angle split osteotomy based on a novel augmented reality navigation using specialized robot-assisted arms – a feasibility study. J Cranio-Maxillofac Surg 44(2):215–223

    Article  Google Scholar 

  • Liu WP, Azizian M, Sorger J, Taylor RH, Reilly BK, Cleary K, Preciado D (2014) Cadaveric feasibility study of da Vinci Si-assisted cochlear implant with augmented visual navigation for otologic surgery. JAMA Otolaryngol Head Neck Surg 140(3):208–214

    Article  PubMed  Google Scholar 

  • Ma L, Zhao Z, Chen F, Zhang B, Fu L, Liao H (2017) Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study. Int J Comput Assist Radiol Surg 12(12):2205–2215

    Article  PubMed  Google Scholar 

  • Ma L, Zhao Z, Zhang B, Jiang W, Fu L, Zhang X, Liao H (2018) Three-dimensional augmented reality surgical navigation with hybrid optical and electromagnetic tracking for distal intramedullary nail interlocking. Int J Medical Rob Comput Assisted Surg 14(4):e1909

    Article  Google Scholar 

  • Ma L, Jiang W, Zhang B, Qu X, Ning G, Zhang X, Liao H (2019) Augmented reality surgical navigation with accurate CBCT-patient registration for dental implant placement. Med Biol Eng Comput 57(1):47–57

    Article  PubMed  Google Scholar 

  • Mahmoud N, Grasa OG, Nicolau SA, Doignon C, Soler L, Marescaux J, Montiel JMM (2017) On-patient see-through augmented reality based on visual SLAM. Int J Comput Assist Radiol Surg 12(1):1–11

    Article  PubMed  Google Scholar 

  • Mahvash M, Tabrizi LB (2013) A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochir 155(5):943–947

    Article  PubMed  Google Scholar 

  • Marmulla R, Hoppe H, Muhling J, Eggers G (2005) An augmented reality system for image-guided surgery. This article is derived from a previous article published in the journal International Congress Series. Int J Oral Maxillofac Surg 34(6):594–596

    Article  CAS  PubMed  Google Scholar 

  • Martins S, Vairinhos M, Eliseu S, Borgerson J (2016) Input system interface for image-guided surgery based on augmented reality. In: First International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW); 2016 December 1–3; Vila Real, Portugal. IEEE; 2017. Available from: https://doi.org/10.1109/TISHW.2016.7847779

  • Maruyama K, Watanabe E, Kin T, Saito K, Kumakiri A, Noguchi A, Nagane M, Shiokawa Y (2018) Smart glasses for neurosurgical navigation by augmented reality. Operative Neurosurg (Hagerstown) 15(5):551–556

    Article  Google Scholar 

  • Mezzana P, Scarinci F, Marabottini N (2011) Augmented reality in oculoplastic surgery: first iPhone application. Plastic Reconstruct Surg 127(3):57e–58e

    Article  CAS  Google Scholar 

  • Mischkowski RA, Zinser MJ, Kubler AC, Krug B, Seifert U, Zoller JE (2006) Application of an augmented reality tool for maxillary positioning in orthognathic surgery – a feasibility study. J Cranio-Maxillofac Surg 34(8):478–483

    Article  Google Scholar 

  • Mohring M, Lessig C, Bimber O (2004) Video see-through AR on consumer cell-phones. In: Third IEEE/ACM International Symposium on Mixed and Augmented Reality, 2004 Nov 5, IEEE, 2005 Jan 24. Available from: https://doi.org/10.1109/ISMAR.2004.63

  • Mondal SB, Gao S, Zhu N, Sudlow GP, Liang K, Som A, Akers WJ, Fields RC, Margenthaler J, Liang R, Gruev V, Achilefu S (2015) Binocular goggle augmented imaging and navigation system provides real-time fluorescence image guidance for tumor resection and sentinel lymph node mapping Sci Rep 5. Available from: https://doi.org/10.1038/srep12117

  • Müller M, Rassweiler M-C, Klein J, Seitel A, Gondan M, Baumhauer M, Teber D, Rassweiler JJ, Meinzer H-P, Maier-Hein L (2013) Mobile augmented reality for computer-assisted percutaneous nephrolithotomy. Int J Comput Assist Radiol Surg 8(4):663–675

    Article  PubMed  Google Scholar 

  • Parrini S, Cutolo F, Freschi C, Ferrari M, Ferrari V (2014) Augmented reality system for freehand guide of magnetic endovascular devices. IEEE Engineering in Medicine and Biology Society, pp 490–493

    Google Scholar 

  • Pauly O, Diotte B, Fallavollita P, Weidert S, Euler E, Navab N (2015) Machine learning-based augmented reality for improved surgical scene understanding. Comput Med Imag Graph 41:55–60

    Article  Google Scholar 

  • Pessaux P, Diana M, Soler L, Piardi T, Mutter D, Marescaux J (2015) Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbeck’s Arch Surg 400(3):381–385

    Article  Google Scholar 

  • Poulin F, Amiot L (2002) Interference during the use of an electromagnetic tracking system under OR conditions. J Biomech 35(6):733–737

    Article  PubMed  Google Scholar 

  • Profeta AC, Schilling C, McGurk M (2016) Augmented reality visualization in head and neck surgery: an overview of recent findings in sentinel node biopsy and future perspectives. Br J Oral Maxillofac Surg 54(6):694–696

    Article  PubMed  Google Scholar 

  • Qu M, Hou Y, Xu Y, Shen C, Zhu M, Xie L, Wang H, Zhang Y, Chai G (2015) Precise positioning of an intraoral distractor using augmented reality in patients with hemifacial microsomia. J Cranio-Maxillo-Facial Surg 43(1):106–112

    Article  Google Scholar 

  • Rodriguez PS., Becker BC, Lobes Jr LA, Riviere CN (2012) Comparative evaluation of monocular augmented-reality display for surgical microscopes. In: Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp 1409–1412

    Google Scholar 

  • Sayadi LR, Naides A, Eng M, Fijany A, Chopan M, Sayadi JJ, Shaterian A, Banyard DA, Evans GRD, Vyas R, Widgerow AD (2019) The new frontier: a review of augmented reality and virtual reality in plastic surgery. Aesthet Surg J 39(9):1007–1016

    Article  PubMed  Google Scholar 

  • Scolozzi P, Bijlenga P (2017) Removal of recurrent intraorbital tumour using a system of augmented reality. Br J Oral Maxillofac Surg 55(9):962–964

    Article  CAS  PubMed  Google Scholar 

  • Shamir RR, Horn M, Blum T, Mehrkens J, Shoshan Y, Joskowicz L, Navab N (2011) Trajectory planning with Augmented Reality for improved risk assessment in image-guided keyhole neurosurgery. In: International Symposium on Biomedical Imaging: from Nano to Macro. 2011 April 2–March 30, IEEE, 2011 June 9. Available from: https://doi.org/10.1109/ISBI.2011.5872773

  • Shao P, Ding H, Wang J, Liu P, Ling Q, Chen J, Xu J, Zhang S, Xu R (2014) Designing a wearable navigation system for image-guided cancer resection surgery. Ann Biomed Eng 42(11):2228–2237

    Article  PubMed  PubMed Central  Google Scholar 

  • Si W, Liao X, Qian Y, Wang Q (2018) Mixed reality guided radiofrequency needle placement: a pilot study. IEEE Access 6:31493–31502

    Article  Google Scholar 

  • Suenaga H, Hoang Tran H, Liao H, Masamune K, Dohi T, Hoshi K, Mori Y, Takato T (2013) Real-time in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study. Int J Oral Sci 5(2):98–102

    Article  PubMed  PubMed Central  Google Scholar 

  • Suenaga H, Tran HH, Liao H, Masamune K, Dohi T, Hoshi K, Takato T (2015) Vision-based markerless registration using stereo vision and an augmented reality surgical navigation system: a pilot study. BMC Med Imaging 15:51

    Article  PubMed  PubMed Central  Google Scholar 

  • Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T, Kosugi C, Higuchi R, Watayo Y, Yagawa Y, Uemura S, Tsuchiya H, Azuma T (2010) Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci 17(5):629–636

    Article  PubMed  Google Scholar 

  • Sun G, Wang F, Chen X, Yu X, Ma X, Zhou D, Zhu R, Xu B (2016) Impact of virtual and augmented reality based on intraoperative magnetic resonance imaging and functional neuronavigation in glioma surgery involving eloquent areas. World Neurosurg 96:375–382

    Article  PubMed  Google Scholar 

  • Sun G, Chen X, Hou Y, Yu X, Ma X, Liu G, Liu L, Zhang J, Tang H, Zhu R, Zhou D, Xu B (2017) Image-guided endoscopic surgery for spontaneous supratentorial intracerebral hematoma. J Neurosurg 127(3):537–542

    Article  PubMed  Google Scholar 

  • Tang R, Ma L, Xiang C, Wang X, Li A, Liao H, Dong J (2017) Augmented reality navigation in open surgery for hilar cholangiocarcinoma resection with hemihepatectomy using video-based in situ three-dimensional anatomical modeling: a case report. Medicine 96(37):e8083

    Article  PubMed  PubMed Central  Google Scholar 

  • Tepper OM, Rudy HL, Lefkowitz A, Weimer KA, Marks SM, Stern CS, Garfein ES (2017) Mixed reality with HoloLens: where virtual reality meets augmented reality in the operating room. Plast Reconstr Surg 140(5):1066–1070

    Article  CAS  PubMed  Google Scholar 

  • Tran HH, Suenaga H, Kuwana K, Masamune K, Dohi T, Nakajima S, Liao H (2011) Augmented reality system for oral surgery using 3D auto stereoscopic visualization. In: Medical image computing and computer-assisted intervention: MICCAI ...International Conference on Medical Image Computing and Computer-Assisted Intervention, vol 14, no. Pt 1, pp 81–88

    Google Scholar 

  • Vávra P, Roman J, Zonča P, Ihnát P, Němec M, Kumar J, Habib N, El-Gendi A (2017) Recent development of augmented reality in surgery: a review. J Healthcare Eng 2017:4574172. Available from: https://doi.org/10.1155/2017/4574172

  • Vogt S, Khamene A, Sauer F (2006) Reality augmentation for medical procedures: system architecture, single camera marker tracking, and system evaluation. Int J Comput Vis 70(2):179–190

    Article  Google Scholar 

  • Volonte F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P (2011) Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 18(4):506–509

    Article  PubMed  Google Scholar 

  • Wacker F, Vogt S, Khamene A, Sauer F, Wendt M, Duerk J, Lewin J, Wolf K (2005) MR image-guided needle biopsies with a combination of augmented reality and MRI: a pilot study in phantoms and animals. In: CARS 2005: Computerized Assisted Radiology Surgery, vol 1281, pp 424–428

    Google Scholar 

  • Wang J, Suenaga H, Hoshi K, Yang L, Kobayashi E, Sakuma I, Liao H (2014) Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery. IEEE Trans Biomed Eng 61(4):1295–1304

    Article  PubMed  Google Scholar 

  • Wang J, Suenaga H, Liao H, Hoshi K, Yang L, Kobayashi E, Sakuma I (2015) Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation. Comput Med Imag Graph 40:147–159

    Article  Google Scholar 

  • Wang H, Wang F, Leong APY, Xu L, Chen X, Wang Q (2016) Precision insertion of percutaneous sacroiliac screws using a novel augmented reality-based navigation system: a pilot study. Int Orthop 40(9):1941–1947

    Article  PubMed  Google Scholar 

  • Wang J, Suenaga H, Yang L, Kobayashi E, Sakuma I (2017) Video see-through augmented reality for oral and maxillofacial surgery. Int J Med Rob Comput Assisted Surg 13(2). Available from: https://doi.org/10.1002/rcs.1754

  • Wen R, Chui C-K, Ong S-H, Lim K-B, Chang SK-Y (2013) Projection-based visual guidance for robot-aided RF needle insertion. Int J Comput Assist Radiol Surg 8(6):1015–1025

    Article  PubMed  Google Scholar 

  • Wen R, Tay W-L, Nguyen BP, Chng C-B, Chui C-K (2014) Hand gesture guided robot-assisted surgery based on a direct augmented reality interface. Comput Methods Prog Biomed 116(2):68–80

    Article  Google Scholar 

  • Wen R, Chng C, Chui C (2017) Augmented reality guidance with multimodality imaging data and depth-perceived interaction for robot-assisted surgery. Robotics 6(2):13

    Article  Google Scholar 

  • Wesarg S, Firle E, Schwald B, Seibert H, Zogal P, Roeddiger S (2004) Accuracy of needle implantation in brachytherapy using a medical AR system – a phantom study. In: Medical imaging 2004: visualization, image-guided procedures, and display, vol 5367, pp 341–352

    Google Scholar 

  • West JB, Fitzpatrick JM, Toms SA, Maurer CR Jr, Maciunas RJ (2001) Fiducial point placement and the accuracy of point-based, rigid body registration. Neurosurgery 48(4):810–816. discussion 816-7

    Google Scholar 

  • Wong K, Yee HM, Xavier BA, Grillone GA (2018) Applications of augmented reality in otolaryngology: a systematic review. Otolaryngol Head Neck Surg 159(6):956–967

    Article  PubMed  Google Scholar 

  • Wu J, Wang M, Liu K, Hu M, Lee P (2014) Real-time advanced spinal surgery via visible patient model and augmented reality system. Comput Methods Prog Biomed 113(3):869–881

    Article  Google Scholar 

  • Wu ML, Chien JC, Wu CT, Lee JD (2018) An augmented reality system using improved-iterative closest point algorithm for on-patient medical image visualization. Sensors (Basel) 18(8):E2505 [pii]

    Google Scholar 

  • Yang G, Hu H, Wang B, Wen C, Huang Y, Fu Y, Su Y, Wu J (2018) A novel method and system for stereotactic surgical procedures. In: IEEE Signal Processing in Medicine and Biology Symposium (SPMB), 2017 December 2, IEEE, 2018 January 15. Available from: https://doi.org/10.1109/SPMB.2017.8257036

  • Yoon J, Chen R, Kim E, Akinduro O, Kerezoudis P, Han P, Si P, Freeman W, Diaz R, Komotar R, Pirris S, Brown B, Bydon M, Wang M, Wharen R, Quinones-Hinojosa A (2018) Augmented reality for the surgeon: systematic review. Int J Med Rob Comput Assisted Surg 14(4):e1914

    Article  Google Scholar 

  • Yoshino M, Saito T, Kin T, Nakagawa D, Nakatomi H, Oyama H, Saito N (2015) A microscopic optically tracking navigation system that uses high-resolution 3D computer graphics. Neurol Med Chir 55(8):674–679

    Article  Google Scholar 

  • Zeng B, Meng F, Ding H, Wang G (2017) A surgical robot with augmented reality visualization for stereoelectroencephalography electrode implantation. Int J Comput Assist Radiol Surg 12(8):1355–1368

    Article  PubMed  Google Scholar 

  • Zhang X, Chen G, Liao H (2015) A high-accuracy surgical augmented reality system using enhanced integral videography image overlay. In: Conference proceedings: ...Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, vol 2015, pp 4210–4213

    Google Scholar 

  • Zhang X, Chen G, Liao H (2017) High-quality see-through surgical guidance system using enhanced 3-D autostereoscopic augmented reality. IEEE Trans Biomed Eng 64(8):1815–1825

    Article  PubMed  Google Scholar 

  • Zhu M, Chai G, Zhang Y, Ma X, Gan J (2011) Registration strategy using occlusal splint based on augmented reality for mandibular angle oblique split osteotomy. J Craniofac Surg 22(5):1806–1809

    Article  PubMed  Google Scholar 

  • Zhu M, Chai G, Lin L, Xin Y, Tan A, Bogari M, Zhang Y, Li Q (2016) Effectiveness of a novel augmented reality-based navigation system in treatment of orbital hypertelorism. Ann Plast Surg 77(6):662–668

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank the staff of the Medical Library of the University of Aberdeen for their advice and Prof. Jennifer Cleland and Dr. Jenny Gregory for discussion and support. This work was funded by the Roland Sutton Academic Trust (0053/R/17) and an Elphinstone PhD Scholarship from the University of Aberdeen.

Author information

Authors and Affiliations

  1. School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, Aberdeen, UK

    Laura Pérez-Pachón, Terry Lowe & Flora Gröning

  2. School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK

    Matthieu Poyade

  3. Head and Neck Oncology Unit, Aberdeen Royal Infirmary (NHS Grampian), Aberdeen, UK

    Terry Lowe

Authors
  1. Laura Pérez-Pachón
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthieu Poyade
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Terry Lowe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Flora Gröning
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laura Pérez-Pachón .

Editor information

Editors and Affiliations

  1. College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK

    Paul M. Rea

10.1 Supplementary Electronic Material (S)

S10.1 Appendix

. Database of reviewed studies categorised according to the variables considered in this review (XLSX 23 kb)

Appendix (Tables 10.5, 10.6, 10.7, 10.8, and 10.9)

Appendix (Tables 10.5, 10.6, 10.7, 10.8, and 10.9)

Table 10.5 Search strategy used in this systematic review illustrated by the search done in MEDLINE
Full size table
Table 10.6 Reviewed studies organised according to surgical procedure
Full size table
Table 10.7 Classification of reviewed automatic optical tracking studies according to the tracking method
Full size table
Table 10.8 Reviewed studies organised according to the system’s usability
Full size table
Table 10.9 Classification of reviewed automatic optical tracking studies according to display device
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Pérez-Pachón, L., Poyade, M., Lowe, T., Gröning, F. (2020). Image Overlay Surgery Based on Augmented Reality: A Systematic Review. In: Rea, P.M. (eds) Biomedical Visualisation. Advances in Experimental Medicine and Biology, vol 1320. Springer, Cham. https://doi.org/10.1007/978-3-030-47483-6_10

Download citation

Publish with us

Policies and ethics

Search

Navigation