Skip to main content

Advertisement

SpringerLink
Log in

Augmented reality in neurosurgery: a systematic review

  • Review
  • Published:
Neurosurgical Review Aims and scope Submit manuscript

Abstract

Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only a few AR systems have been tested in a clinical setting. The aim is to review such devices. We performed a PubMed search of reports restricted to human studies of in vivo applications of AR in any neurosurgical procedure using the search terms “Augmented reality” and “Neurosurgery.” Eligibility assessment was performed independently by two reviewers in an unblinded standardized manner. The systems were qualitatively evaluated on the basis of the following: neurosurgical subspecialty of application, pathology of treated lesions and lesion locations, real data source, virtual data source, tracking modality, registration technique, visualization processing, display type, and perception location. Eighteen studies were included during the period 1996 to September 30, 2015. The AR systems were grouped by the real data source: microscope (8), hand- or head-held cameras (4), direct patient view (2), endoscope (1), and X-ray fluoroscopy (1) head-mounted display (1). A total of 195 lesions were treated: 75 (38.46 %) were neoplastic, 77 (39.48 %) neurovascular, and 1 (0.51 %) hydrocephalus, and 42 (21.53 %) were undetermined. Current literature confirms that AR is a reliable and versatile tool when performing minimally invasive approaches in a wide range of neurosurgical diseases, although prospective randomized studies are not yet available and technical improvements are needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Badiali G, Ferrari V, Cutolo F, Freschi C, Caramella D, Bianchi A, Marchetti C (2014) Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Cranio-Maxillofac Surg. doi:10.1016/j.jcms.2014.09.001

    Google Scholar 

  2. Besharati Tabrizi L, Mahvash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123:206–211. doi:10.3171/2014.9.JNS141001

    Article  PubMed  Google Scholar 

  3. Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery 10(Suppl 2):252–260. doi:10.1227/NEU.0000000000000328, discussion 260–251

    Article  PubMed  Google Scholar 

  4. Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir 156:1769–1774. doi:10.1007/s00701-014-2183-9

    Article  PubMed  Google Scholar 

  5. Cabrilo I, Schaller K, Bijlenga P (2015) Augmented reality-assisted bypass surgery: embracing minimal invasiveness. World Neurosurgery 83:596–602. doi:10.1016/j.wneu.2014.12.020

    Article  PubMed  Google Scholar 

  6. Craig AB (2013) Understanding augmented reality: concepts and applications. Morgan Kaufmann, Waltham

    Google Scholar 

  7. Cutolo F, Badiali G, Ferrari V (2015) Human-PnP: ergonomic AR interaction paradigm for manual placement of rigid bodies. In: Augmented environments for computer-assisted interventions. Springer International Publishing, pp 50–60

  8. Deng W, Li F, Wang M, Song Z (2014) Easy-to-use augmented reality neuronavigation using a wireless tablet PC. Stereotact Funct Neurosurg 92:17–24. doi:10.1159/000354816

    Article  PubMed  Google Scholar 

  9. Doyle WK (1996) Low end interactive image-directed neurosurgery. Update on rudimentary augmented reality used in epilepsy surgery. Stud Health Technol Inform 29:1–11. doi:10.3233/978-1-60750-873-1-1

    CAS  PubMed  Google Scholar 

  10. Drouin S, Kersten-Oertel M, Collins DL (2015) Interaction-based registration correction for improved augmented reality overlay in neurosurgery. In: Linte C, Yaniv Z, Fallavollita P (eds) Augmented environments for computer-assisted interventions, vol 9365. Lecture Notes in Computer Science. Springer International Publishing, pp 21–29. doi:10.1007/978-3-319-24601-7_3

  11. Edwards PJ, Johnson LG, Hawkes DJ, Fenlon MR, Strong A, Gleeson M (2004) Clinical experience and perception in stereo augmented reality surgical navigation. In: YG Z, Jiang T (eds) MIAR. Springer-Verlag, Berlin, pp 369–376

    Google Scholar 

  12. Ferrari V, Cutolo F (2016) Letter to the Editor on “Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique”. Accepted by Journal of Neurosurgery

  13. Ferrari V, Megali G, Troia E, Pietrabissa A, Mosca F (2009) A 3-D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng 56:2627–2633. doi:10.1109/TBME.2009.2028013

    Article  PubMed  Google Scholar 

  14. Inoue D, Cho B, Mori M, Kikkawa Y, Amano T, Nakamizo A, Yoshimoto K, Mizoguchi M, Tomikawa M, Hong J, Hashizume M, Sasaki T (2013) Preliminary study on the clinical application of augmented reality neuronavigation. J Neurol Surg Part A, Central European Neurosurg 74:71–76. doi:10.1055/s-0032-1333415

    Article  CAS  Google Scholar 

  15. Iseki H, Masutani Y, Iwahara M, Tanikawa T, Muragaki Y, Taira T, Dohi T, Takakura K (1997) Volumegraph (overlaid three-dimensional image-guided navigation). Clinical application of augmented reality in neurosurgery. Stereotact Funct Neurosurg 68:18–24. doi:10.1159/000099897

    Article  CAS  PubMed  Google Scholar 

  16. Kantelhardt SR, Gutenberg A, Neulen A, Keric N, Renovanz M, Giese A (2015) Video-assisted navigation for adjustment of image-guidance accuracy to slight brain shift. Neurosurgery. doi:10.1227/NEU.0000000000000921

    Google Scholar 

  17. Kawamata T, Iseki H, Shibasaki T, Hori T (2002) Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery 50:1393–1397

    PubMed  Google Scholar 

  18. Kersten-Oertel M, Chen SJ, Collins DL (2014) An evaluation of depth enhancing perceptual cues for vascular volume visualization in neurosurgery. IEEE Trans Vis Comput Graph 20:391–403. doi:10.1109/TVCG.2013.240

    Article  PubMed  Google Scholar 

  19. Kersten-Oertel M, Jannin P, Collins DL (2010) DVV: towards a taxonomy for mixed reality visualization in image guided surgery. Lect Notes Comput Sc 6326:334–343

    Article  Google Scholar 

  20. Kersten-Oertel M, Jannin P, Collins DL (2012) DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Trans Vis Comput Graph 18:332–352. doi:10.1109/TVCG.2011.50

    Article  PubMed  Google Scholar 

  21. Kersten-Oertel M, Jannin P, Collins DL (2013) The state of the art of visualization in mixed reality image guided surgery. Comput Med Imaging Graph 37:98–112. doi:10.1016/j.compmedimag.2013.01.009

    Article  PubMed  Google Scholar 

  22. King AP, Edwards PJ, Maurer CR Jr, de Cunha DA, Hawkes DJ, Hill DL, Gaston RP, Fenlon MR, Strong AJ, Chandler CL, Richards A, Gleeson MJ (1999) A system for microscope-assisted guided interventions. Stereotact Funct Neurosurg 72:107–111

    Article  CAS  PubMed  Google Scholar 

  23. Kockro RA, Tsai YT, Ng I, Hwang P, Zhu C, Agusanto K, Hong LX, Serra L (2009) Dex-ray: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery 65:795–807. doi:10.1227/01.NEU.0000349918.36700.1C, discussion 807–798

    Article  PubMed  Google Scholar 

  24. Kruijff E, Swan JE, Feiner S Perceptual issues in augmented reality revisited. In: Mixed and Augmented Reality (ISMAR), 2010 9th IEEE International Symposium on, 13–16 Oct. 2010 2010. pp 3–12. doi:10.1109/ISMAR.2010.5643530

  25. Le Moigne J, Netanyahu NS, Eastman RD (2010) Image registration for remote sensing. Cambridge University Press, Cambridge

    Google Scholar 

  26. Levoy M (2006) Light fields and computational imaging. Computer:46–55

  27. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 339:b2700. doi:10.1136/bmj.b2700

    Article  PubMed  PubMed Central  Google Scholar 

  28. Lippmann G (1908) Epreuves reversibles donnant la sensation du relief. J Phys Theor Appl 7:821–825

    Article  Google Scholar 

  29. Lovo EE, Quintana JC, Puebla MC, Torrealba G, Santos JL, Lira IH, Tagle P (2007) A novel, inexpensive method of image coregistration for applications in image-guided surgery using augmented reality. Neurosurgery 60:366–371. doi:10.1227/01.NEU.0000255360.32689.FA, discussion 371–362

    PubMed  Google Scholar 

  30. Low D, Lee CK, Dip LL, Ng WH, Ang BT, Ng I (2010) Augmented reality neurosurgical planning and navigation for surgical excision of parasagittal, falcine and convexity meningiomas. Br J Neurosurg 24:69–74. doi:10.3109/02688690903506093

    Article  PubMed  Google Scholar 

  31. Mahvash M, Besharati Tabrizi L (2013) A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochir 155:943–947. doi:10.1007/s00701-013-1668-2

    Article  PubMed  Google Scholar 

  32. Masutani Y, Dohi T, Yamane F, Iseki H, Takakura K (1998) Augmented reality visualization system for intravascular neurosurgery. Comput Aided Surg 3:239–247. doi:10.1002/(SICI)1097-0150(1998)3:5<239::AID-IGS3>3.0.CO;2-B

    Article  CAS  PubMed  Google Scholar 

  33. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. Ieice T Inf Syst E77d:1321–1329

    Google Scholar 

  34. Paul P, Fleig O, Jannin P (2005) Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation. IEEE Trans Med Imaging 24:1500–1511. doi:10.1109/TMI.2005.857029

    Article  PubMed  Google Scholar 

  35. Perneczky A, Reisch R, Tschabitscher M (2008) Keyhole approaches in neurosurgery. Springer, Wien

    Google Scholar 

  36. Reichelt S, Häussler R, Fütterer G, Leister N Depth cues in human visual perception and their realization in 3D displays. In: SPIE Defense, Security, and Sensing, 2010. International Society for Optics and Photonics, pp 76900B-76900B-76912

  37. Rhoton AL, Rhoton AL, Congress of Neurological Surgeons. (2003) Rhoton cranial anatomy and surgical approaches. Neurosurgery,, vol 53. Lippincott Williams & Wilkins, Philadelphia

  38. Stadie AT, Reisch R, Kockro RA, Fischer G, Schwandt E, Boor S, Stoeter P (2009) Minimally invasive cerebral cavernoma surgery using keyhole approaches—solutions for technique-related limitations. Minimally Invasive Neurosurgery: MIN 52:9–16. doi:10.1055/s-0028-1103305

    Article  CAS  PubMed  Google Scholar 

  39. Wang J, Suenaga H, Hoshi K, Yang L, Kobayashi E, Sakuma I, Liao H (2014) Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery. IEEE Trans Biomed Eng 61:1295–1304. doi:10.1109/TBME.2014.2301191

    Article  PubMed  Google Scholar 

  40. Zhang X, Chen G, Liao H A High-accuracy surgical augmented reality system using enhanced integral videography image overlay. In: 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society EMBC, Milano, 2015.

Download references

Acknowledgments

Dr. Meola is supported by an NIH award (R25CA089017). We sincerely thank Nina Geller, PhD, for the careful and rigorous editing of the present manuscript.

Author information

Authors and Affiliations

  1. Department of Neurosurgery, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis Street, 02115, Boston, MA, USA

    Antonio Meola

  2. Department of Translational Research and New Technologies in Medicine and Surgery, EndoCAS Center, University of Pisa, Pisa, Italy

    Fabrizio Cutolo, Marina Carbone, Mauro Ferrari & Vincenzo Ferrari

  3. Department of Neurological Surgery, University of Pisa, Pisa, Italy

    Federico Cagnazzo

  4. Department of Vascular Surgery, Pisa University Medical School, Pisa, Italy

    Mauro Ferrari

  5. Department of Information Engineering, University of Pisa, Pisa, Italy

    Vincenzo Ferrari

Authors
  1. Antonio Meola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabrizio Cutolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marina Carbone
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Federico Cagnazzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mauro Ferrari
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vincenzo Ferrari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Meola.

Additional information

Comments

Yavor Enchev, Varna, Bulgaria

Neuronavigation exemplifies one of the newest and most rapidly developing neurosurgical technologies. Neuronavigation gradually became inseparable part of the efforts of neurosurgeons to achieve minimal invasiveness with maximal effect simultaneously reducing the hazards for the patients’ safety. It is extremely diverse technique with multiple forms and subtypes. Augmented reality represents a separate direction in the development of the image-guided technology allowing incorporation of virtual image data into the real surgical field. However, the clinical experience with the augmented reality in neurosurgery is quite limited.

The authors performed meticulous review of the literature in PUBMED pertinent to the augmented reality in neurosurgery. The eligible papers were analyzed according to the relevant neurosurgical subspecialty, type of pathologies and their location as well as many additional related technical aspects. Quantitative assessment of the clinical usefulness and feasibility were not available from the selected data. Significant matter is the lack of data for the accuracy of the augmented reality devices due to the inconsistency of its definition in the different papers.

Meaningful and useful for the practice conclusions, from this interesting review, could not be draught due to the limited patient population, the lack of data for quantitative assessment and the impossibility for statistical analysis. Future, more numerous series would be crucial for the potentially wider distribution and application of this approach.

Uwe Spetzger, Karlsruhe, Germany

The paper augment reality in Neurosurgery provides a perfect and systematic overview and demonstrates the development and improvement of neuronavigation systems with the implementation of AR in the last years. Augmented reality is a helpful device to visualize hidden structures in the skull. However the paper of A. Meola et al., demonstrate that AR is not only a device or tool, it is more a strategy or philosophy to improve our surgical planning and provides a high-end simulation of the procedure. The auxiliary to look through or behind anatomical structures is the key-benefit and AR will be utilized more frequently in the near future.

During the 90es neuronavigation gets more and more in the focus and meanwhile is a routine tool in our daily neurosurgical practice (1). Initially, arm based and consecutively also the first optical navigations systems allowed a detailed depiction of radiological data and integrated them into the real anatomy and the microscopic view of the neurosurgeons. The integration of neuronavigation in our routine work in cranial and also in spinal neurosurgery was one of the milestones of modern neurosurgery, and the acceptance of AR in modern neurosurgery will increase continuously.

The paper perfectly compares different augmented reality systems and also shows different philosophies of AR and their focus on cranial neurosurgery. This review gives detailed information about the development and also demonstrates the usefulness of the indication in different pathologies. The authors also point out that AR is an additional part in modern neuronavigation and also indicated in their review, that further efforts are necessary to make these systems more user-friendly and intuitive. Another aspect could be, using AR as a platform for integration of functional data to enhance these systems.

In this review, I miss the really important aspect that augmented reality systems are perfect tools for education and especially for practical surgical training (2). The capability of high-end visual representation of the anatomy and the combined radiological data, will create a perfect simulation and educational tool to learn the surgical anatomy much better as in textbooks. Therefore, I want to point out the importance to integrate AR systems more into the education and training of our young neurosurgeons. However, as a common warning, we always should be aware, that all augmented reality tools bear the potential risk of inaccuracy and errors and we have to keep an eye on the precise registration and the exact handling. Just as is all other navigation systems, accuracy of the registration and the navigation are the basis of all our AR data (3).

Also the planning and manufacturing of 3D implants for the reconstruction of the skull is an upcoming field for using AR (4). Just as modern computer-assisted pre-planning and especially the exact surgical implantation of individualized and patient-specific 3D laser printed spinal implants will be the next important issue for AR in spine surgery (5).

References

(1) Spetzger U, Laborde G, Gilsbach JM. Frameless neuronavigation in modern neurosurgery. Minim Invasive Neurosurg. 1995 Dec. 38(4):163–6. Review

(2) Krombach G, Ganser A, Fricke C, Rohde V, Reinges M, Gilsbach J, Spetzger U. Virtual placement of frontal ventricular catheters using frameless neuronavigation: an “unbloody training” for young neurosurgeons. Minim Invasive Neurosurg. 2000 Dec. 43(4):171–5

(3) Spetzger U, Hubbe U, Struffert T, Reinges MH, Krings T, Krombach GA, Zentner J, Gilsbach JM, Stiehl HS. Error analysis in cranial neuronavigation. Minim Invasive Neurosurg. 2002 Mar; 45(1):6–10

(4) Vougioukas VI, Hubbe U, van Velthoven V, Freiman TM, Schramm A, Spetzger U. Neuronavigation-assisted cranial reconstruction. Neurosurgery. 2004 Jul; 55(1):162–7; discussion 167

(5) Spetzger U, Frasca M, König SA. Surgical planning, manufacturing and implantation of an individualized cervical fusion titanium cage using patient-specific data. Eur Spine J. 2016 Mar 1 [Epub ahead of print]

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meola, A., Cutolo, F., Carbone, M. et al. Augmented reality in neurosurgery: a systematic review. Neurosurg Rev 40, 537–548 (2017). https://doi.org/10.1007/s10143-016-0732-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10143-016-0732-9

Keywords

Search

Navigation