Advertisement

Neurosurgical Review

, Volume 40, Issue 4, pp 537–548 | Cite as

Augmented reality in neurosurgery: a systematic review

  • Antonio MeolaEmail author
  • Fabrizio Cutolo
  • Marina Carbone
  • Federico Cagnazzo
  • Mauro Ferrari
  • Vincenzo Ferrari
Review

Abstract

Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only a few AR systems have been tested in a clinical setting. The aim is to review such devices. We performed a PubMed search of reports restricted to human studies of in vivo applications of AR in any neurosurgical procedure using the search terms “Augmented reality” and “Neurosurgery.” Eligibility assessment was performed independently by two reviewers in an unblinded standardized manner. The systems were qualitatively evaluated on the basis of the following: neurosurgical subspecialty of application, pathology of treated lesions and lesion locations, real data source, virtual data source, tracking modality, registration technique, visualization processing, display type, and perception location. Eighteen studies were included during the period 1996 to September 30, 2015. The AR systems were grouped by the real data source: microscope (8), hand- or head-held cameras (4), direct patient view (2), endoscope (1), and X-ray fluoroscopy (1) head-mounted display (1). A total of 195 lesions were treated: 75 (38.46 %) were neoplastic, 77 (39.48 %) neurovascular, and 1 (0.51 %) hydrocephalus, and 42 (21.53 %) were undetermined. Current literature confirms that AR is a reliable and versatile tool when performing minimally invasive approaches in a wide range of neurosurgical diseases, although prospective randomized studies are not yet available and technical improvements are needed.

Keywords

Augmented reality Virtual reality Tumor Aneurysm Arterovenous malformation Cavernous malformation Navigation 

Notes

Acknowledgments

Dr. Meola is supported by an NIH award (R25CA089017). We sincerely thank Nina Geller, PhD, for the careful and rigorous editing of the present manuscript.

References

  1. 1.
    Badiali G, Ferrari V, Cutolo F, Freschi C, Caramella D, Bianchi A, Marchetti C (2014) Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Cranio-Maxillofac Surg. doi: 10.1016/j.jcms.2014.09.001 Google Scholar
  2. 2.
    Besharati Tabrizi L, Mahvash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123:206–211. doi: 10.3171/2014.9.JNS141001 CrossRefPubMedGoogle Scholar
  3. 3.
    Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery 10(Suppl 2):252–260. doi: 10.1227/NEU.0000000000000328, discussion 260–251CrossRefPubMedGoogle Scholar
  4. 4.
    Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir 156:1769–1774. doi: 10.1007/s00701-014-2183-9 CrossRefPubMedGoogle Scholar
  5. 5.
    Cabrilo I, Schaller K, Bijlenga P (2015) Augmented reality-assisted bypass surgery: embracing minimal invasiveness. World Neurosurgery 83:596–602. doi: 10.1016/j.wneu.2014.12.020 CrossRefPubMedGoogle Scholar
  6. 6.
    Craig AB (2013) Understanding augmented reality: concepts and applications. Morgan Kaufmann, WalthamGoogle Scholar
  7. 7.
    Cutolo F, Badiali G, Ferrari V (2015) Human-PnP: ergonomic AR interaction paradigm for manual placement of rigid bodies. In: Augmented environments for computer-assisted interventions. Springer International Publishing, pp 50–60Google Scholar
  8. 8.
    Deng W, Li F, Wang M, Song Z (2014) Easy-to-use augmented reality neuronavigation using a wireless tablet PC. Stereotact Funct Neurosurg 92:17–24. doi: 10.1159/000354816 CrossRefPubMedGoogle Scholar
  9. 9.
    Doyle WK (1996) Low end interactive image-directed neurosurgery. Update on rudimentary augmented reality used in epilepsy surgery. Stud Health Technol Inform 29:1–11. doi: 10.3233/978-1-60750-873-1-1 PubMedGoogle Scholar
  10. 10.
    Drouin S, Kersten-Oertel M, Collins DL (2015) Interaction-based registration correction for improved augmented reality overlay in neurosurgery. In: Linte C, Yaniv Z, Fallavollita P (eds) Augmented environments for computer-assisted interventions, vol 9365. Lecture Notes in Computer Science. Springer International Publishing, pp 21–29. doi: 10.1007/978-3-319-24601-7_3
  11. 11.
    Edwards PJ, Johnson LG, Hawkes DJ, Fenlon MR, Strong A, Gleeson M (2004) Clinical experience and perception in stereo augmented reality surgical navigation. In: YG Z, Jiang T (eds) MIAR. Springer-Verlag, Berlin, pp 369–376Google Scholar
  12. 12.
    Ferrari V, Cutolo F (2016) Letter to the Editor on “Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique”. Accepted by Journal of NeurosurgeryGoogle Scholar
  13. 13.
    Ferrari V, Megali G, Troia E, Pietrabissa A, Mosca F (2009) A 3-D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng 56:2627–2633. doi: 10.1109/TBME.2009.2028013 CrossRefPubMedGoogle Scholar
  14. 14.
    Inoue D, Cho B, Mori M, Kikkawa Y, Amano T, Nakamizo A, Yoshimoto K, Mizoguchi M, Tomikawa M, Hong J, Hashizume M, Sasaki T (2013) Preliminary study on the clinical application of augmented reality neuronavigation. J Neurol Surg Part A, Central European Neurosurg 74:71–76. doi: 10.1055/s-0032-1333415 CrossRefGoogle Scholar
  15. 15.
    Iseki H, Masutani Y, Iwahara M, Tanikawa T, Muragaki Y, Taira T, Dohi T, Takakura K (1997) Volumegraph (overlaid three-dimensional image-guided navigation). Clinical application of augmented reality in neurosurgery. Stereotact Funct Neurosurg 68:18–24. doi: 10.1159/000099897 CrossRefPubMedGoogle Scholar
  16. 16.
    Kantelhardt SR, Gutenberg A, Neulen A, Keric N, Renovanz M, Giese A (2015) Video-assisted navigation for adjustment of image-guidance accuracy to slight brain shift. Neurosurgery. doi: 10.1227/NEU.0000000000000921 Google Scholar
  17. 17.
    Kawamata T, Iseki H, Shibasaki T, Hori T (2002) Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery 50:1393–1397PubMedGoogle Scholar
  18. 18.
    Kersten-Oertel M, Chen SJ, Collins DL (2014) An evaluation of depth enhancing perceptual cues for vascular volume visualization in neurosurgery. IEEE Trans Vis Comput Graph 20:391–403. doi: 10.1109/TVCG.2013.240 CrossRefPubMedGoogle Scholar
  19. 19.
    Kersten-Oertel M, Jannin P, Collins DL (2010) DVV: towards a taxonomy for mixed reality visualization in image guided surgery. Lect Notes Comput Sc 6326:334–343CrossRefGoogle Scholar
  20. 20.
    Kersten-Oertel M, Jannin P, Collins DL (2012) DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Trans Vis Comput Graph 18:332–352. doi: 10.1109/TVCG.2011.50 CrossRefPubMedGoogle Scholar
  21. 21.
    Kersten-Oertel M, Jannin P, Collins DL (2013) The state of the art of visualization in mixed reality image guided surgery. Comput Med Imaging Graph 37:98–112. doi: 10.1016/j.compmedimag.2013.01.009 CrossRefPubMedGoogle Scholar
  22. 22.
    King AP, Edwards PJ, Maurer CR Jr, de Cunha DA, Hawkes DJ, Hill DL, Gaston RP, Fenlon MR, Strong AJ, Chandler CL, Richards A, Gleeson MJ (1999) A system for microscope-assisted guided interventions. Stereotact Funct Neurosurg 72:107–111CrossRefPubMedGoogle Scholar
  23. 23.
    Kockro RA, Tsai YT, Ng I, Hwang P, Zhu C, Agusanto K, Hong LX, Serra L (2009) Dex-ray: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery 65:795–807. doi: 10.1227/01.NEU.0000349918.36700.1C, discussion 807–798CrossRefPubMedGoogle Scholar
  24. 24.
    Kruijff E, Swan JE, Feiner S Perceptual issues in augmented reality revisited. In: Mixed and Augmented Reality (ISMAR), 2010 9th IEEE International Symposium on, 13–16 Oct. 2010 2010. pp 3–12. doi: 10.1109/ISMAR.2010.5643530
  25. 25.
    Le Moigne J, Netanyahu NS, Eastman RD (2010) Image registration for remote sensing. Cambridge University Press, CambridgeGoogle Scholar
  26. 26.
    Levoy M (2006) Light fields and computational imaging. Computer:46–55Google Scholar
  27. 27.
    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 339:b2700. doi: 10.1136/bmj.b2700 CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Lippmann G (1908) Epreuves reversibles donnant la sensation du relief. J Phys Theor Appl 7:821–825CrossRefGoogle Scholar
  29. 29.
    Lovo EE, Quintana JC, Puebla MC, Torrealba G, Santos JL, Lira IH, Tagle P (2007) A novel, inexpensive method of image coregistration for applications in image-guided surgery using augmented reality. Neurosurgery 60:366–371. doi: 10.1227/01.NEU.0000255360.32689.FA, discussion 371–362PubMedGoogle Scholar
  30. 30.
    Low D, Lee CK, Dip LL, Ng WH, Ang BT, Ng I (2010) Augmented reality neurosurgical planning and navigation for surgical excision of parasagittal, falcine and convexity meningiomas. Br J Neurosurg 24:69–74. doi: 10.3109/02688690903506093 CrossRefPubMedGoogle Scholar
  31. 31.
    Mahvash M, Besharati Tabrizi L (2013) A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochir 155:943–947. doi: 10.1007/s00701-013-1668-2 CrossRefPubMedGoogle Scholar
  32. 32.
    Masutani Y, Dohi T, Yamane F, Iseki H, Takakura K (1998) Augmented reality visualization system for intravascular neurosurgery. Comput Aided Surg 3:239–247. doi: 10.1002/(SICI)1097-0150(1998)3:5<239::AID-IGS3>3.0.CO;2-B CrossRefPubMedGoogle Scholar
  33. 33.
    Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. Ieice T Inf Syst E77d:1321–1329Google Scholar
  34. 34.
    Paul P, Fleig O, Jannin P (2005) Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation. IEEE Trans Med Imaging 24:1500–1511. doi: 10.1109/TMI.2005.857029 CrossRefPubMedGoogle Scholar
  35. 35.
    Perneczky A, Reisch R, Tschabitscher M (2008) Keyhole approaches in neurosurgery. Springer, WienGoogle Scholar
  36. 36.
    Reichelt S, Häussler R, Fütterer G, Leister N Depth cues in human visual perception and their realization in 3D displays. In: SPIE Defense, Security, and Sensing, 2010. International Society for Optics and Photonics, pp 76900B-76900B-76912Google Scholar
  37. 37.
    Rhoton AL, Rhoton AL, Congress of Neurological Surgeons. (2003) Rhoton cranial anatomy and surgical approaches. Neurosurgery,, vol 53. Lippincott Williams & Wilkins, PhiladelphiaGoogle Scholar
  38. 38.
    Stadie AT, Reisch R, Kockro RA, Fischer G, Schwandt E, Boor S, Stoeter P (2009) Minimally invasive cerebral cavernoma surgery using keyhole approaches—solutions for technique-related limitations. Minimally Invasive Neurosurgery: MIN 52:9–16. doi: 10.1055/s-0028-1103305 CrossRefPubMedGoogle Scholar
  39. 39.
    Wang J, Suenaga H, Hoshi K, Yang L, Kobayashi E, Sakuma I, Liao H (2014) Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery. IEEE Trans Biomed Eng 61:1295–1304. doi: 10.1109/TBME.2014.2301191 CrossRefPubMedGoogle Scholar
  40. 40.
    Zhang X, Chen G, Liao H A High-accuracy surgical augmented reality system using enhanced integral videography image overlay. In: 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society EMBC, Milano, 2015.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Antonio Meola
    • 1
    Email author
  • Fabrizio Cutolo
    • 2
  • Marina Carbone
    • 2
  • Federico Cagnazzo
    • 3
  • Mauro Ferrari
    • 2
    • 4
  • Vincenzo Ferrari
    • 2
    • 5
  1. 1.Department of Neurosurgery, Brigham and Women’s HospitalHarvard Medical SchoolBostonUSA
  2. 2.Department of Translational Research and New Technologies in Medicine and Surgery, EndoCAS CenterUniversity of PisaPisaItaly
  3. 3.Department of Neurological SurgeryUniversity of PisaPisaItaly
  4. 4.Department of Vascular SurgeryPisa University Medical SchoolPisaItaly
  5. 5.Department of Information EngineeringUniversity of PisaPisaItaly

Personalised recommendations