Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography

  • 1115 Accesses

  • 12 Citations



Augmented reality has potential to enhance surgical navigation and visualization. We determined whether head-mounted display augmented reality (HMD-AR) with superimposed computed tomography (CT) data could allow the wearer to percutaneously guide pedicle screw placement in an opaque lumbar model with no real-time fluoroscopic guidance.


CT imaging was obtained of a phantom composed of L1–L3 Sawbones vertebrae in opaque silicone. Preprocedural planning was performed by creating virtual trajectories of appropriate angle and depth for ideal approach into the pedicle, and these data were integrated into the Microsoft HoloLens using the Novarad OpenSight application allowing the user to view the virtual trajectory guides and CT images superimposed on the phantom in two and three dimensions. Spinal needles were inserted following the virtual trajectories to the point of contact with bone. Repeat CT revealed actual needle trajectory, allowing comparison with the ideal preprocedural paths.


Registration of AR to phantom showed a roughly circular deviation with maximum average radius of 2.5 mm. Users took an average of 200 s to place a needle. Extrapolation of needle trajectory into the pedicle showed that of 36 needles placed, 35 (97%) would have remained within the pedicles. Needles placed approximated a mean distance of 4.69 mm in the mediolateral direction and 4.48 mm in the craniocaudal direction from pedicle bone edge.


To our knowledge, this is the first peer-reviewed report and evaluation of HMD-AR with superimposed 3D guidance utilizing CT for spinal pedicle guide placement for the purpose of cannulation without the use of fluoroscopy.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. 1.

    Thakkar SC, Thakkar RS, Sirisreetreerux N, Carrino JA, Shafiq B, Hasenboehler EA (2017) 2D versus 3D fluoroscopy-based navigation in posterior pelvic fixation: review of the literature on current technology. Int J Comput Assist Radiol Surg 12(1):69–76

  2. 2.

    Wolf MK, Rostetter C, Stadlinger B, Locher M, Damerau G (2015) Preoperative 3D imaging in maxillary sinus: brief review of the literature and case report. Quintessence Int 46(7):627–631

  3. 3.

    Adamczak SE, Bova FJ, Hoh DJ (2017) Intraoperative 3D computed tomography: spine surgery. Neurosurg Clin N Am 28(4):585–594

  4. 4.

    Javan R, Zeman MN (2017) A prototype educational model for hepatobiliary interventions: unveiling the role of graphic designers in medical 3D printing. J Digit Imaging 23(9):1183–1189

  5. 5.

    Wilcox B, Mobbs RJ, Wu AM, Phan K (2017) Systematic review of 3D printing in spinal surgery: the current state of play. J Spine Surg 3(3):433–443

  6. 6.

    Ballard DH, Trace AP, Ali S, Hodgdon T, Zygmont ME, DeBenedectis CM, Smith SE, Richardson ML, Patel MJ, Decker SJ, Lenchik L (2018) Clinical applications of 3D printing: primer for radiologists. Acad Radiol 25(1):52–65

  7. 7.

    Bourgeois A, Faulkner A, Bradley YC, Pasciak AS, Barlow PB, Gash JR, Reid WS (2015) Improved accuracy of minimally invasive transpedicular screw placement in the lumbar spine with 3-dimensional stereotactic image guidance: a comparative meta-analysis. Clin Spine Surg 28(9):324–329

  8. 8.

    Simpfendörfer T, Baumhauer M, Müller M, Gutt CN, Meinzer HP, Rassweiler JJ, Guven S, Teber D (2011) Augmented reality visualization during laparoscopic radical prostatectomy. J Endourol 25(12):1841–1845

  9. 9.

    Faiella E, Frauenfelder G, Santucci D, Luppi G, Schena E, Beomonte ZB, Grasso RF (2017) Percutaneous low-dose CT-guided lung biopsy with an augmented reality navigation system: validation of the technique on 496 suspected lesions. Clin Imaging 49:101–105

  10. 10.

    Grasso RF, Faiella E, Luppi G, Schena E, Giurazza F, Del Vescovo R, D’Agostino F, Cazzato RL, Beomonte ZB (2013) Percutaneous lung biopsy: comparison between an augmented reality CT navigation system and standard CT-guided technique. Int J Comput Assist Radiol Surg 8(5):837–848

  11. 11.

    Lee SC, Fuerst B, Tateno K, Johnson A, Fotouhi J, Osgood G, Tombari F, Navab N (2017) Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery. Healthc Technol Lett 4(5):168–173

  12. 12.

    Terander AE, Nachabe R, Skulason H, Pedersen K, Söderman M, Racadio J, Babic D, Gerdhem P, Edström E (2017) Feasibility and accuracy of thoracolumbar minimally invasive pedicle screw placement with augmented reality navigation technology. Spine J.

  13. 13.

    Pfandler M, Lazarovici M, Stefan P, Wucherer P, Weigl M (2017) Virtual reality-based simulators for spine surgery: a systematic review. Spine J 17(9):1352–1363

  14. 14.

    Léger E, Drouin S, Collins DL, Popa T, Kersten-Oertel M (2017) Quantifying attention shifts in augmented reality image-guided neurosurgery. Healthc Technol Lett 4(5):188–192

  15. 15.

    Kosterhon M, Gutenberg A, Kantelhardtt SR, Archavlis E, Giese A (2017) Navigation and image injection for control of bone removal and osteotomy planes in spine surgery. Oper Neurosurg 13(2):297–304

  16. 16.

    Singla R, Edgcumbe P, Pratt P, Nguan C, Rohling R (2017) Intra-operative ultrasound-based augmented reality guidance for laparoscopic surgery. Healthc Technol Lett 4(5):204–209

  17. 17.

    Sauer IM, Queisner M, Tang P, Moosburner S, Hoepfner O, Horner R, Lohmann R, Pratschke J (2017) Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg 266(5):706–712

  18. 18.

    Galvis V, Berrospi RD, Arias JD, Tello A, Bernal JC (2017) Heads up Descemet membrane endothelial keratoplasty performed using a 3D visualization system. J Surg Case Rep 11(1):1–4

  19. 19.

    Chu Y, Yang J, Ma S, Ai D, Li W, Song H, Li L, Chen D, Chen L, Wang Y (2017) Registration and fusion quantification of augmented reality based nasal endoscopic surgery. Med Image Anal 42:241–256

  20. 20.

    Scolozzi P, Bijlenga P (2017) Removal of recurrent intraorbital tumour using a system of augmented reality. Br J Oral Maxillofac Surg 55(9):962–964

  21. 21.

    Aydin A, Raison N, Khan MS, Dasgupta P, Ahmed K (2016) Simulation-based training and assessment in urological surgery. Nat Rev Urol 13(9):503–519

  22. 22.

    Kim Y, Kim H, Kim YO (2017) Virtual reality and augmented reality in plastic surgery: a review. Arch Plast Surg 44(3):179–187

  23. 23.

    Vávra P, Roman J, Zonča P, Ihnát P, Němec M, Kumar J, Habib N, El-Gendi A (2017) Recent development of augmented reality in surgery: a review. J Healthc Eng.

  24. 24.

    Gasco J, Patel A, Ortega-Barnett J, Branch D, Desai S, Kuo YF, Luciano C, Rizzi S, Kania P, Matuyauskas M, Banerjee P, Roitberg BZ (2014) Virtual reality spine surgery simulation: an empirical study of its usefulness. Neurol Res 36(11):968–973

  25. 25.

    Johnston MJ, Paige JT, Aggarwal R, Stefanidis D, Tsuda S, Khajuria A, Arora S, Association for Surgical Education Simulation Committee (2016) An overview of research priorities in surgical simulation: what the literature shows has been achieved during the 21st century and what remains. Am J Surg 211(1):214–225

  26. 26.

    Dawe SR, Pena GN, Windsor JA, Broeders JA, Cregan PC, Hewett PJ, Maddern GJ (2014) Systematic review of skills transfer after surgical simulation-based training. Br J Surg 101(9):1063–1076

  27. 27.

    Pulijala Y, Ma M, Pears M, Peebles D, Ayoub A (2017) Effectiveness of immersive virtual reality in surgical training—a randomized control trial. J Oral Maxillofac Surg.

  28. 28.

    Barsom EZ, Graafland M, Schijven MP (2016) Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 30(10):4174–4183

  29. 29.

    Kwon OH, Muelder C, Lee K, Ma KL (2017) A study of layout, rendering, and interaction methods for immersive graph visualization. IEEE Trans Vis Comput Graphics 22(7):1802–1815

  30. 30.

    Qian L, Barthel A, Johnson A, Osgood G, Kazanzides P, Navab N, Fuerst B (2017) Comparison of optical see-through head-mounted displays for surgical interventions with object-anchored 2D-display. Int J Comput Assist Radiol Surg 12(6):901–910

  31. 31.

    Nan C, Pradosh K, Viktor G (2017) Augmented reality with Microsoft HoloLens holograms for near infrared fluorescence based image guided surgery. Proc SPIE.

  32. 32.

    Kuhlemann I, Kleemann M, Jauer P, Schweikard A, Ernst F (2017) Towards X-ray free endovascular interventions—using HoloLens for on-line holographic visualisation. Healthc Technol Lett 4(5):184–187

  33. 33.

    Bederman SS, Le VH, Pahlavan S (2016) An approach to lumbar revision spine surgery in adults. J Am Acad Orthop Surg 24(7):433–442

  34. 34.

    Chandra RV, Maingard J, Asadi H, Slater LA, Mazwi TL, Marcia S, Barr J, Hirsch JA (2017) Vertebroplasty and kyphoplasty for osteoporotic vertebral fractures: what are the latest data? Am J Neuroradiol.

  35. 35.

    Chan A, Parent E, Narvacan K, San C, Lou E (2017) Intraoperative image guidance compared with free-hand methods in adolescent idiopathic scoliosis posterior spinal surgery: a systematic review on screw-related complications and breach rates. Spine J 17(9):1215–1229

  36. 36.

    Chaudhari A, Lakhani K, Deulkar K (2015) Transforming the world using holograms. Int. J Comput Appl Eng Sci 130(1):30–32

  37. 37.

    Evans G, Miller J, Pena MI, MacAllister A, Winer E (2017) Evaluating the Microsoft HoloLens through an augmented reality assembly application. Proc SPIE.

  38. 38.

    Matsukawa K, Yato Y (2017) Lumbar pedicle screw fixation with cortical bone trajectory: a review from anatomical and biomechanical standpoints. Spine Surg Relat Res 1(4):164–173

  39. 39.

    Gabbita A, Usman MM, Kishan A, Varadaraju DN, Patil SG, Hosmath AV (2016) Pedicle screw placement in the thoracic and lumbar spine by the c-arm guided navigation and the free hand method: a technical and outcome analysis. J Spine Surg 3(3):90–95

  40. 40.

    Bernard TN, Seibert CE (1992) Pedicle diameter determined by computed tomography. Its relevance to pedicle screw fixation in the lumbar spine. Spine J 17(6):160–163

  41. 41.

    Lai DM, Shih YT, Chen YH, Chien A, Wang JL (2017) Effect of pedicle screw diameter on screw fixation efficacy in human osteoporotic thoracic vertebrae. J Biomech.

  42. 42.

    Theologis AA, Burch S, Pekmezci M (2016) Placement of iliosacral screws using 3D image-guided (O-Arm) technology and Stealth Navigation: comparison with traditional fluoroscopy. Bone Joint J 98-B(5):696–702

  43. 43.

    Spitz SM, Sandhu FA, Voyadzis JM (2015) Percutaneous “K-wireless” pedicle screw fixation technique: an evaluation of the initial experience of 100 screws with assessment of accuracy, radiation exposure, and procedure time. J Neurosurg Spine 22(4):422–431

  44. 44.

    Wray S, Mimran R, Vadapalli S, Shetve SS, McGilvray KC, Puttlitz CM (2015) Pedicle screw placement in the lumbar spine: effect of trajectory and screw design on acute biomechanical purchase. J Neurosurg Spine 22(5):503–510

  45. 45.

    Chung KJ, Sug SW, Desai S, Song HR (2008) Ideal entry point for the thoracic pedicle screw during the free hand technique. Int Orthop 32(5):657–662

  46. 46.

    Zindrick MR, Wiltse LL, Doornik A, Widell EH, Knight GW, Patwardhan AG, Thomas JC, Rothman SL, Fields BT (1987) Analysis of the morphometric characteristics of the thoracic and lumbar pedicles. Spine 12(2):160–166

  47. 47.

    Labadie RF, Davis BM, Fitzpatrick JM (2005) Image-guided surgery: what is the accuracy? Curr Opin Otolaryngol Head Neck Surg 13(1):27–31

  48. 48.

    Afshari E, Rostami M, Farahmand F (2017) Review on different experimental techniques developed for recording force-deformation behaviour of soft tissues; with a view to surgery simulation applications. J Med Eng Technol 41(4):257–274

  49. 49.

    Tonutti M, Gras G, Yang GZ (2017) A machine learning approach for real-time modelling of tissue deformation in image-guided neurosurgery. Artif Intell Med 80:39–47

  50. 50.

    Pan IW, Harris DA, Luerssen TG, Lam SK (2017) Comparative effectiveness of surgical treatments for pediatric hydrocephalus. Neurosurgery.

  51. 51.

    Hervey-Jumper SL, Berger MS (2016) Maximizing safe resection of low- and high-grade glioma. J Neurooncol 130(2):269–282

  52. 52.

    Jensen ME, McGraw JK, Cardella JF, Hirsch JA (2009) Position statement on percutaneous vertebral augmentation: a consensus statement developed by the American Society of Interventional and Therapeutic Neuroradiology, Society of Interventional Radiology, American Association of Neurological Surgeons/Congress of Neurological Surgeons, and American Society of Spine Radiology. J Vasc Interv Radiol 20(7):326–331

  53. 53.

    Alaraj A, Charbel FT, Birk D, Tobin M, Luciano C, Banerjee PP, Rizzi S, Sorenson J, Foley K, Slavin K, Roitberg B (2013) Role of cranial and spinal virtual and augmented reality simulation using immersive touch modules in neurosurgical training. Neurosurgery 72(1):115–123

Download references


We would like to thank Novarad for allowing us to use the Novarad PACS software and the OpenSight application to perform this research.

Author information

Correspondence to Ramin Javan.

Ethics declarations

Conflict of interest

Steve Cvetko is an employee of Novarad. The remaining authors declare that they have no conflict of interest.

Research involving human participants and/or animals

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gibby, J.T., Swenson, S.A., Cvetko, S. et al. Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography. Int J CARS 14, 525–535 (2019) doi:10.1007/s11548-018-1814-7

Download citation


  • Augmented reality
  • Head-mounted display
  • HoloLens
  • OpenSight
  • Spinal pedicle
  • Orthopedic surgery