Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study

  • Longfei Ma
  • Zhe Zhao
  • Fang Chen
  • Boyu Zhang
  • Ligong Fu
  • Hongen LiaoEmail author
Original Article



We present a novel augmented reality (AR) surgical navigation system based on ultrasound-assisted registration for pedicle screw placement. This system provides the clinically desired targeting accuracy and reduces radiation exposure.


Ultrasound (US) is used to perform registration between preoperative computed tomography (CT) images and patient, and the registration is performed by least-squares fitting of these two three-dimensional (3D) point sets of anatomical landmarks taken from US and CT images. An integral videography overlay device is calibrated to accurately display naked-eye 3D images for surgical navigation. We use a 3.0-mm Kirschner wire (K-wire) instead of a pedicle screw in this study, and the K-wire is calibrated to obtain its orientation and tip location. Based on the above registration and calibration, naked-eye 3D images of the planning path and the spine are superimposed onto patient in situ using our AR navigation system. Simultaneously, a 3D image of the K-wire is overlaid accurately on the real one to guide the insertion procedure. The targeting accuracy is evaluated postoperatively by performing a CT scan.


An agar phantom experiment was performed. Eight K-wires were inserted successfully after US-assisted registration, and the mean targeting error and angle error were 3.35 mm and \(2.74{^{\circ }}\), respectively. Furthermore, an additional sheep cadaver experiment was performed. Four K-wires were inserted successfully. The mean targeting error was 3.79 mm and the mean angle error was \(4.51{^{\circ }}\), and US-assisted registration yielded better targeting results than skin markers-based registration (targeting errors: 2.41 vs. 5.18 mm, angle errors: \(3.13{^{\circ }}\) vs. \(5.89{^{\circ }})\).


Experimental outcomes demonstrate that the proposed navigation system has acceptable targeting accuracy. In particular, the proposed navigation method reduces repeated radiation exposure to the patient and surgeons. Therefore, it has promising prospects for clinical use.


Augmented reality Surgical navigation Pedicle screw placement Ultrasound assistance 



This work is supported in part by National Natural Science Foundation of China (Grant Nos. 81427803, 61361160417, 81271735), Grant-in-Aid of Project 985, and Beijing Municipal Science & Technology Commission (Z151100003915079).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human participants or animals

This article does not contain studies with human participants or animals performed by any of the authors.

Informed consent

This articles does not contain patient data.


  1. 1.
    Phillips FM, Cheng I, Rampersaud YR, Akbarnia BA, Pimenta L, Rodgers WB, Uribe JS, Khanna N, Smith WD, Youssef JA (2016) Breaking through the “Glass Ceiling” of minimally invasive spine surgery. SPINE 41 Suppl 8(1):35–58Google Scholar
  2. 2.
    Laine T, Lund T, Ylikoski M, Lohikoski J, Schlenzka D (2000) Accuracy of pedicle screw insertion with and without computer assistance: a randomised controlled clinical study in 100 consecutive patients. Eur Spine J 9(3):235–240CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Smith H, Welsch M, Sasso R, Vaccaro A (2008) Comparison of radiation exposure in lumbar pedicle screw placement with fluoroscopy vs computer-assisted image guidance with intraoperative three-dimensional imaging. J Spinal Cord Med 31(5):532–537Google Scholar
  4. 4.
    Mastrangelo G, Fedeli U, Fadda E, Giovanazzi A, Scoizzato L, Saia B (2005) Increased cancer risk among surgeons in an orthopaedic hospital. Occup Med 55(6):498–500CrossRefGoogle Scholar
  5. 5.
    Gebhard FT, Kraus MD, Schneider E, Liener UC, Kinzl L, Arand M (2006) Does computer-assisted spine surgery reduce intraoperative radiation doses? SPINE 31(17):2024–2027CrossRefPubMedGoogle Scholar
  6. 6.
    Härtl R, Lam KS, Wang J, Korge A, Kandziora F, Audigé L (2013) Worldwide survey on the use of navigation in spine surgery. World Neurosurg 79(1):162–172CrossRefPubMedGoogle Scholar
  7. 7.
    Manbachi A, Cobbold RS, Ginsberg HJ (2014) Guided pedicle screw insertion: techniques and training. Spine J 14(1):165–179CrossRefPubMedGoogle Scholar
  8. 8.
    Hott JS, Deshmukh VR, Klopfenstein JD, Sonntag VK, Dickman CA, Spetzler RF, Papadopoulos SM (2004) Intraoperative Iso-C C-arm navigation in craniospinal surgery: the first 60 cases. Neurosurgery 54(5):1131–1136 1136-1137CrossRefPubMedGoogle Scholar
  9. 9.
    Merloz P, Troccaz J, Vouaillat H, Vasile C, Tonetti J, Eid A, Plaweski S (2007) Fluoroscopy-based navigation system in spine surgery. Proc Inst Mech Eng [H] 221(7):813–820CrossRefGoogle Scholar
  10. 10.
    Galiano K, Obwegeser AA, Bodner G, Freund M, Maurer H, Kamelger FS, Schatzer R, Ploner F (2005) Ultrasound guidance for facet joint injections in the lumbar spine: a computed tomography-controlled feasibility study. Anesth Analg 101(2):579–583CrossRefPubMedGoogle Scholar
  11. 11.
    Loizides A, Peer S, Plaikner M, Spiss V, Galiano K, Obernauer J, Gruber H (2011) Ultrasound-guided injections in the lumbar spine. Med Ultrasonogr 13(1):54Google Scholar
  12. 12.
    Moore J, Clarke C, Bainbridge D, Wedlake C, Wiles A, Pace D, Peters T (2009) Image guidance for spinal facet injections using tracked ultrasound. Med Image Comput Comput Assist Interv 12:516–23PubMedGoogle Scholar
  13. 13.
    Ungi T, Abolmaesumi P, Jalal R, Welch M, Ayukawa I, Nagpal S, Lasso A, Jaeger M, Borschneck DP, Fichtinger G (2012) Spinal needle navigation by tracked ultrasound snapshots. IEEE Trans Bio-med Eng 59(10):2766–2772CrossRefGoogle Scholar
  14. 14.
    Ungi T, Beiko D, Fuoco M, King F, Holden MS, Fichtinger G, Siemens DR (2014) Tracked ultrasonography snapshots enhance needle guidance for percutaneous renal access: a pilot study. J Endourol 28(9):1040–104515CrossRefPubMedGoogle Scholar
  15. 15.
    Blackwell M, Nikou C, Digioia AM, Kanade T (2000) An image overlay system for medical data visualization. Med Image Anal 4(1):67–72CrossRefPubMedGoogle Scholar
  16. 16.
    Lippmann G, Lippmann G (1908) Epreuves reversibles donnant la sensation du relief. J Phys Theor Appl 7(1):821–825CrossRefGoogle Scholar
  17. 17.
    Liao, H, Hata, N, Iwahara, M, Nakajima, S, Sakuma, I, Dohi, T (2002) Integral videography: a new 3-D autostereoscopic medical image. In: Icbme’2002 of the international conference on biomedical engineering and rehabilitation engineeringGoogle Scholar
  18. 18.
    Hongen Liao SNMI, (2001) Intra-operative Real-Time 3-D Information Display System Based on Integral Videography. In: Medical image computing and computer-assisted intervention–Miccai, (2001) International Conference, Utrecht, October 14–17. Proceedings 2001:2001392–400Google Scholar
  19. 19.
    Liao H, Inomata T, Sakuma I, Dohi T (2010) 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Bio-Med Eng 57(6):1476–1486CrossRefGoogle Scholar
  20. 20.
    Wang J, Suenaga H, Hoshi K, Yang L (2014) Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery. IEEE Trans Bio-Med Eng 61(4):1295–1304CrossRefGoogle Scholar
  21. 21.
    Hsu PW, Prager RW, Gee AH, Treece GM (2009) Freehand 3D ultrasound calibration: a review. Adv Imaging Biol Med 2009:47–84CrossRefGoogle Scholar
  22. 22.
    Arun KS, Huang TS, Blostein SD (1987) Least-squares fitting of two 3-D point sets. IEEE Trans Pattern Anal Mach Intell 9(5):698–700CrossRefPubMedGoogle Scholar
  23. 23.
    Yaniv, Z (2015) Which pivot calibration? In: SPIE medical imaging, International Society for Optics and Photonics, vol. 9415Google Scholar
  24. 24.
    Wang J, Suenaga H, Liao H, Hoshi K, Yang L, Kobayashi E, Sakuma I (2015) Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation. Comput Med Imaging Graph 40:147–159CrossRefPubMedGoogle Scholar
  25. 25.
    Rampersaud YR, Simon DA, Foley KT (2001) Accuracy requirements for image-guided spinal pedicle screw placement. Spine 26(4):352–359CrossRefPubMedGoogle Scholar
  26. 26.
    Glossop ND, Hu RW (1998) Practical accuracy assessment of image guided spine surgery. Presented at the 2nd annual north American program on computer-assisted orthopaedic surgery, Pittsburgh, Pennsylvania, June 1998Google Scholar
  27. 27.
    Mason A, Paulsen R, Babuska JM, Rajpal S, Burneikiene S, Nelson EL, Villavicencio AT (2014) The accuracy of pedicle screw placement using intraoperative image guidance systems. J Neurosurg Spine 20(2):196–203CrossRefPubMedGoogle Scholar
  28. 28.
    Gill S, Mousavi P, Fichtinger G, Chen E, Boisvert J, Pichora D, Abolmaesumi P (2009) Biomechanically constrained groupwise US to CT registration of the lumbar spine. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 803–810Google Scholar
  29. 29.
    Echeverría R, Cortes C, Bertelsen A, Macia I, Ruíz ÓE, Flórez J (2015) Robust CT to US 3D-3D registration by using principal component analysis and kalman filtering. In: International workshop on computational methods and clinical applications for spine imaging. Springer, Berlin, pp 52–63Google Scholar
  30. 30.
    Koo TK, Kwok WE (2016) Hierarchical CT to ultrasound registration of the lumbar spine: a comparison with other registration methods. Ann Biomed Eng 44(10):2887–2900CrossRefPubMedGoogle Scholar
  31. 31.
    Nagpal S, Abolmaesumi P, Rasoulian A, Ungi T, Hacihaliloglu I, Osborn J, Borschneck DP, Lessoway VA, Rohling RN, Mousavi P (2014) CT to US registration of the lumbar spine: a clinical feasibility study. In: International conference on information processing in computer-assisted interventions. Springer, Berlin, pp 108–117Google Scholar
  32. 32.
    Nagpal S, Abolmaesumi P, Rasoulian A, Hacihaliloglu I, Ungi T, Osborn J, Lessoway VA, Rudan J, Jaeger M, Rohling RN (2015) A multi-vertebrae CT to US registration of the lumbar spine in clinical data. Int J Comput Assist Radiol Surg 10(9):1371–1381CrossRefPubMedGoogle Scholar

Copyright information

© CARS 2017

Authors and Affiliations

  • Longfei Ma
    • 1
  • Zhe Zhao
    • 2
  • Fang Chen
    • 1
  • Boyu Zhang
    • 1
  • Ligong Fu
    • 2
  • Hongen Liao
    • 1
    Email author
  1. 1.Department of Biomedical Engineering, School of MedicineTsinghua UniversityBeijingChina
  2. 2.Department of Orthopedics, Beijing Tsinghua Changgung HospitalTsinghua UniversityBeijingChina

Personalised recommendations