Skip to main content

Advertisement

SpringerLink
Log in

Augmented and Virtual Reality Navigation for Interventions in the Musculoskeletal System

  • Musculoskeletal Imaging (J Fritz, Section Editor)
  • Published:
Current Radiology Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

Augmented reality (AR) and virtual reality (VR) are cutting-edge technologies that offer advanced navigational solutions. These systems are just starting to be used in interventional radiology and the literature has been limited to pre-clinical and translational experiments. In this paper, we present a review of AR and VR for musculoskeletal interventions.

Recent Findings

Mixed reality systems have evolved from costly research systems to fully commercialized clinical instruments designed to facilitate operators in navigating complex anatomy. Studies within interventional radiology have demonstrated safety, improved accuracy, and decreased exposure to ionizing radiation.

Summary

We review the progression of mixed reality systems from their early origins in computer science through their current day surgical applications, with a special focus on landmark studies within radiologic interventions for the musculoskeletal system. We highlight the instrumentation, clinical workflow, benefits and drawbacks, and suggested future directions for the two main AR systems: head-mounted display and image-overlay.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Roberts DW, et al. A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope. J Neurosurg. 1986;65(4):545–9.

    Article  PubMed  CAS  Google Scholar 

  2. •• Sutherland IE. A head-mounted three dimensional display, in Proceedings of the December 9-11, 1968, fall joint computer conference, part I. 1968, ACM: San Francisco, California: 757–764. The original paper that presented the concept of a headmounted display to augment reality.

  3. Cho HS, et al. Augmented reality in bone tumour resection. Bone & Joint Res. 2017;6(3):137–43.

    Article  CAS  Google Scholar 

  4. Faiella E, et al. Percutaneous low-dose CT-guided lung biopsy with an augmented reality navigation system: validation of the technique on 496 suspected lesions. Clin Imaging. 2017;49:101–5.

    Article  PubMed  Google Scholar 

  5. Hou, Y., et al., Effectiveness of the thoracic pedicle screw placement using the virtual surgical training system: a cadaver study. Oper Neurosurg. 2018.

  6. Robison RA, Liu CY, Apuzzo ML. Man, mind, and machine: the past and future of virtual reality simulation in neurologic surgery. World Neurosurg. 2011;76(5):419–30.

    Article  PubMed  Google Scholar 

  7. Guha D, et al. Augmented reality in neurosurgery: a review of current concepts and emerging applications. Can J Neurol Sci. 2017;44(3):235–45.

    Article  PubMed  Google Scholar 

  8. Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery. 2014;10(2):252–60 discussion 260-1.

    Article  PubMed  Google Scholar 

  9. Gildenberg PL, Labuz J. Use of a volumetric target for image-guided surgery. Neurosurgery. 2006;59(3):651–9 discussion 651-9.

    Article  PubMed  Google Scholar 

  10. Bernhardt S, et al. The status of augmented reality in laparoscopic surgery as of 2016. Med Image Anal. 2017;37:66–90.

    Article  PubMed  Google Scholar 

  11. Diana M, et al. Enhanced-reality video fluorescence: a real-time assessment of intestinal viability. Ann Surg. 2014;259(4):700–7.

    Article  PubMed  Google Scholar 

  12. Hansen C, et al. Illustrative visualization of 3D planning models for augmented reality in liver surgery. Int J Comput Assist Radiol Surg. 2010;5(2):133–41.

    Article  PubMed  Google Scholar 

  13. Nicolaou M, et al. Invisible shadow for navigation and planning in minimal invasive surgery. Med Image Comput Comput Assist Interv. 2005;8(Pt 2):25–32.

    PubMed  Google Scholar 

  14. Lee SC, et al. Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery. Healthc Technol Lett. 2017;4(5):168–73.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Fischer M, et al. Preclinical usability study of multiple augmented reality concepts for K-wire placement. Int J Comput Assist Radiol Surg. 2016;11(6):1007–14.

    Article  PubMed  Google Scholar 

  16. Navab N, et al. First deployments of augmented reality in operating rooms. Computer. 2012;45(7):48–55.

    Article  Google Scholar 

  17. Elmi-Terander A et al. Feasibility and accuracy of thoracolumbar minimally invasive pedicle screw placement with augmented reality navigation technology. Spine (Phila Pa 1976), 2017.

  18. Parker PJ, Copeland C. Percutaneous fluoroscopic screw fixation of acetabular fractures. Injury. 1997;28(9–10):597–600.

    Article  PubMed  CAS  Google Scholar 

  19. Hiranaka T, et al. The use of smart glasses for surgical video streaming. Surg Innov. 2017;24(2):151–4.

    Article  PubMed  Google Scholar 

  20. Chimenti PC, Mitten DJ. Google glass as an alternative to standard fluoroscopic visualization for percutaneous fixation of hand fractures: a pilot study. Plast Reconstr Surg. 2015;136(2):328–30.

    Article  PubMed  CAS  Google Scholar 

  21. Hiranaka T, et al. Augmented reality: the use of the PicoLinker smart glasses improves wire insertion under fluoroscopy. World J Orthop. 2017;8(12):891–4.

    Article  PubMed  PubMed Central  Google Scholar 

  22. • Ogawa H, et al. A pilot study of augmented reality technology applied to the acetabular cup placement during total hip arthroplasty. J Arthroplast. 2018;33(6):1833–9. New innovative way using AR through smartphones in the OR.

  23. Weiss CR, Nour SG, Lewin JS. MR-guided biopsy: a review of current techniques and applications. J Magn Reson Imaging. 2008;27(2):311–25.

    Article  PubMed  Google Scholar 

  24. Fritz J, Pereira PL. MR-Guided pain therapy: principles and clinical applications. Rofo. 2007;179(9):914–24.

    Article  PubMed  CAS  Google Scholar 

  25. Fritz J, et al. Magnetic resonance imaging-guided osseous biopsy in children with chronic recurrent multifocal osteomyelitis. Cardiovasc Intervent Radiol. 2012;35(1):146–53.

    Article  PubMed  Google Scholar 

  26. Sonnow L, et al. Instrument visualization using conventional and compressed sensing SEMAC for interventional MRI at 3T. J Magn Reson Imaging. 2018;47(5):1306–15.

    Article  PubMed  Google Scholar 

  27. • Weiss CR, et al. Augmented reality visualization using image-overlay for MR-guided interventions: system description, feasibility, and initial evaluation in a spine phantom. AJR Am J Roentgenol. 2011;196(3):W305–7. MSK IR specific experiment demonstrating image-overlay is safe and effective.

  28. Anand M, et al. Design and development of a mobile image overlay system for needle interventions. Conf Proc IEEE Eng Med Biol Soc. 2014;2014:6159–62.

    PubMed  PubMed Central  CAS  Google Scholar 

  29. Fichtinger G, et al. Image overlay guidance for needle insertion in CT scanner. IEEE Trans Biomed Eng. 2005;52(8):1415–24.

    Article  PubMed  Google Scholar 

  30. Fichtinger G, et al. Image overlay for CT-guided needle insertions. Comput Aided Surg. 2005;10(4):241–55.

    Article  PubMed  Google Scholar 

  31. • Fischer GS, et al. MRI image overlay: applications to arthrography needle insertion. Stud Health Technol Inform. 2006;119:150–5. MSK IR specific experiment that expanded AR applications to arthrograms.

  32. Fischer GS, et al. MRI image overlay: application to arthrography needle insertion. Comput Aided Surg. 2007;12(1):2–14.

    Article  PubMed  Google Scholar 

  33. Fischer GS, et al. Validation system of MR image overlay and other needle insertion techniques. Stud Health Technol Inform. 2007;125:130–5.

    PubMed  Google Scholar 

  34. • Fischer GS, Cole G, Su H. Approaches to creating and controlling motion in MRI. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:6687–90. Proposes solutions to the major limiting effects caused by motion that lead to image distortion.

  35. • Fritz J, et al. Augmented reality visualization with image overlay for MRI-guided intervention: accuracy for lumbar spinal procedures with a 1.5-T MRI system. AJR Am J Roentgenol. 2012;198(3):W266–73. MSK IR specific experiment that expanded upon earlier experiments of AR within the spine.

  36. Fritz J, et al. Augmented reality visualisation using an image overlay system for MR-guided interventions: technical performance of spine injection procedures in human cadavers at 1.5 Tesla. Eur Radiol. 2013;23(1):235–45.

    Article  PubMed  Google Scholar 

  37. Fritz J, et al. Augmented reality visualization with use of image overlay technology for MR imaging-guided interventions: assessment of performance in cadaveric shoulder and hip arthrography at 1.5 T. Radiology. 2012;265(1):254–9.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Fritz J, et al. Augmented reality visualization using image overlay technology for MR-guided interventions: cadaveric bone biopsy at 1.5 T. Invest Radiol. 2013;48(6):464–70.

    Article  PubMed  Google Scholar 

  39. •• Fritz J, et al. MR-guided vertebroplasty with augmented reality image overlay navigation. Cardiovasc Interv Radiol. 2014;37(6):1589–96. MSK IR specific experiment that is one of the most recent translational experiments in the field.

  40. •• Marker DR, et al. 1.5 T augmented reality navigated interventional MRI: paravertebral sympathetic plexus injections. Diagn Interv Radiol. 2017;23(3):227–32. The most recent MSK IR specific experiment that expanded AR applications to the nervous system

  41. Wendt M, et al. A head-mounted display system for augmented reality: initial evaluation for interventional MRI. Rofo. 2003;175(3):418–21.

    Article  PubMed  CAS  Google Scholar 

  42. Khamene A, et al. An Augmented Reality system for MRI-guided needle biopsies. Stud Health Technol Inform. 2003;94:151–7.

    PubMed  Google Scholar 

  43. Wacker FK, et al. An augmented reality system for MR image-guided needle biopsy: initial results in a swine model. Radiology. 2006;238(2):497–504.

    Article  PubMed  Google Scholar 

  44. Das M, et al. Augmented reality visualization for CT-guided interventions: system description, feasibility, and initial evaluation in an abdominal phantom. Radiology. 2006;240(1):230–5.

    Article  PubMed  Google Scholar 

  45. Maruyama K. et al. Smart glasses for neurosurgical navigation by augmented reality. Oper Neurosurg (Hagerstown), 2018.

  46. Ianchulev T, et al. Wearable technology with head-mounted displays and visual function. JAMA. 2014;312(17):1799–801.

    Article  PubMed  Google Scholar 

  47. Mitrasinovic S, et al. Clinical and surgical applications of smart glasses. Technol Health Care. 2015;23(4):381–401.

    Article  PubMed  Google Scholar 

  48. Shao P, et al. Designing a wearable navigation system for image-guided cancer resection surgery. Ann Biomed Eng. 2014;42(11):2228–37.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Arora A, et al. Virtual reality simulation training in Otolaryngology. Int J Surg. 2014;12(2):87–94.

    Article  PubMed  Google Scholar 

  50. Wiet GJ, Stredney D, Wan D. Training and simulation in otolaryngology. Otolaryngol Clin North Am. 2011;44(6):1333–50.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Haque S, Srinivasan S. A meta-analysis of the training effectiveness of virtual reality surgical simulators. IEEE Trans Inf Technol Biomed. 2006;10(1):51–8.

    Article  PubMed  Google Scholar 

  52. Zhao YC, et al. Can virtual reality simulator be used as a training aid to improve cadaver temporal bone dissection? Results of a randomized blinded control trial. Laryngoscope. 2011;121(4):831–7.

    Article  PubMed  Google Scholar 

  53. Fried MP, et al. From virtual reality to the operating room: the endoscopic sinus surgery simulator experiment. Otolaryngol Head Neck Surg. 2010;142(2):202–7.

    Article  PubMed  Google Scholar 

Download references

Funding

Clifford R. Weiss reports grants from the NIH, Clear Guide Medical, and Siemens Healthcare and is a former medical board member for Clear Guide Medical.

Author information

Authors and Affiliations

  1. Vascular and Interventional Radiology Center, Russell H. Morgan Department of Radiology and Radiological Science, The Johns Hopkins Hospital/The Johns Hopkins University, 1800 Orleans Street, Zayed 7203, Baltimore, MD, 21287, USA

    Stephen Belmustakov, Christopher Bailey & Clifford R. Weiss

Authors
  1. Stephen Belmustakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Bailey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clifford R. Weiss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Clifford R. Weiss.

Ethics declarations

Conflict of interest

Stephen Belmustakov and Christopher Bailey each declare no potential conflicts of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

This article is part of the Topical collection on Musculoskeletal Imaging.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Belmustakov, S., Bailey, C. & Weiss, C.R. Augmented and Virtual Reality Navigation for Interventions in the Musculoskeletal System. Curr Radiol Rep 6, 33 (2018). https://doi.org/10.1007/s40134-018-0293-5

Download citation

  • Published:

  • DOI: https://doi.org/10.1007/s40134-018-0293-5

Keywords

Search

Navigation