Skip to main content

Extended Reality in Medical Practice

Current Treatment Options in Cardiovascular Medicine volume 21, Article number: 18 (2019) Cite this article

Abstract

Purpose of review

Advances in display technology and computing have led to new devices capable of overlaying digital information onto the physical world or incorporating aspects of the physical world into virtual scenes. These combinations of digital and physical environments are referred to as extended realities. Extended reality (XR) devices offer many advantages for medical applications including realistic 3D visualization and touch-free interfaces that can be used in sterile environments. This review introduces extended reality and describes how it can be applied to medical practice.

Recent findings

The 3D displays of extended reality devices are valuable in situations where spatial information such as patient anatomy and medical instrument position is important. Applications that take advantage of these 3D capabilities include teaching and pre-operative planning. The utility of extended reality during interventional procedures has been demonstrated with through 3D visualizations of patient anatomy, scar visualization, and real-time catheter tracking with touch-free software control.

Summary

Extended reality devices have been applied to education, pre-procedural planning, and cardiac interventions. These devices excel in settings where traditional devices are difficult to use, such as in the cardiac catheterization lab. New applications of extended reality in cardiology will continue to emerge as the technology improves.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

References and Recommended Reading

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. The Stanford virtual heart – revolutionizing education on congenital heart defects [Internet]. Available from: https://www.stanfordchildrens.org/en/innovation/virtual-reality/stanford-virtual-heart

  2. Bruckheimer E, Rotschild C, Dagan T, Amir G, Kaufman A, Gelman S, et al. Computer-generated real-time digital holography: first time use in clinical medical imaging. Eur Heart J Cardiovasc Imaging. 2016;17:845–9.

    Article  Google Scholar 

  3. • Silva JN, Southworth MK, Dalal A, Van Hare GF. Improving visualization and interaction during transcatheter ablation using an augmented reality system: first-in-human experience (abstr). Circulation. 2017;136 Our prototype electrophysiology mapping software developed for the Microsoft HoloLens demonstrates the power of XR by visualizing patient-specific 3D anatomy and real-time catheter locations with a completely sterile user interface.

  4. Loy Rodas N, Barrera F, Padoy N. See it with your own eyes: markerless mobile augmented reality for radiation awareness in the hybrid room. IEEE Trans Biomed Eng. 2017;64:429–40.

    Article  Google Scholar 

  5. Doug’s great demo 1968 [Internet]. Available from: http://www.dougengelbart.org/content/view/209/448/

  6. Over 50 years of Moore’s law [Internet]. Available from: https://www.intel.com/content/www/us/en/silicon-innovations/moores-law-technology.html

  7. Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst. 1994;E77-D(12):1321–9.

    Google Scholar 

  8. What is mixed reality? [Internet]. Available from: https://docs.microsoft.com/en-us/windows/mixed-reality/mixed-reality

  9. HoloAnatomy app wins top honors [Internet]. Available from: http://engineering.case.edu/HoloAnatomy-honors

  10. Peterson DC, Mlynarczyk GSA. Analysis of traditional versus three-dimensional augmented curriculum on anatomical learning outcome measures. Anat Sci Educ. 2016;9:529–36. https://doi.org/10.1002/ase.1612

    Article  Google Scholar 

  11. Chan F, Aguirre S, Bauser H. Head tracked stereoscopic pre-surgical evaluation of major aortopulmonary collateral arteries in the newborns. Chicago: Paper presented at: Radiological Society of North America 2013 Scientific Assembly and Annual Meeting; 2013.

    Google Scholar 

  12. Jang J, Tschabrunn CM, Barkagan M, Anter E. Three-dimensional holographic visualization of high-resolution myocardial scar on HoloLens. PLoS One. 2018;13:e0205188. https://doi.org/10.1371/journal.pone.0205188

    Article  Google Scholar 

  13. Kiss G, Palmer CL, Torp H. Patient adapted augmented reality system for real-time echocardiographic applications. In: Linte CA, Yaniv Z, Fallavollita P, editors. Augmented environments for computer-assisted interventions. Cham: Springer International Publishing; 2015. p. 145–54.

    Chapter  Google Scholar 

  14. Opolski MP, Debski A, Borucki BA, Szpak M, Staruch AD, Kepka C, et al. First-in-man computed tomography-guided percutaneous revascularization of coronary chronic total occlusion using a wearable computer: proof of concept. Can J Cardiol. 2016;32:11–3.

    Article  Google Scholar 

  15. Katz A, Shtub A, Roguin A. Minimizing ionizing radiation exposure in invasive cardiology safety training for medical doctors. J Nucl Eng Radiat Sci. 2017;3:030905.

    Article  Google Scholar 

  16. Linte CA, Moore J, Wedlake C, Bainbridge D, Guiraudon GM, Jones DL, et al. Inside the beating heart: an in vivo feasibility study on fusing pre- and intra-operative imaging for minimally invasive therapy. Int J Comput Assist Radiol Surg. 2009;4:113–23.

    Article  Google Scholar 

  17. Moro C, Štromberga Z, Raikos A, Stirling A. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat Sci Educ. 2017;10:549–59.

    Article  Google Scholar 

  18. Statt N. Realmax’s prototype AR goggles fix one of the HoloLens’ biggest issues. Available from: https://www.theverge.com/2018/1/9/16872110/realmax-ar-goggles-microsoft-hololens-prototype-field-of-view-ces-2018

  19. Vukicevic M, Mosadegh B, Min JK, Little SH. Cardiac 3D printing and its future directions. JACC Cardiovasc Imaging. 2017;10:171–84. https://doi.org/10.1016/j.jcmg.2016.12.001

    PubMed  Google Scholar 

  20. van der MOAJ, Schijven MP. The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review. Surg Endosc. 2009;23:1180–90.

    Article  Google Scholar 

  21. King C-H, Culjat MO, Franco ML, Lewis CE, Dutson EP, Grundfest WS, et al. Tactile feedback induces reduced grasping force in robot-assisted surgery. IEEE Trans Haptics. 2009;2:103–10.

    Article  Google Scholar 

  22. Stetten G, Wu B, Klatzky R, Galeotti J, Siegel M, Lee R, et al. Hand-held force magnifier for surgical instruments 2011;90–100.

  23. • Silva JNA, Southworth M, Raptis C, Silva J. Emerging applications of virtual reality in cardiovascular medicine. JACC Basic Transl Sci. 2018;3:420–30. Our previous review of this subject includes additional information about extended reality display hardware.

    Article  Google Scholar 

  24. • Peters TM, Linte CA, Yaniv Z, Williams J. Mixed and Augmented Reality in Medicine: CRC Press; 2018. Readers interested in a thorough treatment of XR in medical practice should consider this book.

Download references

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Washington University in St Louis School of Engineering and Applied Science, 1 Brookings Drive, Campus Box 1097, St. Louis, MO, 63130-4899, USA

    Christopher Andrews PhD, Jennifer N. A. Silva MD & Jonathan R. Silva PhD

  2. SentiAR, Inc, 212 N Kingshighway Blvd, Suite #115, St. Louis, MO, 63108, USA

    Michael K. Southworth MS, Jennifer N. A. Silva MD & Jonathan R. Silva PhD

  3. Department of Pediatrics, Division of Cardiology, Washington University in St Louis School of Medicine, 1 Children’s Place, CB 8116 NWT, St Louis, MO, 63110, USA

    Jennifer N. A. Silva MD

Authors
  1. Christopher Andrews PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael K. Southworth MS
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennifer N. A. Silva MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jonathan R. Silva PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jennifer N. A. Silva MD or Jonathan R. Silva PhD.

Ethics declarations

Conflict of Interest

Christopher Andrews, Michael K. Southworth, Jennifer N. A. Silva, and Jonathan R. Silva each report grants from Children’s Discovery Institute and National Institute of Health.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on State-of-the-Art Informatics

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Andrews, C., Southworth, M.K., Silva, J.N.A. et al. Extended Reality in Medical Practice. Curr Treat Options Cardio Med 21, 18 (2019). https://doi.org/10.1007/s11936-019-0722-7

Download citation

  • Published:

  • DOI: https://doi.org/10.1007/s11936-019-0722-7

Keywords

  • Extended reality devices
  • Informatics
  • Display technology