Abstract
Purpose of Review
Augmented reality (AR) has gained popularity in various sectors, including gaming, entertainment, and healthcare. The desire for improved surgical navigation within orthopaedic surgery has led to the evaluation of the feasibility and usability of AR in the operating room (OR). However, the safe and effective use of AR technology in the OR necessitates a proper understanding of its capabilities and limitations. This review aims to describe the fundamental elements of AR, highlight limitations for use within the field of orthopaedic surgery, and discuss potential areas for development.
Recent Findings
To date, studies have demonstrated evidence that AR technology can be used to enhance navigation and performance in orthopaedic procedures. General hardware and software limitations of the technology include the registration process, ergonomics, and battery life. Other limitations are related to the human response factors such as inattentional blindness, which may lead to the inability to see complications within the surgical field. Furthermore, the prolonged use of AR can cause eye strain and headache due to phenomena such as the vergence-convergence conflict.
Summary
AR technology may prove to be a better alternative to current orthopaedic surgery navigation systems. However, the current limitations should be mitigated to further improve the feasibility and usability of AR in the OR setting. It is important for both non-clinicians and clinicians to work in conjunction to guide the development of future iterations of AR technology and its implementation into the OR workflow.
Similar content being viewed by others
Data Availability
This declaration is not applicable.
References
Cherian JJ, Kapadia BH, Banerjee S, Jauregui JJ, Issa K, Mont MA. Mechanical, anatomical, and kinematic axis in TKA: concepts and practical applications. Curr Rev Musculoskelet Med. 2014;7:89–95. https://doi.org/10.1007/s12178-014-9218-y.
Liebmann F, Roner S, von Atzigen M, Scaramuzza D, Sutter R, Snedeker J, Farshad M, Furnstahl P. Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J Comput Assist Radiol Surg. 2019;14:1157–65. https://doi.org/10.1007/s11548-019-01973-7.
Casari FA, Navab N, Hruby LA, Kriechling P, Nakamura R, Tori R, Nunes F de LDS, Queiroz MC, Furnstahl P, Farshad M. Augmented reality in orthopedic surgery is emerging from proof of concept towards clinical studies: a literature review explaining the technology and current state of the art. Curr Rev Musculoskelet Med. 2021;14:192–203. https://doi.org/10.1007/s12178-021-09699-3.
Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Information Systems 1994;E77-D, no. 12:1321–1329.
Cipresso P, Giglioli IAC, Raya MA, Riva G. The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature. Front Psychol. 2018;9:2086. https://doi.org/10.3389/fpsyg.2018.02086.
Alexander C, Loeb AE, Fotouhi J, Navab N, Armand M, Khanuja HS. Augmented reality for acetabular component placement in direct anterior total hip arthroplasty. J Arthroplasty. 2020;35:1636-1641.e3. https://doi.org/10.1016/J.ARTH.2020.01.025.
Cho HS, Park YK, Gupta S, Yoon C, Han I, Kim HS, Choi H, Hong J. Augmented reality in bone tumour resection: an experimental study. Bone Joint Res. 2017;6:137–43. https://doi.org/10.1302/2046-3758.63.BJR-2016-0289.R1.
Fotouhi J, Alexander CP, Unberath M, Taylor G, Lee SC, Fuerst B, Johnson A, Osgood G, Taylor RH, Khanuja H, Armand M, Navab N. Plan in 2-D, execute in 3-D: an augmented reality solution for cup placement in total hip arthroplasty. J Med Imaging (Bellingham). 2018;5:21205–21205. https://doi.org/10.1117/1.Jmi.5.2.021205.
Kriechling P, Roner S, Liebmann F, Casari F, Furnstahl P, Wieser K. Augmented reality for base plate component placement in reverse total shoulder arthroplasty: a feasibility study. Arch Orthop Trauma Surg. 2020;141:1447–53. https://doi.org/10.1007/s00402-020-03542-z.
Molina CA, Phillips FM, Colman MW, Ray WZ, Khan M, Orru E, Poelstra K, Khoo L. A cadaveric precision and accuracy analysis of augmented reality-mediated percutaneous pedicle implant insertion. J Neurosurg Spine. 2020;1–9. https://doi.org/10.3171/2020.6.SPINE20370.
Ogawa H, Kurosaka K, Sato A, Hirasawa N, Matsubara M, Tsukada S. Does an augmented reality-based portable navigation system improve the accuracy of acetabular component orientation during THA? A randomized controlled trial. Clin Orthop Relat Res. 2020;478:935–43. https://doi.org/10.1097/CORR.0000000000001083.
Tu P, Gao Y, Lungu AJ, Li D, Wang H, Chen X. Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2. Comput Biol Med. 2021;133:104402–104402. https://doi.org/10.1016/j.compbiomed.2021.104402.
Vávra P, Roman J, Zonča P, Ihnát P, Němec M, Kumar J, Habib N, El-Gendi A. Recent development of augmented reality in surgery: a review. J Healthc Eng. 2017;2017:4574172. https://doi.org/10.1155/2017/4574172.
Roth T, Carrillo F, Wieczorek M, Ceschi G, Esfandiari H, Sutter R, Vlachopoulos L, Wein W, Fucentese SF, Fürnstahl P. Three-dimensional preoperative planning in the weight-bearing state: validation and clinical evaluation. Insights Imaging. 2021;12:44–44. https://doi.org/10.1186/s13244-021-00994-8.
Fida B, Cutolo F, di Franco G, Ferrari M, Ferrari V. Augmented reality in open surgery. Updates Surg. 2018;70:389–400. https://doi.org/10.1007/S13304-018-0567-8.
Fischer M, Fuerst B, Lee SC, Fotouhi J, Habert S, Weidert S, Euler E, Osgood G, Navab N. Preclinical usability study of multiple augmented reality concepts for K-wire placement. Int J Comput Assist Radiol Surg. 2016;11:1007–14. https://doi.org/10.1007/s11548-016-1363-x.
Keating TC, Jacobs JJ. Augmented reality in orthopedic practice and education. Orthop Clin North Am. 2021;52:15–26. https://doi.org/10.1016/j.ocl.2020.08.002.
Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X. A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev Med Devices. 2021;18:47–62. https://doi.org/10.1080/17434440.2021.1860750.
Park BJ, Hunt SJ, Martin C 3rd, Nadolski GJ, Wood BJ, Gade TP. Augmented and mixed reality: technologies for enhancing the future of IR. J Vasc Interv Radiol. 2020;31:1074–82. https://doi.org/10.1016/j.jvir.2019.09.020.
Silva ACDH, Gaber M, Dolenc M, Silva ACDH, Gaber M, Dolenc M. Using augmented reality in different BIM workflows. In: Augmented reality and its application. IntechOpen; 2021.
Elmi-Terander A, Burstrom G, Nachabe R, Fagerlund M, Stahl F, Charalampidis A, Edstrom E, Gerdhem P. Augmented reality navigation with intraoperative 3D imaging vs fluoroscopy-assisted free-hand surgery for spine fixation surgery: a matched-control study comparing accuracy. Sci Rep. 2020;10:707–707. https://doi.org/10.1038/s41598-020-57693-5.
Guo Q, Li X, Tang Y, Huang Y, Luo L. Augmented reality and three-dimensional plate library-assisted posterior minimally invasive surgery for scapula fracture. Int Orthop. 2022;46:875–82. https://doi.org/10.1007/s00264-022-05303-6.
Hiranaka T, Fujishiro T, Hida Y, Shibata Y, Tsubosaka M, Nakanishi Y, Okimura K, Uemoto H. Augmented reality: the use of the PicoLinker smart glasses improves wire insertion under fluoroscopy. World J Orthop. 2017;8:891–4. https://doi.org/10.5312/wjo.v8.i12.891.
Liu H, Auvinet E, Giles J, Rodriguez YBF. Augmented reality based navigation for computer assisted hip resurfacing: a proof of concept study. Ann Biomed Eng. 2018;46:1595–605. https://doi.org/10.1007/s10439-018-2055-1.
Silva R, Oliveira JC, Giraldi GA. Introduction to augmented reality. National laboratory for Scientific Computation 2003; 11:1–11.
Tsukada S, Ogawa H, Nishino M, Kurosaka K, Hirasawa N. Augmented reality-assisted femoral bone resection in total knee arthroplasty. JB JS Open Access 2021;6. https://doi.org/10.2106/JBJS.OA.21.00001.
Aydındoğan G, Kavaklı K, Şahin A, Artal P, Ürey H. Applications of augmented reality in ophthalmology [Invited]. Biomed Opt Express. 2021;12:511–38. https://doi.org/10.1364/boe.405026.
Palumbo A. Microsoft HoloLens 2 in medical and healthcare context: state of the art and future prospects. Sensors (Basel). 2022;22:7709. https://doi.org/10.3390/s22207709.
Furman AA, Hsu WK. Augmented reality (AR) in orthopedics: current applications and future directions. Curr Rev Musculoskelet Med. 2021;14:397–405. https://doi.org/10.1007/s12178-021-09728-1.
Iacono V, Farinelli L, Natali S, Piovan G, Screpis D, Gigante A, Zorzi C. The use of augmented reality for limb and component alignment in total knee arthroplasty: systematic review of the literature and clinical pilot study. J Exp Orthop. 2021;8:52–52. https://doi.org/10.1186/s40634-021-00374-7.
Lee SC, Fuerst B, Tateno K, Johnson A, Fotouhi J, Osgood G, Tombari F, Navab N. Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery. Healthc Technol Lett. 2017;4:168–73. https://doi.org/10.1049/htl.2017.0066.
Tanzer M, Laverdiere C, Barimani B, Hart A. Augmented reality in arthroplasty: an overview of clinical applications, benefits, and limitations. J Am Acad Orthop Surg. 2022;30:e760–8. https://doi.org/10.5435/JAAOS-D-21-00964.
Cutolo F, Mamone V, Carbonaro N, Ferrari V, Tognetti A. Ambiguity-free optical-inertial tracking for augmented reality headsets. Sensors (Basel) 2020;20. https://doi.org/10.3390/s20051444.
Hoff WA, Vincent TL. Analysis of head pose accuracy in augmented reality. IEEE Trans Vis Comput Graph. 2000;6:319–34.
Jud L, Fotouhi J, Andronic O, Aichmair A, Osgood G, Navab N, Farshad M. Applicability of augmented reality in orthopedic surgery - a systematic review. BMC Musculoskelet Disord. 2020; 21. https://doi.org/10.1186/S12891-020-3110-2.
Sutherland J, Belec J, Sheikh A, Chepelev L, Althobaity W, Chow BJW, Mitsouras D, Christensen A, Rybicki FJ, Russa DJL. Applying modern virtual and augmented reality technologies to medical images and models. J Digit Imaging. 2019;32:38–53. https://doi.org/10.1007/s10278-018-0122-7.
Barcali E, Iadanza E, Manetti L, Francia P, Nardi C, Bocchi L. Augmented reality in surgery: a scoping review. Appl Sci. 2022;12. https://doi.org/10.3390/app12146890.
Fucentese SF, Koch PP. A novel augmented reality-based surgical guidance system for total knee arthroplasty. Arch Orthop Trauma Surg. 2021;141:2227–33. https://doi.org/10.1007/S00402-021-04204-4.
Butler AJ, Colman MW, Lynch J, Phillips FM. Augmented reality in minimally invasive spine surgery: early efficiency and complications of percutaneous pedicle screw instrumentation. Spine J. 2022. https://doi.org/10.1016/j.spinee.2022.09.008.
Yoon JW, Chen RE, Han PK, Si P, Freeman WD, Pirris SM. Technical feasibility and safety of an intraoperative head-up display device during spine instrumentation. Int J Med Robot 2017;13. https://doi.org/10.1002/rcs.1770.
Muller F, Roner S, Liebmann F, Spirig JM, Furnstahl P, Farshad M. Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging. Spine J. 2020;20:621–8. https://doi.org/10.1016/j.spinee.2019.10.012.
Logishetty K, Western L, Morgan R, Iranpour F, Cobb JP, Auvinet E. Can an augmented reality headset improve accuracy of acetabular cup orientation in simulated THA? A randomized trial. Clin Orthop Relat Res. 2019;477:1190–9. https://doi.org/10.1097/CORR.0000000000000542.
Farshad M, Furnstahl P, Spirig JM. First in man in-situ augmented reality pedicle screw navigation. N Am Spine Soc J. 2021;6:100065–100065. https://doi.org/10.1016/j.xnsj.2021.100065.
Kriechling P, Loucas R, Loucas M, Casari F, Fürnstahl P, Wieser K. Augmented reality through head-mounted display for navigation of baseplate component placement in reverse total shoulder arthroplasty: a cadaveric study. Arch Orthop Trauma Surg. 2021. https://doi.org/10.1007/s00402-021-04025-5.
Schlueter-Brust K, Henckel J, Katinakis F, Buken C, Opt-Eynde J, Pofahl T, Rodriguez YBF, Tatti F. Augmented-reality-assisted K-wire placement for glenoid component positioning in reversed shoulder arthroplasty: a proof-of-concept study. J Pers Med. 2021;11. https://doi.org/10.3390/jpm11080777.
Weidert S, Wang L, Landes J, Sandner P, Suero EM, Navab N, Kammerlander C, Euler E, Heide A von D. Video-augmented fluoroscopy for distal interlocking of intramedullary nails decreased radiation exposure and surgical time in a bovine cadaveric setting. Int J Med Robot. 2019;15:e1995–e1995. https://doi.org/10.1002/rcs.1995.
Gregory T, Hurst SA, Moslemi A. Mixed reality assisted percutaneous scaphoid fixation: a proposed new surgical technique. Tech Hand Up Extrem Surg. 2021;26:32–6. https://doi.org/10.1097/bth.0000000000000353.
Chen F, Cui X, Han B, Liu J, Zhang X, Liao H. Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. Comput Methods Programs Biomed. 2021;201:105952–105952. https://doi.org/10.1016/j.cmpb.2021.105952.
Chan A, Parent E, Narvacan K, San C, Lou E. Intraoperative image guidance compared with free-hand methods in adolescent idiopathic scoliosis posterior spinal surgery: a systematic review on screw-related complications and breach rates. Spine J. 2017;17:1215–29. https://doi.org/10.1016/j.spinee.2017.04.001.
van Dijk JD, van den Ende RPJ, Stramigioli S, Köchling M, Höss N. Clinical pedicle screw accuracy and deviation from planning in robot-guided spine surgery: robot-guided pedicle screw accuracy. Spine (Phila Pa 1976) 2015;40:E986-991. https://doi.org/10.1097/BRS.0000000000000960.
Laudato PA, Pierzchala K, Schizas C. Pedicle screw insertion accuracy using O-arm, robotic guidance, or freehand technique: a comparative study. Spine (Phila Pa 1976) 2018;43:E373–E378. https://doi.org/10.1097/BRS.0000000000002449.
Mason A, Paulsen R, Babuska JM, Rajpal S, Burneikiene S, Nelson EL, Villavicencio AT. The accuracy of pedicle screw placement using intraoperative image guidance systems. J Neurosurg Spine. 2014;20:196–203. https://doi.org/10.3171/2013.11.SPINE13413.
Nevzati E, Marbacher S, Soleman J, Perrig WN, Diepers M, Khamis A, Fandino J. Accuracy of pedicle screw placement in the thoracic and lumbosacral spine using a conventional intraoperative fluoroscopy-guided technique: a national neurosurgical education and training center analysis of 1236 consecutive screws. World Neurosurg. 2014;82(866–871):e1-2. https://doi.org/10.1016/j.wneu.2014.06.023.
Carucci LR. Imaging obese patients: problems and solutions. Abdom Imaging. 2013;38:630–46. https://doi.org/10.1007/s00261-012-9959-2.
James CR, Peterson BE, Crim JR, Cook JL, Crist BD. The use of fluoroscopy during direct anterior hip arthroplasty: powerful or misleading? J Arthroplasty. 2018;33:1775–9. https://doi.org/10.1016/j.arth.2018.01.040.
Laverdière C, Corban J, Khoury J, Ge SM, Schupbach J, Harvey EJ, Reindl R, Martineau PA. Augmented reality in orthopaedics: a systematic review and a window on future possibilities. Bone Joint J. 2019;101-b:1479–1488. https://doi.org/10.1302/0301-620x.101b12.Bjj-2019-0315.R1.
Cho HS, Park MS, Gupta S, Han I, Kim H-S, Choi H, Hong J. Can augmented reality be helpful in pelvic bone cancer surgery? An in vitro study. Clin Orthop Relat Res. 2018;476:1719–25. https://doi.org/10.1007/s11999.0000000000000233.
Gavaghan K, Oliveira-Santos T, Peterhans M, Reyes M, Kim H, Anderegg S, Weber S. Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies. Int J Comput Assist Radiol Surg. 2012;7:547–56. https://doi.org/10.1007/S11548-011-0660-7.
Rochlen LR, Levine R, Tait AR. First-person point-of-view-augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc. 2017;12:57–62. https://doi.org/10.1097/SIH.0000000000000185.
Parsons D, Maccallum K. Current perspectives on augmented reality in medical education: applications, affordances and limitations. Adv Med Educ Pract. 2021;12:77–77. https://doi.org/10.2147/AMEP.S249891.
Li N, Wakim J, Koethe Y, Huber T, Schenning R, Gade TP, Hunt SJ, Park BJ. Multicenter assessment of augmented reality registration methods for image-guided interventions. Radiol Med. 2022;127:857–65. https://doi.org/10.1007/s11547-022-01515-3.
Condino S, Turini G, Parchi PD, Viglialoro RM, Piolanti N, Gesi M, Ferrari M, Ferrari V. How to build a patient-specific hybrid simulator for orthopaedic open surgery: benefits and limits of mixed-reality using the Microsoft HoloLens. J Healthc Eng. 2018;2018:5435097–5435097. https://doi.org/10.1155/2018/5435097.
Lin CJ, Canny S. Effects of virtual target size, position, and parallax on vergence-accommodation conflict as estimated by actual gaze. Sci Rep. 2022;12:20100–20100. https://doi.org/10.1038/s41598-022-24450-9.
Lin Y-H, Huang T-W, Huang H-H, Wang Y-J. Liquid crystal lens set in augmented reality systems and virtual reality systems for rapidly varifocal images and vision correction. Optics Express. 2022;30:22768–78. https://doi.org/10.1364/OE.461378.
Matthews JH, Shields JS. The clinical application of augmented reality in orthopaedics: where do we stand? Curr Rev Musculoskelet Med. 2021;14:316–9. https://doi.org/10.1007/s12178-021-09713-8.
Ha J, Parekh P, Gamble D, Masters J, Jun P, Hester T, Daniels T, Halai M. Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: a narrative review. J Clin Orthop Trauma. 2021;18:209–15. https://doi.org/10.1016/j.jcot.2021.04.031.
Babichenko D, Andrews EG, Canton SP, Littleton EB, Patel R, Labaze D, Mills A. Evaluating effect of Microsoft HoloLens on extraneous cognitive load during simulated cervical lateral mass screw placement. In: Yang X-S, Sherratt S, Dey N, Joshi A, editors. Proceedings of Seventh International Congress on Information and Communication Technology. Singapore: Springer Nature; 2023. pp. 191–201.
Dixon BJ, Daly MJ, Chan HH, Vescan A, Witterick IJ, Irish JC. Inattentional blindness increased with augmented reality surgical navigation. Am J Rhinol Allergy. 2014;28:433–7. https://doi.org/10.2500/ajra.2014.28.4067.
Dennler C, Bauer DE, Scheibler AG, Spirig J, Gotschi T, Furnstahl P, Farshad M. Augmented reality in the operating room: a clinical feasibility study. BMC Musculoskelet Disord. 2021;22:451–451. https://doi.org/10.1186/s12891-021-04339-w.
Khan T, Biehl JT, Andrews EG, Babichenko D. A systematic comparison of the accuracy of monocular RGB tracking and LiDAR for neuronavigation. Healthc Technol Lett. 2022;9:91–101. https://doi.org/10.1049/htl2.12036.
Author information
Authors and Affiliations
Contributions
S.C., C.A., F.S. wrote the main manuscript text. S.C., C.A. and F.S. prepared the table. All authors reviewed and revised it critically for important intellectual content.
Corresponding author
Ethics declarations
Ethical Approval
This declaration is not applicable.
Conflict of Interest
Stephen P. Canton, Confidence Njoku Austin, Fritz Steuer, Edward G. Andrews, Srujan Dadi, David Fogg, Elizabeth Clayton, Onaje Cunningham, Devon Scott, Dukens LabBaze, Nicolás M. Kass, Nikhil Sharma, and MaCalus V. Hogan declare that they have no conflict of interest.
Human and Animal Rights and Informed Consent
This article does not contain any studies with human or animal subjects performed by any of the authors.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Stephen P. Canton and Confidence Njoku Austin are co-first authors.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Canton, S.P., Austin, C.N., Steuer, F. et al. Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room. Curr Rev Musculoskelet Med 17, 117–128 (2024). https://doi.org/10.1007/s12178-024-09888-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12178-024-09888-w