Abstract
Purpose
The treatment of pelvic and acetabular fractures remains technically demanding, and traditional surgical navigation systems suffer from the hand–eye mis-coordination. This paper describes a multi-view interactive virtual-physical registration method to enhance the surgeon’s depth perception and a mixed reality (MR)-based surgical navigation system for pelvic and acetabular fracture fixation.
Methods
First, the pelvic structure is reconstructed by segmentation in a preoperative CT scan, and an insertion path for the percutaneous LC-II screw is computed. A custom hand-held registration cube is used for virtual-physical registration. Three strategies are proposed to improve the surgeon’s depth perception: vertices alignment, tremble compensation and multi-view averaging. During navigation, distance and angular deviation visual cues are updated to help the surgeon with the guide wire insertion. The methods have been integrated into an MR module in a surgical navigation system.
Results
Phantom experiments were conducted. Ablation experimental results demonstrated the effectiveness of each strategy in the virtual-physical registration method. The proposed method achieved the best accuracy in comparison with related works. For percutaneous guide wire placement, our system achieved a mean bony entry point error of 2.76 ± 1.31 mm, a mean bony exit point error of 4.13 ± 1.74 mm, and a mean angular deviation of 3.04 ± 1.22°.
Conclusions
The proposed method can improve the virtual-physical fusion accuracy. The developed MR-based surgical navigation system has clinical application potential. Cadaver and clinical experiments will be conducted in future.
Similar content being viewed by others
References
Butterwick D, Papp S, Gofton W, Liew A, Beaulé PE (2015) Acetabular fractures in the elderly: evaluation and management. J Bone Joint Surg 97(9):758–768
Wong JML, Bewsher S, Yew J, Bucknill A, Steiger R (2015) Fluoroscopically assisted computer navigation enables accurate percutaneous screw placement for pelvic and acetabular fracture fixation. Injury 46(6):1064–1068
Karkenny AJ, Mendelis JR, Geller DS, Gomez JA (2019) The role of intraoperative navigation in orthopaedic surgery. JAAOS-J Am Acad Orthopaedic Surg 27(19):e849–e858
Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X (2021) A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev Med Devices 18(1):47–62
Chen X, Xu L, Wang Y, Wang H, Wang F, Zeng X, Egger J (2015) Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. J Biomed Inform 55:124–131
Liebmann F, Roner S, Atzigen M, Scaramuzza D, Sutter R, Snedeker J, Fürnstahl P (2019) Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J Comp Assist Radiol Surg 14(7):1157–1165.
Gsaxner C, Pepe A, Li J, Ibrahimpasic U, Wallner J, Schmalstieg D, Egger J (2021) Augmented reality for head and neck carcinoma imaging: Description and feasibility of an instant calibration, markerless approach. Comput Methods Programs Biomed 200:105854
Atzigen M, Liebmann F, Hoch A, Spirig JM, Farshad M, Snedeker J, Fürnstahl P (2022) Marker-free surgical navigation of rod bending using a stereo neural network and augmented reality in spinal fusion. Med Image Anal 77:102365
Gsaxner C, Pepe A, Wallner J, Schmalstieg D, Egger J (2019) Markerless image-to-face registration for untethered augmented reality in head and neck surgery. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 236–244.
Sun Q, Mai Y, Yang R, Ji T, Jiang X, Chen X (2020) Fast and accurate online calibration of optical see-through head-mounted display for AR-based surgical navigation using Microsoft HoloLens. Int J Comp Assist Radiol Surg 15(11):1907–1919.
Li R, Tong Y, Yang T, Guo J, Si W, Zhang Y, Heng PA (2021) Towards quantitative and intuitive percutaneous tumor puncture via augmented virtual reality. Comput Med Imaging Graph 90:101905
Song T, Yang C, Dianat O, Azimi E (2018) Endodontic guided treatment using augmented reality on a head-mounted display system. Healthcare Technol Lett 5(5):201–207
Tu P, Qin C, Guo Y, Li D, Lungu AJ, Wang H, Chen X (2022) Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement. IEEE Trans Biomed Eng 68(8):2593–2603
Oliveira ME, Debarba HG, Lädermann A, Chagué S, Charbonnier C (2019) A hand‐eye calibration method for augmented reality applied to computer‐assisted orthopedic surgery. Int J Med Robot Computer Assisted Surg 15(2):e1969.
Fotouhi J, Mehrfard A, Song T, Johnson A, Osgood G, Unberath M, Navab N (2020) Development and pre-clinical analysis of spatiotemporal-aware augmented reality in orthopedic interventions. IEEE Trans Med Imaging 40(2):765–778
Scherer J, Guy P, Lefaivre KA, Pape HC, Werner CM, Osterhoff G (2017) Guide wire insertion for percutaneous LC2 screws in acetabular and pelvic ring fixation using a transpedicular working cannula. Injury 48(10):2360–2364
Wang H, Wang F, Newman S, Lin Y, Chen X, Xu L, Wang Q (2016) Application of an innovative computerized virtual planning system in acetabular fracture surgery: a feasibility study. Injury 47(8):1698–1701
Tu P, Gao Y, Lungu AJ, Li D, Wang H, Chen X (2021) Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2. Comput Biol Med 133:104402
Lane CG, Warren R, Pearle AD (2008) The pivot shift. JAAOS-J Am Acad Orthopaed Surgeons 16(12):679–688
Moakher M (2002) Means and averaging in the group of rotations. SIAM J Matrix Anal Appl 24(1):1–16
Gao Y, Wang H, Tu P, Hu J, Wang Q, Chen X (2021) A novel dynamic electromagnetic tracking navigation system for distal locking of intramedullary nails. Comput Methods Programs Biomed 209:106326
Qin C, Cao Z, Fan S, Wu Y, Sun Y, Politis C, Chen X (2019) An oral and maxillofacial navigation system for implant placement with automatic identification of fiducial points. Int J Comp Assist Radiol Surg 14(2):281–289.
Fan X, Zhu Q, Tu P, Joskowicz L, Chen X (2022) A review of advances in image-guided orthopedic surgery. Phys Med Biolo 68:02TR01.
Acknowledgments
This work was supported by grants from National Key R&D Program of China (2022YFE0197900), National Natural Science Foundation of China (81971709; M-0019; 82011530141; 82202302), the Foundation of Science and Technology Commission of Shanghai Municipality (20490740700), Shanghai Jiao Tong University Foundation on Medical and Technological Joint Science Research (YG2019ZDA06; YG2021ZD21; YG2021QN72; YG2022QN056), SJTU Global Strategic Partnership Fund (2023 SJTU-CORNELL, 2021 SJTU-HUJI), and a Joint China-Israel grant from the Ministry of Science and Technology, Israel, 2021-2023.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent
There was no informed consent required for the work reported in this manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Tu, P., Wang, H., Joskowicz, L. et al. A multi-view interactive virtual-physical registration method for mixed reality based surgical navigation in pelvic and acetabular fracture fixation. Int J CARS 18, 1715–1724 (2023). https://doi.org/10.1007/s11548-023-02884-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-023-02884-4