Abstract
Ultrasound (US) imaging provides a safe and accessible solution to procedural guidance and diagnostic imaging. The effective usage of conventional 2D US for interventional guidance requires extensive experience to project the image plane onto the patient, and the interpretation of images in diagnostics suffers from high intra- and inter-user variability. 3D US reconstruction allows for more consistent diagnosis and interpretation, but existing solutions are limited in terms of equipment and applicability in real-time navigation. To address these issues, we propose HoloPOCUS—a mixed reality US system (MR-US) that overlays rich US information onto the user’s vision in a point-of-care setting. HoloPOCUS extends existing MR-US methods beyond placing a US plane in the user’s vision to include a 3D reconstruction and projection that can aid in procedural guidance using conventional probes. We validated a tracking pipeline that demonstrates higher accuracy compared to existing MR-US works. Furthermore, user studies conducted via a phantom task showed significant improvements in navigation duration when using our proposed methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Caraiani, C., Petresc, B., Dong, Y., Dietrich, C.F.: Contraindications and adverse effects in abdominal imaging. Med. Ultrason. 21, 456–463 (2019)
Lentz, B., Fong, T., Rhyne, R., Risko, N.: A systematic review of the cost-effectiveness of ultrasound in emergency care settings. Ultrasound J. 13, 1–9 (2021)
Salonen, R., Haapanen, A., Salonen, J.T.: Measurement of intima-media thickness of common carotid arteries with high-resolution B-mode ultrasonography: Inter- and intra-observer variability. Ultrasound Med. Biol. 17, 225–230 (1991)
Yoon, H.K., et al.: Effects of practitioner’s experience on the clinical performance of ultrasound-guided central venous catheterization: a randomized trial. Sci. Rep. 11, 1–8 (2021)
Gonçalves, L.F., et al.: Applications of 2D matrix array for 3D and 4D examination of the fetus: a pictorial essay. J. Ultrasound Med. 25, 745 (2006)
Daoud, M.I., Alshalalfah, A.L., Awwad, F., Al-Najar, M.: Freehand 3D ultrasound imaging system using electromagnetic tracking. In: 2015 International Conference on Open Source Software Computing, OSSCOM 2015 (2016)
Kim, T., et al.: Versatile low-cost volumetric 3D ultrasound imaging using gimbal-assisted distance sensors and an inertial measurement unit. Sens. (Basel). 20, 1–15 (2020)
Prevost, R., et al.: 3D freehand ultrasound without external tracking using deep learning. Med Image Anal. 48, 187–202 (2018)
Krönke, M., et al.: Tracked 3D ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry. PLoS ONE 17, e0268550 (2022)
Fraser, J.F., Schwartz, T.H., Kaplitt, M.G.: BrainLab image guided system. In: Textbook of Stereotactic and Functional Neurosurgery, pp. 567–581 (2009)
Glas, H.H., Kraeima, J., van Ooijen, P.M.A., Spijkervet, F.K.L., Yu, L., Witjes, M.J.H.: Augmented reality visualization for image-guided surgery: a validation study using a three-dimensional printed phantom. J. Oral Maxillofac. Surg. 79(1943), e1-1943.e10 (2021)
Ameri, G., et al.: Development and evaluation of an augmented reality ultrasound guidance system for spinal anesthesia: preliminary results. Ultrasound Med Biol. 45, 2736–2746 (2019)
Rosenthal, M., et al.: Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms. Med. Image Anal. 6, 313–320 (2002)
Nguyen, T., Plishker, W., Matisoff, A., Sharma, K., Shekhar, R.: HoloUS: augmented reality visualization of live ultrasound images using HoloLens for ultrasound-guided procedures. Int. J. Comput. Assist. Radiol. Surg. 17, 385–391 (2022)
von Haxthausen, F., Moreta-Martinez, R., Pose Díez de la Lastra, A., Pascau, J., Ernst, F.: UltrARsound: in situ visualization of live ultrasound images using HoloLens 2. Int. J. Comput. Assist. Radiol. Surg. 17, 2081 (2022)
Cattari, N., Condino, S., Cutolo, F., Ghilli, M., Ferrari, M., Ferrari, V.: Wearable AR and 3D ultrasound: towards a novel way to guide surgical dissections. IEEE Access. 9, 156746–156757 (2021)
Ungureanu, D., et al.: HoloLens 2 research mode as a tool for computer vision research (2020)
Dibene, J.C., Dunn, E.: HoloLens 2 sensor streaming (2022)
Tölgyessy, M., Dekan, M., Chovanec, Ľ, Hubinský, P.: Evaluation of the azure kinect and its comparison to kinect V1 and kinect V2. Sens. (Basel). 21, 1–25 (2021)
Hübner, P., Clintworth, K., Liu, Q., Weinmann, M., Wursthorn, S.: Evaluation of HoloLens tracking and depth sensing for indoor mapping applications. Sensors 20, 1021 (2020)
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit. 47, 2280–2292 (2014)
Kedilioglu, O., Bocco, T.M., Landesberger, M., Rizzo, A., Franke, J.: ArUcoE: enhanced ArUco marker. In: International Conference on Control, Automation and Systems, vol. 2021-October, pp. 878–881 (2021)
Wang, Y., Zheng, Z., Su, Z., Yang, G., Wang, Z., Luo, Y.: An improved ArUco marker for monocular vision ranging. In: Proceedings of the 32nd Chinese Control and Decision Conference, CCDC 2020, pp. 2915–2919 (2020)
Rijlaarsdam, D.D.W., Zwick, M., Kuiper, J.M.: A novel encoding element for robust pose estimation using planar fiducials. Front. Robot. AI 9, 227 (2022)
Zhang, Z., Hu, Y., Yu, G., Dai, J.: DeepTag: a general framework for fiducial marker design and detection. IEEE Trans. Pattern Anal. Mach. Intell. 45, 2931–2944 (2021)
Duda, A.: Accurate detection and localization of checkerboard corners for calibration (2018)
West, J.B., Fitzpatrick, J.M., Toms, S.A., Maurer, C.R., Maciunas, R.J.: Fiducial point placement and the accuracy of point-based, rigid body registration. Neurosurgery 48, 810–817 (2001)
Fitzpatrick, J.M.: Fiducial registration error and target registration error are uncorrelated. In: Medical Imaging 2009: Visualization, Image-Guided Procedures, and Modeling, vol. 7261, p. 726102 (2009)
Gsaxner, C., Pepe, A., Schmalstieg, D., Li, J., Egger, J.: Inside-out instrument tracking for surgical navigation in augmented reality. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST, p. 11 (2021)
Earle, M., Portu, G.D., Devos, E.: Agar ultrasound phantoms for low-cost training without refrigeration. Afr. J. Emerg. Med. 6, 18–23 (2016)
Weerakkody, Y., Morgan, M.: ATA guidelines for assessment of thyroid nodules. Radiopaedia.org (2016)
Lindseth, F., et al.: Ultrasound-based guidance and therapy. Advancements and Breakthroughs in Ultrasound Imaging (2013)
Azizi, G., et al.: 3-D ultrasound and thyroid cancer diagnosis: a prospective study. Ultrasound Med. Biol. 47, 1299–1309 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ng, K.W. et al. (2023). HoloPOCUS: Portable Mixed-Reality 3D Ultrasound Tracking, Reconstruction and Overlay. In: Kainz, B., Noble, A., Schnabel, J., Khanal, B., Müller, J.P., Day, T. (eds) Simplifying Medical Ultrasound. ASMUS 2023. Lecture Notes in Computer Science, vol 14337. Springer, Cham. https://doi.org/10.1007/978-3-031-44521-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-44521-7_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44520-0
Online ISBN: 978-3-031-44521-7
eBook Packages: Computer ScienceComputer Science (R0)
