Abstract
We present the first system that provides real-time probe movement guidance for acquiring standard planes in routine freehand obstetric ultrasound scanning. Such a system can contribute to the worldwide deployment of obstetric ultrasound scanning by lowering the required level of operator expertise. The system employs an artificial neural network that receives the ultrasound video signal and the motion signal of an inertial measurement unit (IMU) that is attached to the probe, and predicts a guidance signal. The network termed US-GuideNet predicts either the movement towards the standard plane position (goal prediction), or the next movement that an expert sonographer would perform (action prediction). While existing models for other ultrasound applications are trained with simulations or phantoms, we train our model with real-world ultrasound video and probe motion data from 464 routine clinical scans by 17 accredited sonographers. Evaluations for 3 standard plane types show that the model provides a useful guidance signal with an accuracy of 88.8% for goal prediction and 90.9% for action prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahner, D.P., et al.: Language of transducer manipulation. J. Ultrasound Med. 35(1), 183–188 (2016)
Baumgartner, C.F., et al.: SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound. IEEE Trans. Med. Imag. 36(11), 2204–2215 (2017)
Britton, N., Miller, M.A., Safadi, S., Siegel, A., Levine, A.R., McCurdy, M.T.: Tele-ultrasound in resource-limited settings: a systematic review. Front. Public Health 7, 244 (2019)
Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. In: Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-8), pp. 103–111 (2014)
Housden, R., Treece, G.M., Gee, A.H., Prager, R.W.: Calibration of an orientation sensor for freehand 3D ultrasound and its use in a hybrid acquisition system. BioMed. Eng. OnLine 7(1), 5 (2008)
Jarosik, P., Lewandowski, M.: Automatic ultrasound guidance based on deep reinforcement learning. In: IEEE International Ultrasonics Symposium (IUS), pp. 475–478 (2019)
Li, Y., et al.: Standard plane detection in 3D fetal ultrasound using an iterative transformation network. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 392–400. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_45
Liang, K., Rogers, A.J., Light, E.D., von Allmen, D., Smith, S.W.: Three-dimensional ultrasound guidance of autonomous robotic breast biopsy: feasibility study. Ultrasound Med. Biol. 36(1), 173–177 (2010)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR) (2019)
Maraci, M.A., Bridge, C.P., Napolitano, R., Papageorghiou, A., Noble, J.A.: A framework for analysis of linear ultrasound videos to detect fetal presentation and heartbeat. Med. Image Anal. 37, 22–36 (2017)
Mebarki, R., Krupa, A., Chaumette, F.: 2-D ultrasound probe complete guidance by visual servoing using image moments. IEEE Trans. Robot. 26(2), 296–306 (2010)
Milletari, F., Birodkar, V., Sofka, M.: Straight to the point: reinforcement learning for user guidance in ultrasound. In: Wang, Q., et al. (eds.) PIPPI/SUSI -2019. LNCS, vol. 11798, pp. 3–10. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32875-7_1
Mustafa, A.S.B., et al.: Development of robotic system for autonomous liver screening using ultrasound scanning device. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 804–809 (2013)
Pan, Y., et al.: Agile autonomous driving using end-to-end deep imitation learning. In: Robotics: Science and Systems (RSS) (2019)
Pavllo, D., Feichtenhofer, C., Auli, M., Grangier, D.: Modeling human motion with quaternion-based neural networks. Int. J. Comput. Vis. 128, 855–872 (2020)
Prevost, R., et al.: 3D freehand ultrasound without external tracking using deep learning. Med. Image Anal. 48, 187–202 (2018)
Rahmatullah, B., Papageorghiou, A., Noble, J.A.: Automated selection of standardized planes from ultrasound volume. In: Suzuki, K., Wang, F., Shen, D., Yan, P. (eds.) MLMI 2011. LNCS, vol. 7009, pp. 35–42. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24319-6_5
Salomon, L.J., et al.: Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan. Ultrasound Obstet. Gynecol. 37(1), 116–126 (2011)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4510–4520 (2018)
Shah, S., Bellows, B.A., Adedipe, A.A., Totten, J.E., Backlund, B.H., Sajed, D.: Perceived barriers in the use of ultrasound in developing countries. Crit. Ultrasound J. 7(1), 1–5 (2015). https://doi.org/10.1186/s13089-015-0028-2
Toporek, G., Wang, H., Balicki, M., Xie, H.: Autonomous image-based ultrasound probe positioning via deep learning. In: Hamlyn Symposium on Medical Robotics (2018)
Vilchis, A., Troccaz, J., Cinquin, P., Masuda, K., Pellissier, F.: A new robot architecture for tele-echography. IEEE Trans. Robot. Autom. 19(5), 922–926 (2003)
Wang, S., et al.: Robotic-assisted ultrasound for fetal imaging: evolution from single-arm to dual-arm system. In: Althoefer, K., Konstantinova, J., Zhang, K. (eds.) TAROS 2019. LNCS (LNAI), vol. 11650, pp. 27–38. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25332-5_3
Yaqub, M., Kelly, B., Papageorghiou, A.T., Noble, J.A.: A deep learning solution for automatic fetal neurosonographic diagnostic plane verification using clinical standard constraints. Ultrasound Med. Biol. 43(12), 2925–2933 (2017)
Acknowledgements
We acknowledge the ERC (ERC-ADG-2015 694581, project PULSE) and the NIHR Oxford Biomedical Research Centre.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Droste, R., Drukker, L., Papageorghiou, A.T., Noble, J.A. (2020). Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12263. Springer, Cham. https://doi.org/10.1007/978-3-030-59716-0_56
Download citation
DOI: https://doi.org/10.1007/978-3-030-59716-0_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59715-3
Online ISBN: 978-3-030-59716-0
eBook Packages: Computer ScienceComputer Science (R0)
