Abstract
Fetal motion is the dominant challenge to reliable performance and diagnostic quality of fetal magnetic resonance imaging (MRI). The fetus can move unpredictably and rapidly, leading to severe image artifacts. Consequently, MR acquisitions are largely limited to so-called single-shot techniques in an attempt to “freeze” fetal motion through fast imaging, while the problem due to motion occur between slices still exists. In this work, we propose a deep learning method for fetal pose estimation from MR volumes using the paradigm of conditional generative adversarial network which consists of two networks, a generator and a discriminator. The generator is responsible for estimating keypoint heatmaps from input MRI and the discriminator tries to learn the features of plausible fetal pose and distinguish ground-truth heatmaps from generated ones. With this adversarial training scheme, the generator can robustly produce realistic heatmaps for fetal pose inference. Besides, we use adaptive variance to model the difference in intensity of motion of different keypoints. Evaluation shows that the proposed method can improve the performance of pose estimation in 3D MRI, achieving quantitatively an average error of 2.64 mm and 98.31% accuracy (with error less than 10 mm). The proposed method can process volumes with latency less than 300 ms, potentially enabling low-latency online tracking of fetal pose during MR scans.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alexander, G.R., Himes, J.H., Kaufman, R.B., Mor, J., Kogan, M.: A united states national reference for fetal growth. Obstet. Gynecol. 87(2), 163–168 (1996)
Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: new benchmark and state of the art analysis. In: Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, pp. 3686–3693 (2014)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017)
Biglari, H., Sameni, R.: Fetal motion estimation from noninvasive cardiac signal recordings. Physiol. Meas. 37(11), 2003 (2016)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Fai, F.Y., Singh, K., Malcus, P., Biswas, A., Arulkumaran, S., Ratnam, S.: Assessment of fetal health should be based on maternal perception of clusters rather than episodes of fetal movements. J. Obstet. Gynaecol. Res. 22(3), 299–304 (1996)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Heazell, A.P., Frøen, J.: Methods of fetal movement counting and the detection of fetal compromise. J. Obstet. Gynaecol. 28(2), 147–154 (2008)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Itti, L., Chang, L., Ernst, T.: Automatic scan prescription for brain MRI. Magn. Reson. Med. Off. J. Int. Soc. Magn. Reson. Med. 45(3), 486–494 (2001)
Jokhi, R.P., Whitby, E.H.: Magnetic resonance imaging of the fetus. Dev. Med. Child Neurol. 53(1), 18–28 (2011)
Khan, N.U., Wan, W.: A review of human pose estimation from single image. In: 2018 International Conference on Audio, Language and Image Processing (ICALIP), pp. 230–236. IEEE (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101 (2017)
Luo, J., et al.: In vivo quantification of placental insufficiency by bold MRI: a human study. Sci. Rep. 7(1), 3713 (2017)
Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Paul Smolley, S.: Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2794–2802 (2017)
Matsuo, K., Shimoya, K., Ushioda, N., Kimura, T.: Maternal positioning and fetal positioning in utero. J. Obstet. Gynaecol. Res. 33(3), 279–282 (2007)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Payer, C., Štern, D., Bischof, H., Urschler, M.: Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Med. Image Anal. 54, 207–219 (2019)
Saleem, S.N.: Fetal MRI: an approach to practice: a review. J. Adv. Res. 5(5), 507–523 (2014)
Toussaint, N., et al.: Weakly supervised localisation for fetal ultrasound images. In: Stoyanov, D., et al. (eds.) DLMIA 2018, ML-CDS 2018. LNCS, vol. 11045, pp. 192–200. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_22
Xu, J., et al.: Fetal pose estimation in volumetric MRI using a 3D convolution neural network. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 403–410. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_44
Yang, X., et al.: FetusMap: fetal pose estimation in 3D ultrasound. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 281–289. Springer, Cham (2019)
Yen, C.J., Mehollin-Ray, A.R., Bernardo, F., Zhang, W., Cassady, C.I.: Correlation between maternal meal and fetal motion during fetal MRI. Pediatr. Radiol. 49(1), 46–50 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, J., Zhang, M., Turk, E.A., Grant, P.E., Golland, P., Adalsteinsson, E. (2020). 3D Fetal Pose Estimation with Adaptive Variance and Conditional Generative Adversarial Network. In: Hu, Y., et al. Medical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis. ASMUS PIPPI 2020 2020. Lecture Notes in Computer Science(), vol 12437. Springer, Cham. https://doi.org/10.1007/978-3-030-60334-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-60334-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60333-5
Online ISBN: 978-3-030-60334-2
eBook Packages: Computer ScienceComputer Science (R0)
