Advertisement

Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound

  • Haoran Dou
  • Xin Yang
  • Jikuan Qian
  • Wufeng Xue
  • Hao Qin
  • Xu Wang
  • Lequan Yu
  • Shujun Wang
  • Yi Xiong
  • Pheng-Ann Heng
  • Dong NiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11768)

Abstract

Standard plane localization is crucial for ultrasound (US) diagnosis. In prenatal US, dozens of standard planes are manually acquired with a 2D probe. It is time-consuming and operator-dependent. In comparison, 3D US containing multiple standard planes in one shot has the inherent advantages of less user-dependency and more efficiency. However, manual plane localization in US volume is challenging due to the huge search space and large fetal posture variation. In this study, we propose a novel reinforcement learning (RL) framework to automatically localize fetal brain standard planes in 3D US. Our contribution is two-fold. First, we equip the RL framework with a landmark-aware alignment module to provide warm start and strong spatial bounds for the agent actions, thus ensuring its effectiveness. Second, instead of passively and empirically terminating the agent inference, we propose a recurrent neural network based strategy for active termination of the agent’s interaction procedure. This improves both the accuracy and efficiency of the localization system. Extensively validated on our in-house large dataset, our approach achieves the accuracy of 3.4 mm/9.6\(^{\circ }\) and 2.7 mm/9.1\(^{\circ }\) for the transcerebellar and transthalamic plane localization, respectively. Our proposed RL framework is general and has the potential to improve the efficiency and standardization of US scanning.

Notes

Acknowledgments

The work in this paper was supported by the grant from National Natural Science Foundation of China (No. 61571304), Shenzhen Peacock Plan (No. KQTD2016053112051497, KQJSCX20180328095606003), Medical Scientific Research Foundation of Guangdong Province, China (No. B2018031) and National Natural Science Foundation of China (Project No. U1813204).

References

  1. 1.
    Alansary, A., et al.: Automatic view planning with multi-scale deep reinforcement learning agents. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 277–285. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00928-1_32CrossRefGoogle Scholar
  2. 2.
    Chykeyuk, K., Yaqub, M., Alison Noble, J.: Class-specific regression random forest for accurate extraction of standard planes from 3D echocardiography. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Tu, Z. (eds.) MCV 2013. LNCS, vol. 8331, pp. 53–62. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-05530-5_6CrossRefGoogle Scholar
  3. 3.
    Ghesu, F.C., et al.: Multi-scale deep reinforcement learning for real-time 3D-landmark detection in CT scans. IEEE TPAMI 41(1), 176–189 (2019)CrossRefGoogle Scholar
  4. 4.
    Li, Y., et al.: Standard plane detection in 3D fetal ultrasound using an iterative transformation network. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 392–400. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00928-1_45CrossRefGoogle Scholar
  5. 5.
    Lorenz, C., Brosch, T., et al.: Automated abdominal plane and circumference estimation in 3D US for fetal screening. In: Medical Imaging 2018: Image Processing, vol. 10574, p. 105740I (2018)Google Scholar
  6. 6.
    Mnih, V., Kavukcuoglu, K., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)CrossRefGoogle Scholar
  7. 7.
    Namburete, A.I., Stebbing, R.V., Noble, J.A.: Diagnostic plane extraction from 3D parametric surface of the fetal cranium. In: MIUA, pp. 27–32 (2014)Google Scholar
  8. 8.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  9. 9.
    Ryou, H., Yaqub, M., Cavallaro, A., Roseman, F., Papageorghiou, A., Noble, J.A.: Automated 3D ultrasound biometry planes extraction for first trimester fetal assessment. In: Wang, L., Adeli, E., Wang, Q., Shi, Y., Suk, H.-I. (eds.) MLMI 2016. LNCS, vol. 10019, pp. 196–204. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-47157-0_24CrossRefGoogle Scholar
  10. 10.
    Schmidt-Richberg, A., Schadewaldt, N., et al.: Offset regression networks for view plane estimation in 3D fetal ultrasound. In: Medical Imaging 2019: Image Processing, vol. 10949, p. 109493K (2019)Google Scholar
  11. 11.
    Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: AAAI, pp. 1234–1241 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Haoran Dou
    • 1
    • 2
  • Xin Yang
    • 3
  • Jikuan Qian
    • 1
    • 2
  • Wufeng Xue
    • 1
    • 2
  • Hao Qin
    • 1
    • 2
  • Xu Wang
    • 1
    • 2
  • Lequan Yu
    • 3
  • Shujun Wang
    • 3
  • Yi Xiong
    • 4
  • Pheng-Ann Heng
    • 3
  • Dong Ni
    • 1
    • 2
    Email author
  1. 1.National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science CenterShenzhen UniversityShenzhenChina
  2. 2.Medical UltraSound Image Computing (MUSIC) LabShenzhen UniversityShenzhenChina
  3. 3.Department of Computer Science and EngineeringThe Chinese University of Hong KongHong KongChina
  4. 4.Department of UltrasoundLuohu People’s HospitalShenzhenChina

Personalised recommendations