Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image

  • Chen QinEmail author
  • Wenjia Bai
  • Jo Schlemper
  • Steffen E. Petersen
  • Stefan K. Piechnik
  • Stefan Neubauer
  • Daniel Rueckert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11074)


Accelerating the acquisition of magnetic resonance imaging (MRI) is a challenging problem, and many works have been proposed to reconstruct images from undersampled k-space data. However, if the main purpose is to extract certain quantitative measures from the images, perfect reconstructions may not always be necessary as long as the images enable the means of extracting the clinically relevant measures. In this paper, we work on jointly predicting cardiac motion estimation and segmentation directly from undersampled data, which are two important steps in quantitatively assessing cardiac function and diagnosing cardiovascular diseases. In particular, a unified model consisting of both motion estimation branch and segmentation branch is learned by optimising the two tasks simultaneously. Additional corresponding fully-sampled images are incorporated into the network as a parallel sub-network to enhance and guide the learning during the training process. Experimental results using cardiac MR images from 220 subjects show that the proposed model is robust to undersampled data and is capable of predicting results that are close to that from fully-sampled ones, while bypassing the usual image reconstruction stage.


  1. 1.
    Bai, W., Sinclair, M., Tarroni, G., et al.: Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J. Cardiovasc. Magn. Reson. (2018)Google Scholar
  2. 2.
    Caballero, J., Bai, W., Price, A.N., Rueckert, D., Hajnal, J.V.: Application-driven MRI: joint reconstruction and segmentation from undersampled MRI data. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 106–113. Springer, Cham (2014). Scholar
  3. 3.
    Caballero, J., Ledig, C., Aitken, A., et al.: Real-time video super-resolution with spatio-temporal networks and motion compensation. In: CVPR (2017)Google Scholar
  4. 4.
    Cheng, J., Tsai, Y.H., Wang, S., Yang, M.H.: SegFlow: Joint learning for video object segmentation and optical flow. In: ICCV, pp. 686–695 (2017)Google Scholar
  5. 5.
    Oksuz, I., Dharmakumar, R., Tsaftaris, S.A.: Joint myocardial registration and segmentation of cardiac BOLD MRI. In: Pop, M., Sermesant, M., Jodoin, P.-M., Lalande, A., Zhuang, X., Yang, G., Young, A., Bernard, O. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 12–20. Springer, Cham (2018). Scholar
  6. 6.
    Qin, C., et al.: Joint learning of motion estimation and segmentation for cardiac MR image sequences. arXiv preprint arXiv:1806.04066 (2018)
  7. 7.
    Qin, C., Schlemper, J., Caballero, J., Price, A., Hajnal, J.V., Rueckert, D.: Convolutional recurrent neural networks for dynamic MR image reconstruction. arXiv preprint arXiv:1712.01751 (2017)
  8. 8.
    Rueckert, D., Sonoda, L.I., Hayes, C.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999)CrossRefGoogle Scholar
  9. 9.
    Schlemper, J., Caballero, J., Hajnal, J.V., Price, A., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2018)CrossRefGoogle Scholar
  10. 10.
    Schlemper, J., Oktay, O., Bai, W., et al.: Cardiac MR segmentation from undersampled k-space using deep latent representation learning. In: MICCAI (2018)Google Scholar
  11. 11.
    Shi, W., Zhuang, X., Wang, H.: A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration. IEEE Trans. Med. Imaging 31(6), 1263–1275 (2012)CrossRefGoogle Scholar
  12. 12.
    Tobon-Gomez, C., De Craene, M., Mcleod, K.: Benchmarking framework for myocardial tracking and deformation algorithms: an open access database. Med. Image Anal. 17(6), 632–648 (2013)CrossRefGoogle Scholar
  13. 13.
    Tsai, Y.H., Yang, M.H., Black, M.J.: Video segmentation via object flow. In: CVPR, pp. 3899–3908 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Chen Qin
    • 1
    Email author
  • Wenjia Bai
    • 1
  • Jo Schlemper
    • 1
  • Steffen E. Petersen
    • 2
  • Stefan K. Piechnik
    • 3
  • Stefan Neubauer
    • 3
  • Daniel Rueckert
    • 1
  1. 1.Department of ComputingImperial College LondonLondonUK
  2. 2.NIHR Biomedical Research Centre at BartsQueen Mary University of LondonLondonUK
  3. 3.Division of Cardiovascular Medicine, Radcliffe Department of MedicineUniversity of OxfordOxfordUK

Personalised recommendations