Abstract
Motion-compensated MR reconstruction (MCMR) is a powerful concept with considerable potential, consisting of two coupled sub-problems: Motion estimation, assuming a known image, and image reconstruction, assuming known motion. In this work, we propose a learning-based self-supervised framework for MCMR, to efficiently deal with non-rigid motion corruption in cardiac MR imaging. Contrary to conventional MCMR methods in which the motion is estimated prior to reconstruction and remains unchanged during the iterative optimization process, we introduce a dynamic motion estimation process and embed it into the unrolled optimization. We establish a cardiac motion estimation network that leverages temporal information via a group-wise registration approach, and carry out a joint optimization between the motion estimation and reconstruction. Experiments on 40 acquired 2D cardiac MR CINE datasets demonstrate that the proposed unrolled MCMR framework can reconstruct high quality MR images at high acceleration rates where other state-of-the-art methods fail. We also show that the joint optimization mechanism is mutually beneficial for both sub-tasks, i.e., motion estimation and image reconstruction, especially when the MR image is highly undersampled.
T. Küstner and K. Hammernik—Contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, H.K., Mani, M.P., Jacob, M.: Model based image reconstruction using deep learned priors (MODL). In: IEEE International Symposium on Biomedical Imaging (ISBI), pp. 671–674 (2018)
Ahmad, R., Xue, H., Giri, S., et al.: Variable density incoherent spatiotemporal acquisition (VISTA) for highly accelerated cardiac MRI. Magn. Reson. Med. 74(5), 1266–1278 (2015)
Aviles-Rivero, A.I., Debroux, N., Williams, G., et al.: Compressed sensing plus motion (CS + M): a new perspective for improving undersampled MR image reconstruction. Med. Image Anal. 68, 101933 (2021)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., et al.: An unsupervised learning model for deformable medical image registration. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9252–9260 (2018)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., et al.: Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)
Batchelor, P., Atkinson, D., Irarrazaval, P., Hill, D., et al.: Matrix description of general motion correction applied to multishot images. Magn. Reson. Med. 54, 1273–1280 (2005)
Bustin, A., Rashid, I., Cruz, G., et al.: 3D whole-heart isotropic sub-millimeter resolution coronary magnetic resonance angiography with non-rigid motion-compensated prost. J. Cardiovasc. Magn. Reson. 22(1) (2020)
Cruz, G., Hammernik, K., Kuestner, T., et al.: One-heartbeat cardiac cine imaging via jointly regularized non-rigid motion corrected reconstruction. In: Proceedings of International Society for Magnetic Resonance in Medicine (ISMRM), p. 0070 (2021)
Cruz, G., Atkinson, D., Henningsson, M., et al.: Highly efficient nonrigid motion-corrected 3D whole-heart coronary vessel wall imaging. Magn. Reson. Med. 77(5), 1894–1908 (2017)
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med. Image Anal. 57, 226–236 (2019)
Hammernik, K., Klatzer, T., Kobler, E., et al.: Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 79(6), 3055–3071 (2018)
Hammernik, K., Pan, J., Rueckert, D., Küstner, T.: Motion-guided physics-based learning for cardiac MRI reconstruction. In: Asilomar Conference on Signals, Systems, and Computers (2021)
Huang, W., Ke, Z., Cui, Z.X., et al.: Deep low-rank plus sparse network for dynamic MR imaging. Med. Image Anal. 73, 102190 (2021)
Jung, H., Sung, K., Nayak, K.S., et al.: k-t FOCUSS: a general compressed sensing framework for high resolution dynamic MRI. Magn. Reson. Med. 61(1), 103–116 (2009)
Klein, S., Staring, M., Murphy, K., et al.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2009)
von Knobelsdorff-Brenkenhoff, F., Pilz, G., Schulz-Menger, J.: Representation of cardiovascular magnetic resonance in the AHA/ACC guidelines. J. Cardiovasc. Magn. Reson. 19(1), 1–21 (2017)
Küstner, T., Fuin, N., Hammernik, K., et al.: CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions. Sci. Rep. 10(1), 1–13 (2020)
Lee, D., Markl, M., Dall’Armellina, E., et al.: The growth and evolution of cardiovascular magnetic resonance: a 20-year history of the society for cardiovascular magnetic resonance (SCMR) annual scientific sessions. J. Cardiovasc. Magn. Reson. 20(1) (2018)
Liu, F., Li, D., Jin, X., et al.: Dynamic cardiac MRI reconstruction using motion aligned locally low rank tensor (MALLRT). Magn. Reson. Imaging 66, 104–115 (2020)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (19) (2017)
Modat, M., Ridgway, G.R., Taylor, Z.A., et al.: Fast free-form deformation using graphics processing units. Comput. Methods Programs Biomed. 98(3), 278–284 (2010)
Mok, T.C.W., Chung, A.C.S.: Conditional deformable image registration with convolutional neural network. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 35–45. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_4
Odille, F., Vuissoz, P., Marie, P., Felblinger, J.: Generalized reconstruction by inversion of coupled systems (GRICS) applied to free-breathing MRI. Magn. Reson. Med. 60, 146–157 (2008)
Odille, F., Menini, A., Escanyé, J.M., et al.: Joint reconstruction of multiple images and motion in MRI: application to free-breathing myocardial \({\rm t}_{2}\) quantification. IEEE Trans. Med. Imaging 35(1), 197–207 (2016)
Pan, J., Rueckert, D., Küstner, T., Hammernik, K.: Efficient image registration network for non-rigid cardiac motion estimation. In: Haq, N., Johnson, P., Maier, A., Würfl, T., Yoo, J. (eds.) Machine Learning for Medical Image Reconstruction, pp. 14–24 (2021)
Poddar, S., Jacob, M.: Dynamic MRI using smoothness regularization on manifolds (SToRM). IEEE Trans. Med. Imaging 35(4), 1106–1115 (2016)
Pruessmann, K.P., Weiger, M., Börnert, P., Boesiger, P.: Advances in sensitivity encoding with arbitrary k-space trajectories. Magn. Reson. Med. 46, 638–651 (2001)
Qi, H., Fuin, N., Cruz, G., et al.: Non-rigid respiratory motion estimation of whole-heart coronary MR images using unsupervised deep learning. IEEE Trans. Med. Imaging 40(1), 444–454 (2021)
Qi, H., Hajhosseiny, R., Cruz, G., et al.: End-to-end deep learning nonrigid motion-corrected reconstruction for highly accelerated free-breathing coronary MRA. Magn. Reson. Med. 86(1), 1983–1996 (2021)
Qin, C., Duan, J., Hammernik, K., et al.: Complementary time-frequency domain networks for dynamic parallel MR image reconstruction. Magn. Reson. Med. 86(6), 3274–3291 (2021)
Sandino, C.M., Lai, P., Vasanawala, S.S., Cheng, J.Y.: Accelerating cardiac cine MRI using a deep learning-based ESPIRiT reconstruction. Magn. Reson. Med. 85(1), 152–167 (2021)
Schmoderer, T., Aviles-Rivero, A.I., Corona, V., et al.: Learning optical flow for fast MRI reconstruction. Inverse Probl. 37(9), 095007 (2021)
Sun, D., Yang, X., Liu, M., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8934–8943 (2018)
Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. Neuro Image 45(1), S61–S72 (2009)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Acknowledgements
This work was supported in part by the European Research Council (Grant Agreement no. 884622).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pan, J., Rueckert, D., Küstner, T., Hammernik, K. (2022). Learning-Based and Unrolled Motion-Compensated Reconstruction for Cardiac MR CINE Imaging. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13436. Springer, Cham. https://doi.org/10.1007/978-3-031-16446-0_65
Download citation
DOI: https://doi.org/10.1007/978-3-031-16446-0_65
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16445-3
Online ISBN: 978-3-031-16446-0
eBook Packages: Computer ScienceComputer Science (R0)
