A Supervised Image Registration Approach for Late Gadolinium Enhanced MRI and Cine Cardiac MRI Using Convolutional Neural Networks

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1248)


Late gadolinium enhanced (LGE) cardiac magnetic resonance (CMR) imaging is the current gold standard for assessing myocardium viability for patients diagnosed with myocardial infarction, myocarditis or cardiomyopathy. This imaging method enables the identification and quantification of myocardial tissue regions that appear hyper-enhanced. However, the delineation of the myocardium is hampered by the reduced contrast between the myocardium and the left ventricle (LV) blood-pool due to the gadolinium-based contrast agent. The balanced-Steady State Free Precession (bSSFP) cine CMR imaging provides high resolution images with superior contrast between the myocardium and the LV blood-pool. Hence, the registration of the LGE CMR images and the bSSFP cine CMR images is a vital step for accurate localization and quantification of the compromised myocardial tissue. Here, we propose a Spatial Transformer Network (STN) inspired convolutional neural network (CNN) architecture to perform supervised registration of bSSFP cine CMR and LGE CMR images. We evaluate our proposed method on the 2019 Multi-Sequence Cardiac Magnetic Resonance Segmentation Challenge (MS-CMRSeg) dataset and use several evaluation metrics, including the center-to-center LV and right ventricle (RV) blood-pool distance, and the contour-to-contour blood-pool and myocardium distance between the LGE and bSSFP CMR images. Specifically, we showed that our registration method reduced the bSSFP to LGE LV blood-pool center distance from 3.28 mm before registration to 2.27 mm post registration and RV blood-pool center distance from 4.35 mm before registration to 2.52 mm post registration. We also show that the average surface distance (ASD) between bSSFP and LGE is reduced from 2.53 mm to 2.09 mm, 1.78 mm to 1.40 mm and 2.42 mm to 1.73 mm for LV blood-pool, LV myocardium and RV blood-pool, respectively.


Image registration Late gadolinium enhanced MRI Cine cardiac MRI Deep learning 



Research reported in this publication was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award No. R35GM128877 and by the Office of Advanced Cyber-infrastructure of the National Science Foundation under Award No. 1808530.


  1. 1.
    Benjamin, E.J., et al.: Heart disease and stroke statistics-2017 update: a report from the american heart association. Circulation 135(10), e146–e603 (2017)CrossRefGoogle Scholar
  2. 2.
    Chen, C., et al.: Unsupervised multi-modal style transfer for cardiac MR segmentation. arXiv preprint arXiv:1908.07344 (2019)
  3. 3.
    Chenoune, Y., et al.: Rigid registration of delayed-enhancement and cine cardiac MR images using 3D normalized mutual information. In: 2010 Computing in Cardiology, pp. 161–164. IEEE (2010)Google Scholar
  4. 4.
    Dangi, S., Linte, C.A., Yaniv, Z.: Cine cardiac MRI slice misalignment correction towards full 3D left ventricle segmentation. In: Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 10576, p. 1057607. International Society for Optics and Photonics (2018)Google Scholar
  5. 5.
    Dangi, S., Linte, C.A., Yaniv, Z.: A distance map regularized CNN for cardiac cine MR image segmentation. Med. Phys. 46(12), 5637–5651 (2019)CrossRefGoogle Scholar
  6. 6.
    Dangi, S., Yaniv, Z., Linte, C.A.: Left ventricle segmentation and quantification from cardiac cine MR images via multi-task learning. In: Pop, M., et al. (eds.) STACOM 2018. LNCS, vol. 11395, pp. 21–31. Springer, Cham (2019). Scholar
  7. 7.
    Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)Google Scholar
  8. 8.
    Guo, F., Li, M., Ng, M., Wright, G., Pop, M.: Cine and multicontrast late enhanced MRI registration for 3D heart model construction. In: Pop, M., et al. (eds.) STACOM 2018. LNCS, vol. 11395, pp. 49–57. Springer, Cham (2019). Scholar
  9. 9.
    Hasan, S.K., Linte, C.A.: CondenseUNet: a memory-efficient condensely-connected architecture for bi-ventricular blood pool and myocardium segmentation. In: Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 11315, p. 113151J. International Society for Optics and Photonics (2020)Google Scholar
  10. 10.
    Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 (2015)Google Scholar
  11. 11.
    Juan, L.J., Crean, A.M., Wintersperger, B.J.: Late gadolinium enhancement imaging in assessment of myocardial viability: techniques and clinical applications. Radiol. Clin. 53(2), 397–411 (2015)CrossRefGoogle Scholar
  12. 12.
    Khalil, A., Ng, S.C., Liew, Y.M., Lai, K.W.: An overview on image registration techniques for cardiac diagnosis and treatment. Cardiol. Res. Pract. 2018, 1437125 (2018)CrossRefGoogle Scholar
  13. 13.
    Lee, M.C.H., Oktay, O., Schuh, A., Schaap, M., Glocker, B.: Image-and-spatial transformer networks for structure-guided image registration. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 337–345. Springer, Cham (2019). Scholar
  14. 14.
    Liu, Y., Wang, W., Wang, K., Ye, C., Luo, G.: An automatic cardiac segmentation framework based on multi-sequence MR image. arXiv preprint arXiv:1909.05488 (2019)
  15. 15.
    Campello, V.M., Martín-Isla, C., Izquierdo, C., Petersen, S.E., Ballester, M.A.G., Lekadir, K.: Combining multi-sequence and synthetic images for improved segmentation of late gadolinium enhancement cardiac MRI. In: Pop, M., et al. (eds.) STACOM 2019. LNCS, vol. 12009, pp. 290–299. Springer, Cham (2020). Scholar
  16. 16.
    Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B.P.F., Išgum, I., Staring, M.: Nonrigid image registration using multi-scale 3D convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 232–239. Springer, Cham (2017). Scholar
  17. 17.
    Tao, X., Wei, H., Xue, W., Ni, D.: Segmentation of multimodal myocardial images using shape-transfer GAN. arXiv preprint arXiv:1908.05094 (2019)
  18. 18.
    Upendra, R.R., Dangi, S., Linte, C.A.: An adversarial network architecture using 2D U-Net models for segmentation of left ventricle from cine cardiac MRI. In: Coudière, Y., Ozenne, V., Vigmond, E., Zemzemi, N. (eds.) FIMH 2019. LNCS, vol. 11504, pp. 415–424. Springer, Cham (2019). Scholar
  19. 19.
    Upendra, R.R., Dangi, S., Linte, C.A.: Automated segmentation of cardiac chambers from cine cardiac MRI using an adversarial network architecture. In: Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 11315, p. 113152Y. International Society for Optics and Photonics (2020)Google Scholar
  20. 20.
    Wei, D., Sun, Y., Chai, P., Low, A., Ong, S.H.: Myocardial segmentation of late gadolinium enhanced MR images by propagation of contours from cine MR images. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6893, pp. 428–435. Springer, Heidelberg (2011). Scholar
  21. 21.
    Zhuang, X.: Multivariate mixture model for cardiac segmentation from multi-sequence MRI. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 581–588. Springer, Cham (2016). Scholar
  22. 22.
    Zhuang, X.: Multivariate mixture model for myocardial segmentation combining multi-source images. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 2933–2946 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Center for Imaging ScienceRochester Institute of TechnologyRochesterUSA
  2. 2.Biomedical EngineeringRochester Institute of TechnologyRochesterUSA

Personalised recommendations