Abstract
In cardiac magnetic resonance (CMR) imaging, a 3D high-resolution segmentation of the heart is essential for detailed description of its anatomical structures. However, due to the limit of acquisition duration and respiratory/cardiac motion, stacks of multi-slice 2D images are acquired in clinical routine. The segmentation of these images provides a low-resolution representation of cardiac anatomy, which may contain artefacts caused by motion. Here we propose a novel latent optimisation framework that jointly performs motion correction and super resolution for cardiac image segmentations. Given a low-resolution segmentation as input, the framework accounts for inter-slice motion in cardiac MR imaging and super-resolves the input into a high-resolution segmentation consistent with input. A multi-view loss is incorporated to leverage information from both short-axis view and long-axis view of cardiac imaging. To solve the inverse problem, iterative optimisation is performed in a latent space, which ensures the anatomical plausibility. This alleviates the need of paired low-resolution and high-resolution images for supervised learning. Experiments on two cardiac MR datasets show that the proposed framework achieves high performance, comparable to state-of-the-art super-resolution approaches and with better cross-domain generalisability and anatomical plausibility. The codes are available at https://github.com/shuowang26/SRHeart.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anwar, S., Khan, S., Barnes, N.: A Deep journey into super-resolution: a survey. ACM Comput. Surv. 53(3), 1–34 (2020)
Avendi, M., Kheradvar, A., Jafarkhani, H.: A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI. Med. Image Anal. 30, 108–119 (2016)
Bai, W., Sinclair, M., Tarroni, G., Oktay, O., et al.: Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J. Cardiovasc. Magn. Reson. 20(1), 65 (2018)
Bhatia, K.K., Price, A.N., Shi, W., Hajnal, J.V., Rueckert, D.: Super-resolution reconstruction of cardiac MRI using coupled dictionary learning. In: IEEE International Symposium on Biomedical Imaging (ISBI), pp. 947–950 (2014)
Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a GAN to learn how to do image degradation first. In: Proceedings of the European conference on computer vision (ECCV), pp. 185–200 (2018)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)
Duan, J., Bello, G., Schlemper, J., Bai, W., et al.: Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach. IEEE Trans. Med. Imaging 38(9), 2151–2164 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Larrazabal, A.J., Martínez, C., Glocker, B., Ferrante, E.: Post-DAE: anatomically plausible segmentation via post-processing with denoising autoencoders. IEEE Trans. Med. Imaging 39, 3813–3820 (2020)
Law, M.H., Jain, A.K.: Incremental nonlinear dimensionality reduction by manifold learning. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 377–391 (2006)
Menon, S., Damian, A., Hu, S., Ravi, N., Rudin, C.: PULSE: self-supervised photo upsampling via latent space exploration of generative models. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Odille, F., Bustin, A., Chen, B., Vuissoz, P.-A., Felblinger, J.: Motion-corrected, super-resolution reconstruction for high-resolution 3D cardiac cine MRI. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 435–442. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_52
Oktay, O., et al.: Multi-input cardiac image super-resolution using convolutional neural networks. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 246–254. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_29
Oktay, O., Ferrante, E., Kamnitsas, K., Heinrich, M., et al.: Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation. IEEE Trans. Med. Imaging 37(2), 384–395 (2017)
Painchaud, N., Skandarani, Y., Judge, T., Bernard, O., et al.: Cardiac segmentation with strong anatomical guarantees. IEEE Trans. Med. Imaging 39(11), 3703–3713 (2020)
Petersen, S.E., Matthews, P.M., Francis, J.M., Robson, M.D., et al.: UK Biobank’s cardiovascular magnetic resonance protocol. J. Cardiovasc. Magn. Reson. 18(1), 8 (2015)
Qiu, Y., Wang, R., Tao, D., Cheng, J.: Embedded block residual network: a recursive restoration model for single-image super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4180–4189 (2019)
Raya, S.P., Udupa, J.K.: Shape-based interpolation of multidimensional objects. IEEE Trans. Med. Imaging 9(1), 32–42 (1990)
Tarroni, G., Bai, W., Oktay, O., Schuh, A., et al.: Large-scale quality control of cardiac imaging in population Studies: application to UK Biobank. Sci. Rep. 10(1), 1–11 (2020)
Tarroni, G., Oktay, O., Bai, W., Schuh, A., et al.: Learning-based quality control for cardiac MR images. IEEE Trans. Med. Imaging 38(5), 1127–1138 (2018)
Tezcan, K.C., Baumgartner, C.F., Luechinger, R., Pruessmann, K.P., Konukoglu, E.: MR image reconstruction using deep density priors. IEEE Trans. Med. Imaging 38(7), 1633–1642 (2018)
Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: IEEE International Conference on Computer Vision (ICCV), pp. 4799–4807 (2017)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2018)
Wang, S., et al.: Deep generative model-based quality control for cardiac MRI segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12264, pp. 88–97. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59719-1_9
Wang, Z., Chen, J., Hoi, S.C.: Deep learning for image super-resolution: a survey. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Yue, L., Shen, H., Li, J., Yuan, Q., et al.: Image super-resolution: the techniques, applications, and future. Signal Process. 128, 389–408 (2016)
Zhang, Y., Li, K., Li, K., Wang, L., et al.: Image super-resolution using very deep residual channel attention networks. In: European Conference on Computer Vision (ECCV), pp. 286–301 (2018)
Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487–492 (2018)
Acknowledgements
This research has been conducted using the UK Biobank Resource under Application Number 18545. The authors also acknowledge funding by EPSRC Programme (EP/P001009/1).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, S. et al. (2021). Joint Motion Correction and Super Resolution for Cardiac Segmentation via Latent Optimisation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12903. Springer, Cham. https://doi.org/10.1007/978-3-030-87199-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-87199-4_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87198-7
Online ISBN: 978-3-030-87199-4
eBook Packages: Computer ScienceComputer Science (R0)
