Abstract
The thick-slice magnetic resonance (MR) images are often structurally blurred in coronal and sagittal views, which causes harm to diagnosis and image post-processing. Deep learning (DL) has shown great potential to reconstruct the high-resolution (HR) thin-slice MR images from those low-resolution (LR) cases, which we refer to as the slice interpolation task in this work. However, since it is generally difficult to sample abundant paired LR-HR MR images, the classical fully supervised DL-based models cannot be effectively trained to get robust performance. To this end, we propose a novel Two-stage Self-supervised Cycle-consistency Network (TSCNet) for MR slice interpolation, in which a two-stage self-supervised learning (SSL) strategy is developed for unsupervised DL network training. The paired LR-HR images are synthesized along the sagittal and coronal directions of input LR images for network pretraining in the first-stage SSL, and then a cyclic interpolation procedure based on triplet axial slices is designed in the second-stage SSL for further refinement. More training samples with rich contexts along all directions are exploited as guidance to guarantee the improved interpolation performance. Moreover, a new cycle-consistency constraint is proposed to supervise this cyclic procedure, which encourages the network to reconstruct more realistic HR images. The experimental results on a real MRI dataset indicate that TSCNet achieves superior performance over the conventional and other SSL-based algorithms, and obtains competitive qualitative and quantitative results compared with the fully supervised algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Plenge, E., Poot, D.H., Bernsen, M., et al.: Super-resolution methods in MRI: can they improve the trade-off between resolution, signal-to-noise ratio, and acquisition time? Magn. Reson. Med. 68(6), 1983–1993 (2012)
Jia, Y., Gholipour, A., He, Z., et al.: A new sparse representation framework for reconstruction of an isotropic high spatial resolution MR volume from orthogonal anisotropic resolution scans. IEEE Trans. Med. Imag. 36(5), 1182–1193 (2017)
Van Reeth, E., Tham, I.W., Tan, C.H., et al.: Super-resolution in magnetic resonance imaging: a review. Concepts Magn. Reson. Part A 40(6), 306–325 (2012)
Yang, W., Zhang, X., Tian, Y., et al.: Deep learning for single image super-resolution: a brief review. IEEE Trans. Multimedia 21(12), 3106–3121 (2019)
Shi, J., Li, Z., Ying, S., et al.: MR image super-resolution via wide residual networks with fixed skip connection. IEEE J. Biomed. Health inform. 23(3), 1129–1140 (2018)
Li, Z., et al.: A two-stage multi-loss super-resolution network for arterial spin labeling magnetic resonance imaging. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 12–20. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_2
Li, Z., Wang, Y., Yu, J.: Reconstruction of thin-slice medical images using generative adversarial network. In: Wang, Q., Shi, Y., Suk, H.-I., Suzuki, K. (eds.) MLMI 2017. LNCS, vol. 10541, pp. 325–333. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67389-9_38
Chen, Y., Xie, Y., Zhou, Z., et al.: Brain MRI super resolution using 3D deep densely connected neural networks. In: ISBI, pp. 739–742 (2018)
Du, J., He, Z., Wang, L., et al.: Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network. Neurocomputing 392, 209–220 (2020)
Peng, C., Lin, W.A., Liao, H., et al.: SAINT: spatially aware interpolation network for medical slice synthesis. In: CVPR, pp. 7750–7759 (2020)
Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR, pp. 5197–5206 (2015)
Yuan, Y., Liu, S., Zhang, J., et al.: Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks. In: CVPR, pp. 701–710 (2018)
Reda, F.A., Sun, D., Dundar, A., et al.: Unsupervised video interpolation using cycle consistency. In: ICCV, pp. 892–900 (2019)
Zhao, C., Carass, A., Dewey, B.E., et al.: Self super-resolution for magnetic resonance images using deep networks. In: ISBI, pp. 365–368 (2018)
Zhao, C., et al.: A deep learning based anti-aliasing self super-resolution algorithm for MRI. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 100–108. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_12
Zhao, C., Dewey, B.E., Pham, D.L., et al.: SMORE: a self-supervised anti-aliasing and super-resolution algorithm for MRI using deep learning. IEEE Trans. Med. Imag. 40(3), 805–817 (2020)
Zhang, Y., Tian, Y., Kong, Y., et al.: Residual dense network for image super-resolution. In: CVPR, pp. 2472–2481 (2018)
Ledig, C., Theis, L., Huszár, F., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR, pp. 4681–4690 (2017)
Jack, Jr., C.R., Bernstein, M.A., Fox, N.C., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imag. 27(4), 685–691 (2008)
Acknowledgements
This work is supported by National Natural Science Foundation of China (81830058) and the 111 Project (D20031).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, Z., Li, Z., Wang, J., Shi, J., Shen, D. (2021). Two-Stage Self-supervised Cycle-Consistency Network for Reconstruction of Thin-Slice MR Images. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12906. Springer, Cham. https://doi.org/10.1007/978-3-030-87231-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-87231-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87230-4
Online ISBN: 978-3-030-87231-1
eBook Packages: Computer ScienceComputer Science (R0)
