Abstract
Reconstructing magnetic resonance (MR) images from under-sampled data is a challenging problem due to various artifacts introduced by the under-sampling operation. Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture which captures low-level features at the initial layers and high-level features at the deeper layers. Such networks focus much on global features which may not be optimal to reconstruct the fully-sampled image. In this paper, we propose an Over-and-Under Complete Convolutional Recurrent Neural Network (OUCR), which consists of an overcomplete and an undercomplete Convolutional Recurrent Neural Network (CRNN). The overcomplete branch gives special attention in learning local structures by restraining the receptive field of the network. Combining it with the undercomplete branch leads to a network which focuses more on low-level features without losing out on the global structures. Extensive experiments on two datasets demonstrate that the proposed method achieves significant improvements over the compressed sensing and popular deep learning-based methods with less number of trainable parameters.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akçakaya, M., Moeller, S., Weingärtner, S., Uğurbil, K.: Scan-specific robust artificial-neural-networks for k-space interpolation (RAKI) reconstruction: database-free deep learning for fast imaging. Magn. Reson. Med. 81(1), 439–453 (2019)
Chen, E.Z., Chen, T., Sun, S.: MRI image reconstruction via learning optimization using neural ODEs. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 83–93. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_9
Edmund, J.M., Nyholm, T.: A review of substitute CT generation for MRI-only radiation therapy. Radiat. Oncol. 12(1), 1–15 (2017)
Eo, T., Jun, Y., Kim, T., Jang, J., Lee, H.J., Hwang, D.: Kiki-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn. Reson. Med. 80(5), 2188–2201 (2018)
Fisher, R.B.: Cvonline: The evolving, distributed, non-proprietary, on-line compendium of computer vision (2008). https://homepages.inf.ed.ac.uk/rbf/CVonline. Accessed 28 Jan 2006
Guo, P., Wang, P., Yasarla, R., Zhou, J., Patel, V.M., Jiang, S.: Anatomic and molecular MR image synthesis using confidence guided CNNS. IEEE Trans. Med. Imaging, 1 (2020). https://doi.org/10.1109/TMI.2020.3046460
Guo, P., Wang, P., Zhou, J., Jiang, S., Patel, V.M.: Multi-institutional collaborations for improving deep learning-based magnetic resonance image reconstruction using federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2423–2432 (2021)
Guo, P., Wang, P., Zhou, J., Patel, V.M., Jiang, S.: Lesion mask-based simultaneous synthesis of anatomic and molecular MR Images Using a GAN. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 104–113. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_11
Haldar, J.P., Hernando, D., Liang, Z.P.: Compressed-sensing MRI with random encoding. IEEE Trans. Med. Imaging 30(4), 893–903 (2010)
Jiang, S., et al.: Identifying recurrent malignant glioma after treatment using amide proton transfer-weighted MR imaging: a validation study with image-guided stereotactic biopsy. Clin. Cancer Res. 25(2), 552–561 (2019)
Knoll, F., et al.: fastmri: A publicly available raw k-space and DICOM dataset of knee images for accelerated MR image reconstruction using machine learning. Radiol. Artif. Intell. 2(1), e190007 (2020)
Lee, D., Yoo, J., Tak, S., Ye, J.C.: Deep residual learning for accelerated MRI using magnitude and phase networks. IEEE Trans. Biomed. Eng. 65(9), 1985–1995 (2018)
Lewicki, M.S., Sejnowski, T.J.: Learning overcomplete representations. Neural Comput. 12(2), 337–365 (2000)
Liang, D., Cheng, J., Ke, Z., Ying, L.: Deep magnetic resonance image reconstruction: inverse problems meet neural networks. IEEE Sign. Process. Mag. 37(1), 141–151 (2020)
Liang, D., Liu, B., Wang, J., Ying, L.: Accelerating sense using compressed sensing. Magn. Reson. Med. Offic. J. Int. Soc. Magn. Reson. Med. 62(6), 1574–1584 (2009)
Majumdar, A.: Improving synthesis and analysis prior blind compressed sensing with low-rank constraints for dynamic MRI reconstruction. Magn. Reson. Imaging 33(1), 174–179 (2015)
Mezrich, R.: A perspective on k-space. Radiology 195(2), 297–315 (1995)
Patel, V.M., Chellappa, R.: Sparse representations, compressive sensing and dictionaries for pattern recognition. In: The First Asian Conference on Pattern Recognition, pp. 325–329. IEEE (2011)
Patel, V.M., Maleh, R., Gilbert, A.C., Chellappa, R.: Gradient-based image recovery methods from incomplete fourier measurements. IEEE Trans. Image Process. 21(1), 94–105 (2011)
Pruessmann, K.P., Weiger, M., Scheidegger, M.B., Boesiger, P.: Sense: sensitivity encoding for fast MRI. Magn. Reson. Med. Offic. J. Int. Soc. Magn. Reson. Med. 42(5), 952–962 (1999)
Putzky, P., Welling, M.: Invert to learn to invert. arXiv preprint arXiv:1911.10914 (2019)
Qin, C., et al.: Convolutional recurrent neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 38(1), 280–290 (2019). https://doi.org/10.1109/TMI.2018.2863670
Ravishankar, S., Bresler, Y.: MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Med. Imaging 30(5), 1028–1041 (2010)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schlemper, J., et al.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2), 491–503 (2017)
Tamir, J.I., Ong, F., Cheng, J.Y., Uecker, M., Lustig, M.: Generalized magnetic resonance image reconstruction using the Berkeley advanced reconstruction toolbox. In: ISMRM Workshop on Data Sampling & Image Reconstruction, Sedona, AZ (2016)
Valanarasu, J.M.J., Patel, V.M.: Overcomplete deep subspace clustering networks. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 746–755 (2021)
Valanarasu, J.M.J., Sindagi, V.A., Hacihaliloglu, I., Patel, V.M.: Kiu-net: overcomplete convolutional architectures for biomedical image and volumetric segmentation. arXiv preprint arXiv:2010.01663 (2020)
Valanarasu, J.M.J., Sindagi, V.A., Hacihaliloglu, I., Patel, V.M.: KiU-Net: towards accurate segmentation of biomedical images using over-complete representations. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12264, pp. 363–373. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59719-1_36
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103 (2008)
Wang, P., Chen, E.Z., Chen, T., Patel, V.M., Sun, S.: Pyramid convolutional RNN for MRI reconstruction. arXiv preprint arXiv:1912.00543 (2019)
Yasarla, R., Valanarasu, J.M.J., Patel, V.M.: Exploring overcomplete representations for single image deraining using CNNS. IEEE J. Select. Top. Sign. Process. 15(2), 229–239 (2020)
Acknowledgment
This work was supported by grants from the National Science Foundation (1910141) and the National Institutes of Health (R37CA248077).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, P., Valanarasu, J.M.J., Wang, P., Zhou, J., Jiang, S., Patel, V.M. (2021). Over-and-Under Complete Convolutional RNN for MRI Reconstruction. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12906. Springer, Cham. https://doi.org/10.1007/978-3-030-87231-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-87231-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87230-4
Online ISBN: 978-3-030-87231-1
eBook Packages: Computer ScienceComputer Science (R0)
