Skip to main content
SpringerLink
Log in

Feature Fusion for Multi-Coil Compressed MR Image Reconstruction

  • Original Paper
  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

Magnetic resonance imaging (MRI) occupies a pivotal position within contemporary diagnostic imaging modalities, offering non-invasive and radiation-free scanning. Despite its significance, MRI’s principal limitation is the protracted data acquisition time, which hampers broader practical application. Promising deep learning (DL) methods for undersampled magnetic resonance (MR) image reconstruction outperform the traditional approaches in terms of speed and image quality. However, the intricate inter-coil correlations have been insufficiently addressed, leading to an underexploitation of the rich information inherent in multi-coil acquisitions. In this article, we proposed a method called “Multi-coil Feature Fusion Variation Network” (MFFVN), which introduces an encoder to extract the feature from multi-coil MR image directly and explicitly, followed by a feature fusion operation. Coil reshaping enables the 2D network to achieve satisfactory reconstruction results, while avoiding the introduction of a significant number of parameters and preserving inter-coil information. Compared with VN, MFFVN yields an improvement in the average PSNR and SSIM of the test set, registering enhancements of 0.2622 dB and 0.0021 dB respectively. This uplift can be attributed to the integration of feature extraction and fusion stages into the network’s architecture, thereby effectively leveraging and combining the multi-coil information for enhanced image reconstruction quality. The proposed method outperforms the state-of-the-art methods on fastMRI dataset of multi-coil brains under a fourfold acceleration factor without incurring substantial computation overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Griswold M A, Jakob P M, Nittka M, et al. Partially parallel imaging with localized sensitivities (PILS). Magnetic Resonance in Medicine, 2000; 44(4): 602–9. https://doi.org/10.1002/1522-2594(200010)44:4<602::aid-mrm14>3.0.co;2-5.

    Article  CAS  PubMed  Google Scholar 

  2. Donoho D L. Compressed sensing. IEEE Transactions on information theory, 2006; 52(4): 1289–306. https://doi.org/10.1109/TIT.2006.871582.

    Article  MathSciNet  Google Scholar 

  3. Lustig M, Donoho D L, Santos J M, et al. Compressed sensing MRI. IEEE signal Processing Magazine, 2008; 25(2): 72–82. https://doi.org/10.1109/MSP.2007.914728.

    Article  ADS  Google Scholar 

  4. Sodickson DK, Manning WJ. Simultaneous acquisition of spatial harmonics (SMASH): fast imaging with radiofrequency coil arrays. Magnetic Resonance in Medicine, 1997; 38(4): 591–603. https://doi.org/10.1002/mrm.1910380414.

    Article  CAS  PubMed  Google Scholar 

  5. Pruessmann KP, Weiger M, Scheidegger MB, et al. SENSE: sensitivity encoding for fast MRI. Magnetic Resonance in Medicine, 1999; 42(5): 952–62.

    Article  CAS  PubMed  Google Scholar 

  6. Griswold M, Jakob P, Heidemann R, et al. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magnetic Resonance in Medicine, 2014; 71(3): 990–1001. https://doi.org/10.1002/mrm.10171.

    Article  Google Scholar 

  7. Uecker M, Lai P, Murphy MJ, et al. ESPIRiT-an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magnetic Resonance in Medicine, 2014; 71. https://doi.org/10.1002/mrm.24751.

  8. Wang S, Su Z, Ying L, et al. Accelerating magnetic resonance imaging via deep learning. 2016 IEEE 13th international symposium on biomedical imaging (ISBI). IEEE, 2016: 514–517. https://doi.org/10.1109/ISBI.2016.7493320.

    Article  Google Scholar 

  9. Schlemper J, Caballero J, Hajnal Jv, et al. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction [J]. IEEE Trans Med Imaging. 2018, 37(2):491–503. https://doi.org/10.1109/TMI.2017.2760978

  10. Gan W, Sun Y, Eldeniz C, et al. Deep image reconstruction using unregistered measurements without groundtruth. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE, 2021: 1531–1534. https://doi.org/10.1109/ISBI48211.2021.9434079

    Article  Google Scholar 

  11. Duan C, Deng H, Xiao S, et al. Accelerate gas diffusion-weighted MRI for lung morphometry with deep learning. European Radiology. 2022, 32(1):702–713. https://doi.org/10.1007/s00330-021-08126-y.

    Article  PubMed  Google Scholar 

  12. Guo P, Valanarasu J M J, Wang P, et al. Over-and-under complete convolutional RNN for MRI reconstruction. Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VI 24. Springer International Publishing, 2021: 13–23. https://arxiv.org/abs/2106.08886

  13. Chen E Z, Wang P, Chen X, et al. Pyramid convolutional RNN for MRI image reconstruction. IEEE Transactions on Medical Imaging, 2022, 41(8): 2033–2047. https://doi.org/10.1109/TMI.2022.3153849.

    Article  PubMed  Google Scholar 

  14. Yang G, Yu S, Dong H, et al. DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction. IEEE Trans Med Imaging. 2018, 37(6):1310–1321. https://doi.org/10.1109/TMI.2017.2785879.

  15. Belov A, Stadelmann J, Kastryulin S, et al. Towards ultrafast MRI via extreme k-space undersampling and superresolution. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2021: 254–264. https://arxiv.org/abs/2103.02940

  16. Li G, Lv J, Wang C. A modified generative adversarial network using spatial and channel-wise attention for CS-MRI reconstruction. IEEE Access, 2021, 9: 83185–83198. https://doi.org/10.1109/access.2021.3086839

    Article  Google Scholar 

  17. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015: 234–241. https://doi.org/10.1016/j.compbiomed.2021.104699

  18. Hyun CM, Kim HP, Lee SM, et al. Deep learning for undersampled MRI reconstruction. Physics in Medicine & Biology. 2018, 63(13):135007. https://doi.org/10.1088/1361-6560/aac71a

    Article  ADS  Google Scholar 

  19. Yang J, Küstner T, Hu P, et al. End-to-End Deep Learning of Non-rigid Groupwise Registration and Reconstruction of Dynamic MRI. Frontiers in Cardiovascular Medicine, 2022, 9. https://doi.org/10.3389/fcvm.2022.880186.

  20. Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magnetic Resonance in Medicine, 2018; 79(6): 3055–71. https://doi.org/10.1002/mrm.26977.

    Article  PubMed  Google Scholar 

  21. Yiasemis G, Sonke J J, Sánchez C, et al. Recurrent variational network: a deep learning inverse problem Solver applied to the task of accelerated MRI reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 732–41. https://arxiv.org/abs/2111.09639.

  22. Zhang Z X, Du H W, Qiu B S. FFVN: An explicit feature fusion-based variational network for accelerated multi-coil MRI reconstruction. Magnetic Resonance Imaging, 2023, 97: 31–45. https://doi.org/10.1016/j.mri.2022.12.018.

    Article  ADS  PubMed  Google Scholar 

  23. Duan J, Schlemper J, Qin C, et al. VS-Net: Variable splitting network for accelerated parallel MRI reconstruction. Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part IV 22. Springer International Publishing, 2019: 713–722. https://arxiv.org/abs/1907.10033.

  24. Murugesan B, Ramanarayanan S, Vijayarangan S, et al. A deep cascade of ensemble of dual domain networks with gradient-based T1 assistance and perceptual refinement for fast MRI reconstruction. Computerized Medical Imaging and Graphics, 2021, 91: 101942. https://doi.org/10.1016/j.compmedimag.2021.101942.

    Article  PubMed  Google Scholar 

  25. Küstner T, Fuin N, Hammernik K, et al. CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions. Scientific reports, 2020, 10(1): 13710. https://doi.org/10.1038/s41598-020-70551-8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. WANG Y D, SONG Y, XIE H B, et al. Reconstruction of under-sampled magnetic resonance image based on convolution neural network. Chin J Magn Reson Imaging, 2018, 9 (06): 453–459. https://doi.org/10.12015/issn.1674-8034.2018.06.010

  27. Liu Y, Niu H, Ren P, et al. Generation of quantification maps and weighted images from synthetic magnetic resonance imaging using deep learning network. Physics in Medicine & Biology, 2022, 67(2): 025002. https://doi.org/10.1088/1361-6560/ac46dd

    Article  ADS  Google Scholar 

  28. Sriram A, Zbontar J, Murrell T, et al. End-to-end variational networks for accelerated MRI reconstruction. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2020: 64–73. https://doi.org/10.1016/j.media.2017.06.012

  29. Ramzi Z., Ciuciu P., and Starck J.-L., “XPDNet for MRI reconstruction: An application to the 2020 fastMRI challenge,” in Proc. ISMRM, 2020, pp. 1–4. Available: https://arxiv.org/abs/2010.07290

  30. Ramzi Z, Chaithya G R, Starck J L, et al. NC-PDNet: A density-compensated unrolled network [[41(7): 1625–1638. https://doi.org/10.1109/tmi.2022.3144619

  31. Sun L, Fan Z, Fu X, et al. A deep information sharing network for multi-contrast compressed sensing MRI reconstruction. IEEE Transactions on Image Processing, 2019, 28(12): 6141–6153. https://doi.org/10.1109/tip.2019.2925288

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  32. Xiang L, Chen Y, Chang W, et al. Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Transactions on Biomedical Engineering, 2018, 66(7): 2105–2114. https://doi.org/10.1109/tbme.2018.2883958

    Article  Google Scholar 

  33. Zbontar J, Knoll F, Sriram A, et al. fastMRI: An open dataset and benchmarks for accelerated MRI. https://arxiv.org/abs/1811.08839

  34. Wang S, Ke Z, Cheng H, et al. DIMENSION: dynamic MR imaging with both k‐space and spatial prior knowledge obtained via multi‐supervised network training[J]. NMR in Biomedicine, 2022, 35(4): e4131. https://doi.org/10.1002/nbm.4131

    Article  PubMed  Google Scholar 

  35. Trabelsi C, Bilaniuk O, Serdyuk D, et al. Deep complex networks. 2018, https://arxiv.org/abs/1705.09792

Download references

Funding

The study was supported by research grants from The National Natural Science Foundation of China [Grant No. 81830052] and Shanghai Key Laboratory of Molecular Imaging [18DZ2260400].

Author information

Author notes

  1. Hang Cheng and Xuewen Hou contributed to the work equally and should be regarded as co-first authors.

Authors and Affiliations

  1. School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Hang Cheng & Shengdong Nie

  2. Radiotherapy Business Unit, Shanghai United Imaging Healthcare Co., Ltd., Shanghai, 201807, China

    Xuewen Hou

  3. Shanghai Key Laboratory of Molecular Imaging, Shanghai University of Medicine and Health Sciences, Shanghai, 201318, China

    Gang Huang

  4. Department of Radiology, Jinan People’s Hospital Affiliated to Shandong First Medical University, Jinan Shandong, 271199, China

    Shouqiang Jia

  5. Shanghai Key Laboratory of Magnetic Resonance, Department of Physics, East China Normal University, Shanghai, 200062, China

    Guang Yang

Authors
  1. Hang Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuewen Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shouqiang Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shengdong Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shouqiang Jia, Guang Yang or Shengdong Nie.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, H., Hou, X., Huang, G. et al. Feature Fusion for Multi-Coil Compressed MR Image Reconstruction. J Digit Imaging. Inform. med. (2024). https://doi.org/10.1007/s10278-024-01057-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10278-024-01057-2

Keywords

Search

Navigation