Advertisement

Adversarial and Perceptual Refinement for Compressed Sensing MRI Reconstruction

  • Maximilian SeitzerEmail author
  • Guang Yang
  • Jo Schlemper
  • Ozan Oktay
  • Tobias Würfl
  • Vincent Christlein
  • Tom Wong
  • Raad Mohiaddin
  • David Firmin
  • Jennifer Keegan
  • Daniel Rueckert
  • Andreas Maier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11070)

Abstract

Deep learning approaches have shown promising performance for compressed sensing-based Magnetic Resonance Imaging. While deep neural networks trained with mean squared error (MSE) loss functions can achieve high peak signal to noise ratio, the reconstructed images are often blurry and lack sharp details, especially for higher undersampling rates. Recently, adversarial and perceptual loss functions have been shown to achieve more visually appealing results. However, it remains an open question how to (1) optimally combine these loss functions with the MSE loss function and (2) evaluate such a perceptual enhancement. In this work, we propose a hybrid method, in which a visual refinement component is learnt on top of an MSE loss-based reconstruction network. In addition, we introduce a semantic interpretability score, measuring the visibility of the region of interest in both ground truth and reconstructed images, which allows us to objectively quantify the usefulness of the image quality for image post-processing and analysis. Applied on a large cardiac MRI dataset simulated with 8-fold undersampling, we demonstrate significant improvements (\(p<0.01\)) over the state-of-the-art in both a human observer study and the semantic interpretability score.

References

  1. 1.
    Dahl, R., et al.: Pixel recursive super resolution. In: ICCV, pp. 5449–5458 (2017)Google Scholar
  2. 2.
    Dosovitskiy, A., Brox, T.: Generating images with perceptual similarity metrics based on deep networks. In: NIPS (2016)Google Scholar
  3. 3.
    Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS (2014)Google Scholar
  4. 4.
    Isola, P., et al.: Image-to-image translation with conditional adversarial networks. In: IEEE CVPR, pp. 5967–5976 (2017)Google Scholar
  5. 5.
    Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46475-6_43CrossRefGoogle Scholar
  6. 6.
    Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE CVPR, pp. 105–114 (2017)Google Scholar
  7. 7.
    Lee, D., et al.: Deep residual learning for compressed sensing MRI. In: IEEE 14th International Symposium on Biomedical Imaging, pp. 15–18 (2017)Google Scholar
  8. 8.
    Pfau, D., Vinyals, O.: Connecting generative adversarial networks and actor-critic methods. arXiv preprint arXiv:1610.01945 (2016)
  9. 9.
    Ravishankar, S., Bresler, Y.: MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE TMI 30, 1028–1041 (2011)Google Scholar
  10. 10.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  11. 11.
    Salimans, T., et al.: Improved Techniques for Training GANs. In: NIPS (2016)Google Scholar
  12. 12.
    Schlemper, J., et al.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE TMI (2017)Google Scholar
  13. 13.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)Google Scholar
  14. 14.
    Yang, G., et al.: DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE TMI (2018)Google Scholar
  15. 15.
    Yang, Y., et al.: Deep ADMM-net for compressive sensing MRI. In: NIPS (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Maximilian Seitzer
    • 1
    • 2
    Email author
  • Guang Yang
    • 3
    • 4
  • Jo Schlemper
    • 2
  • Ozan Oktay
    • 2
  • Tobias Würfl
    • 1
  • Vincent Christlein
    • 1
  • Tom Wong
    • 3
    • 4
  • Raad Mohiaddin
    • 3
    • 4
  • David Firmin
    • 3
    • 4
  • Jennifer Keegan
    • 3
    • 4
  • Daniel Rueckert
    • 2
  • Andreas Maier
    • 1
  1. 1.Pattern Recognition LabFriedrich-Alexander-UniversitätErlangenGermany
  2. 2.Biomedical Image Analysis GroupImperial CollegeLondonUK
  3. 3.National Heart and Lung InstituteImperial CollegeLondonUK
  4. 4.Cardiovascular Research CentreRoyal Brompton HospitalLondonUK

Personalised recommendations