Skip to main content

Adversarial Robustness of MR Image Reconstruction Under Realistic Perturbations

  • Conference paper
  • First Online:
Machine Learning for Medical Image Reconstruction (MLMIR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13587))

Abstract

Deep Learning (DL) methods have shown promising results for solving ill-posed inverse problems such as MR image reconstruction from undersampled k-space data. However, these approaches currently have no guarantees for reconstruction quality and the reliability of such algorithms is only poorly understood. Adversarial attacks offer a valuable tool to understand possible failure modes and worst case performance of DL-based reconstruction algorithms. In this paper we describe adversarial attacks on multi-coil k-space measurements and evaluate them on the recently proposed E2E-VarNet and a simpler UNet-based model. In contrast to prior work, the attacks are targeted to specifically alter diagnostically relevant regions. Using two realistic attack models (adversarial k-space noise and adversarial rotations) we are able to show that current state-of-the-art DL-based reconstruction algorithms are indeed sensitive to such perturbations to a degree where relevant diagnostic information may be lost. Surprisingly, in our experiments the UNet and the more sophisticated E2E-VarNet were similarly sensitive to such attacks. Our findings add further to the evidence that caution must be exercised as DL-based methods move closer to clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/NikolasMorshuis/AdvRec

References

  1. Antun, V., Renna, F., Poon, C., Adcock, B., Hansen, A.C.: On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc. Natl. Acad. Sci. 117(48), 30088–30095 (2020). https://doi.org/10.1073/pnas.1907377117

    Article  Google Scholar 

  2. Calivá, F., Cheng, K., Shah, R., Pedoia, V.: Adversarial robust training of deep learning MRI reconstruction models. arXiv preprint arXiv:2011.00070 (2020)

  3. Cheng, K., Calivá, F., Shah, R., Han, M., Majumdar, S., Pedoia, V.: Addressing the false negative problem of deep learning MRI reconstruction models by adversarial attacks and robust training. In: Arbel, T., Ben Ayed, I., de Bruijne, M., Descoteaux, M., Lombaert, H., Pal, C. (eds.) Proceedings of the Third Conference on Medical Imaging with Deep Learning. Proceedings of Machine Learning Research, vol. 121, pp. 121–135. PMLR, 06–08 July 2020. https://proceedings.mlr.press/v121/cheng20a.html

  4. Fabian, Z., Heckel, R., Soltanolkotabi, M.: Data augmentation for deep learning based accelerated MRI reconstruction with limited data. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 3057–3067. PMLR, 18–24 July 2021. https://proceedings.mlr.press/v139/fabian21a.html

  5. Firbank, M., Coulthard, A., Harrison, R., Williams, E.: A comparison of two methods for measuring the signal to noise ratio on MR images. Phys. Med. Biol. 44(12), N261 (1999)

    Article  Google Scholar 

  6. Genzel, M., Macdonald, J., März, M.: Solving inverse problems with deep neural networks - robustness included? (2020). https://doi.org/10.48550/ARXIV.2011.04268. https://arxiv.org/abs/2011.04268

  7. Hyun, C.M., Kim, H.P., Lee, S.M., Lee, S., Seo, J.K.: Deep learning for undersampled MRI reconstruction. Phys. Med. Biol. 63(13), 135007 (2018). https://doi.org/10.1088/1361-6560/aac71a

    Article  Google Scholar 

  8. Johnson, P.M., et al.: Evaluation of the robustness of learned MR image reconstruction to systematic deviations between training and test data for the models from the fastMRI challenge. In: Haq, N., Johnson, P., Maier, A., Würfl, T., Yoo, J. (eds.) MLMIR 2021. LNCS, vol. 12964, pp. 25–34. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88552-6_3

    Chapter  Google Scholar 

  9. Liu, X., Glocker, B., McCradden, M.M., Ghassemi, M., Denniston, A.K., Oakden-Rayner, L.: The medical algorithmic audit. Lancet Digit. Health 4(5), e384–e397 (2022)

    Article  Google Scholar 

  10. Macovski, A.: Noise in MRI. Magn. Reson. Med. 36(3), 494–497 (1996)

    Article  Google Scholar 

  11. Muckley, M.J., et al.: Results of the 2020 fastMRI challenge for machine learning MR image reconstruction. IEEE Trans. Med. Imaging 40(9), 2306–2317 (2021). https://doi.org/10.1109/TMI.2021.3075856

    Article  Google Scholar 

  12. Roemer, P.B., Edelstein, W.A., Hayes, C.E., Souza, S.P., Mueller, O.M.: The NMR phased array. Magn. Reson. Med. 16(2), 192–225 (1990)

    Article  Google Scholar 

  13. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  14. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1–4), 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  15. Schlemper, J., Caballero, J., Hajnal, J.V., Price, A., Rueckert, D.: A deep cascade of convolutional neural networks for MR image reconstruction. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 647–658. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_51

    Chapter  Google Scholar 

  16. Sriram, A., et al.: End-to-end variational networks for accelerated MRI reconstruction. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 64–73. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_7

    Chapter  Google Scholar 

  17. Tezcan, K.C., Baumgartner, C.F., Luechinger, R., Pruessmann, K.P., Konukoglu, E.: MR image reconstruction using deep density priors. IEEE Trans. Med. Imaging 38(7), 1633–1642 (2018)

    Article  Google Scholar 

  18. Darestani, M.Z., Chaudhari, A.S., Heckel, R.: Measuring robustness in deep learning based compressive sensing. In: International Conference on Machine Learning (ICML) (2021)

    Google Scholar 

  19. Zbontar, J., et al.: fastMRI: an open dataset and benchmarks for accelerated MRI (2018)

    Google Scholar 

  20. Zhao, R., et al.: fastMRI+, clinical pathology annotations for knee and brain fully sampled magnetic resonance imaging data. Sci. Data 9(1), 152 (2022). https://doi.org/10.1038/s41597-022-01255-z

    Article  Google Scholar 

  21. Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487–492 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy - EXC number 2064/1 - Project number 390727645. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Jan Nikolas Morshuis.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jan Nikolas Morshuis .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Morshuis, J.N., Gatidis, S., Hein, M., Baumgartner, C.F. (2022). Adversarial Robustness of MR Image Reconstruction Under Realistic Perturbations. In: Haq, N., Johnson, P., Maier, A., Qin, C., Würfl, T., Yoo, J. (eds) Machine Learning for Medical Image Reconstruction. MLMIR 2022. Lecture Notes in Computer Science, vol 13587. Springer, Cham. https://doi.org/10.1007/978-3-031-17247-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-17247-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-17246-5

  • Online ISBN: 978-3-031-17247-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics