Skip to main content

Unsupervised Heteromodal Physics-Informed Representation of MRI Data: Tackling Data Harmonisation, Imputation and Domain Shift

  • Conference paper
  • First Online:
Simulation and Synthesis in Medical Imaging (SASHIMI 2023)

Abstract

Clinical MR imaging is typically qualitative, i.e. the observed signal reflects the underlying tissue contrast, but the measurements are not meaningful when taken in isolation. Quantitative MR imaging maps are rarely acquired due to time and complexity constraints but directly measure intrinsic tissue properties, allowing for explicit tissue characterisation and MR contrast simulation. A machine learning network trained on quantitative MRI would circumvent the need to design contrast-agnostic models, minimise domain shift issues, and reduce complex data pre-processing. Such models would also be future-proof by design, anticipating new qualitative sequence developments and changes in acquisition parameters. In this work, we propose a new Bloch-equation-based physics-informed unsupervised network that learns to map qualitative MR data to their quantitative equivalents without the need for paired qualitative-quantitative data. Furthermore, we make the proposed model robust to missing data, enabling us to map any arbitrary set of qualitative data from a patient into quantitative Multi-Parametric Maps (MPMs). We demonstrate that the estimated MPMs are a robust and invariant data representation, are self-consistent, enable missing data imputation, and facilitate data harmonisation from multiple sites while bridging algorithmic domain gaps.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Project MONAI. DOI: 10.5281/zenodo.4323059

    Google Scholar 

  2. Abd-Ellah, M.K., Awad, A.I., Khalaf, A.A.M., Hamed, H.F.A.: A review on brain tumor diagnosis from MRI images: practical implications, key achievements, and lessons learned. Magn. Reson. Imaging 61, 300–318 (2019). https://doi.org/10.1016/j.mri.2019.05.028

    Article  Google Scholar 

  3. Andreasen, N.C., et al.: T1 and T2 relaxation times in schizophrenia as measured with magnetic resonance imaging. Schizophr. Res. 5(3), 223–232 (1991). https://doi.org/10.1016/0920-9964(91)90080-B

    Article  MathSciNet  Google Scholar 

  4. Bandettini, P.A.: Twenty years of functional MRI: the science and the stories. Neuroimage 62(2), 575–588 (2012). https://doi.org/10.1016/j.neuroimage.2012.04.026

    Article  Google Scholar 

  5. Baudrexel, S., et al.: Quantitative mapping of T1 and T2* discloses nigral and brainstem pathology in early Parkinson’s disease. Neuroimage 51(2), 512–520 (2010). https://doi.org/10.1016/j.neuroimage.2010.03.005

    Article  Google Scholar 

  6. Borges, P., et al.: Physics-informed brain MRI segmentation. 11827 LNCS, 100–109 (2020). https://doi.org/10.1007/978-3-030-32778-1_11

  7. Brudfors, M., Balbastre, Y., Nachev, P., Ashburner, J.: MRI Super-Resolution using Multi-Channel Total Variation. https://brain-development.org/ixi-dataset/

  8. Brudfors, M., Balbastre, Y., Nachev, P., Ashburner, J.: A Tool for Super-Resolving Multimodal Clinical MRI (2019). 10.48550/arxiv.1909.01140. https://arxiv.org/abs/1909.01140v1

  9. Cai, C., et al.: Single-shot T2 mapping using overlapping-echo detachment planar imaging and a deep convolutional neural network. Magn. Reson. Med. 80(5), 2202–2214 (2018). https://doi.org/10.1002/MRM.27205. https://onlinelibrary.wiley.com/doi/full/10.1002/mrm.27205

  10. Cardoso, M.J., et al.: Geodesic information flows: spatially-variant graphs and their application to segmentation and fusion. IEEE Trans. Med. Imaging 34(9), 1976–1988 (2015)

    Article  Google Scholar 

  11. Chavhan, G.B.: Appropriate selection of MRI sequences for common scenarios in clinical practice. Pediatr. Radiol. 46(6), 740–747 (2016). https://doi.org/10.1007/s00247-016-3556-4

  12. Cohen, O., Zhu, B., Rosen, M.S.: MR fingerprinting Deep RecOnstruction NEtwork (DRONE). Magn. Reson. Med. 80(3), 885–894 (2018). https://doi.org/10.1002/MRM.27198, https://onlinelibrary.wiley.com/doi/full/10.1002/mrm.27198

  13. Deichmann, R., Good, C.D., Josephs, O., Ashburner, J., Turner, R.: Optimization of 3-D MP-RAGE sequences for structural brain imaging. NeuroImage 12(1), 112–127 (2000). https://doi.org/10.1006/NIMG.2000.0601,https://pubmed.ncbi.nlm.nih.gov/10875908/

  14. Desikan, R., et al.: Automated MRI measures identify individuals with mild cognitive impairment and Alzheimer’s disease. Brain : J. Neurol. 132(Pt 8), 2048–2057 (2009). https://doi.org/10.1093/BRAIN/AWP123, https://pubmed.ncbi.nlm.nih.gov/19460794/

  15. Fang, Z., et al.: Deep learning for fast and spatially constrained tissue quantification from highly accelerated data in magnetic resonance fingerprinting. IEEE Trans. Med. Imaging 38(10), 2364–2374 (2019). https://doi.org/10.1109/TMI.2019.2899328

  16. Hornak, J.: The Basics of MRI. https://www.cis.rit.edu/htbooks/mri/

  17. Jia, X., Liu, Y., Yang, Z., Yang, D.: Multi-modality self-attention aware deep network for 3D biomedical segmentation. BMC Medical Informatics and Decision Making 20(3), 1–7 (jul 2020). https://doi.org/10.1186/s12911-020-1109-0

  18. Jog, A., et al.: MR image synthesis by contrast learning on neighborhood ensembles. Med. Image Anal. 24(1), 63–76 (2015)

    Article  Google Scholar 

  19. Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19(9), 1498–1507 (2007). https://doi.org/10.1162/JOCN.2007.19.9.1498

  20. Protti, A., et al.: Development and validation of a new MRI simulation technique that can reliably estimate optimal in vivo scanning parameters in a glioblastoma murine model. PloS ONE 13(7) (2018). https://doi.org/10.1371/JOURNAL.PONE.0200611, https://pubmed.ncbi.nlm.nih.gov/30036367/

  21. Rohrer, J.D., et al.: Presymptomatic cognitive and neuroanatomical changes in genetic frontotemporal dementia in the Genetic Frontotemporal dementia Initiative (GENFI) study: a cross-sectional analysis. Articles Lancet Neurol. 14, 253–62 (2015). https://doi.org/10.1016/S1474-4422(14)70324-2, https://dx.doi.org/10.1016/

  22. Sudlow, C., et al.: UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLOS Med. 12(3), e1001779 (2015). https://doi.org/10.1371/JOURNAL.PMED.1001779, https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001779

  23. Tillin, T., et al.: Southall and Brent Revisited: Cohort profile of SABRE, a UK population-based comparison of cardiovascular disease and diabetes in people of European, Indian Asian and African Caribbean origins. Int. J. epidemiol. 41(1), 33–42 (2012). https://doi.org/10.1093/IJE/DYQ175, https://pubmed.ncbi.nlm.nih.gov/21044979/

  24. Ulyanov, D., Vedaldi, A., Lempitsky, V., Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. Int. J. Comput. Vis. 128(7), 1867–1888 (2017). https://doi.org/10.1007/s11263-020-01303-4, arXiv:1711.10925v4

  25. Varadarajan, D., Bouman, K.L., van der Kouwe, A., Fischl, B., Dalca, A.V.: Unsupervised learning of MRI tissue properties using MRI physics models (2021). arXiv.org:2107.02704v1

  26. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 586–595 (2018). https://doi.org/10.1109/CVPR.2018.00068

  27. Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain-transform manifold learning. Nature 2018 555:7697 555(7697), 487–492 (2018). https://doi.org/10.1038/nature25988, https://www.nature.com/articles/nature25988

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Pedro Borges .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 573 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Borges, P., Fernandez, V., Tudosiu, P.D., Nachev, P., Ourselin, S., Cardoso, M.J. (2023). Unsupervised Heteromodal Physics-Informed Representation of MRI Data: Tackling Data Harmonisation, Imputation and Domain Shift. In: Wolterink, J.M., Svoboda, D., Zhao, C., Fernandez, V. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2023. Lecture Notes in Computer Science, vol 14288. Springer, Cham. https://doi.org/10.1007/978-3-031-44689-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44689-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44688-7

  • Online ISBN: 978-3-031-44689-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics