Abstract
Recent development of image-to-image translation techniques has enabled the generation of rare medical images (e.g., PET) from common ones (e.g., MRI). Beyond the potential benefits of the reduction in scanning time, acquisition cost, and radiation exposure risks, the translation models in themselves are inscrutable black boxes. In this work, we propose two approaches to demystify the image translation process, where we particularly focus on the T1-MRI to PET translation. First, we adopt the representational similarity analysis and discover that the process of T1-MR to PET image translation includes the stages of brain tissue segmentation and brain region recognition, which unravels the relationship between the structural and functional neuroimaging data. Second, based on our findings, an Explainable and Simplified Image Translation (ESIT) model is proposed to demonstrate the capability of deep learning models for extracting gray matter volume information and identifying brain regions related to normal aging and Alzheimer’s disease, which untangles the biological plausibility hidden in deep learning models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One 10, e0130140 (2015)
Bailly, M., et al.: Precuneus and cingulate cortex atrophy and hypometabolism in patients with Alzheimer’s disease and mild cognitive impairment: MRI and 18F-FDG PET quantitative analysis using FreeSurfer. BioMed Res. Int. (2015)
Berti, V., Mosconi, L., Pupi, A.: Brain: normal variations and benign findings in fluorodeoxyglucose-PET/computed tomography imaging. PET Clin. 9, 129–140 (2014)
Chételat, G., et al.: Direct voxel-based comparison between grey matter hypometabolism and atrophy in Alzheimer’s disease. Brain 131, 60–71 (2008)
Dale, P., George, A., David, F., Lawrence, K., Anthony-Samuel, L., James, M., S, W.: Neuroscience, 2nd edn. Sinauer Associates, Sunderland (2001)
Driscoll, M.E., Bollu, P.C., Tadi, P.: Neuroanatomy, Nucleus Caudate. StatPearls Publishing, Treasure Island (FL) (2020)
Gaser, C., Dahnke, R.: Cat-a computational anatomy toolbox for the analysis of structural MRI data. Hum. Brain Mapp. (2016)
Hammers, A., et al.: Three-dimensional maximum probability atlas of the human brain, with particular reference to the temporal lobe. Hum. Brain Mapp. 19, 224–247 (2003)
Jack Jr., C.R., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging (2008)
Lan, H., Toga, A., Sepehrband, F.: SC-GAN: 3D self-attention conditional GAN with spectral normalization for multi-modal neuroimaging synthesis. bioRxiv:2020.06.09.143297 (2020)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR) (2019)
Lowe, V.J., et al.: Association of hypometabolism and amyloid levels in aging, normal subjects. Neurology 82, 1959–1967 (2014)
Manninen, S., et al.: Cerebral grey matter density is associated with neuroreceptor and neurotransporter availability: a combined PET and MRI study. bioRxiv:2020.01.29.924530 (2020)
Marcus, C., Mena, E., Subramaniam, R.M.: Brain PET in the diagnosis of Alzheimer’s disease. Clin. Nucl. Med. 39, e413 (2014)
Márquez, F., Yassa, M.A.: Neuroimaging biomarkers for Alzheimer’s disease. Mol. Neurodegeneration (2019)
Mosconi, L.: Glucose metabolism in normal aging and Alzheimer’s disease: methodological and physiological considerations for PET studies. Clin. Transl. Imaging 1, 217–233 (2013)
Nie, D., et al.: Medical image synthesis with context-aware generative adversarial networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 417–425. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_48
Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, Y., Shen, D.: Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 455–463. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_52
Penny, W.D., Friston, K.J., Ashburner, J.T., Kiebel, S.J., Nichols, T.E.: Statistical Parametric Mapping: The Analysis of Functional Brain Images. Academic Press, Cambridge (2011)
Rolls, E.T., Huang, C.C., Lin, C.P., Feng, J., Joliot, M.: Automated anatomical labelling atlas 3. NeuroImage 206, 116189 (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Shamchi, S.P., et al.: Normal patterns of regional brain 18F-FDG uptake in normal aging. Hell. J. Nucl. Med. (2018)
Shulman, R.G., Rothman, D.L., Behar, K.L., Hyder, F.: Energetic basis of brain activity: implications for neuroimaging. Trends Neurosci. 27, 489–495 (2004)
Sikka, A., Peri, S.V., Bathula, D.R.: MRI to FDG-PET: cross-modal synthesis using 3D U-Net for multi-modal Alzheimer’s classification. In: Gooya, A., Goksel, O., Oguz, I., Burgos, N. (eds.) SASHIMI 2018. LNCS, vol. 11037, pp. 80–89. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00536-8_9
Sun, H., et al.: Dual-glow: conditional flow-based generative model for modality transfer. In: IEEE International Conference on Computer Vision (ICCV) (2019)
Wei, W., et al.: Learning myelin content in multiple sclerosis from multimodal MRI through adversarial training. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 514–522. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_59
Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. ArXiv:1506.06579 (2015)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kao, CH., Chen, YS., Chen, LF., Chiu, WC. (2021). Demystifying T1-MRI to FDG\(^{18}\)-PET Image Translation via Representational Similarity. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12903. Springer, Cham. https://doi.org/10.1007/978-3-030-87199-4_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-87199-4_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87198-7
Online ISBN: 978-3-030-87199-4
eBook Packages: Computer ScienceComputer Science (R0)
