Skip to main content

GSR-Net: Graph Super-Resolution Network for Predicting High-Resolution from Low-Resolution Functional Brain Connectomes

  • Conference paper
  • First Online:
Machine Learning in Medical Imaging (MLMI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12436))

Included in the following conference series:

Abstract

Catchy but rigorous deep learning architectures were tailored for image super-resolution (SR), however, these fail to generalize to non-Euclidean data such as brain connectomes. Specifically, building generative models for super-resolving a low-resolution brain connectome at a higher resolution (i.e., adding new graph nodes/edges) remains unexplored —although this would circumvent the need for costly data collection and manual labelling of anatomical brain regions (i.e. parcellation). To fill this gap, we introduce GSR-Net (Graph Super-Resolution Network), the first super-resolution framework operating on graph-structured data that generates high-resolution brain graphs from low-resolution graphs. First, we adopt a U-Net like architecture based on graph convolution, pooling and unpooling operations specific to non-Euclidean data. However, unlike conventional U-Nets where graph nodes represent samples and node features are mapped to a low-dimensional space (encoding and decoding node attributes or sample features), our GSR-Net operates directly on a single connectome: a fully connected graph where conventionally, a node denotes a brain region, nodes have no features, and edge weights denote brain connectivity strength between two regions of interest (ROIs). In the absence of original node features, we initially assign identity feature vectors to each brain ROI (node) and then leverage the learned local receptive fields to learn node feature representations. Specifically, for each ROI, we learn a node feature embedding by locally averaging the features of its neighboring nodes based on their connectivity weights. Second, inspired by spectral theory, we break the symmetry of the U-Net architecture by topping it up with a graph super-resolution (GSR) layer and two graph convolutional network layers to predict a HR (high-resolution) graph while preserving the characteristics of the LR (low-resolution) input. Our proposed GSR-Net framework outperformed its variants for predicting high-resolution brain functional connectomes from low-resolution connectomes. Our Python GSR-Net code is available on BASIRA GitHub at https://github.com/basiralab/GSR-Net.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bahrami, K., Shi, F., Rekik, I., Gao, Y., Shen, D.: 7T-guided super-resolution of 3T mri. Med. Phys. 44, 1661–1677 (2017)

    Article  Google Scholar 

  2. Chen, Y., Xie, Y., Zhou, Z., Shi, F., Christodoulou, A.G., Li, D.: Brain mri super resolution using 3d deep densely connected neural networks. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 739–742. IEEE (2018)

    Google Scholar 

  3. Ebner, M., et al.: An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain mri. NeuroImage 206, 116324 (2020)

    Article  Google Scholar 

  4. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks (2014)

    Google Scholar 

  5. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690 (2016)

    Google Scholar 

  6. Bahrami, K., Shi, F., Rekik, I.: Convolutional neural network for reconstruction of 7T-like images from 3T MRI using appearance and anatomical features. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, (2016)

    Google Scholar 

  7. Lyu, Q., Shan, H., Wang, G.: Mri super-resolution with ensemble learning and complementary priors. IEEE Trans. Comput. Imaging 6, 615–624 (2020)

    Article  Google Scholar 

  8. Bassett, D.S., Sporns, O.: Network neuroscience. Nat. Neurosci. 20, 353 (2017)

    Article  Google Scholar 

  9. Fornito, A., Zalesky, A., Breakspear, M.: The connectomics of brain disorders. Nat. Rev. Neurosci. 16, 159–172 (2015)

    Article  Google Scholar 

  10. Van den Heuvel, M.P., Sporns, O.: A cross-disorder connectome landscape of brain dysconnectivity. Nat. Rev. Neurosci. 20, 435–446 (2019)

    Article  Google Scholar 

  11. Qi, S., Meesters, S., Nicolay, K., ter Haar Romeny, B.M., Ossenblok, P.: The influence of construction methodology on structural brain network measures: a review. J. Neurosci. Methods 253, 170–182 (2015)

    Article  Google Scholar 

  12. Bressler, S.L., Menon, V.: Large-scale brain networks in cognition: emerging methods and principles. Trends Cognitive Sci. 14, 277–290 (2010)

    Article  Google Scholar 

  13. Cui, P., Wang, X., Pei, J., Zhu, W.: A survey on network embedding. IEEE Trans. Knowl. Data Eng. 31(5), 833–852 (2017)

    Article  Google Scholar 

  14. Gao, H., Ji, S.: Graph u-nets. In: Chaudhuri, K., Salakhutdinov, R., (eds.) Proceedings of the 36th International Conference on Machine Learning. Volume 97 of Proceedings of Machine Learning Research., Long Beach, CA, USA, PMLR, pp. 2083–2092 (2019)

    Google Scholar 

  15. Cengiz, K., Rekik, I.: Predicting high-resolution brain networks using hierarchically embedded and aligned multi-resolution neighborhoods. In: Rekik, I., Adeli, E., Park, S.H. (eds.) PRIME 2019. LNCS, vol. 11843, pp. 115–124. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32281-6_12

    Chapter  Google Scholar 

  16. Mhiri, I., Khalifa, A.B., Mahjoub, M.A., Rekik, I.: Brain graph super-resolution for boosting neurological disorder diagnosis using unsupervised multi-topology connectional brain template learning. Med. Image Anal. 65, 101768 (2020)

    Article  Google Scholar 

  17. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20, 61–80 (2009)

    Article  Google Scholar 

  18. Tanaka, Y.: Spectral domain sampling of graph signals. IEEE Trans. Signal Process. 66, 3752–3767 (2018)

    Article  MathSciNet  Google Scholar 

  19. Chepuri, S.P., Leus, G.: Subsampling for graph power spectrum estimation. In: 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM), pp. 1–5. IEEE (2016)

    Google Scholar 

  20. Liu, W., et al.: Longitudinal test-retest neuroimaging data from healthy young adults in southwest china. Scientific Data. 4(1), 1–9 (2017)

    Article  Google Scholar 

  21. Dosenbach, N.U., et al.: Prediction of individual brain maturity using fMRI. Sci. 329, 1358–1361 (2010)

    Article  Google Scholar 

  22. Shen, X., Tokoglu, F., Papademetris, X., Constable, R.: Groupwise whole-brain parcellation from resting-state fMRI data for network node identification. NeuroImage 82, 403–415 (2013)

    Article  Google Scholar 

  23. Van den Heuvel, M.P., Bullmore, E.T., Sporns, O.: Comparative connectomics. Trends Cognitive Sci. 20, 345–361 (2016)

    Article  Google Scholar 

  24. Hammond, D.K., Vandergheynst, P., Gribonval, R.: Wavelets on graphs via spectral graph theory. Appl. Comput. Harmonic Anal. 30(2), 129–150 (2009)

    Article  MathSciNet  Google Scholar 

  25. Dhifallah, S., Rekik, I., Initiative, A.D.N., et al.: Estimation of connectional brain templates using selective multi-view network normalization. Med. Image Anal. 59, 101567 (2020)

    Article  Google Scholar 

Download references

Acknowledgement

This project has been funded by the 2232 International Fellowship for Outstanding Researchers Program of TUBITAK (Project No:118C288, http://basira-lab.com/reprime/) supporting I. Rekik. However, all scientific contributions made in this project are owned and approved solely by the authors.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Islem Rekik .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Isallari, M., Rekik, I. (2020). GSR-Net: Graph Super-Resolution Network for Predicting High-Resolution from Low-Resolution Functional Brain Connectomes. In: Liu, M., Yan, P., Lian, C., Cao, X. (eds) Machine Learning in Medical Imaging. MLMI 2020. Lecture Notes in Computer Science(), vol 12436. Springer, Cham. https://doi.org/10.1007/978-3-030-59861-7_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59861-7_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59860-0

  • Online ISBN: 978-3-030-59861-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics