Abstract
Having access to brain connectomes at various resolutions is important for clinicians, as they can reveal vital information about brain anatomy and function. However, the process of deriving the graphs from magnetic resonance imaging (MRI) is computationally expensive and error-prone. Furthermore, an existing challenge in the medical domain is the small amount of data that is available, as well as privacy concerns. In this work, we propose a new federated learning framework, named RepFL. At its core, RepFL is a replica-based federated learning approach for heterogeneous models, which creates replicas of each participating client by copying its model architecture and perturbing its local training dataset. This solution enables learning from limited data with a small number of participating clients by aggregating multiple local models and diversifying the data distributions of the clients. Specifically, we apply the framework for graph super-resolution using heterogeneous model architectures. In addition, to the best of our knowledge, this is the first federated multi-resolution graph generation approach. Our experiments prove that the method outperforms other federated learning methods on the task of brain graph super-resolution. Our RepFL code is available at https://github.com/basiralab/RepFL.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fornito, A., Zalesky, A., Bullmore, E.: Fundamentals of Brain Network Analysis. Academic Press (2016)
Bessadok, A., Mahjoub, M.A., Rekik, I.: Graph neural networks in network neuroscience. IEEE Trans. Pattern Anal. Mach. Intell. 45, 5833–5848 (2022)
Zhou, J., et al.: Graph neural networks: a review of methods and applications. AI Open 1, 57–81 (2020)
Veličković, P.: Everything is connected: graph neural networks. Curr. Opin. Struct. Biol. 79, 102538 (2023)
Kairouz, P., et al.: Advances and open problems in federated learning. Found. Trends® Mach. Learn. 14, 1–210 (2021)
Pfitzner, B., Steckhan, N., Arnrich, B.: Federated learning in a medical context: a systematic literature review. ACM Trans. Internet Technol. (TOIT) 21, 1–31 (2021)
Isallari, M., Rekik, I.: GSR-net: Graph super-resolution network for predicting high-resolution from low-resolution functional brain connectomes. In: Liu, M., Yan, P., Lian, C., Cao, X. (eds.) MLMI 2020. LNCS, vol. 12436, pp. 139–149. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59861-7_15
Mhiri, I., Nebli, A., Mahjoub, M.A., Rekik, I.: Non-isomorphic inter-modality graph alignment and synthesis for holistic brain mapping. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds.) IPMI 2021. LNCS, vol. 12729, pp. 203–215. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78191-0_16
Mhiri, I., Mahjoub, M.A., Rekik, I.: Stairwaygraphnet for inter-and intra-modality multi-resolution brain graph alignment and synthesis. In: Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, 27 September 2021, pp. 140–150. Springer, Cham (2021)
Pala, F., Mhiri, I., Rekik, I.: Template-based inter-modality super-resolution of brain connectivity. In: Rekik, I., Adeli, E., Park, S.H., Schnabel, J. (eds.) PRIME 2021. LNCS, vol. 12928, pp. 70–82. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87602-9_7
Chaari, N., Akdağ, H.C., Rekik, I.: Comparative survey of multigraph integration methods for holistic brain connectivity mapping. Med. Image Anal. 102741 (2023)
Demir, B., Bessadok, A., Rekik, I.: Inter-domain alignment for predicting high-resolution brain networks using teacher-student learning. In: Albarqouni, S., et al. (eds.) DART/FAIR -2021. LNCS, vol. 12968, pp. 203–215. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87722-4_19
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, PMLR, pp. 1273–1282 (2017)
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proc. Mach. Learn. Syst. 2, 429–450 (2020)
Acar, D.A.E., Zhao, Y., Matas, R., Mattina, M., Whatmough, P., Saligrama, V.: Federated learning based on dynamic regularization. In: International Conference on Learning Representations (2020)
Kulkarni, V., Kulkarni, M., Pant, A.: Survey of personalization techniques for federated learning. In: 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), pp. 794–797. IEEE (2020)
Hanzely, F., Richtárik, P.: Federated learning of a mixture of global and local models. arXiv preprint arXiv:2002.05516 (2020)
Deng, Y., Kamani, M.M., Mahdavi, M.: Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461 (2020)
Sattler, F., Müller, K.R., Samek, W.: Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints. IEEE Trans. Neural Netw. Learn. Syst. 32, 3710–3722 (2020)
Ghosh, A., Chung, J., Yin, D., Ramchandran, K.: An efficient framework for clustered federated learning. Adv. Neural Inf. Process. Syst. 33, 19586–19597 (2020)
Arivazhagan, M.G., Aggarwal, V., Singh, A.K., Choudhary, S.: Federated learning with personalization layers. arXiv preprint arXiv:1912.00818 (2019)
Diao, E., Ding, J., Tarokh, V.: Heterofl: Computation and communication efficient federated learning for heterogeneous clients. In: International Conference on Learning Representations (2020)
Shamsian, A., Navon, A., Fetaya, E., Chechik, G.: Personalized federated learning using hypernetworks. In: International Conference on Machine Learning, PMLR, pp. 9489–9502 (2021)
Litany, O., Maron, H., Acuna, D., Kautz, J., Chechik, G., Fidler, S.: Federated learning with heterogeneous architectures using graph hypernetworks. arXiv preprint arXiv:2201.08459 (2022)
Li, D., Wang, J.: Fedmd: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019)
Wang, S., Xie, J., Lu, M., Xiong, N.N.: Fedgraph-kd: an effective federated graph learning scheme based on knowledge distillation. In: 2023 IEEE 9th International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing, (HPSC) and IEEE International Conference on Intelligent Data and Security (IDS), pp. 130–134. IEEE (2023)
Kamp, M., Fischer, J., Vreeken, J.: Federated learning from small datasets. In: The Eleventh International Conference on Learning Representations (2022)
Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3693–3702 (2017)
Liu, W., et al.: Longitudinal test-retest neuroimaging data from healthy young adults in Southwest China. Sci. Data 4, 1–9 (2017)
Dosenbach, N.U., et al.: Prediction of individual brain maturity using FMRI. Science 329, 1358–1361 (2010)
Shen, X., Tokoglu, F., Papademetris, X., Constable, R.: Groupwise whole-brain parcellation from resting-state FMRI data for network node identification. Neuroimage 82, 403–415 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ghilea, R., Rekik, I. (2024). Replica-Based Federated Learning with Heterogeneous Architectures for Graph Super-Resolution. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds) Machine Learning in Medical Imaging. MLMI 2023. Lecture Notes in Computer Science, vol 14349. Springer, Cham. https://doi.org/10.1007/978-3-031-45676-3_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-45676-3_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45675-6
Online ISBN: 978-3-031-45676-3
eBook Packages: Computer ScienceComputer Science (R0)
