Abstract
As a promising machine learning framework in the big data era, federated learning (FL) allows multiple mobile devices to collaboratively train a model without transmitting raw data, thus has attracted widespread attention in both academia and industry. Considering that heterogeneous mobile devices with limited resources and data diversity are bound to impact the actual performance of some training nodes. However, conventional FL could not support collaborative training with multi-granularity neural networks. To this end, we propose multi-granularity federated learning (MGFL) that contains two mechanisms serving for same-granularity FL and cross-granularity FL. MGFL customizes a personalized model for each device by designing a divergence-based similarity measurement method in same-granularity FL. Further, it adjusts the empirical risk loss function to break the restriction of cross-granularity FL. Experimental evaluations demonstrate the positive guidance of the fine-granularity model to the coarse-granularity model, which significantly improves the performance of the coarse-granularity model. Besides, our method shows superiority on both independently identically distribution (IID) and non-IID data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, T., Luo, H., Zeng, X., Yu, Z., Liu, A., Sangaiah, A.K.: Mobility based trust evaluation for heterogeneous electric vehicles network in smart cities. IEEE Trans. Intell. Transp. Syst. 22(3), 1797–1806 (2020)
Wang, T., et al.: Propagation modeling and defending of a mobile sensor worm in wireless sensor and actuator networks. Sensors 17(1), 139 (2017)
Wang, X., Li, R., Wang, C., Li, X., Taleb, T., Leung, V.C.: Attention-weighted federated deep reinforcement learning for device-to-device assisted heterogeneous collaborative edge caching. IEEE J. Sel. Areas Commun. 39(1), 154–169 (2020)
Wang, X., Wang, C., Li, X., Leung, V.C., Taleb, T.: Federated deep reinforcement learning for internet of things with decentralized cooperative edge caching. IEEE Internet Things J. 7(10), 9441–9455 (2020)
Wang, X., Han, Y., Leung, V.C., Niyato, D., Yan, X., Chen, X.: Convergence of edge computing and deep learning: a comprehensive survey. IEEE Commun. Surv. Tut. 22(2), 869–904 (2020)
Alsheikh, M.A., Niyato, D., Lin, S., Tan, H., Han, Z.: Mobile big data analytics using deep learning and apache spark. IEEE Netw. 30(3), 22–29 (2016)
Bhowmick, A., Duchi, J., Freudiger, J., Kapoor, G., Rogers, R.: Protection against reconstruction and its applications in private federated learning. arXiv preprint arXiv:1812.00984 (2018)
Bonawitz, K., et al.: Practical secure aggregation for federated learning on user-held data. arXiv preprint arXiv:1611.04482 (2016)
Wang, T., Bhuiyan, M.Z.A., Wang, G., Qi, L., Wu, J., Hayajneh, T.: Preserving balance between privacy and data integrity in edge-assisted internet of things. IEEE Internet Things J. 7(4), 2679–2689 (2019)
Hard, A., et al.: Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604 (2018)
Kairouz, P., et al.: Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 (2019)
Tan, A.Z., Yu, H., Cui, L., Yang, Q.: Towards personalized federated learning. arXiv preprint arXiv:2103.00710 (2021)
Huang, Y., et al.: Personalized cross-silo federated learning on non-IID data. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 7865–7873 (2021)
Nagalapatti, L., Narayanam, R.: Game of gradients: mitigating irrelevant clients in federated learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 9046–9054 (2021)
Sattler, F., Müller, K.R., Samek, W.: Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints. IEEE Trans. Neural Netw. Learn. Syst. 32, 3710–3722 (2020)
Ouyang, X., Xie, Z., Zhou, J., Huang, J., Xing, G.: ClusterFL: a similarity-aware federated learning system for human activity recognition. In: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, pp. 54–66 (2021)
Nishio, T., Yonetani, R.: Client selection for federated learning with heterogeneous resources in mobile edge. In: 2019 IEEE International Conference on Communications (ICC), ICC 2019, pp. 1–7 (2019)
Smith, V., Chiang, C.K., Sanjabi, M., Talwalkar, A.: Federated multi-task learning. arXiv preprint arXiv:1705.10467 (2017)
Liu, F., Wu, X., Ge, S., Fan, W., Zou, Y.: Federated learning for vision-and-language grounding problems. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11572–11579 (2020)
Corinzia, L., Beuret, A., Buhmann, J.M.: Variational federated multi-task learning. arXiv preprint arXiv:1906.06268 (2019)
Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146 (2016)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with non-IID data. arXiv preprint arXiv:1806.00582 (2018)
Ghosh, A., Hong, J., Yin, D., Ramchandran, K.: Robust federated learning in a heterogeneous environment. arXiv preprint arXiv:1906.06629 (2019)
Jiang, B., Pei, J., Tao, Y., Lin, X.: Clustering uncertain data based on probability distribution similarity. IEEE Trans. Knowl. Data Eng. 25(4), 751–763 (2011)
Acknowledgement
This work is supported by research on key technologies of electrical cloud-edge-end collaborative AI model sharing in Science and Technology Project of State Grid Headquarters (2021, Power base support technology - 30).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Cai, S., Zhao, Y., Liu, Z., Qiu, C., Wang, X., Hu, Q. (2022). MGFL: Multi-granularity Federated Learning in Edge Computing Systems. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13155. Springer, Cham. https://doi.org/10.1007/978-3-030-95384-3_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-95384-3_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95383-6
Online ISBN: 978-3-030-95384-3
eBook Packages: Computer ScienceComputer Science (R0)