Skip to main content
SpringerLink
Log in

FLSwitch: Towards Secure and Fast Model Aggregation for Federated Deep Learning with a Learning State-Aware Switch

  • Conference paper
  • First Online:
Applied Cryptography and Network Security (ACNS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13905))

Included in the following conference series:

  • 765 Accesses

Abstract

Security and efficiency are two desirable properties of federated learning (FL). To enforce data security for FL participants, homomorphic encryption (HE) is widely adopted. However, existing solutions based on HE treat FL as a general computation task and apply HE protections indiscriminately at each step without considering FL computations’ inherent characteristics, leading to unsatisfactory efficiency. In contrast, we find that the convergence process of FL generally consists of two phases, and the differences between these two phases can be exploited to improve the efficiency of secure FL solutions. In this paper, we propose a secure and fast FL solution named FLSwitch by tailoring different security protections for different learning phases. FLSwitch consists of three novel components, a new secure aggregation protocol based on the Pailliar HE and a residue number coding system outperforming the state-of-the-art HE-based solutions, a fast FL aggregation protocol with an extremely light overhead of learning on ciphertexts, and a learning state-aware decision model to switch between two protocols during an FL task. Since exploiting FL characteristics is orthogonal to optimizing HE techniques, FLSwitch can be applied to the existing HE-based FL solutions with cutting-edge optimizations, which could further boost secure FL efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amiri, M.M., Gündüz, D., Kulkarni, S.R., Poor, H.V.: Update aware device scheduling for federated learning at the wireless edge. In: 2020 IEEE International Symposium on Information Theory (ISIT), pp. 2598–2603. IEEE (2020)

    Google Scholar 

  2. Amiri, M.M., Gündüz, D., Kulkarni, S.R., Poor, H.V.: Convergence of update aware device scheduling for federated learning at the wireless edge. IEEE Trans. Wireless Commun. 20(6), 3643–3658 (2021)

    Article  Google Scholar 

  3. Aono, Y., Hayashi, T., Wang, L., Moriai, S., et al.: Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2017)

    Google Scholar 

  4. Baskin, C., et al.: UNIQ: uniform noise injection for non-uniform quantization of neural networks. ACM Trans. Comput. Syst. (TOCS) 37(1–4), 1–15 (2021)

    Google Scholar 

  5. Bell, J.H., Bonawitz, K.A., Gascón, A., Lepoint, T., Raykova, M.: Secure single-server aggregation with (poly) logarithmic overhead. In: ACM SIGSAC Conference on Computer and Communications Security, pp. 1253–1269 (2020)

    Google Scholar 

  6. Bonawitz, K., et al.: Towards federated learning at scale: system design. Proc. Mach. Learn. Syst. 1, 374–388 (2019)

    Google Scholar 

  7. Bonawitz, K., et al.: Practical secure aggregation for privacy-preserving machine learning. In: ACM SIGSAC Conference on Computer and Communications Security, pp. 1175–1191 (2017)

    Google Scholar 

  8. Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 409–437. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70694-8_15

    Chapter  Google Scholar 

  9. Cominetti, E.L., Simplicio, M.A.: Fast additive partially homomorphic encryption from the approximate common divisor problem. IEEE Trans. Inf. Forensics Secur. 15, 2988–2998 (2020)

    Article  Google Scholar 

  10. Fang, C., Guo, Y., Hu, Y., Ma, B., Feng, L., Yin, A.: Privacy-preserving and communication-efficient federated learning in internet of things. Comput. Secur. 103, 102199 (2021)

    Article  Google Scholar 

  11. Fang, H., Qian, Q.: Privacy preserving machine learning with homomorphic encryption and federated learning. Future Internet 13(4), 94 (2021)

    Article  Google Scholar 

  12. Fang, M., Cao, X., Jia, J., Gong, N.: Local model poisoning attacks to \(\{\)Byzantine-Robust\(\}\) federated learning. In: USENIX Security Symposium, pp. 1605–1622 (2020)

    Google Scholar 

  13. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135 (2017)

    Google Scholar 

  14. Guo, X., et al.: VeriFL: communication-efficient and fast verifiable aggregation for federated learning. IEEE Trans. Inf. Forensics Secur. 16, 1736–1751 (2020)

    Article  Google Scholar 

  15. Gurari, E.M., Ibarra, O.H.: An NP-complete number-theoretic problem. J. ACM (JACM) 26(3), 567–581 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  16. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  17. Hilbert, D.: Mathematische probleme. In: Dritter Band: Analysis \(\cdot \) Grundlagen der Mathematik \(\cdot \) Physik Verschiedenes, pp. 290–329. Springer, Berlin (1935). https://doi.org/10.1007/978-3-662-38452-7_19

  18. Hitaj, B., Ateniese, G., Perez-Cruz, F.: Deep models under the GAN: information leakage from collaborative deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 603–618 (2017)

    Google Scholar 

  19. Horváth, S., Kovalev, D., Mishchenko, K., Richtárik, P., Stich, S.: Stochastic distributed learning with gradient quantization and double-variance reduction. Optim. Methods Softw., 1–16 (2022)

    Google Scholar 

  20. Huang, Y., Gupta, S., Song, Z., Li, K., Arora, S.: Evaluating gradient inversion attacks and defenses in federated learning. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  21. Jiang, P., Agrawal, G.: A linear speedup analysis of distributed deep learning with sparse and quantized communication. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  22. Jiang, Z., Wang, W., Liu, Y.: FLASHE: additively symmetric homomorphic encryption for cross-silo federated learning. arXiv preprint: arXiv:2109.00675 (2021)

  23. Kaya, Y., Dumitras, T.: When does data augmentation help with membership inference attacks? In: International Conference on Machine Learning, pp. 5345–5355 (2021)

    Google Scholar 

  24. Krause, A., Guestrin, C.: Nonmyopic active learning of gaussian processes: an exploration-exploitation approach. In: International Conference on Machine Learning, pp. 449–456 (2007)

    Google Scholar 

  25. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  26. Lai, F., Zhu, X., Madhyastha, H.V., Chowdhury, M.: Oort: efficient federated learning via guided participant selection. In: USENIX Symposium on Operating Systems Design and Implementation, pp. 19–35 (2021)

    Google Scholar 

  27. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  28. Liu, Z., Guo, J., Yang, W., Fan, J., Lam, K.Y., Zhao, J.: Privacy-preserving aggregation in federated learning: a survey. IEEE Trans. Big Data (2022)

    Google Scholar 

  29. Luo, B., Li, X., Wang, S., Huang, J., Tassiulas, L.: Cost-effective federated learning design. In: IEEE Conference on Computer Communications, pp. 1–10 (2021)

    Google Scholar 

  30. Luo, X., Wu, Y., Xiao, X., Ooi, B.C.: Feature inference attack on model predictions in vertical federated learning. In: International Conference on Data Engineering (ICDE), pp. 181–192 (2021)

    Google Scholar 

  31. Ma, J., Naas, S.A., Sigg, S., Lyu, X.: Privacy-preserving federated learning based on multi-key homomorphic encryption. Int. J. Intell. Syst. 37(9), 5880–5901 (2022)

    Article  Google Scholar 

  32. Mao, Y., Hong, W., Zhu, B., Zhu, Z., Zhang, Y., Zhong, S.: Secure deep neural network models publishing against membership inference attacks via training task parallelism. IEEE Trans. Parallel Distrib. Syst. 33(11), 3079–3091 (2021)

    Google Scholar 

  33. Mao, Y., Yuan, X., Zhao, X., Zhong, S.: Romoa: robust Model Aggregation for the resistance of federated learning to model poisoning attacks. In: Bertino, E., Shulman, H., Waidner, M. (eds.) Computer Security—ESORICS 2021. ESORICS 2021. Lecture Notes in Computer Science(), vol. 12972, pp. 476–496 . Springer, Cham. https://doi.org/10.1007/978-3-030-88418-5_23

  34. Matijasevič, Y., Robinson, J.: Reduction of an arbitrary Diophantine equation to one in 13 unknowns. 6, 235 (1996). The Collected Works of Julia Robinson

    Google Scholar 

  35. McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282 (2017)

    Google Scholar 

  36. Melis, L., Song, C., De Cristofaro, E., Shmatikov, V.: Exploiting unintended feature leakage in collaborative learning. In: IEEE Symposium on Security and Privacy (SP), pp. 691–706 (2019)

    Google Scholar 

  37. Mishchenko, K., Gorbunov, E., Takáč, M., Richtárik, P.: Distributed learning with compressed gradient differences. arXiv preprint: arXiv:1901.09269 (2019)

  38. Mouchet, C., Troncoso-Pastoriza, J.R., Hubaux, J.P.: Multiparty homomorphic encryption: from theory to practice. IACR Cryptol. ePrint Arch. 2020, 304 (2020)

    Google Scholar 

  39. Nasr, M., Shokri, R., Houmansadr, A.: Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning. In: IEEE Symposium on Security and Privacy (SP), pp. 739–753 (2019)

    Google Scholar 

  40. Nguyen, H.T., Sehwag, V., Hosseinalipour, S., Brinton, C.G., Chiang, M., Poor, H.V.: Fast-convergent federated learning. IEEE J. Sel. Areas Commun. 39(1), 201–218 (2020)

    Article  Google Scholar 

  41. Pasquini, D., Ateniese, G., Bernaschi, M.: Unleashing the tiger: inference attacks on split learning. In: ACM SIGSAC Conference on Computer and Communications Security, pp. 2113–2129 (2021)

    Google Scholar 

  42. Ren, J., He, Y., Wen, D., Yu, G., Huang, K., Guo, D.: Scheduling for cellular federated edge learning with importance and channel awareness. IEEE Trans. Wireless Commun. 19(11), 7690–7703 (2020)

    Article  Google Scholar 

  43. Sav, S., et al.: POSEIDON: privacy-preserving federated neural network learning. In: Network and Distributed System Security Symposium, NDSS (2021)

    Google Scholar 

  44. Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321 (2015)

    Google Scholar 

  45. Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: IEEE Symposium on Security and Privacy (SP), pp. 3–18 (2017)

    Google Scholar 

  46. So, J., Güler, B., Avestimehr, A.S.: Turbo-aggregate: breaking the quadratic aggregation barrier in secure federated learning. IEEE J. Sel. Areas Inf. Theory 2(1), 479–489 (2021)

    Article  Google Scholar 

  47. Sun, J., Chen, T., Giannakis, G.B., Yang, Q., Yang, Z.: Lazily aggregated quantized gradient innovation for communication-efficient federated learning. IEEE Trans. Pattern Anal. Mach. Intell. 44(4), 2031–2044 (2020)

    Article  Google Scholar 

  48. Sun, L., Qian, J., Chen, X.: LDP-FL: practical private aggregation in federated learning with local differential privacy. In: International Joint Conference on Artificial Intelligence, IJCAI, pp. 1571–1578 (2021)

    Google Scholar 

  49. Wang, Z., Song, M., Zhang, Z., Song, Y., Wang, Q., Qi, H.: Beyond inferring class representatives: User-level privacy leakage from federated learning. In: IEEE Conference on Computer Communications, pp. 2512–2520 (2019)

    Google Scholar 

  50. Wei, K., et al.: Federated learning with differential privacy: algorithms and performance analysis. IEEE Trans. Inf. Forensics Secur. 15, 3454–3469 (2020)

    Article  Google Scholar 

  51. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint: arXiv:1708.07747 (2017)

  52. Xu, G., Li, H., Liu, S., Yang, K., Lin, X.: VerifyNet: secure and verifiable federated learning. IEEE Trans. Inf. Forensics Secur. 15, 911–926 (2019)

    Article  Google Scholar 

  53. Yang, W., Liu, B., Lu, C., Yu, N.: Privacy preserving on updated parameters in federated learning. In: Proceedings of the ACM Turing Celebration Conference-China, pp. 27–31 (2020)

    Google Scholar 

  54. Yu, S., Nguyen, P., Abebe, W., Qian, W., Anwar, A., Jannesari, A.: SPATL: salient parameter aggregation and transfer learning for heterogeneous federated learning. In: 2022 SC22: International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pp. 495–508. IEEE Computer Society (2022)

    Google Scholar 

  55. Zhang, C., Li, S., Xia, J., Wang, W., Yan, F., Liu, Y.: BatchCrypt: efficient homomorphic encryption for cross-silo federated learning. In: USENIX Annual Technical Conference, pp. 493–506 (2020)

    Google Scholar 

  56. Zhang, W., Tople, S., Ohrimenko, O.: Leakage of dataset properties in \(\{\)Multi-Party\(\}\) machine learning. In: USENIX Security Symposium, pp. 2687–2704 (2021)

    Google Scholar 

  57. Zheng, Q., Chen, S., Long, Q., Su, W.: Federated f-differential privacy. In: International Conference on Artificial Intelligence and Statistics, pp. 2251–2259 (2021)

    Google Scholar 

Download references

Acknowledgement

The authors would like to thank the anonymous reviewers for the time and efforts they have kindly made in this paper. This work was supported in part by the National Key R &D Program of China under Grants 2020YFB1005900, the Leading-edge Technology Program of Jiangsu-NSF under Grant BK20222001 and BK20202001, the National Natural Science Foundation of China under Grants NSFC-62272222, NSFC-61902176, NSFC-62272215.

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Yunlong Mao, Ziqin Dang, Yu Lin, Tianling Zhang, Yuan Zhang, Jingyu Hua & Sheng Zhong

Authors
  1. Yunlong Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziqin Dang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jingyu Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sheng Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunlong Mao .

Editor information

Editors and Affiliations

  1. NTT Social Informatics Laboratories, Tokyo, Japan

    Mehdi Tibouchi

  2. Indiana University at Bloomington, Bloomington, IN, USA

    XiaoFeng Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mao, Y. et al. (2023). FLSwitch: Towards Secure and Fast Model Aggregation for Federated Deep Learning with a Learning State-Aware Switch. In: Tibouchi, M., Wang, X. (eds) Applied Cryptography and Network Security. ACNS 2023. Lecture Notes in Computer Science, vol 13905. Springer, Cham. https://doi.org/10.1007/978-3-031-33488-7_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33488-7_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33487-0

  • Online ISBN: 978-3-031-33488-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation