Skip to main content
SpringerLink
Log in

FedDAA: a robust federated learning framework to protect privacy and defend against adversarial attack

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Federated learning (FL) has emerged to break data-silo and protect clients’ privacy in the field of artificial intelligence. However, deep leakage from gradient (DLG) attack can fully reconstruct clients’ data from the submitted gradient, which threatens the fundamental privacy of FL. Although cryptology and differential privacy prevent privacy leakage from gradient, they bring negative effect on communication overhead or model performance. Moreover, the original distribution of local gradient has been changed in these schemes, which makes it difficult to defend against adversarial attack. In this paper, we propose a novel federated learning framework with model decomposition, aggregation and assembling (FedDAA), along with a training algorithm, to train federated model, where local gradient is decomposed into multiple blocks and sent to different proxy servers to complete aggregation. To bring better privacy protection performance to FedDAA, an indicator is designed based on image structural similarity to measure privacy leakage under DLG attack and an optimization method is given to protect privacy with the least proxy servers. In addition, we give defense schemes against adversarial attack in FedDAA and design an algorithm to verify the correctness of aggregated results. Experimental results demonstrate that FedDAA can reduce the structural similarity between the reconstructed image and the original image to 0.014 and remain model convergence accuracy as 0.952, thus having the best privacy protection performance and model training effect. More importantly, defense schemes against adversarial attack are compatible with privacy protection in FedDAA and the defense effects are not weaker than those in the traditional FL. Moreover, verification algorithm of aggregation results brings about negligible overhead to FedDAA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. McMahan H B, Moore E, Ramage D, Arcas B A Y. Federated learning of deep networks using model averaging. 2016, arXiv preprint arXiv: 1602.05629

  2. McMahan H B, Moore E, Ramage D, Hampson S, Arcas B A Y. Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 2017, 1273–1282

  3. Geiping J, Bauermeister H, Dröge H, Moeller M. Inverting gradients-how easy is it to break privacy in federated learning?. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1421

  4. Jeon J, Kim J, Lee K, Oh S, Ok J. Gradient inversion with generative image prior. In: Proceedings of the 35th Conference on Neural Information Processing Systems. 2021, 29898–29908

  5. Yin H, Mallya A, Vahdat A, Alvarez J M, Kautz J, Molchanov P. See through gradients: image batch recovery via gradInversion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 16332–16341

  6. Zhao B, Mopuri K R, Bilen H. iDLG: improved deep leakage from gradients. 2020, arXiv preprint arXiv: 2001.02610

  7. Zhu L, Liu Z, Han S. Deep leakage from gradients. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 1323

  8. Bhagoji A N, Chakraborty S, Mittal P, Calo S B. Analyzing federated learning through an adversarial lens. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 634–643

  9. Fung C, Yoon C J M, Beschastnikh I. Mitigating sybils in federated learning poisoning. 2018, arXiv preprint arXiv: 1808.04866

  10. Lyu L, Yu H, Yang Q. Threats to federated learning: a survey. 2020, arXiv preprint arXiv: 2003.02133

  11. Tolpegin V, Truex S, Gursoy M E, Liu L. Data poisoning attacks against federated learning systems. In: Proceedings of the 25th European Symposium on Research in Computer Security. 2020, 480–501

  12. Bagdasaryan E, Veit A, Hua Y, Estrin D, Shmatikov V. How to backdoor federated learning. In: Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics. 2020, 2938–2948

  13. Sun Z, Kairouz P, Suresh A T, McMahan H B. Can you really backdoor federated learning?. 2019, arXiv preprint arXiv: 1911.07963

  14. Wang H, Sreenivasan K, Rajput S, Vishwakarma H, Agarwal S, Sohn J Y, Lee K, Papailiopoulos D. Attack of the tails: yes, you really can backdoor federated learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1348

  15. Fang M, Cao X, Jia J, Gong N Z. Local model poisoning attacks to byzantine-robust federated learning. In: Proceedings of the 29th USENIX Conference on Security Symposium (USENIX Security 20). 2020, 92

  16. Li S, Cheng Y, Wang W, Liu Y, Chen T. Learning to detect malicious clients for robust federated learning. 2020, arXiv preprint arXiv: 2002.00211

  17. So J, Güler B, Avestimehr A S. Byzantine-resilient secure federated learning. IEEE Journal on Selected Areas in Communications, 2021, 39(7): 2168–2181

    Article  Google Scholar 

  18. Fang H, Qian Q. Privacy preserving machine learning with homomorphic encryption and federated learning. Future Internet, 2021, 13(4): 94

    Article  Google Scholar 

  19. Hardy S, Henecka W, Ivey-Law H, Nock R, Patrini G, Smith G, Thorne B. Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. 2017, arXiv preprint arXiv: 1711.10677

  20. Jiang Z, Wang W, Liu Y. FLASHE: additively symmetric homomorphic encryption for cross-silo federated learning. 2021, arXiv preprint arXiv: 2109.00675

  21. Girgis A, Data D, Diggavi S, Kairouz P, Suresh A T. Shuffled model of differential privacy in federated learning. In: Proceedings of the 24th International Conference on Artificial Intelligence and Statistics. 2021, 2521–2529

  22. Sun L, Qian J, Chen X. LDP-FL: practical private aggregation in federated learning with local differential privacy. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence. 2021, 1571–1578

  23. Truex S, Liu L, Chow K H, Gursoy M E, Wei W. LDP-Fed: federated learning with local differential privacy. In: Proceedings of the 3rd ACM International Workshop on Edge Systems, Analytics and Networking. 2020, 61–66

  24. Wei K, Li J, Ding M, Ma C, Yang H H, Farokhi F, Jin S, Quek T Q S, Poor H V. Federated learning with differential privacy: algorithms and performance analysis. IEEE Transactions on Information Forensics and Security, 2020, 15: 3454–3469

    Article  Google Scholar 

  25. Zhao Y, Zhao J, Yang M, Wang T, Wang N, Lyu L, Niyato D, Lam K Y. Local differential privacy-based federated learning for internet of things. IEEE Internet of Things Journal, 2021, 8(11): 8836–8853

    Article  Google Scholar 

  26. Bonawitz K, Ivanov V, Kreuter B, Marcedone A, McMahan H B, Patel S, Ramage D, Segal A, Seth K. Practical secure aggregation for federated learning on user-held data. 2016, arXiv preprint arXiv: 1611.04482

  27. Choi B, Sohn J Y, Han D J, Moon J. Communication-computation efficient secure aggregation for federated learning. 2020, arXiv preprint arXiv: 2012.05433

  28. Fereidooni H, Marchal S, Miettinen M, Mirhoseini A, Möllering H, Nguyen T D, Rieger P, Sadeghi A R, Schneider T, Yalame H, Zeitouni S. SAFELearn: secure aggregation for private FEderated learning. In: Proceedings of 2021 IEEE Security and Privacy Workshops (SPW). 2021, 56–62

  29. Truex S, Baracaldo N, Anwar A, Steinke T, Ludwig H, Zhang R, Zhou Y. A hybrid approach to privacy-preserving federated learning. In: Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security. 2019, 1–11

  30. Xu G, Li H, Liu S, Yang K, Lin X D. VerifyNet: secure and verifiable federated learning. IEEE Transactions on Information Forensics and Security, 2020, 15: 911–926

    Article  Google Scholar 

  31. Dong Y, Chen X, Shen L, Wang D. EaSTFLy: efficient and secure ternary federated learning. Computers & Security, 2020, 94: 101824

    Article  Google Scholar 

  32. Fang C, Guo Y, Wang N, Ju A. Highly efficient federated learning with strong privacy preservation in cloud computing. Computers & Security, 2020, 96: 101889

    Article  Google Scholar 

  33. Blanchard P, El Mhamdi E M, Guerraoui R, Stainer J. Machine learning with adversaries: byzantine tolerant gradient descent. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 118–128

  34. El Mhamdi E M, Guerraoui R, Rouault S. The hidden vulnerability of distributed learning in Byzantium. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 3521–3530

  35. Yin D, Chen Y, Kannan R, Bartlett P. Byzantine-robust distributed learning: towards optimal statistical rates. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 5650–5659

  36. Andreina S, Marson G A, Möllering H, Karame G. BaFFle: backdoor detection via feedback-based federated learning. In: Proceedings of the 41st International Conference on Distributed Computing Systems (ICDCS). 2021, 852–863

  37. Chen C, Zhang J, Tung A K H, Kankanhalli M, Chen G. Robust federated recommendation system. 2020, arXiv preprint arXiv: 2006.08259

  38. Melis L, Song C, De Cristofaro E, Shmatikov V. Exploiting unintended feature leakage in collaborative learning. In: Proceedings of 2019 IEEE Symposium on Security and Privacy (SP). 2019, 691–706

  39. Shokri R, Stronati M, Song C, Shmatikov V. Membership inference attacks against machine learning models. In: Proceedings of the IEEE Symposium on Security and Privacy (SP). 2017, 3–18

  40. Yang D, Zhang D, Yu Z, Yu Z. Fine-grained preference-aware location search leveraging crowdsourced digital footprints from LBSNs. In: Proceedings of 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2013, 479–488

  41. Huang G B, Mattar M, Berg T, Learned-Miller E. Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Proceedings of the Workshop on Faces in ‘Real-Life’ Images: detection, Alignment, and Recognition. 2008

  42. Fredrikson M, Jha S, Ristenpart T. Model inversion attacks that exploit confidence information and basic countermeasures. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015, 1322–1333

  43. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial networks. Communications of the ACM, 2020, 63(11): 139–144

    Article  MathSciNet  Google Scholar 

  44. Phong L T, Aono Y, Hayashi T, Wang L, Moriai S. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, 2018, 13(5): 1333–1345

    Article  Google Scholar 

  45. Lin Y, Han S, Mao H, Wang Y, Dally W J. Deep gradient compression: reducing the communication bandwidth for distributed training. In: Proceedings of the 6th International Conference on Learning Representations. 2018

  46. Tsuzuku Y, Imachi H, Akiba T. Variance-based gradient compression for efficient distributed deep learning. In: Proceedings of the 6th International Conference on Learning Representations. 2018

  47. Kairouz P, McMahan H B, Avent B, Bellet A, Bennis M, et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 2021, 14(1–2): 1–210

    Article  Google Scholar 

  48. Stallkamp J, Schlipsing M, Salmen J, Igel C. Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, 2012, 32: 323–332

    Article  Google Scholar 

  49. Li L, Xu W, Chen T, Giannakis G B, Ling Q. RSA: byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 1544–1551

  50. Wu Z, Ling Q, Chen T, Giannakis G B. Federated variance-reduced stochastic gradient descent with robustness to byzantine attacks. IEEE Transactions on Signal Processing, 2020, 68: 4583–4596

    Article  MathSciNet  Google Scholar 

  51. Lorenz E N. Section of planetary sciences: the predictability of hydrodynamic flow. Transactions of the New York Academy of Sciences, 1963, 25(4 Series II): 409–432

    Article  Google Scholar 

  52. May R M. Simple mathematical models with very complicated dynamics. In: Hunt B R, Li T Y, Kennedy J A, Nusse H E, eds. The Theory of Chaotic Attractors. New York: Springer, 2004, 85–93

    Chapter  Google Scholar 

  53. Hsu T M H, Qi H, Brown M. Measuring the effects of non-identical data distribution for federated visual classification. 2019, arXiv preprint arXiv: 1909.06335

  54. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770–778

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation of China (Grand Nos. 62072128, 11901579, 11801564) and the Natural Science Foundation of Shaanxi (2022JQ-046, 2021JQ-335, 2021JM-216).

Author information

Authors and Affiliations

  1. Fundamentals Department, Air Force Engineering University, Xi’an, 710051, China

    Shiwei Lu & Ruihu Li

  2. Institute of Advanced Computational Science and Technology, Guangzhou University, Guangzhou, 510006, China

    Wenbin Liu

Authors
  1. Shiwei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruihu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenbin Liu.

Additional information

Shiwei Lu is a Doctor at Department of Basic Sciences, Air Force Engineering University, China. He received his MS from Air Force Engineering University, China and achieved his BS degree in computer science and technology from Zhejiang University, China in 2017. His major interests include cyberspace security and machine learning.

Ruihu Li received his PhD degree from Northwestern Polytechnical University of Technology, China in 2004. He is currently a professor in Department of Basic Sciences, Air Force Engineering University, China. His research interests include group theory, coding theory and cryptography.

Wenbin Liu (Member, IEEE) received the PhD degree in systems engineering from the Huazhong University of Science and Technology, China in 2004. In 2004, he joined the College of Mathematics, Physics and Electronic Information Engineering, Wenzhou University, China. He was a Visiting Scholar with the Institute for Systems Biology, USA in 2006, and Texas A&M University, USA in 2013. He is currently a Professor with Guangzhou University, China. His work was supported by four NSFC grant and other funds from Zhejiang Province. His research interests include pattern recognition, data mining, Image encryption storage, and DNA storage.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, S., Li, R. & Liu, W. FedDAA: a robust federated learning framework to protect privacy and defend against adversarial attack. Front. Comput. Sci. 18, 182307 (2024). https://doi.org/10.1007/s11704-023-2283-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-023-2283-x

Keywords

Search

Navigation