Skip to main content
SpringerLink
Log in

Federated learning attack surface: taxonomy, cyber defences, challenges, and future directions

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Federated learning (FL) has received a great deal of research attention in the context of privacy protection restrictions. By jointly training deep learning models, a variety of training tasks can be competently performed with the help of invited participants. However, FL is concerned with a large number of attacks involving privacy and security aspects. This paper shows a federated learning workflow process and how a malicious client can exploit vulnerabilities in the FL system to attack the system. A systematic survey of existing research on the taxonomy of federated learning attack surface and the classification is presented. As with the FL attack surface, attackers compromise security, privacy, gain free incentives and abuse the Confidentiality, Integrity, and Availability (CIA) security triad. In addition, state-of-the-art defensive approaches against FL attacks are elaborated which help to protect and minimize the likelihood of attacks. FL models and tools for privacy attacks are explained, along with their best aspects and drawbacks. Finally, technical challenges and possible research guidelines are discussed as future work to build robust FL systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Araki T, Furukawa J, Lindell Y, Nof A, Ohara K (2016) High-throughput semi-honest secure three-party computation with an honest majority. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, https://doi.org/10.1145/2976749.2978331

  • Ács G, Castelluccia C (2011) I have a DREAM! (DiffeRentially privatE smArt metering). In Information Hiding, pages 118–132. Springer Berlin Heidelberg, https://doi.org/10.1007/978-3-642-24178-9_9

  • Bagdasaryan E, Veit A, Hua Y, Estrin D, Shmatikov V (2020) How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pp 2938–2948. PMLR

  • Baruch M, Baruch G, Goldberg Y (2019) A little is enough: Circumventing defenses for distributed learning. arXiv preprint arXiv:1902.06156

  • Berlioz A, Friedman A, Kaafar MA, Boreli R, Berkovsky S (2015) Applying differential privacy to matrix factorization. In Proceedings of the 9th ACM Conference on Recommender Systems. ACM, https://doi.org/10.1145/2792838.2800173

  • Bertino E (2021) Attacks on artificial intelligence [last word]. IEEE Secur Privacy 19(1):103–104

    Article  Google Scholar 

  • Bhagoji AN, Chakraborty S, Mittal P, Calo S (2019) Analyzing federated learning through an adversarial lens. In International Conference on Machine Learning, pp 634–643. PMLR

  • Bhowmick A, Duchi J, Freudiger J, Kapoor G, Rogers R (2018) Protection against reconstruction and its applications in private federated learning. arXiv preprint arXiv:1812.00984

  • Blanchard P, Mhamdi EM, Guerraoui R, Stainer J (2017) Machine learning with adversaries: Byzantine tolerant gradient descent. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp 118–128

  • Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, Bernstein MS, Bohg J, Bosselut A, Brunskill E, Brynjolfsson E et al. (2021) On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

  • Bonawitz K, Ivanov V, Kreuter B, Marcedone A, McMahan HB, Patel S, Ramage D, Segal A, Seth K (2016) Practical secure aggregation for federated learning on user-held data. arXiv preprint arXiv:1611.04482

  • Bonawitz K, Ivanov V, Kreuter B, Marcedone A, McMahan HB, Patel S, Ramage D, Segal A, Seth K (2017) Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM.https://doi.org/10.1145/3133956.3133982

  • CPRA (2020) California privacy rights act, https://www.caprivacy.org/

  • Caldas S, Duddu Sai MK, Wu P, Li T, Konečnỳ J, McMahan HB, Smith V, Talwalkar A (2018) Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097

  • Cao X, Fang M, Liu J, Gong NZ (2020) Fltrust: Byzantine-robust federated learning via trust bootstrapping. arXiv preprint arXiv:2012.13995

  • Chai D, Wang L, Chen K, Yang Q (2020) Secure federated matrix factorization. IEEE Intelligent Systems, https://doi.org/10.1109/mis.2020.3014880

  • Chen Y, Luo F, Li T, Xiang T, Liu Z, Li J (2020) A training-integrity privacy-preserving federated learning scheme with trusted execution environment. Inf Sci 522:69–79. https://doi.org/10.1016/j.ins.2020.02.037

    Article  Google Scholar 

  • Chen Y, Qin X, Wang J, Chaohui Yu, Gao W (2020) FedHealth: A federated transfer learning framework for wearable healthcare. IEEE Intell Syst 35(4):83–93. https://doi.org/10.1109/mis.2020.2988604

    Article  Google Scholar 

  • Chen J, Zhang J, Zhao Y, Han H, Zhu K, Chen B (2020) Beyond model-level membership privacy leakage: an adversarial approach in federated learning. In 2020 29th International Conference on Computer Communications and Networks (ICCCN). IEEE, https://doi.org/10.1109/icccn49398.2020.9209744

  • Cheng Y, Liu Y, Chen T, Yang Q (2020) Federated learning for privacy-preserving AI. Commun ACM 63(12):33–36. https://doi.org/10.1145/3387107

    Article  Google Scholar 

  • Cheng K, Fan T, Jin Y, Liu Y, Chen T, Papadopoulos D, Yang Q (2019) Secureboost: A lossless federated learning framework. arXiv preprint arXiv:1901.08755

  • Chik WB (2013) The singapore personal data protection act and an assessment of future trends in data privacy reform. Comput Law Secur Rev 29(5):554–575. https://doi.org/10.1016/j.clsr.2013.07.010

    Article  Google Scholar 

  • Cohen G, Afshar S, Tapson J, Van Schaik A (2017) Emnist: Extending mnist to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 2921–2926. IEEE

  • Developers TensorFlow (2021) Tensorflow. https://doi.org/10.5281/ZENODO.4724125

  • Dua D, Graff C (2017) Machine learning repository, URL: http://archive.ics.uci.edu/ml/index.php

  • El Mhamdi EM, Guerraoui R, Rouault SL (2018) The hidden vulnerability of distributed learning in byzantium. arXiv preprint arXiv:1802.07927

  • FATE (2021) An industrial gradefederated learning framework, URL: https://fate.fedai.org/

  • Fang M, Cao J, Jia J, Gong N (2020) Local model poisoning attacks to byzantine-robust federated learning. In 29th USENIX Security Symposium (USENIX Security 20), pp 1605–1622

  • FeatureCloud (2021) Transforming health care and medical research with federated learning, URL: https://featurecloud.eu/about/our-vision/

  • FedAI (2020) Webank and swiss re signed cooperation mou, URL: https://www.fedai.org/news/webank-and-swiss-re-signed-cooperation-mou/

  • Feldman M, Papadimitriou C, Chuang J, Stoica I (2006) Free-riding and whitewashing in peer-to-peer systems. IEEE J Sel Areas Commun 24(5):1010–1019. https://doi.org/10.1109/jsac.2006.872882

    Article  Google Scholar 

  • Fernandes K, Vinagre P, Cortez P (2015) A proactive intelligent decision support system for predicting the popularity of online news. In Progress in Artificial Intelligence, pages 535–546. Springer International Publishing, https://doi.org/10.1007/978-3-319-23485-4_53

  • Fraboni Y, Vidal R, Lorenzi M (2021) Free-rider attacks on model aggregation in federated learning. In International Conference on Artificial Intelligence and Statistics, pp 1846–1854. PMLR

  • Fredrikson M, Jha S, Ristenpart T (2015) Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM. https://doi.org/10.1145/2810103.2813677

  • Fu S, Xie C, Li B, Chen Q (2019) Attack-resistant federated learning with residual-based reweighting. arXiv preprint arXiv:1912.11464

  • Fung C, Yoon CJM, Beschastnikh I (2020) The limitations of federated learning in sybil settings. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (\(\{\)RAID\(\}\)2020), pp 301–316

  • Fung C, Yoon CJ, Beschastnikh I (2018) Mitigating sybils in federated learning poisoning. arXiv preprint arXiv:1808.04866

  • Geyer Robin C, Klein Tassilo, Nabi Moin (2017) Differentially private federated learning: A client level perspective. arxiv preprint arXiv:1712.07557

  • Goodfellow IJ, Erhan D, Carrier PL, Courville A, Mirza M, Hamner B, Cukierski W, Tang Y, Thaler D, Lee DH, Zhou Y et al. (2013) Challenges in representation learning: A report on three machine learning contests. In International conference on neural information processing, pp 117–124. Springer

  • Google BigQuery (2017) Reddit dataset, URL: https://www.reddit.com/r/bigquery/wiki/datasets

  • Guowen X, Li H, Liu S, Yang K, Lin X (2020) VerifyNet: Secure and verifiable federated learning. IEEE Trans Inf Forensics Secur 15:911–926. https://doi.org/10.1109/tifs.2019.2929409

    Article  Google Scholar 

  • Hahn SJ, Lee J (2020) Graffl: Gradient-free federated learning of a bayesian generative model. arXiv preprint arXiv:2008.12925

  • Hardy S, Henecka W, Ivey-Law H, Nock R, Patrini G, Smith G, Thorne B (2017) Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv preprint arXiv:1711.10677

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • He Z, Zhang T, Lee RB (2019) Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference. ACM, https://doi.org/10.1145/3359789.3359824

  • Hitaj B, Ateniese G, Perez-Cruz F (2017) Deep models under the GAN. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, https://doi.org/10.1145/3133956.3134012

  • House W (2012) Consumer data privacy in a networked world: A framework for protecting privacy and promoting innovation in the global digital economy. White House, Washington, DC, pp 1–62

    Google Scholar 

  • Huang W, Li T, Wang D, Du S, Zhang J (2020) Fairness and accuracy in federated learning. arXiv preprint arXiv:2012.10069

  • Huang L, Joseph AD, Nelson B, Rubinstein BIP, Tygar JD (2011) Adversarial machine learning. In Proceedings of the 4th ACM workshop on Security and artificial intelligence - AISec ’11. ACM Press, https://doi.org/10.1145/2046684.2046692

  • Jie X, Glicksberg BS, Chang S, Walker P, Bian J, Wang F (2020) Federated learning for healthcare informatics. J Healthcare Informatics Res 5(1):1–19. https://doi.org/10.1007/s41666-020-00082-4

    Article  Google Scholar 

  • Kaggle (2013) Acquire valued shoppers challenge, URL: https://www.kaggle.com/c/acquire-valued-shoppers-challenge/data

  • Kairouz P, McMahan HB, Avent B, Bellet A, Bennis M, Bhagoji AN, Bonawitz K, Charles Z, Cormode G, Cummings R et al. (2019) Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977

  • Kang J, Xiong Z, Niyato D, Yu H, Liang YC, Kim DI (2019) Incentive design for efficient federated learning in mobile networks: A contract theory approach. In 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS). IEEE, https://doi.org/10.1109/vts-apwcs.2019.8851649

  • Kanwendy. Lending club loan data, 2019. URL: https://www.kaggle.com/wendykan/lending-club-loan-data

  • Karimireddy SP, Jaggi M, Kale S, Mohri M, Reddi SJ, Stich SU, Suresh AT (2020) Mime: Mimicking centralized stochastic algorithms in federated learning. arXiv preprint arXiv:2008.03606

  • Khazbak Y, Tan T, Cao G (2020) MLGuard: Mitigating poisoning attacks in privacy preserving distributed collaborative learning. In 2020 29th International Conference on Computer Communications and Networks (ICCCN). IEEE, https://doi.org/10.1109/icccn49398.2020.9209670

  • Kim S, Kim J, Koo D, Kim Y, Yoon H, Shin J (2016) Efficient privacy-preserving matrix factorization via fully homomorphic encryption. In Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. ACM, https://doi.org/10.1145/2897845.2897875

  • Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37. https://doi.org/10.1109/mc.2009.263

    Article  Google Scholar 

  • Krizhevsky Alex, Hinton Geoffrey, et al. (2009) Learning multiple layers of features from tiny images

  • Kuchler H (2019) Pharma groups combine to promote drug discovery with ai, URL: https://www.ft.com/content/ef7be832-86d0-11e9-a028-86cea8523dc2

  • Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  • Li H, Ota K, Dong M (2018) Learning IoT in edge: Deep learning for the internet of things with edge computing. IEEE Network 32(1):96–101. https://doi.org/10.1109/mnet.2018.1700202

    Article  Google Scholar 

  • Li T, Sahu AK, Talwalkar A, Smith V (2020) IEEE Signal Process Mag. Federated learning: challenges, methods, and future directions. 37(3):50–60. https://doi.org/10.1109/msp.2020.2975749

    Article  Google Scholar 

  • Li Z, Sharma V, Mohanty SP (2020) Preserving data privacy via federated learning: Challenges and solutions. IEEE Consumer Electron Mag 9(3):8–16. https://doi.org/10.1109/mce.2019.2959108

    Article  Google Scholar 

  • Li L, Wei X, Chen T, Giannakis GB, Ling Q (2019) RSA: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets. Proceed AAAI Conf Artif Intell 33:1544–1551. https://doi.org/10.1609/aaai.v33i01.33011544

    Article  Google Scholar 

  • Li Q, Zhu W, Wu C, Pan X, Yang F, Zhou Y, Zhang Y (2020) InvisibleFL: Federated learning over non-informative intermediate updates against multimedia privacy leakages. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, https://doi.org/10.1145/3394171.3413923

  • Li S, Cheng Y, Liu Y, Wang W, Chen T (2019) Abnormal client behavior detection in federated learning. arXiv preprint arXiv:1910.09933

  • Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2018) Federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127

  • Lim HK, Kim JB, Kim CM, Hwang GY, Choi HB, Han YH (2020) Federated reinforcement learning for controlling multiple rotary inverted pendulums in edge computing environments. In 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). IEEE. https://doi.org/10.1109/icaiic48513.2020.9065233

  • Lin J, Du M, Liu J (2019) Free-riders in federated learning: Attacks and defenses. arXiv preprint arXiv:1911.12560

  • Lin Y, Han S, Mao H, Wang Y, Dally WJ (2017) Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887

  • Liu Y, Huang A, Luo Y, Huang H, Liu Y, Chen Y, Feng L, Chen T, Han Yu, Yang Q (2020) FedVision: An online visual object detection platform powered by federated learning. Proceed AAAI Conf Artif Intell 34(08):13172–13179. https://doi.org/10.1609/aaai.v34i08.7021

    Article  Google Scholar 

  • Liu Y, Kang Y, Xing C, Chen T, Yang Q (2020) A secure federated transfer learning framework. IEEE Intell Syst 35(4):70–82. https://doi.org/10.1109/mis.2020.2988525

    Article  Google Scholar 

  • Long G, Tan Y, Jiang J, Zhang C (2020) Federated learning for open banking. In Lecture Notes in Computer Science, pages 240–254. Springer International Publishing, https://doi.org/10.1007/978-3-030-63076-8_17

  • Luo X, Wu Y, Xiao X, Ooi BC (2020) Feature inference attack on model predictions in vertical federated learning. arXiv preprint arXiv:2010.10152

  • Luo X , Zhu X (2020) Exploiting defenses against gan-based feature inference attacks in federated learning. arXiv preprint arXiv:2004.12571

  • Lyu L, Yu H, Ma X, Sun L, Zhao J, Yang Q, Yu PS (2020) Threats to federated learning. In Lecture Notes in Computer Science, pages 3–16. Springer International Publishing, https://doi.org/10.1007/978-3-030-63076-8_1

  • Ma C, Li J, Ding M, Yang HH, Shu F, Quek TQS, Vincent Poor H (2020) On safeguarding privacy and security in the framework of federated learning. IEEE Network 34(4):242–248. https://doi.org/10.1109/mnet.001.1900506

    Article  Google Scholar 

  • Ma Y, Zhu X, Hsu J (2019) Data poisoning against differentially-private learners: Attacks and defenses. arXiv preprint arXiv:1903.09860

  • Mallah RA, Lopez D, Farooq B (2021) Untargeted poisoning attack detection in federated learning via behavior attestation. arXiv preprint arXiv:2101.10904

  • McMahan HB, Ramage D, Talwar K, Zhang L (2017) Learning differentially private recurrent language models. arXiv preprint arXiv:1710.06963

  • McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, pages 1273–1282. PMLR

  • McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2016) Federated learning of deep networks using model averaging. arXiv preprint arXiv:1602.05629

  • Melis L, Song C, De Cristofaro E, Shmatikov V (2019) Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy (SP). IEEE, https://doi.org/10.1109/sp.2019.00029

  • Mo F, Haddadi H, Katevas K, Marin E, Perino D, Kourtellis N (2021) Ppfl: Privacy-preserving federated learning with trusted execution environments. arXiv preprint arXiv:2104.14380

  • Moro S, Cortez P, Rita P (2014) A data-driven approach to predict the success of bank telemarketing. Decis Support Syst 62:22–31. https://doi.org/10.1016/j.dss.2014.03.001

    Article  Google Scholar 

  • Musketeer. Smart manufacturing and health care, 2020. URL: https://musketeer.eu/project/

  • Nadiger C, Kumar A, Abdelhak S (2019) Federated reinforcement learning for fast personalization. In 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE). IEEE, https://doi.org/10.1109/aike.2019.00031

  • Naseri M, Hayes J, Emiliano DC (2020) Toward robustness and privacy in federated learning: Experimenting with local and central differential privacy. arXiv preprint arXiv:2009.03561

  • Nasr M, Shokri R, Houmansadr A (2019) Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE Symposium on Security and Privacy (SP). IEEE. https://doi.org/10.1109/sp.2019.00065

  • Nguyen TD, Rieger P, Yalame H, Mollering H, Fereidooni H, Marchal S, Miettinen M, Mirhoseini A, Sadeghi AR, Schneider T et al. (2021) Flguard: Secure and private federated learning. arXiv preprint arXiv:2101.02281

  • Nilsson A, Smith S, Gustavsson E, Jirstrand M (2018) A performance evaluation of federated learning algorithms. In Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning. ACM, https://doi.org/10.1145/3286490.3286559

  • Nishio T, Yonetani R (2019) Client selection for federated learning with heterogeneous resources in mobile edge. In ICC 2019 - 2019 IEEE International Conference on Communications (ICC). IEEE, https://doi.org/10.1109/icc.2019.8761315

  • Nock R, Hardy S, Henecka W, Ivey-Law H, Patrini G, Smith G, Thorne B (2018) Entity resolution and federated learning get a federated resolution. arXiv preprint arXiv:1803.04035

  • OpenMined (2021) Let’s solve privacy, URL: https://www.openmined.org/

  • Owkin. Federated learning, 2021. URL: https://owkin.com/federated-learning/

  • O’Driscoll A (2021) 30+ data breach statistics and facts, https://www.comparitech.com/blog/vpn-privacy/data-breach-statistics-facts/

  • Paul V, von dem Axel B (2017) The EU General data protection regulation (GDPR). Springer International Publishing, Berlin. https://doi.org/10.1007/978-3-319-57959-7

    Book  Google Scholar 

  • Phong LT, Aono Y, Hayashi T, Wang L, Moriai S (2018) Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans Inf Forensics Secur 13(5):1333–1345. https://doi.org/10.1109/tifs.2017.2787987

    Article  Google Scholar 

  • Pustozerova A, Mayer R (2020) Information leaks in federated learning. In Proceedings of the Network and Distributed System Security Symposium

  • Radanliev P, De Roure D (2021) Review of algorithms for artificial intelligence on low memory devices. IEEE Access 9:109986–109993

    Article  Google Scholar 

  • Radanliev P, De Roure D, Burnap P, Santos O (2021) Epistemological equation for analysing uncontrollable states in complex systems: Quantifying cyber risks from the internet of things. The Review of Socionetwork Strategies, pp 1–31

  • Richardson A, Filos-Ratsikas A, Faltings B (2019) Rewarding high-quality data via influence functions. arXiv preprint arXiv:1908.11598

  • Samarakoon S, Bennis M, Saad W, Debbah M (2020) Distributed federated learning for ultra-reliable low-latency vehicular communications. IEEE Trans Commun 68(2):1146–1159. https://doi.org/10.1109/tcomm.2019.2956472

    Article  Google Scholar 

  • Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In Proceedings of 1994 IEEE Workshop on Applications of Computer Vision. IEEE Comput Soc Press https://doi.org/10.1109/acv.1994.341300

  • Satariano A (2019) Google is fined 57 million under europe’s data privacy law URL: https://www.nytimes.com/2019/01/21/technology/google-europe-gdpr-fine.html

  • Sherpa.ai. (2021) We research and build artificial intelligence technology and services, URL: https://sherpa.ai/

  • Shokri R, Stronati M, Song C, Shmatikov V (2017) Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE

  • Smith SL, Kindermans PJ, Ying C, Le QV (2017) Don’t decay the learning rate, increase the batch size. arXiv preprint arXiv:1711.00489

  • So J, Guler B, Avestimehr AS (2020) Byzantine-resilient secure federated learning. IEEE J Sel Areas Commun. https://doi.org/10.1109/jsac.2020.3041404

    Article  Google Scholar 

  • Song M, Wang Z, Zhang Z, Song Y, Wang Q, Ren J, Qi H (2019) Beyond inferring class representatives: User-level privacy leakage from federated learning. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications. IEEE, https://doi.org/10.1109/infocom.2019.8737416

  • Stich SU (2018) Local sgd converges fast and communicates little. arXiv preprint arXiv:1805.09767

  • Subramanyan P, Sinha R, Lebedev I, Devadas S, Seshia SA (2017) A formal foundation for secure remote execution of enclaves. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, https://doi.org/10.1145/3133956.3134098

  • Sun Z, Kairouz P, Suresh AT, McMahan HB (2019) Can you really backdoor federated learning? arXiv preprint arXiv:1911.07963

  • Tan K, Bremner D, Le Kernec J , Imran M (2020) Federated machine learning in vehicular networks: A summary of recent applications. In 2020 International Conference on UK-China Emerging Technologies (UCET). IEEE, https://doi.org/10.1109/ucet51115.2020.9205482

  • Tolpegin V, Truex S, Gursoy ME, Liu L (2020) Data poisoning attacks against federated learning systems. In Computer Security – ESORICS 2020, pages 480–501. Springer International Publishing. https://doi.org/10.1007/978-3-030-58951-6_24

  • Truex S, Liu L, Chow K-H, Gursoy ME, Wei W (2020) LDP-fed. In Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking. ACM, https://doi.org/10.1145/3378679.3394533

  • Truex S, Liu L, Gursoy ME, Yu L, Wei W (2019) Demystifying membership inference attacks in machine learning as a service. IEEE Transactions on Services Computing, pages 1–1.https://doi.org/10.1109/tsc.2019.2897554

  • Tschandl P, Rosendahl C, Kittler H (2018) The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientif Data 5(1):1–9

    Article  Google Scholar 

  • Tseng Y-M, Chen F-G (2011) A free-rider aware reputation system for peer-to-peer file-sharing networks. Expert Syst Appl 38(3):2432–2440. https://doi.org/10.1016/j.eswa.2010.08.032

    Article  Google Scholar 

  • Wang H (2019) Baidu paddlepaddle releases 21 new capabilities to accelerate industry-grade model development, URL: http://research.baidu.com/Blog/index-view?id=126

  • Wang H, Yurochkin M, Sun Y, Papailiopoulos D, Khazaeni Y (2020) Federated learning with matched averaging. arXiv preprint arXiv:2002.06440

  • Wang L, Xu S, Wang X, Zhu Q (2019) Eavesdrop the composition proportion of training labels in federated learning. arXiv preprint arXiv:1910.06044

  • Wei O, Zeng J, Guo Z, Yan W, Liu D, Fuentes S (2020) A homomorphic-encryption-based vertical federated learning scheme for rick management. Comput Sci Inf Syst 17(3):819–834. https://doi.org/10.2298/csis190923022o

    Article  Google Scholar 

  • Wu D, Pan M, Xu Z, Zhang Y, Han Z (2020) Towards efficient secure aggregation for model update in federated learning. In GLOBECOM 2020 - 2020 IEEE Global Communications Conference. IEEE, https://doi.org/10.1109/globecom42002.2020.9347960

  • Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  • Xie C, Huang K, Chen PY, Li B (2019) Dba: Distributed backdoor attacks against federated learning. In International Conference on Learning Representations

  • Xu X, Lyu L (2020) Towards building a robust and fair federated learning system. arXiv preprint arXiv:2011.10464

  • Xu R, Baracaldo N, Zhou Y, Anwar A, Ludwig H (2019) HybridAlpha. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security - AISec’19. ACM Press, https://doi.org/10.1145/3338501.3357371

  • Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning. ACM Trans Intell Syst Technol 10(2):1–19. https://doi.org/10.1145/3298981

    Article  Google Scholar 

  • Yang D, Zhang D, Chen L, Qu B (2015) NationTelescope: Monitoring and visualizing large-scale collective behavior in LBSNs. J Netw Comput Appl 55:170–180. https://doi.org/10.1016/j.jnca.2015.05.010

    Article  Google Scholar 

  • Yang Z, Zhang J, Chang EC (2019) Adversarial neural network inversion via auxiliary knowledge alignment. arXiv preprint arXiv:1902.08552

  • Yeh I-C, Lien C (2009) The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Syst Appl 36(2):2473–2480. https://doi.org/10.1016/j.eswa.2007.12.020

    Article  Google Scholar 

  • Yelp. Yelp open dataset, 2020. URL: https://www.yelp.com/dataset

  • Zhang C, Li S, Xia J, Wang W, Yan F, Liu Y (2020) Batchcrypt: Efficient homomorphic encryption for cross-silo federated learning. In 2020 USENIX Annual Technical Conference (USENIXATC 20), pp 493–506

  • Zhang W, Tople S, Ohrimenko O (2020) Dataset-level attribute leakage in collaborative learning. arXiv preprint arXiv:2006.07267

  • Zhao Y, Chen J, Zhang J, Wu D, Teng J, Yu S (2020) PDGAN: A novel poisoning defense method in federated learning using generative adversarial network. In Algorithms and Architectures for Parallel Processing, pages 595–609. Springer International Publishing, https://doi.org/10.1007/978-3-030-38991-8_39

  • Zhao B, Mopuri KR, Bilen H (2020) idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610

  • Zheng Z, Zhou Y, Sun Y, Wang Z, Liu B, Li K (2021) Federated learning in smart cities: A comprehensive survey. arXiv e-prints, pages arXiv–2102

  • Zhou X, Ming X, Yiming W, Zheng N (2021) Deep model poisoning attack on federated learning. Future Internet 13(3):73. https://doi.org/10.3390/fi13030073

    Article  Google Scholar 

  • Zhu L, Han S (2020) Deep leakage from gradients. In Lecture Notes in Computer Science, pages 17–31. Springer International Publishing, https://doi.org/10.1007/978-3-030-63076-8_2

  • Zong B, Song Q, Min MR, Cheng W, Lumezanu C, Cho D, Chen H (2018) Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations

Download references

Author information

Authors and Affiliations

  1. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China

    Attia Qammar & Huansheng Ning

  2. Department of Computer Science, Blekinge Institute of Technology, Karlskrona, Sweden

    Jianguo Ding

Authors
  1. Attia Qammar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianguo Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huansheng Ning
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huansheng Ning.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qammar, A., Ding, J. & Ning, H. Federated learning attack surface: taxonomy, cyber defences, challenges, and future directions. Artif Intell Rev 55, 3569–3606 (2022). https://doi.org/10.1007/s10462-021-10098-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-021-10098-w

Keywords

Search

Navigation