Skip to main content
SpringerLink
Log in

Leveraging deep learning-assisted attacks against image obfuscation via federated learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Obfuscation techniques (e.g., blurring) are employed to protect sensitive information (SI) in images such as individuals’ faces. Recent works demonstrated that adversaries can perform deep learning-assisted (DL) attacks to re-identify obfuscated face images. Adversaries are modeled by their goals, knowledge (e.g., background knowledge), and capabilities (e.g., DL-assisted attacks). Nevertheless, enhancing the evaluation methodology of obfuscation techniques and improving the defense strategies against adversaries requires considering more "pessimistic” attacking scenario, i.e., stronger adversaries. According to a 2019 article published by the European Union Agency for Cybersecurity (ENISA), adversaries tend to perform more sophisticated and dangerous attacks when collaborating together. To address these concerns, our paper investigates a novel privacy challenge in the context of image obfuscation. Specifically, we examine whether adversaries, when collaborating together, can amplify their DL-assisted attacks and cause additional privacy breaches against a target dataset of obfuscated images. We empirically demonstrate that federated learning (FL) can be used as a collaborative attack/adversarial strategy to (i) leverage the attacking capabilities of an adversary, (ii) increase the privacy breaches, and (iii) remedy the lack of background knowledge and data shortage without the need to share/disclose the local training datasets in a centralized location. To the best of our knowledge, we are the first to consider collaborative and more specifically FL-based attacks in the context of face obfuscation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The dataset used throughout our experiments belong to the publicly available dataset FaceScrub. The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Code availability

y: The code that we implemented will be publicly available as a GitHub repository and is referenced in the manuscript.

Notes

  1. Several studies showed that deep neural networks outperform traditional learning-based approaches for image recognition and restoration tasks [1, 16]. Hence, from a privacy perspective, deep learning-based (DL) techniques are considered strong attacks [8, 17, 18].

  2. Throughout the rest of this paper, we will use the terms obfuscation and anonymization interchangeably.

  3. The authors define a network providing multiple services by several core nodes in the scenario. Some nodes in the network might be constructed by virtualization technology and/or deployed with proactive and reactive defense resources according to the defense strategy.

  4. The central server can train the DL model in a centralized manner or by employing parallelism over different worker nodes (e.g., traditional distributed machine learning, DML)

  5. In some cases, the clients are in a peer to peer communication (no server).

  6. The communication between the FL server and the clients can be either synchronous or asynchronous. Throughout a synchronous communication, the server waits for all clients to send their update before he starts with the aggregation process.

  7. All the clients possess the exact neural network architecture.

  8. Issues with regard to attacking the FL setting, e.g., via model poisoning [43] and data poisoning [44, 45], are not part of this study’s scope.

  9. We employed the python module named random.

  10. This dataset is available upon request. It will be published along with our implementation upon acceptance.

  11. The implementation of the Resnet50 architecture is provided by the pytorch framework via https://pytorch.org/vision/stable/models/resnet.html.

  12. https://github.com/jiahuanluo/Federated-Benchmark/tree/master

  13. The code will be available as an open-source project.

  14. Our test set contains five anonymized face images per individual. Hence, we consider that an individual is accurately recognized if L images out of five are recognized (Top-1 recognition) where \(0< L<= 5\). In our experiments, we report the values for L= 3.

  15. Basically in the PyTorch framework, the weights of a network can be saved via two methods state_dict() or params(). state_dict() saves the weights containing both parameters and persistent buffers (e.g., batch normalization’s running mean and var), i.e., the complete weight structure. Whereas params() only saves the parameters without the persistent buffers. In our implementation, when averaging we are saving and loading the parameters via params(). The buffers for each network will not be shared nor aggregated. Therefore after model convergence, the local classifiers’ accuracy will be close but not identical.

  16. In this study, we consider that the adversary does not migrate face images of the newly known identities shared by other adversaries to study the effect of the label distribution skew on the FL-based attack.

  17. The authors define a network providing multiple services by several core nodes in the scenario. Some nodes in the network might be constructed by virtualization technology and/or deployed with proactive and reactive defense resources according to the defense strategy.

  18. The set of adversarial computers, which are located in different geographic or network places, are coordinated to launch attacks against a target at (roughly) the same time.

References

  1. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein MS, Berg AC, Li F (2015) Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  2. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer Vision - ECCV 2016. Springer, Cham, pp 21–37

    Chapter  Google Scholar 

  3. Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  4. Naumann A, Hertlein F, Doerr L, Thoma S, Furmans K (2023) Literature review: computer vision applications in transportation logistics and warehousing. arXiv:2304.06009

  5. Kumar S, Gopi T, Harikeerthana N, Gupta MK, Gaur V, Krolczyk GM, Wu C (2023) Machine learning techniques in additive manufacturing: a state of the art review on design, processes and production control. Journal of Intelligent Manufacturing 34(1):21–55. https://doi.org/10.1007/s10845-022-02029-5

    Article  Google Scholar 

  6. Akar CA, Tekli J, Jess D, Khoury M, Kamradt M, Guthe M (2022) Synthetic object recognition dataset for industries. In: 2022 35th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 1, 150–155. https://doi.org/10.1109/SIBGRAPI55357.2022.9991784

  7. Ayle M, Tekli J, El-Zini J, El-Asmar B, Awad M (2020) Bar - a reinforcement learning agent for bounding-box automated refinement. Proceedings of the AAAI Conference on Artificial Intelligence 34(03):2561–2568. https://doi.org/10.1609/aaai.v34i03.5639

    Article  Google Scholar 

  8. Tekli J, al Bouna B, Couturier R, Tekli G, al Zein Z, Kamradt M A framework for evaluating image obfuscation under deep learning-assisted privacy attacks. In: 17th international conference on privacy, security and trust, PST 2019, Fredericton, NB, Canada, August 26-28, 2019, pp1–10. IEEE. https://doi.org/10.1109/PST47121.2019.8949040

  9. Tekli J, Al Bouna B, Tekli G, Couturier R (2023) A framework for evaluating image obfuscation under deep learning-assisted privacy attacks. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-023-14664-y

    Article  Google Scholar 

  10. Hill S, Zhou Z, Saul LK, Shacham H (2016) On the (in)effectiveness of mosaicing and blurring as tools for document redaction. Proc. Priv. Enhancing Technol. 2016(4):403–417. https://doi.org/10.1515/popets-2016-0047

    Article  Google Scholar 

  11. Frome A, Cheung G, Abdulkader A, Zennaro M, Wu B, Bissacco A, Adam H, Neven H, Vincent L (2009) Large-scale privacy protection in google street view. In: IEEE 12th international conference on computer vision, ICCV 2009, Kyoto, Japan, September 27 - October 4, 2373–2380. IEEE Computer Society, (2009). https://doi.org/10.1109/ICCV.2009.5459413.

  12. Zhang G, Liu B, Zhu T, Zhou A, Zhou W (2022) Visual privacy attacks and defenses in deep learning: a survey. Artif Intell Rev 55(6):4347–4401. https://doi.org/10.1007/s10462-021-10123-y

    Article  Google Scholar 

  13. Hanisch S, Todt J, Patino J, Evans N, Strufe T (2023) A false sense of privacy: Towards a reliable evaluation methodology for the anonymization of biometric data. Proc. Priv. Enhancing Technol. 2024:116–132

    Article  Google Scholar 

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, 770–778. IEEE Computer Society, https://doi.org/10.1109/CVPR.2016.90.

  15. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 105–114. IEEE Computer Society. https://doi.org/10.1109/CVPR.2017.19.

  16. Yang W, Zhang X, Tian Y, Wang W, Xue J, Liao Q (2019) Deep learning for single image super-resolution: A brief review. IEEE Trans Multim 21(12):3106–3121. https://doi.org/10.1109/TMM.2019.2919431

    Article  Google Scholar 

  17. McPherson R, Shokri R, Shmatikov V (2016) Defeating image obfuscation with deep learning. arXiv:1609.00408

  18. Hao H, Güera D, Reibman AR, Delp EJ (2019) Robustness analysis of face obscuration.arXiv:1905.05243

  19. Lander K, Bruce V, Hill H (2001) Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces. Applied Cognitive Psychology 15(1):101–116. https://doi.org/10.1002/1099-0720(200101/02)15:1<101::AID-ACP697>3.0.CO;2.7

    Article  Google Scholar 

  20. Meden B, Rot P, Terhörst P, Damer N, Kuijper A, Scheirer WJ, Ross A, Peer P, Štruc V (2021) Privacy-enhancing face biometrics: a comprehensive survey. IEEE Trans Inf Forensics Secur 16:4147–4183. https://doi.org/10.1109/TIFS.2021.3096024

    Article  Google Scholar 

  21. Todt J, Hanisch S, Strufe T (2022) Fantômas: evaluating reversibility of face anonymizations using a general deep learning attacker. arXiv:2210.10651

  22. Jensen M, Cedric Lauradoux KL (2019) Pseudonymisation techniques and best practices. https://www.enisa.europa.eu/publications/pseudonymisation-techniques-and-best-practices. Accessed: 2021

  23. Cybercriminals are increasing efficiency with coordinated attacks. https://www.enisa.europa.eu/publications/info-notes/cybercriminals-are-increasing-efficiency-with-coordinated-attacks. Accessed: 2023

  24. Motaqy Z, Almashaqbeh G, Bahrak B, Yazdani N (2020) Bet and attack: Incentive compatible collaborative attacks using smart contracts. In: Decision and Game Theory for Security. https://api.semanticscholar.org/CorpusID:237571333

  25. Jingle DJ, ManoPaul P (2021) A collaborative defense protocol against collaborative attacks in wireless mesh networks. International Journal of Enterprise Network Management

  26. Lin FY-S, Wang Y-S, Chang I-T, Hsiao W-w (2014) Effective network defense strategies to assure service continuity under collaborative attacks. https://api.semanticscholar.org/CorpusID:107456049

  27. Feng Y, Hori Y, Sakurai K, Takeuchi J (2013) A behavior-based method for detecting distributed scan attacks in darknets. J. Inf. Process. 21:527–538

    Google Scholar 

  28. Xu S (2008) Collaborative attack vs. collaborative defense. In: Bertino, E., Joshi, J.B.D. (eds.) Collaborative Computing: Networking, Applications and Worksharing, 4th International Conference, CollaborateCom 2008, Orlando, FL, USA, November 13-16, Revised Selected Papers. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 10, 217–228. Springer. https://doi.org/10.1007/978-3-642-03354-4_17.

  29. Chen Y, Chu WW (2008) Protection of database security via collaborative inference detection. IEEE Trans. Knowl. Data Eng. 20(8):1013–1027. https://doi.org/10.1109/TKDE.2007.190642

    Article  Google Scholar 

  30. Duong Q, LeFevre K, Wellman MP (2010) Strategic modeling of information sharing among data privacy attackers. Informatica (Slovenia) 34(2):151–158

    Google Scholar 

  31. Yang Q, Liu Y, Cheng Y, Kang Y, Chen T, Yu H (2019) Federated Learning. Morgan & Claypool Publishers,

  32. McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Singh, A., Zhu, X.J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA. Proceedings of Machine Learning Research,54, 1273–1282. PMLR, http://proceedings.mlr.press/v54/mcmahan17a.html

  33. Abbas H, Pietro RD (2022) Sanitization of visual multimedia content: a survey of techniques, attacks, and future directions. arXiv:2207.02051

  34. Yang K, Yau J, Fei-Fei L, Deng J, Russakovsky O (2022) A Study of Face Obfuscation in ImageNet. arXiv:2103.06191

  35. Caesar H, Bankiti V, Lang AH, Vora S, Liong VE, Xu Q, Krishnan A, Pan Y, Baldan G, Beijbom O (2020) nuScenes: a multimodal dataset for autonomous driving. arXiv:1903.11027

  36. Zhu J, Gu L, Wu SX, Li Z, Harada T, Zhu Y (2023) People taking photos that faces never share: Privacy protection and fairness enhancement from camera to user. In: AAAI Conference on Artificial Intelligence. https://api.semanticscholar.org/CorpusID:259765265

  37. Kairouz P, McMahan HB, Avent B, Bellet A, Bennis M., et al (2019) Advances and open problems in federated learning. arXiv:1912.04977

  38. Hard A, Rao K, Mathews R, Beaufays F, Augenstein S, Eichner H, Kiddon C, Ramage D (2018) Federated learning for mobile keyboard prediction. arXiv:1811.03604

  39. Courtiol P, Maussion C, Moarii M, Pronier E, Pilcer S et al (2019) Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nature medicine 25(10):1519–1525. https://doi.org/10.1038/s41591-019-0583-3

    Article  Google Scholar 

  40. Musketeer, Musketeer,. http://musketeer.eu/project/. Accessed: (2021)

  41. Wu Q, He K, Chen X (2020) Personalized federated learning for intelligent iot applications: A cloud-edge based framework. IEEE Open J. Comput. Soc. 1:35–44. https://doi.org/10.1109/OJCS.2020.2993259

    Article  Google Scholar 

  42. Kulkarni V, Kulkarni M, Pant A (2020) Survey of personalization techniques for federated learning arXiv:2003.08673

  43. Bagdasaryan E, Veit A, Hua Y, Estrin D, Shmatikov V (2020) How to backdoor federated learning. In: Chiappa, S., Calandra, R. (eds.) The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy]. Proceedings of Machine Learning Research, 108, 2938–2948. PMLR, http://proceedings.mlr.press/v108/bagdasaryan20a.html

  44. Biggio B, Nelson B, Laskov P (2012) Poisoning attacks against support vector machines. In: Proceedings of the 29th international conference on machine learning, ICML 2012, Edinburgh, Scotland, UK, June 26 - July 1, 2012. icml.cc / Omnipress, http://icml.cc/2012/papers/880.pdf

  45. Liu Y, Ma S, Aafer Y, Lee W, Zhai J, Wang W, Zhang X (2018) Trojaning attack on neural networks. In: 25th annual network and distributed system security symposium, NDSS 2018, San Diego, California, USA, February 18-21, 2018. The Internet Society,

  46. Ng H. Winkler S (2014) A data-driven approach to cleaning large face datasets. In: 2014 IEEE International Conference on Image Processing, ICIP 2014, Paris, France, October 27-30, 2014, 343–347. IEEE, https://doi.org/10.1109/ICIP.2014.7025068.

  47. Yu T, Bagdasaryan E, Shmatikov V (2020) Salvaging federated learning by local adaptation.arXiv:2002.04758

  48. Luo J, Wu X, Luo Y, Huang A, Huang Y, Liu Y, Yang Q (2019) Real-world image datasets for federated learning. arXiv:1910.11089

  49. Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv:1609.04747

  50. Arivazhagan MG, Aggarwal V, Singh AK, Choudhary S (2019) Federated learning with personalization layers arXiv:1912.00818

  51. Chattopadhyay A, Ruska R, Pfantz L (2021) Determining the robustness of privacy enhancing deid against the reid adversary: an experimental study. In: Proceedings of the 16th international conference on availability, reliability and security, pp 1–11

  52. Chandiramani K, Garg D, Maheswari N (2019) Performance analysis of distributed and federated learning models on private data. Procedia Computer Science

  53. Rezaeifar S, Voloshynovskiy S, Asgari Jirhandeh M, Kinakh V (2022) Privacy-preserving image template sharing using contrastive learning. Entropy 24(5). https://doi.org/10.3390/e24050643

  54. Hsu TH, Qi H, Brown M (2020) Federated visual classification with real-world data distribution. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J. (eds.) Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, Proceedings, Part X. Lecture Notes in Computer Science, 12355, 76–92. Springer. https://doi.org/10.1007/978-3-030-58607-2_5.

  55. Hsu TH, Qi H, Brown M (2019) Measuring the effects of non-identical data distribution for federated visual classification. arXiv:1909.06335

  56. Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2018) Federated learning with non-iid dataarXiv:1806.00582

  57. Hsieh K, Phanishayee A, Mutlu O, Gibbons PB (2020) The non-iid data quagmire of decentralized machine learning. In: Proceedings of the 37th international conference on machine learning, ICML 2020, 13-18 July 2020, Virtual Event. proceedings of machine learning research, 119, pp 4387–4398. PMLR. http://proceedings.mlr.press/v119/hsieh20a.html

Download references

Acknowledgements

The authors especially thank Mr. Marc Kamradt for providing the GPU available at the BMW TechOffice located in Munich to conduct all the experiments.

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Author notes

  1. Bechara Al Bouna and Gilbert Tekli have contributed equally to this work.

Authors and Affiliations

  1. BMW Group, Munich, Germany

    Jimmy Tekli

  2. Université de Franche-Comté, CNRS, institut FEMTO-ST, F-90000 Belfort, France

    Jimmy Tekli & Raphaël Couturier

  3. TICKET Lab, Antonine University, Baabda, Lebanon

    Bechara Al Bouna & Antoine Charbel

  4. University of Balamand, T&R4.0 CoE, Balamand, Lebanon

    Gilbert Tekli

Authors
  1. Jimmy Tekli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bechara Al Bouna
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gilbert Tekli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raphaël Couturier
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antoine Charbel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jimmy Tekli.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical approval

Not applicable.

Consent to participate

Not applicable

Consent for publication

The authors consent that this paper can be published in case of acceptance.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tekli, J., Al Bouna, B., Tekli, G. et al. Leveraging deep learning-assisted attacks against image obfuscation via federated learning. Neural Comput & Applic (2024). https://doi.org/10.1007/s00521-024-09703-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00521-024-09703-0

Keywords

Search

Navigation