Skip to main content
SpringerLink
Log in

Joint reconstruction and deidentification for mobile identity anonymization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The growing use of deep learning methods in various applications has raised concerns about privacy, as these methods heavily rely on large-scale datasets. To address this issue, anonymization methods have been proposed that usually use face detection models to remove the original face and regenerate a new one. Although these methods have achieved promising performance, alternative face detection models significantly increase the training cost and inference time. To solve these problems, we propose an end-to-end mobile anonymization method with a joint reconstruction and deidentification (JRD) framework. We introduce deidentification loss into a generative adversarial network to anonymize the original image and add reconstruction loss to reconstruct the background of the original image. By balancing these two losses, we can anonymize the identity without changing the background and no longer rely on additional face detection models. In addition, to ensure the essential difference between the generated and original images, we generated a random code for each original image. Further, we verified the effectiveness and superiority of JRD on the CelebA dataset. The experimental results show that JRD not only outperforms existing one-to-one methods but is also superior to many-to-one methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

All data included in this study are available upon request by contact with the corresponding author.

References

  1. Li X, Hou Z, Liang J, Chen C (2020) Human action recognition based on 3D body mask and depth spatial-temporal maps. Multimed Tools Appl 79:35761–35778

    Article  Google Scholar 

  2. Cao Y, Liu C, Huang Z, Sheng Y, Ju Y (2021) Skeleton-based action recognition with temporal action graph and temporal adaptive graph convolution structure. Multimed Tools Appl 80(19):29139–29162

    Article  Google Scholar 

  3. Pang Z, Guo J, Ma Z, Sun W, Xiao Y (2021) Median stable clustering and global distance classification for cross-domain person re-identification. IEEE Trans Circ Sys Video Tech 32(5):3164–3177

    Article  Google Scholar 

  4. Sheng W, Li X (2021) Multi-task learning for gait-based identity recognition and emotion recognition using attention enhanced temporal graph convolutional network. Pattern Recog 114:107868

    Article  Google Scholar 

  5. Pang Z, Zhao L, Liu Q, Wang C (2022) Camera invariant feature learning for unsupervised person re-identification. IEEE Trans Multimed

  6. Zhao H, Wang J, Li C, Liu P, Yang R (2023) Fabric defect detection via feature fusion and total variation regularized low-rank decomposition. Multimed Tools Appl 1–25

  7. Li C, Li H, Gao G, Liu Z, Liu P (2023) An accelerating convolutional neural networks via a 2D entropy based-adaptive filter search method for image recognition. Appl Soft Comput 142:110326

    Article  Google Scholar 

  8. Sun L, Zhao C, Yan Z, Liu P, Duckett T, Stolkin R (2022) A Novel Weakly-supervised approach for RGB-D-based Nuclear Waste Object Detection and Categorization

  9. Lee D, Park N (2021) Blockchain based privacy preserving multimedia intelligent video surveillance using secure Merkle tree. Multimed Tools Appl 80(26):34517–34534

    Article  Google Scholar 

  10. Zhang Y, Jiang Y, Qi L, Bhuiyan MZA, Qian P (2021) Epilepsy diagnosis using multi-view & multi-medoid entropy-based clustering with privacy protection. ACM Trans Internet Technol 21(2):1–21

    Article  Google Scholar 

  11. Wei K, Li J, Ding M, Ma C, Yang HH, Farokhi F, ... Poor HV (2020) Federated learning with differential privacy: Algorithms and performance analysis. IEEE Trans Inf Forensic Secur 15:3454–3469

  12. Desiato D (2018) A Methodology for GDPR Compliant Data Processing. In SEBD

  13. Harvey A, LaPlace J (2019) Megapixels: origins, ethics, and privacy implications of publicly available face recognition image datasets. Megapixels 1(2):6

    Google Scholar 

  14. Rodríguez-Triana MJ, Prieto LP, Holzer A, Gillet D (2020) Instruction, student engagement, and learning outcomes: a case study using anonymous social media in a face-to-face classroom. IEEE Trans Learn Technol 13(4):718–733

    Article  Google Scholar 

  15. Newton EM, Sweeney L, Malin B (2005) Preserving privacy by de-identifying face images. IEEE Trans Knowl Data Eng 17(2):232–243

    Article  Google Scholar 

  16. Kuang Z, Liu H, Yu J, Tian A, Wang L, Fan J, Babaguchi N (2021) Effective de-identification generative adversarial network for face anonymization. In Proceedings of the 29th ACM international conference on multimedia, p 3182–3191

  17. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, ... Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139–144

  18. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p 4401–4410

  19. Maximov M, Elezi I, Leal-Taixé L (2020) Ciagan: Conditional identity anonymization generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p 5447–5456

  20. Ma T, Li D, Wang W, Dong J (2021) CFA-Net: controllable face anonymization network with identity representation manipulation. arXiv preprint. arXiv:2105.11137

  21. Hukkelås H, Mester R, Lindseth F (2019) Deepprivacy: A generative adversarial network for face anonymization. International symposium on visual computing. Springer, Cham, pp 565–578

    Chapter  Google Scholar 

  22. Wu Y, Yang F, Xu Y, Ling H (2019) Privacy-protective-GAN for privacy preserving face de-identification. J Comput Sci Technol 34(1):47–60

    Article  Google Scholar 

  23. Kumar A, Kaur A, Kumar M (2019) Face detection techniques: a review. Artif Intell Rev 52(2):927–948

    Article  Google Scholar 

  24. Yang S, Luo P, Loy CC, Tang X (2016) Wider face: A face detection benchmark. In Proceedings of the IEEE conference on computer vision and pattern recognition, p 5525–5533

  25. Newton EM, Sweeney L, Malin B (2005) Preserving privacy by de-identifying face images. IEEE Trans Knowl Data Eng 17(2):232–243

    Article  Google Scholar 

  26. Sweeney L (2002) k-anonymity: A model for protecting privacy. Int J Uncertain Fuzziness Knowl-Based Syst 10(05):557–570

    Article  MathSciNet  Google Scholar 

  27. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, p 1125–1134

  28. Guo J, Pang Z, Bai M, Xie P, Chen Y (2021) Dual generative adversarial active learning. Appl Intell 51(8):5953–5964

    Article  Google Scholar 

  29. Mao X, Li Q, Xie H, Lau RY, Wang Z, Paul Smolley S (2017) Least squares generative adversarial networks. In Proceedings of the IEEE international conference on computer vision, p 2794–2802

  30. Pang Z, Guo J, Sun W, Xiao Y, Yu M (2022) Cross-domain person re-identification by hybrid supervised and unsupervised learning. Appl Intell 52(3):2987–3001

    Article  Google Scholar 

  31. Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, p 3722–3731

  32. Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Adv Neural Inf Process Sys 29

  33. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, p 3730–3738

  34. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv Neural Inf Process Sys 30

  35. Bińkowski M, Sutherland DJ, Arbel M, Gretton A (2018) Demystifying mmd gans. arXiv preprint. arXiv:1801.01401

  36. King DE (2009) Dlib-ml: A machine learning toolkit. J Mach Learn Res 10:1755–1758

    Google Scholar 

  37. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, p 815–823

  38. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI conference on artificial intelligence

  39. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, p 2223–2232

  40. Liu W, Wen Y, Yu Z, Li M, Raj B, Song L (2017) Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p 212–220

  41. Shan S, Wenger E, Zhang J, Li H, Zheng H, Zhao BY (2020) Fawkes: Protecting privacy against unauthorized deep learning models. In 29th USENIX security symposium (USENIX Security 20), p 1589–1604

Download references

Author information

Authors and Affiliations

  1. Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China

    Hyeongbok Kim, Lingling Zhao, Zhiqi Pang & Xiaohong Su

  2. Testworks, Inc., Seoul, South Korea

    Jin Suk Lee

Authors
  1. Hyeongbok Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingling Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiqi Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaohong Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jin Suk Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhiqi Pang or Xiaohong Su.

Ethics declarations

Conflicts of interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, H., Zhao, L., Pang, Z. et al. Joint reconstruction and deidentification for mobile identity anonymization. Multimed Tools Appl 83, 38313–38328 (2024). https://doi.org/10.1007/s11042-023-17107-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17107-w

Keywords

Search

Navigation