Advertisement

Journal of Computer Science and Technology

, Volume 34, Issue 1, pp 47–60 | Cite as

Privacy-Protective-GAN for Privacy Preserving Face De-Identification

  • Yifan WuEmail author
  • Fan Yang
  • Yong Xu
  • Haibin Ling
Regular Paper
  • 15 Downloads

Abstract

Face de-identification has become increasingly important as the image sources are explosively growing and easily accessible. The advance of new face recognition techniques also arises people’s concern regarding the privacy leakage. The mainstream pipelines of face de-identification are mostly based on the k-same framework, which bears critiques of low effectiveness and poor visual quality. In this paper, we propose a new framework called Privacy-Protective-GAN (PP-GAN) that adapts GAN (generative adversarial network) with novel verificator and regulator modules specially designed for the face de-identification problem to ensure generating de-identified output with retained structure similarity according to a single input. We evaluate the proposed approach in terms of privacy protection, utility preservation, and structure similarity. Our approach not only outperforms existing face de-identification techniques but also provides a practical framework of adapting GAN with priors of domain knowledge.

Keywords

face de-identification privacy protection image synthesis generative adversarial network (GAN) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11390_2019_1898_MOESM1_ESM.pdf (643 kb)
ESM 1 (PDF 643 kb)

References

  1. [1]
    Newton E M, Sweeney L, Malin B. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(2): 232-243.CrossRefGoogle Scholar
  2. [2]
    Ribaric S, Ariyaeeinia A, Pavesic N. De-identification for privacy protection in multimedia content: A survey. Signal Processing: Image Communication, 2016, 47: 131-151.Google Scholar
  3. [3]
    Gross R, Airoldi E, Malin B, Sweeney L. Integrating utility into face de-identification. In Proc. the 5th International Conference on Privacy Enhancing Technologies, May 2005, pp.227-242.Google Scholar
  4. [4]
    Gross R, Sweeney L, de la Torre F, Baker S. Model-based face de-identification. In Proc. the 2006 Conference on Computer Vision and Pattern Recognition, Jun. 2006, Article No. 161.Google Scholar
  5. [5]
    Gross R, Sweeney L, de La Torre F, Baker S. Semisupervised learning of multi-factor models for face deidentification. In Proc. the 2008 Conference on Computer Vision and Pattern Recognition, Jun. 2008, Article No. 10.Google Scholar
  6. [6]
    Matthews I, Baker S. Active appearance models revisited. International Journal of Computer Vision, 2004, 60(2): 135-164.CrossRefGoogle Scholar
  7. [7]
    Du L, Yi M, Blasch E, Ling H. GARP-face: Balancing privacy protection and utility preservation in face deidentification. In Proc. IEEE International Joint Conference on Biometrics, Sept. 2014, Article No. 67.Google Scholar
  8. [8]
    Jourabloo A, Yin X, Liu X. Attribute preserved face deidentification. In Proc. IEEE International Conference on Biometrics, May 2015, pp.278-285.Google Scholar
  9. [9]
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In Proc. Annual Conference on Neural Information Processing Systems, Dec. 2014, pp.2672-2680.Google Scholar
  10. [10]
    Isola P, Zhu J Y, Zhou T, Efros A A. Image-to-image translation with conditional adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.5967-5976.Google Scholar
  11. [11]
    Zhu J Y, Park T, Isola P, Efros A A. Unpaired imageto-image translation using cycle-consistent adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.2242-2251.Google Scholar
  12. [12]
    Hadsell R, Chopra S, LeCun Y. Dimensionality reduction by learning an invariant mapping. In Proc. the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2006, pp.1735-1742.Google Scholar
  13. [13]
    Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.CrossRefGoogle Scholar
  14. [14]
    Schroff F, Kalenichenko D, Philbin J. FaceNet: A unified embedding for face recognition and clustering. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.815-823.Google Scholar
  15. [15]
    Zhang K, Zhang Z, Li Z, Qiao Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.CrossRefGoogle Scholar
  16. [16]
    Sun Q, Tewari A, Xu W, Fritz M, Theobalt C, Schiele B. A hybrid model for identity obfuscation by face replacement. In Proc. the 15th European Conference on Computer Vision, Sept. 2018, pp.570-586.Google Scholar
  17. [17]
    Chen D, Chang Y, Yan R, Yang J. Tools for protecting the privacy of specific individuals in video. EURASIP Journal on Advances in Signal Processing, 2007, 2007(1): Article No. 075427.Google Scholar
  18. [18]
    Wilber M J, Boult T E. Secure remote matching with privacy: Scrambled support vector vaulted verification (S2V3). In Proc. IEEE Workshop on Applications of Computer Vision, Jan. 2012, pp.169-176.Google Scholar
  19. [19]
    Chan A B, Liang Z S, Vasconcelos N. Privacy preserving crowd monitoring: Counting people without people models or tracking. In Proc. the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, Article No. 229.Google Scholar
  20. [20]
    Sun Q, Ma L, Oh S J, van Gool L, Schiele B, Fritz M. Natural and effective obfuscation by head inpainting. In Proc. the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp.5050-5059.Google Scholar
  21. [21]
    Sun Q, Schiele B, Fritz M. A domain based approach to social relation recognition. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.435-444.Google Scholar
  22. [22]
    Mirjalili V, Raschka S, Namboodiri A, Ross A. Semiadversarial networks: Convolutional autoencoders for imparting privacy to face images. In Proc. the 2018 International Conference on Biometrics, Feb. 2018 pp.82-89.Google Scholar
  23. [23]
    Ribaric S, Ariyaeeinia A, Pavesic N. De-identification for privacy protection in multimedia content: A survey. Signal Processing: Image Communication, 2016, 47: 131-151.Google Scholar
  24. [24]
    Gross R, Sweeney L, Cohn J, de la Torre F, Baker S. Face de-identification. In Protecting Privacy in Video Surveillance, Senior A (ed.), Springer, 2009, pp.129-146.Google Scholar
  25. [25]
    Frome A, Cheung G, Abdulkader A, Zennaro M, Wu B, Bissacco A, Adam H, Neven H, Vincent L. Large-scale privacy protection in Google street view. In Proc. the 12th International Conference on Computer Vision, Sept. 2009, pp.2373-2380.Google Scholar
  26. [26]
    Sweeney L. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 2002, 10(5): 557-570.MathSciNetCrossRefzbMATHGoogle Scholar
  27. [27]
    Brkic K, Sikiric I, Hrkac T, Kalafatic Z. I know that person: Generative full body and face de-identification of people. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Jul. 2017, pp.1319-1328.Google Scholar
  28. [28]
    Meden B, Mallı R C, Fabijan S, Ekenel H K, Struc V, Peer P. Face deidentification with generative deep neural networks. IET Signal Processing, 2017, 11(9): 1046-54.CrossRefGoogle Scholar
  29. [29]
    Meden B, Ziga E, Struc V, Peer P. k-same-net: k-anonymity with generative deep neural networks for face deidentification. Entropy, 2018, 20(1): Article No. 60.Google Scholar
  30. [30]
    Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proc. the 32nd International Conference on Machine Learning, Jul. 2015, pp.448-456.Google Scholar
  31. [31]
    Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Oct. 2015 pp.234-241.Google Scholar
  32. [32]
    Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In Proc. the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2005, pp.539-546.Google Scholar
  33. [33]
    Sun Y, Chen Y, Wang X, Tang X. Deep learning face representation by joint identification-verification. In Proc. the 2014 Annual Conference on Neural Information Processing Systems, Dec. 2014, pp.1988-1996.Google Scholar
  34. [34]
    Schroff F, Kalenichenko D, Philbin J. FaceNet: A unified embedding for face recognition and clustering. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp.815-823.Google Scholar
  35. [35]
    Parkhi O M, Vedaldi A, Zisserman A. Deep face recognition. In Proc. the 26th British Machine Vision Conference, Sept. 2015, Article No. 6.Google Scholar
  36. [36]
    Wu X, He R, Sun Z, Tan T. A light CNN for deep face representation with noisy labels. IEEE Transactions on Information Forensics and Security, 2018, 13(11): 2884-2896.CrossRefGoogle Scholar
  37. [37]
    Lin M, Chen Q, Yan S. Network in network. arXiv:1312.4400, 2013. https://arxiv.org/pdf/131-2.4400.pdf, Sept. 2018.
  38. [38]
    Du L, Ling H. Preservative license plate de-identification for privacy protection. In Proc. the 2011 International Conference on Document Analysis and Recognition, Sept. 2011, pp.468-472.Google Scholar
  39. [39]
    Ricanek K, Tesafaye T. MORPH: A longitudinal image database of normal adult age-progression. In Proc. the 7th International Conference on Automatic Face and Gesture Recognition, Apr. 2006, pp.341-345.Google Scholar
  40. [40]
    Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv:1412.6980, 2015. https://arxiv.org/pdf/1412.6980.pdf, Sept. 2018.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer and Information SciencesTemple UniversityPhiladelphiaUSA
  2. 2.School of Computer Science and EngineeringSouth China University of TechnologyGuangzhouChina

Personalised recommendations