Advertisement

DeepPrivacy: A Generative Adversarial Network for Face Anonymization

  • Håkon HukkelåsEmail author
  • Rudolf Mester
  • Frank Lindseth
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11844)

Abstract

We propose a novel architecture which is able to automatically anonymize faces in images while retaining the original data distribution. We ensure total anonymization of all faces in an image by generating images exclusively on privacy-safe information. Our model is based on a conditional generative adversarial network, generating images considering the original pose and image background. The conditional information enables us to generate highly realistic faces with a seamless transition between the generated face and the existing background. Furthermore, we introduce a diverse dataset of human faces, including unconventional poses, occluded faces, and a vast variability in backgrounds. Finally, we present experimental results reflecting the capability of our model to anonymize images while preserving the data distribution, making the data suitable for further training of deep learning models. As far as we know, no other solution has been proposed that guarantees the anonymization of faces while generating realistic images.

Keywords

Image anonymization Face de-identification Generative adversarial networks 

References

  1. 1.
    Boyle, M., Edwards, C., Greenberg, S.: The effects of filtered video on awareness and privacy. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, pp. 1–10. ACM (2000).  https://doi.org/10.1145/358916.358935
  2. 2.
    Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=B1xsqj09Fm
  3. 3.
    Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 2672–2680. Curran Associates, Inc. (2014). http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
  4. 4.
    Gross, R., Sweeney, L., de la Torre, F., Baker, S.: Model-based face de-identification. In: 2006 Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2006. IEEE (2006).  https://doi.org/10.1109/cvprw.2006.125
  5. 5.
    Gross, R., Sweeney, L., Cohn, J., de la Torre, F., Baker, S.: Face de-identification. In: Senior, A. (ed.) Protecting Privacy in Video Surveillance, pp. 129–146. Springer, London (2009).  https://doi.org/10.1007/978-1-84882-301-3_8CrossRefGoogle Scholar
  6. 6.
    He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask r-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, October 2017.  https://doi.org/10.1109/iccv.2017.322
  7. 7.
    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 6626–6637. Curran Associates, Inc. (2017)Google Scholar
  8. 8.
    Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1501–1510. IEEE, October 2017.  https://doi.org/10.1109/iccv.2017.167
  9. 9.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017.  https://doi.org/10.1109/cvpr.2017.632
  10. 10.
    Jo, Y., Park, J.: SC-FEGAN: face editing generative adversarial network with user’s sketch and color. arXiv preprint arXiv:1902.06838 (2019)
  11. 11.
    Jourabloo, A., Yin, X., Liu, X.: Attribute preserved face de-identification. In: Proceedings of 2015 International Conference on Biometrics, ICB 2015, pp. 278–285 (2015).  https://doi.org/10.1109/ICB.2015.7139096
  12. 12.
    Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of gans for improved quality, stability, and variation. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=Hk99zCeAb
  13. 13.
    Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4401–4410 (2019)Google Scholar
  14. 14.
    Li, J., et al.: DSFD: dual shot face detector. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  15. 15.
    Li, Y., Liu, S., Yang, J., Yang, M.H.: Generative face completion. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5892–5900. IEEE, July 2017.  https://doi.org/10.1109/cvpr.2017.624
  16. 16.
    Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2117–2125. IEEE, July 2017.  https://doi.org/10.1109/cvpr.2017.106
  17. 17.
    Liu, G., Reda, F.A., Shih, K.J., Wang, T.-C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 89–105. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01252-6_6CrossRefGoogle Scholar
  18. 18.
    Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
  19. 19.
    Neustaedter, C., Greenberg, S., Boyle, M.: Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput. Hum. Interact. 13(1), 1–36 (2006).  https://doi.org/10.1145/1143518.1143519CrossRefGoogle Scholar
  20. 20.
    Newton, E.M., Sweeney, L., Malin, B.: Preserving privacy by de-identifying face images. IEEE Trans. Knowl. Data Eng. 17(2), 232–243 (2005).  https://doi.org/10.1109/tkde.2005.32CrossRefGoogle Scholar
  21. 21.
    NVIDIA: A pyTorch extension: tools for easy mixed precision and distributed training in PyTorch (2019). https://github.com/NVIDIA/apex
  22. 22.
    Ren, Z., Lee, Y.J., Ryoo, M.S.: Learning to anonymize faces for privacy preserving action detection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11205, pp. 639–655. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01246-5_38CrossRefGoogle Scholar
  23. 23.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  24. 24.
    Ruder, M., Dosovitskiy, A., Brox, T.: Artistic style transfer for videos. In: Rosenhahn, B., Andres, B. (eds.) GCPR 2016. LNCS, vol. 9796, pp. 26–36. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-45886-1_3CrossRefGoogle Scholar
  25. 25.
    Sweeney, L.: K-anonymity: a model for protecting privacy. Int. J. Uncertainty Fuzziness Knowl. Based Syst. 10(05), 557–570 (2002)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Thomee, B., et al.: YFCC100M: the new data in multimedia research. arXiv preprint arXiv:1503.01817 (2015). http://arxiv.org/abs/1503.01817
  27. 27.
    Yang, S., Luo, P., Loy, C.C., Tang, X.: WIDER FACE: a face detection benchmark. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2016.  https://doi.org/10.1109/cvpr.2016.596
  28. 28.
    Yazıcı, Y., Foo, C.S., Winkler, S., Yap, K.H., Piliouras, G., Chandrasekhar, V.: The unusual effectiveness of averaging in GAN training. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=SJgw_sRqFQ
  29. 29.
    Yeh, R.A., Chen, C., Lim, T.Y., Schwing, A.G., Hasegawa-Johnson, M., Do, M.N.: Semantic image inpainting with deep generative models. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6882–6890. IEEE, July 2017.  https://doi.org/10.1109/cvpr.2017.728
  30. 30.
    Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: Self-attention generative adversarial networks. In: Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 7354–7563. PMLR (2019). http://proceedings.mlr.press/v97/zhang19d.html
  31. 31.
    Zhang, H., et al.: StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5908–5916 (2017).  https://doi.org/10.1109/ICCV.2017.629
  32. 32.
    Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., Li, S.Z.: \(S^{3}\)FD: single shot scale-invariant face detector. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 192–201. IEEE, October 2017.  https://doi.org/10.1109/iccv.2017.30

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceNorwegian University of Science and TechnologyTrondheimNorway

Personalised recommendations