Skip to main content
SpringerLink
Log in

Unsupervised privacy-enhancement of face representations using similarity-sensitive noise transformations

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Bayardo RJ, Agrawal R (2005) Data privacy through optimal k-anonymization. In: 21st International conference on data engineering (ICDE’05), pp 217–228. https://doi.org/10.1109/ICDE.2005.42

  2. Chhabra S, Singh R, Vatsa M, Gupta G (2018) Anonymizing k- facial attributes via adversarial perturbations. CoRR, arXiv:1805.09380

  3. Dantcheva A, Elia P, Ross A (2016) What else does your biometric data reveal? A survey on soft biometrics. IEEE Trans Inf Forens Secur 11(3):441–467. https://doi.org/10.1109/TIFS.2015.2480381. ISSN 1556-6013

    Article  Google Scholar 

  4. Dwork C (2006) Differential privacy. In: Bugliesi M, Preneel B, Sassone V, Wegener I (eds) Automata, languages and programming. Springer, Berlin, pp 1–12

  5. Gross R, Sweeney L, de la Torre F, Baker S (2006) Model-based face de-identification. In: 2006 Conference on computer vision and pattern recognition workshop (CVPRW’06), p 161–161. https://doi.org/10.1109/CVPRW.2006.125

  6. Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4–5):411–430. https://doi.org/10.1016/S0893-6080(00)00026-5. ISSN 0893-6080

    Article  Google Scholar 

  7. Jourabloo A, Yin X, Liu X (2015) Attribute preserved face de-identification. In: 2015 International conference on biometrics (ICB), pp 278–285. https://doi.org/10.1109/ICB.2015.7139096

  8. King DE (2015) Max-margin object detection. CoRR, arXiv:1502:00046

  9. Li N, Li T, Venkatasubramanian S (2007) t-closeness: privacy beyond k-anonymity and l-diversity. In: 2007 IEEE 23rd international conference on data engineering, pp 106–115. https://doi.org/10.1109/ICDE.2007.367856

  10. Lindell Y, Pinkas B (2000) Privacy preserving data mining. In: Proceedings of the 20th annual international cryptology conference on advances in cryptology, CRYPTO ’00. ISBN 3-540-67907-3. Springer, London, pp 36–54. http://dl.acm.org/citation.cfm?id=646765.704129

  11. Liu W, Wen Y, Yu Z, Li M, Raj B, Le S. (2017) Sphereface: deep hypersphere embedding for face recognition. CoRR, arXiv:1704.08063

  12. Machanavajjhala A, Kifer D, Gehrke J, Venkitasubramaniam M (2007) L-diversity: privacy beyond k-anonymity. ACM Trans Knowl Discov Data 1:1. https://doi.org/10.1145/1217299.1217302. ISSN 1556-4681

    Article  Google Scholar 

  13. Marsaglia G (1972) Choosing a point from the surface of a sphere. Ann Math Statist 43(2):645–646, 04. https://doi.org/10.1214/aoms/1177692644

    Article  MATH  Google Scholar 

  14. Mirjalili V, Ross A (2017) Soft biometric privacy: retaining biometric utility of face images while perturbing gender. In: 2017 IEEE International joint conference on biometrics (IJCB), pp 564–573. https://doi.org/10.1109/BTAS.2017.8272743

  15. Mirjalili V, Raschka S, Namboodiri AM, Ross A (2017) Semi-adversarial networks: Convolutional autoencoders for imparting privacy to face images. CoRR, arXiv:1712.00321

  16. Newton EM, Sweeney L, Malin B (2005) Preserving privacy by de-identifying face images. IEEE Trans Knowl Data Eng 17(2):232– 243. https://doi.org/10.1109/TKDE.2005.32. ISSN 1041-4347

    Article  Google Scholar 

  17. Othman A, Ross A (2015) Privacy of facial soft biometrics: suppressing gender but retaining identity. In: Agapito L, Bronstein MM, Rother C (eds) Computer vision - ECCV 2014 workshops. Springer International Publishing, Cham, pp 682–696

  18. Patil H, Kothari A, Bhurchandi K (2016) Expression invariant face recognition using semidecimated dwt, patch-ldsmt, feature and score level fusion. Appl Intell 44(4):913–930. https://doi.org/10.1007/s10489-015-0735-1. ISSN 1573-7497

    Article  Google Scholar 

  19. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  20. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The feret evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104. https://doi.org/10.1109/34.879790. ISSN 0162-8828

    Article  Google Scholar 

  21. Rozsa A, Günther M, Rudd EM, Boult TE (2016) Are facial attributes adversarially robust? CoRR, arXiv:1605.05411

  22. Rozsa A, Günther M, Rudd EM, Boult TE (2018) Facial attributes: accuracy and adversarial robustness. CoRR, arXiv:1801.02480

  23. Schölkopf B, Smola AJ, Müller K-R (1999) Advances in kernel methods. Chapter Kernel principal component analysis. MIT Press, Cambridge, pp 327–352. ISBN 0-262-19416-3. http://dl.acm.org/citation.cfm?id=299094.299113

    Google Scholar 

  24. Suo J, Lin L, Shan S, Chen X, Gao W (2011) High-resolution face fusion for gender conversion. IEEE Trans Syst Man Cybern - Part A: Syst Humans 41(2):226–237. https://doi.org/10.1109/TSMCA.2010.2064304. ISSN 1083-4427

    Article  Google Scholar 

  25. Tipping ME, Bishop CM (1999) Probabilistic principal component analysis. J R Statist Soc Series B 61 (3):611–622

    Article  MathSciNet  MATH  Google Scholar 

  26. Tripathi BK (2017) On the complex domain deep machine learning for face recognition. Appl Intell 47 (2):382–396. https://doi.org/10.1007/s10489-017-0902-7. ISSN 1573-7497

    Article  Google Scholar 

  27. van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605. http://www.jmlr.org/papers/v9/vandermaaten08a.html

    MATH  Google Scholar 

  28. Wang C-P, Wei W, Zhang J-S, Song H-B (2018) Robust face recognition via discriminative and common hybrid dictionary learning. Appl Intell 48(1):156–165. https://doi.org/10.1007/s10489-017-0956-6. ISSN 1573-7497

    Article  Google Scholar 

  29. Wang Y, Kosinski M (2018) Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. J Pers Soc Psychol 114(3):246–257

    Article  Google Scholar 

  30. Wei W, Yang X-L, Shen P-Y, Zhou B (2012) Holes detection in anisotropic sensornets: topological methods. Int J Distribd Sensor Netw 8(10):135054. https://doi.org/10.1155/2012/135054

    Article  Google Scholar 

  31. Wei W, Qiang Y, Zhang J (2013) A bijection between lattice-valued filters and lattice-valued congruences in residuated lattices. Mathematical Problems in Engineering

  32. Wei W, Fan X, Song H, Wang H (2017) Video tamper detection based on multi-scale mutual information. Multimed Tools Appl, 1–18

Download references

Acknowledgements

This work was supported by the German Federal Ministry of Education and Research (BMBF) as well as by the Hessen State Ministry for Higher Education, Research and the Arts (HMWK) within the Center for Research in Security and Privacy (CRISP). Portions of the research in this paper use the FERET database of facial images collected under the FERET program, sponsored by the DOD Counterdrug Technology Development Program Office.

Author information

Authors and Affiliations

  1. Fraunhofer Institute for Computer Graphics Research IGD, Darmstadt, Germany

    Philipp Terhörst, Naser Damer, Florian Kirchbuchner & Arjan Kuijper

  2. Technical University of Darmstadt, Darmstadt, Germany

    Philipp Terhörst, Naser Damer, Florian Kirchbuchner & Arjan Kuijper

Authors
  1. Philipp Terhörst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Naser Damer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florian Kirchbuchner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arjan Kuijper
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Terhörst.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Terhörst, P., Damer, N., Kirchbuchner, F. et al. Unsupervised privacy-enhancement of face representations using similarity-sensitive noise transformations. Appl Intell 49, 3043–3060 (2019). https://doi.org/10.1007/s10489-019-01432-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01432-5

Keywords

Search

Navigation