Abstract
This paper studies the performance of membership inference attacks against principal component analysis (PCA). In this attack, we assume that the adversary has access to the principal components, and her main goal is to infer whether a given data sample was used to compute these principal components. We show that our attack is successful and achieves high performance when the number of samples used to compute the principal components is small. As a defense strategy, we investigate the use of various differentially private mechanisms. Accordingly, we present experimental results on the performance of Gaussian and Laplace mechanisms under naive and advanced compositions against MIA as well as the utility of these differentially-private PCA solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Due to page limit constraint, we report only the results for Adult and LFW datasets. We refer the reader to the full version of this paper [29].
- 2.
Recall that A is a symmetric matrix.
References
Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS 2016, Vienna, Austria, pp. 308–318. Association for Computing Machinery (2016). ISBN: 9781450341394. https://doi.org/10.1145/2976749.2978318
Balcan, M.-F., et al.: Improved distributed principal component analysis. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, NIPS 2014, Montreal, Canada, vol. 2, pp. 3113–3121. MIT Press (2014)
Blanco-Justicia, A., et al.: A critical review on the use (and misuse) of differential privacy in machine learning (2022). https://arxiv.org/abs/2206.04621
Brand, R., Domingo-Ferrer, J., Mateo-Sanz, J.M.: Reference data sets to test and compare SDC methods for protection of numerical microdata. Technical report. https://research.cbs.nl/casc/CASCrefmicrodata.pdf
Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9(3–4), 211–407 (2014). ISSN: 1551-305X
Dwork, C., et al.: Analyze gauss: optimal bounds for privacy-preserving principal component analysis. In: Proceedings of the Forty-Sixth Annual ACM Symposium on Theory of Computing, STOC 2014, pp. 11–20. Association for Computing Machinery, New York (2014). ISBN: 9781450327107. https://doi.org/10.1145/2591796.2591883
Homer, N., et al.: Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high-density SNP genotyping microarrays. PLoS Genet. 4, e1000167 (2008)
Huang, G.B., et al.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07-49, University of Massachusetts, Amherst, October 2007
Hundepool, A., et al.: Statistical Disclosure Control (2012). Ed. by S. Fischer-Hūbner et al.
Imtiaz, H., Sarwate, A.D.: Differentially private distributed principal component analysis. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2206–2210 (2018). https://doi.org/10.1109/ICASSP.2018.8462519
Imtiaz, H., Sarwate, A.D.: Symmetric matrix perturbation for differentially-private principal component analysis. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2339–2343, March 2016. https://doi.org/10.1109/ICASSP.2016.7472095
Jayaraman, B., Evans, D.E.: Evaluating differentially private machine learning in practice. In: USENIX Security Symposium (2019)
Jayaraman, B., et al.: Revisiting membership inference under realistic assumptions. In: Proceedings on Privacy Enhancing Technologies 2021, pp. 348–368 (2021)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
Long, Y., et al.: Understanding membership inferences on well-generalized learning models. ArXiv, abs/1802.04889 (2018)
Blake, C.L., Newman, D.J., Merz, C.J.: UCI repository of machine learning databases (1998). http://www.ics.uci.edu/~mlearn/MLRepository.html
Parra-Arnau, J., Domingo-Ferrer, J., Soria-Comas, J.: Differentially private data publishing via cross-moment microaggregation. Inf. Fusion 53, 269–288 (2020). ISSN: 1566-2535
Pearson, K.: On lines and planes of closest fit to systems of points in space. Philos. Mag. 2(11), 559–572 (1901)
Rahman, M.A., et al.: Membership inference attack against differentially private deep learning model. Trans. Data Priv. 11, 61–79 (2018)
Sablayrolles, A.: White-box vs black-box: bayes optimal strategies for membership inference. In: ICML (2019)
Salem, A., et al.: ML-Leaks: model and data independent membership inference attacks and defenses on machine learning models. CoRR, abs/1806.01246 (2018). http://arxiv.org/abs/1806.01246
Shokri, R., Stronati, M., Shmatikov, V.: Membership inference attacks against machine learning models. CoRR, abs/1610.05820 (2016). http://arxiv.org/abs/1610.05820
Song, L., Shokri, R., Mittal, P.: Membership inference attacks against adversarially robust deep learning models. In: 2019 IEEE Security and Privacy Workshops (SPW), pp. 50–56 (2019)
Tramèr, F., et al.: Truth serum: poisoning machine learning models to reveal their secrets. ArXiv, abs/2204.00032 (2022)
Truex, S., et al.: Demystifying membership inference attacks in machine learning as a service. IEEE Trans. Serv. Comput. 14, 2073–2089 (2021)
Truex, S., et al.: Effects of differential privacy and data skewness on membership inference vulnerability. In: 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), pp. 82–91 (2019)
Yeom, S., et al.: Privacy risk in machine learning: analyzing the connection to overfitting. In: 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pp. 268–282 (2018)
Ying, Z., Zhang, Y., Liu, X.: Privacy-preserving in defending against membership inference attacks. In: Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice (2020)
Zari, O., et al.: Membership inference attack against principal component analysis (2022). https://www.eurecom.fr/index.php/en/publication/6913
Acknowledgment
This work has been supported by the MESRI-BMBF French-German joint project named PROPOLIS (ANR-20-CYAL-0004-01), the 3IA Côte d’Azur program (ANR19-P3IA-0002). J. Parra-Arnau is an Alexander von Humboldt postdoctoral fellow. The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434) and from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 847648. The fellowship code is LCF/BQ/PR20/11770009. This work was also supported by the Spanish Government under research project “Enhancing Communication Protocols with Machine Learning while Protecting Sensitive Data (COMPROMISE)” (PID2020-113795RB-C31/AEI/10.13039/501100011033).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Zari, O., Parra-Arnau, J., Ünsal, A., Strufe, T., Önen, M. (2022). Membership Inference Attack Against Principal Component Analysis. In: Domingo-Ferrer, J., Laurent, M. (eds) Privacy in Statistical Databases. PSD 2022. Lecture Notes in Computer Science, vol 13463. Springer, Cham. https://doi.org/10.1007/978-3-031-13945-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-13945-1_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13944-4
Online ISBN: 978-3-031-13945-1
eBook Packages: Computer ScienceComputer Science (R0)