Skip to main content
SpringerLink
Log in

Defending Against Machine Learning Based Inference Attacks via Adversarial Examples: Opportunities and Challenges

  • Chapter
  • First Online:
Adaptive Autonomous Secure Cyber Systems

Abstract

As machine learning (ML) becomes more and more powerful and easily accessible, attackers increasingly leverage ML to perform automated large-scale inference attacks in various domains. In such an ML-equipped inference attack, an attacker has access to some data (called public data) of an individual, a software, or a system; and the attacker uses an ML classifier to automatically infer their private data. Inference attacks pose severe privacy and security threats to individuals and systems. Inference attacks are successful because private data are statistically correlated with public data, and ML classifiers can capture such statistical correlations. In this chapter, we discuss the opportunities and challenges of defending against ML-equipped inference attacks via adversarial examples. Our key observation is that attackers rely on ML classifiers in inference attacks. The adversarial machine learning community has demonstrated that ML classifiers have various vulnerabilities. Therefore, we can turn the vulnerabilities of ML into defenses against inference attacks. For example, ML classifiers are vulnerable to adversarial examples, which add carefully crafted noise to normal examples such that an ML classifier makes predictions for the examples as we desire. To defend against inference attacks, we can add carefully crafted noise into the public data to turn them into adversarial examples, such that attackers’ classifiers make incorrect predictions for the private data. However, existing methods to construct adversarial examples are insufficient because they did not consider the unique challenges and requirements for the crafted noise at defending against inference attacks. In this chapter, we take defending against inference attacks in online social networks as an example to illustrate the opportunities and challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Code and dataset of AttriGuard are publicly available: https://github.com/jjy1994/AttriGuard.

  2. 2.

    These attacks are also called attribute inference attacks [30]. To distinguish with attribute inference attacks in online social networks, we call them feature inference attacks.

References

  1. Jahna Otterbacher. Inferring gender of movie reviewers: exploiting writing style, content and metadata. In CIKM, 2010.

    Google Scholar 

  2. Udi Weinsberg, Smriti Bhagat, Stratis Ioannidis, and Nina Taft. Blurme: Inferring and obfuscating user gender based on ratings. In RecSys, 2012.

    Google Scholar 

  3. E. Zheleva and L. Getoor. To join or not to join: The illusion of privacy in social networks with mixed public and private user profiles. In WWW, 2009.

    Google Scholar 

  4. Abdelberi Chaabane, Gergely Acs, and Mohamed Ali Kaafar. You are what you like! information leakage through users’ interests. In NDSS, 2012.

    Google Scholar 

  5. Michal Kosinski, David Stillwell, and Thore Graepel. Private traits and attributes are predictable from digital records of human behavior. PNAS, 2013.

    Google Scholar 

  6. Neil Zhenqiang Gong, Ameet Talwalkar, Lester Mackey, Ling Huang, Eui Chul Richard Shin, Emil Stefanov, Elaine(Runting) Shi, and Dawn Song. Joint link prediction and attribute inference using a social-attribute network. ACM TIST, 5(2), 2014.

    Article  Google Scholar 

  7. Neil Zhenqiang Gong and Bin Liu. You are who you know and how you behave: Attribute inference attacks via users’ social friends and behaviors. In USENIX Security Symposium, 2016.

    Google Scholar 

  8. Jinyuan Jia, Binghui Wang, Le Zhang, and Neil Zhenqiang Gong. AttriInfer: Inferring user attributes in online social networks using markov random fields. In WWW, 2017.

    Google Scholar 

  9. Neil Zhenqiang Gong and Bin Liu. Attribute inference attacks in online social networks. ACM TOPS, 21(1), 2018.

    Google Scholar 

  10. Yang Zhang, Mathias Humbert, Tahleen Rahman, Cheng-Te Li, Jun Pang, and Michael Backes. Tagvisor: A privacy advisor for sharing hashtags. In WWW, 2018.

    Google Scholar 

  11. Arvind Narayanan, Hristo Paskov, Neil Zhenqiang Gong, John Bethencourt, Emil Stefanov, Eui Chul Richard Shin, and Dawn Song. On the feasibility of internet-scale author identification. In IEEE S&P, 2012.

    Google Scholar 

  12. Mathias Payer, Ling Huang, Neil Zhenqiang Gong, Kevin Borgolte, and Mario Frank. What you submit is who you are: A multi-modal approach for deanonymizing scientific publications. IEEE Transactions on Information Forensics and Security, 10(1), 2015.

    Article  Google Scholar 

  13. Aylin Caliskan-Islam, Richard Harang, Andrew Liu, Arvind Narayanan, Clare Voss, Fabian Yamaguchi, and Rachel Greenstadt. De-anonymizing programmers via code stylometry. In USENIX Security Symposium, 2015.

    Google Scholar 

  14. Aylin Caliskan, Fabian Yamaguchi, Edwin Tauber, Richard Harang, Konrad Rieck, Rachel Greenstadt, and Arvind Narayanan. When coding style survives compilation: De-anonymizing programmers from executable binaries. In NDSS, 2018.

    Google Scholar 

  15. Rakshith Shetty, Bernt Schiele, and Mario Fritz. A4nt: Author attribute anonymity by adversarial training of neural machine translation. In USENIX Security Symposium, 2018.

    Google Scholar 

  16. Mohammed Abuhamad, Tamer AbuHmed, Aziz Mohaisen, and DaeHun Nyang. Large-scale and language-oblivious code authorship identification. In CCS, 2018.

    Google Scholar 

  17. Dominik Herrmann, Rolf Wendolsky, and Hannes Federrath. Website fingerprinting: attacking popular privacy enhancing technologies with the multinomial naïve-bayes classifier. In ACM Workshop on Cloud Computing Security, 2009.

    Google Scholar 

  18. Andriy Panchenko, Lukas Niessen, Andreas Zinnen, and Thomas Engel. Website fingerprinting in onion routing based anonymization networks. In ACM workshop on Privacy in the Electronic Society, 2011.

    Google Scholar 

  19. Xiang Cai, Xin Cheng Zhang, Brijesh Joshi, and Rob Johnson. Touching from a distance: Website fingerprinting attacks and defenses. In CCS, 2012.

    Google Scholar 

  20. Marc Juarez, Sadia Afroz, Gunes Acar, Claudia Diaz, and Rachel Greenstadt. A critical evaluation of website fingerprinting attacks. In CCS, 2014.

    Google Scholar 

  21. Tao Wang, Xiang Cai, Rishab Nithyanand, Rob Johnson, and Ian Goldberg. Effective attacks and provable defenses for website fingerprinting. In USENIX Security Symposium, 2014.

    Google Scholar 

  22. Liran Lerman, Gianluca Bontempi, and Olivier Markowitch. Side channel attack: an approach based on machine learning. In COSADE, 2011.

    Google Scholar 

  23. Yinqian Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. Cross-vm side channels and their use to extract private keys. In CCS, 2012.

    Google Scholar 

  24. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership Inference Attacks Against Machine Learning Models. In IEEE S&P, 2017.

    Google Scholar 

  25. Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine Learning with Membership Privacy using Adversarial Regularization. In CCS, 2018.

    Google Scholar 

  26. Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. In NDSS, 2019.

    Google Scholar 

  27. Y. Michalevsky, G. Nakibly, A. Schulman, and D. Boneh. Powerspy: Location tracking using mobile device power analysis. In USENIX Security Symposium, 2015.

    Google Scholar 

  28. Sashank Narain, Triet D. Vo-Huu, Kenneth Block, and Guevara Noubir. Inferring user routes and locations using zero-permission mobile sensors. In IEEE S & P, 2016.

    Google Scholar 

  29. Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In USENIX Security Symposium, 2014.

    Google Scholar 

  30. S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In CSF, 2018.

    Google Scholar 

  31. Guixin Ye, Zhanyong Tang, Dingyi Fang, Zhanxing Zhu, Yansong Feng, Pengfei Xu, Xiaojiang Chen, and Zheng Wang. Yet another text captcha solver: A generative adversarial network based approach. In CCS, 2018.

    Google Scholar 

  32. Elie Bursztein, Romain Beauxis, Hristo Paskov, Daniele Perito, Celine Fabry, and John Mitchell. The failure of noise-based non-continuous audio captchas. In IEEE S & P, 2011.

    Google Scholar 

  33. Elie Bursztein, Matthieu Martin, and John C. Mitchell. Text-based captcha strengths and weaknesses. In CCS, 2011.

    Google Scholar 

  34. Cambridge Analytica. https://goo.gl/PqRjjX, May 2018.

  35. Reza Shokri, George Theodorakopoulos, and Carmela Troncoso. Protecting location privacy: Optimal strategy against localization attacks. In CCS, 2012.

    Google Scholar 

  36. Reza Shokri. Privacy games: Optimal user-centric data obfuscation. In PETS, 2015.

    Google Scholar 

  37. Reza Shokri, George Theodorakopoulos, and Carmela Troncoso. Privacy games along location traces: A game-theoretic framework for optimizing location privacy. ACM TOPS, 19(4), 2016.

    Google Scholar 

  38. Nadia Fawaz Flávio du Pin Calmon. Privacy against statistical inference. In Allerton, 2012.

    Google Scholar 

  39. Jinyuan Jia and Neil Zhenqiang Gong. Attriguard: A practical defense against attribute inference attacks via adversarial machine learning. In USENIX Security Symposium, 2018.

    Google Scholar 

  40. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006.

    Google Scholar 

  41. S. Warner. Randomized response: a survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60(309), 1965.

    Article  Google Scholar 

  42. J. C. Duchi, M. I. Jordan, and M. J. Wainwright. Local privacy and statistical minimax rates. In FOCS, 2013.

    Google Scholar 

  43. Aleksandra Korolova Úlfar Erlingsson, Vasyl Pihur. Rappor: Randomized aggregatable privacy-preserving ordinal response. In CCS, 2014.

    Google Scholar 

  44. R. Bassily and A. D. Smith. Local, private, efficient protocols for succinct histograms. In STOC, 2015.

    Google Scholar 

  45. Tianhao Wang, Jeremiah Blocki, Ninghui Li, and Somesh Jha. Locally differentially private protocols for frequency estimation. In USENIX Security Symposium, 2017.

    Google Scholar 

  46. Jinyuan Jia and Neil Zhenqiang Gong. Calibrate: Frequency estimation and heavy hitter identification with local differential privacy via incorporating prior knowledge. In INFOCOM, 2019.

    Google Scholar 

  47. Salman Salamatian, Amy Zhang, Flavio du Pin Calmon, Sandilya Bhamidipati, Nadia Fawaz, Branislav Kveton, Pedro Oliveira, and Nina Taft. Managing your private and public data: Bringing down inference attacks against your privacy. In IEEE Journal of Selected Topics in Signal Processing, 2015.

    Google Scholar 

  48. Marco Barreno, Blaine Nelson, Russell Sears, Anthony D Joseph, and J Doug Tygar. Can machine learning be secure? In ACM ASIACCS, 2006.

    Google Scholar 

  49. Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim ŚrndićPavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In ECML-PKDD, 2013.

    Google Scholar 

  50. Jonathon Shlens Ian J. Goodfellow and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2014.

    Google Scholar 

  51. Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In AsiaCCS, 2017.

    Google Scholar 

  52. Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In ICLR, 2017.

    Google Scholar 

  53. Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE S & P, 2017.

    Google Scholar 

  54. Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In EuroS&P, 2016.

    Google Scholar 

  55. Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and K Michael Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In CCS, 2016.

    Google Scholar 

  56. Convex Optimization. Cambridge University Press, 2004.

    Google Scholar 

  57. Neil Zhenqiang Gong, Wenchang Xu, Ling Huang, Prateek Mittal, Emil Stefanov, Vyas Sekar, and Dawn Song. Evolution of social-attribute networks: Measurements, modeling, and implications using google+. In IMC, 2012.

    Google Scholar 

  58. Chong Huang, Peter Kairouz, Xiao Chen, Lalitha Sankar, and Ram Rajagopal. Generative adversarial privacy. In Privacy in Machine Learning and Artificial Intelligence Workshop, 2018.

    Google Scholar 

  59. Terence Chen, Roksana Boreli, Mohamed-Ali Kaafar, and Arik Friedman. On the effectiveness of obfuscation techniques in online social networks. In PETS, 2014.

    Google Scholar 

  60. cvxpy. https://www.cvxpy.org/, June 2019.

  61. Mehmet Sinan Inci, Thomas Eisenbarth, and Berk Sunar. Deepcloak: Adversarial crafting as a defensive measure to cloak processes. In arxiv, 2018.

    Google Scholar 

  62. Mohsen Imani, Mohammad Saidur Rahman, Nate Mathews, and Matthew Wright. Mockingbird: Defending against deep-learning-based website fingerprinting attacks with adversarial traces. In arxiv, 2019.

    Google Scholar 

  63. Xiaozhu Meng, Barton P Miller, and Somesh Jha. Adversarial binaries for authorship identification. In arxiv, 2018.

    Google Scholar 

  64. Erwin Quiring, Alwin Maier, and Konrad Rieck. Misleading authorship attribution of source code using adversarial learning. In USENIX Security Symposium, 2019.

    Google Scholar 

  65. Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. In ICML, 2012.

    Google Scholar 

  66. Matthew Jagielski, Alina Oprea, Battista Biggio, Chang Liu, Cristina Nita-Rotaru, and Bo Li. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. In IEEE S & P, 2018.

    Google Scholar 

  67. Bo Li, Yining Wang, Aarti Singh, and Yevgeniy Vorobeychik. Data poisoning attacks on factorization-based collaborative filtering. In NIPS, 2016.

    Google Scholar 

  68. Guolei Yang, Neil Zhenqiang Gong, and Ying Cai. Fake co-visitation injection attacks to recommender systems. In NDSS, 2017.

    Google Scholar 

  69. Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C Lupu, and Fabio Roli. Towards poisoning of deep learning algorithms with back-gradient optimization. In AISec, 2017.

    Google Scholar 

  70. Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks on neural networks. In NeurIPS, 2018.

    Google Scholar 

  71. Octavian Suciu, Radu Marginean, Yigitcan Kaya, Hal Daume III, and Tudor Dumitras. When does machine learning fail? generalized transferability for evasion and poisoning attacks. In Usenix Security Symposium, 2018.

    Google Scholar 

  72. Minghong Fang, Guolei Yang, Neil Zhenqiang Gong, and Jia Liu. Poisoning attacks to graph-based recommender systems. In ACSAC, 2018.

    Google Scholar 

  73. H. Yu, M. Kaminsky, P. B. Gibbons, and A. Flaxman. SybilGuard: Defending against Sybil attacks via social networks. In SIGCOMM, 2006.

    Google Scholar 

  74. Qiang Cao, Michael Sirivianos, Xiaowei Yang, and Tiago Pregueiro. Aiding the detection of fake accounts in large scale social online services. In NSDI, 2012.

    Google Scholar 

  75. Gang Wang, Tristan Konolige, Christo Wilson, and Xiao Wang. You are how you click: Clickstream analysis for sybil detection. In Usenix Security Symposium, 2013.

    Google Scholar 

  76. Neil Zhenqiang Gong, Mario Frank, and Prateek Mittal. Sybilbelief: A semi-supervised learning approach for structure-based sybil detection. IEEE Transactions on Information Forensics and Security, 9(6):976–987, 2014.

    Article  Google Scholar 

  77. Binghui Wang, Le Zhang, and Neil Zhenqiang Gong. Sybilscar: Sybil detection in online social networks via local rule based propagation. In INFOCOM, 2017.

    Google Scholar 

  78. Binghui Wang, Neil Zhenqiang Gong, and Hao Fu. Gang: Detecting fraudulent users in online social networks via guilt-by-association on directed graphs. In ICDM, 2017.

    Google Scholar 

  79. Peng Gao, Binghui Wang, Neil Zhenqiang Gong, Sanjeev R. Kulkarni, Kurt Thomas, and Prateek Mittal. Sybilfuse: Combining local attributes with global structure to perform robust sybil detection. In CNS, 2018.

    Google Scholar 

  80. Binghui Wang, Le Zhang, and Neil Zhenqiang Gong. Sybilblind: Detecting fake users in online social networks without manual labels. In RAID, 2018.

    Google Scholar 

  81. Binghui Wang, Jinyuan Jia, and Neil Zhenqiang Gong. Graph-based security and privacy analytics via collective classification with joint weight learning and propagation. In NDSS, 2019.

    Google Scholar 

  82. Zenghua Xia, Chang Liu, Neil Zhenqiang Gong, Qi Li, Yong Cui, and Dawn Song. Characterizing and detecting malicious accounts in privacy-centric mobile social networks: A case study. In KDD, 2019.

    Google Scholar 

  83. Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischof. On detecting adversarial perturbations. In ICLR, 2017.

    Google Scholar 

  84. Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In NDSS, 2018.

    Google Scholar 

  85. Dongyu Meng and Hao Chen. Magnet: a two-pronged defense against adversarial examples. In CCS, 2017.

    Google Scholar 

  86. Warren He, Bo Li, and Dawn Song. Decision boundary analysis of adversarial examples. In ICLR, 2018.

    Google Scholar 

  87. Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE S & P, 2016.

    Google Scholar 

  88. Xiaoyu Cao and Neil Zhenqiang Gong. Mitigating evasion attacks to deep neural networks via region-based classification. In ACSAC, 2017.

    Google Scholar 

  89. Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In IEEE S & P, 2019.

    Google Scholar 

  90. Jeremy M Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. In ICML, 2019.

    Google Scholar 

  91. Shiqi Wang, Yizheng Chen, Ahmed Abdou, and Suman Jana. Mixtrain: Scalable training of verifiably robust neural networks. In arxiv, 2018.

    Google Scholar 

Download references

Acknowledgement

This work was supported by NSF grant No. 1801584.

Author information

Authors and Affiliations

  1. Duke University, Durham, NC, USA

    Jinyuan Jia & Neil Zhenqiang Gong

Authors
  1. Jinyuan Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Neil Zhenqiang Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Neil Zhenqiang Gong .

Editor information

Editors and Affiliations

  1. Center for Secure Information Systems, George Mason University, Fairfax, VA, USA

    Sushil Jajodia

  2. Thayer School of Engineering, Dartmouth College, Hanover, NH, USA

    George Cybenko

  3. Department of Computer Science, Dartmouth College, Hanover, NH, USA

    V.S. Subrahmanian

  4. MS T310, MITRE Corporation, McLean, VA, USA

    Vipin Swarup

  5. Computing and Information Science Division, Army Research Office, Durham, NC, USA

    Cliff Wang

  6. Computer Science & Engineering, University of Michigan, Ann Arbor, MI, USA

    Michael Wellman

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jia, J., Gong, N.Z. (2020). Defending Against Machine Learning Based Inference Attacks via Adversarial Examples: Opportunities and Challenges. In: Jajodia, S., Cybenko, G., Subrahmanian, V., Swarup, V., Wang, C., Wellman, M. (eds) Adaptive Autonomous Secure Cyber Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-33432-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33432-1_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33431-4

  • Online ISBN: 978-3-030-33432-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation