Skip to main content
SpringerLink
Log in

Practical Poisoning Attacks on Neural Networks

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12372))

Included in the following conference series:

Abstract

Data poisoning attacks on machine learning models have attracted much recent attention, wherein poisoning samples are injected at the training phase to achieve adversarial goals at test time. Although existing poisoning techniques prove to be effective in various scenarios, they rely on certain assumptions on the adversary knowledge and capability to ensure efficacy, which may be unrealistic in practice. This paper presents a new, practical targeted poisoning attack method on neural networks in vision domain, namely BlackCard. BlackCard possesses a set of critical properties for ensuring attacking efficacy in practice, which has never been simultaneously achieved by any existing work, including knowledge-oblivious, clean-label, and clean-test. Importantly, we show that the effectiveness of BlackCard can be intuitively guaranteed by a set of analytical reasoning and observations, through exploiting an essential characteristic of gradient-descent optimization which is pervasively adopted in DNN models. We evaluate the efficacy of BlackCard for generating targeted poisoning attacks via extensive experiments using various datasets and DNN models. Results show that BlackCard is effective with a rather high success rate while preserving all the claimed properties.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that the objective is typically to minimize \(LossFunc(\theta _T, t, L_b)\) such that it falls below \(1 \times 10^{-2}\).

  2. 2.

    If not including the third item in Eq. (1) for calculating poison data x, then x may collide with b under the pre-trained model P (thus A) in feature space. In this case, the fact that A classifies x as b with high confidence may be due to the collision portion in feature space between x and b (i.e., partly due to b’s features), but not solely due to features of t contained in x. Thus, when injecting poison data x at the training phase of target model T, T would learn that x shall be classified as b because of x’s mixed sets of features belonging to both t and b. This would cause ineffectiveness at testing. When target model T classifies input t at testing, it would yield a lower confidence of classifying t as b because according to T, the input t may not collide with b in feature space at all. T may still classify t as b because t’s features are included in its training data x, but with a lower confidence.

  3. 3.

    Note that we choose to evaluate the state-of-the-art, widely adopted models as the target model T for different tasks, and T’s parameter and structure information are unknown to us in all the experiments. For certain tasks, although there may exist other widely recognized models (e.g., the model released on Google Cloud [1] for the MNIST task), we could not use such models for our problem setting, because such models’ APIs are not accessible, thus preventing us to poison the model. For the pre-trained model P, we adopt the ones found either in online repository or our self-built ones. Notably, we intentionally choose the pairs of model T and P which exhibit completely different structure and parameter settings while achieving state-of-art performance.

References

  1. https://cloud.google.com/

  2. https://aws.amazon.com/rds/oracle/

  3. https://github.com/khanhnamle1994/fashion-mnist

  4. https://github.com/Chinmayrane16/Fashion-MNIST-Accuracy-93.4-

  5. https://www.kaggle.com/imrandude/fashion-mnist-cnn-imagedatagenerator

  6. https://www.gradientzoo.com/patrickz3li

  7. https://gluon-cv.mxnet.io/model_zoo/classification.html#cifar10

  8. https://www.kaggle.com/jahongir7174/vgg16-cifar10

  9. https://github.com/apsdehal/traffic-signs-recognition

  10. https://github.com/magnusja/GTSRB-caffe-model

  11. https://github.com/alessiamarcolini/deepstreet

  12. https://gist.github.com/EncodeTS/6bbe8cb8bebad7a672f0d872561782d9

  13. https://github.com/yzhang559/vgg-face

  14. http://www.robots.ox.ac.uk/~albanie/pytorch-models.html

  15. https://github.com/PythonOrR/CASIA-V5

  16. https://www.gradientzoo.com/

  17. Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 265–283 (2016)

    Google Scholar 

  18. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39–57. IEEE (2017)

    Google Scholar 

  19. Chen, C., Seff, A., Kornhauser, A., Xiao, J.: Deepdriving: learning affordance for direct perception in autonomous driving. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2722–2730 (2015)

    Google Scholar 

  20. Chen, X., Liu, C., Li, B., Lu, K., Song, D.: Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017)

  21. Chollet, F., et al.: Keras (2015). https://github.com/fchollet/keras

  22. Diakonikolas, I., Kane, D.M., Stewart, A.: Efficient robust proper learning of log-concave distributions. arXiv preprint arXiv:1606.03077 (2016)

  23. Du, S.S., Lee, J.D., Li, H., Wang, L., Zhai, X.: Gradient descent finds global minima of deep neural networks. arXiv preprint arXiv:1811.03804 (2018)

  24. Emeršič, Ž., Štepec, D., Štruc, V., Peer, P.: Training convolutional neural networks with limited training data for ear recognition in the wild. arXiv preprint arXiv:1711.09952 (2017)

  25. Gu, T., Dolan-Gavitt, B., Garg, S.: Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733 (2017)

  26. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  27. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

    Google Scholar 

  28. Koh, P.W., Liang, P.: Understanding black-box predictions via influence functions. arXiv preprint arXiv:1703.04730 (2017)

  29. Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D.: Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Networks 8(1), 98–113 (1997)

    Article  Google Scholar 

  30. Liu, Y., et al.: Trojaning attack on neural networks (2017)

    Google Scholar 

  31. Mahloujifar, S., Diochnos, D.I., Mahmoody, M.: The curse of concentration in robust learning: Evasion and poisoning attacks from concentration of measure. arXiv preprint arXiv:1809.03063 (2018)

  32. Muñoz-González, L., et al.: Towards poisoning of deep learning algorithms with back-gradient optimization. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security - AISec 2017 (2017). https://doi.org/10.1145/3128572.3140451, http://dx.doi.org/10.1145/3128572.3140451

  33. Rajaraman, S., et al.: Pre-trained convolutional neural networks as feature extractors toward improved malaria parasite detection in thin blood smear images. PeerJ 6, e4568 (2018)

    Article  Google Scholar 

  34. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  35. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525 (2017)

    Google Scholar 

  36. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  37. Shafahi, A., et al.: Poison frogs! targeted clean-label poisoning attacks on neural networks. arXiv preprint arXiv:1804.00792 (2018)

  38. Sharif Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)

    Google Scholar 

  39. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  40. Steinhardt, J., Koh, P.W.W., Liang, P.S.: Certified defenses for data poisoning attacks. In: Advances in Neural Information Processing Systems, pp. 3517–3529 (2017)

    Google Scholar 

  41. Szegedy, C., et al.: Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)

  42. Yang, C., Wu, Q., Li, H., Chen, Y.: Generative poisoning attack method against neural networks. arXiv preprint arXiv:1703.01340 (2017)

Download references

Acknowledgement

This work was supported by NSF grants CNS 1527727 and CNS CAREER 1750263.

Author information

Authors and Affiliations

  1. The University of Texas at Dallas, Richardson, USA

    Junfeng Guo & Cong Liu

Authors
  1. Junfeng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Liu .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 212 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Guo, J., Liu, C. (2020). Practical Poisoning Attacks on Neural Networks. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12372. Springer, Cham. https://doi.org/10.1007/978-3-030-58583-9_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58583-9_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58582-2

  • Online ISBN: 978-3-030-58583-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation