Advertisement

APRICOT: A Dataset of Physical Adversarial Attacks on Object Detection

Conference paper
  • 616 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12366)

Abstract

Physical adversarial attacks threaten to fool object detection systems, but reproducible research on the real-world effectiveness of physical patches and how to defend against them requires a publicly available benchmark dataset. We present APRICOT, a collection of over 1,000 annotated photographs of printed adversarial patches in public locations. The patches target several object categories for three COCO-trained detection models, and the photos represent natural variation in position, distance, lighting conditions, and viewing angle. Our analysis suggests that maintaining adversarial robustness in uncontrolled settings is highly challenging but that it is still possible to produce targeted detections under white-box and sometimes black-box settings. We establish baselines for defending against adversarial patches via several methods, including using a detector supervised with synthetic data and using unsupervised methods such as kernel density estimation, Bayesian uncertainty, and reconstruction error. Our results suggest that adversarial patches can be effectively flagged, both in a high-knowledge, attack-specific scenario and in an unsupervised setting where patches are detected as anomalies in natural images. This dataset and the described experiments provide a benchmark for future research on the effectiveness of and defenses against physical adversarial objects in the wild. The APRICOT project page and dataset are available at apricot.mitre.org.

Keywords

Adversarial attacks Adversarial defense Datasets and evaluation Object detection 

Notes

Acknowledgments

We would like to thank Mikel Rodriguez, David Jacobs, Rama Chellappa, and Abhinav Shrivastava for helpful discussions and feedback on this work. We would also like to thank our MITRE colleagues who participated in collecting and annotating the APRICOT dataset and creating the adversarial patches.

Supplementary material

504479_1_En_3_MOESM1_ESM.pdf (49.1 mb)
Supplementary material 1 (pdf 50313 KB)

References

  1. 1.
    Athalye, A., Engstrom, L., Ilyas, A., Kwok, K.: Synthesizing robust adversarial examples, pp. 284–293 (2018)Google Scholar
  2. 2.
    Berthelot, D., Raffel, C., Roy, A., Goodfellow, I.: Understanding and improving interpolation in autoencoders via an adversarial regularizer. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=S1fQSiCcYm
  3. 3.
    Biggio, B., et al.: Evasion attacks against machine learning at test time. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013. LNCS (LNAI), vol. 8190, pp. 387–402. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40994-3_25CrossRefGoogle Scholar
  4. 4.
    Brown, T.B., Mané, D., Roy, A., Abadi, M., Gilmer, J.: Adversarial patch (2017)Google Scholar
  5. 5.
    Carlini, N., Wagner, D.: Adversarial examples are not easily detected: bypassing ten detection methods. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3–14. ACM (2017)Google Scholar
  6. 6.
    Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39–57. IEEE (2017)Google Scholar
  7. 7.
    Chen, S.T., Cornelius, C., Martin, J., Chau, D.H.: Shapeshifter: robust physical adversarial attack on faster R-CNN object detector (2018)Google Scholar
  8. 8.
    Diederik P. Kingma, J.B.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2015)Google Scholar
  9. 9.
    Dosovitskiy, A., et al.: Flownet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)Google Scholar
  10. 10.
    Eykholt, K., et al.: Robust physical-world attacks on deep learning visual classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625–1634 (2018)Google Scholar
  11. 11.
    Feinman, R., Curtin, R.R., Shintre, S., Gardner, A.B.: Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410 (2017)
  12. 12.
    Fridman, L., et al.: MIT advanced vehicle technology study: large-scale naturalistic driving study of driver behavior and interaction with automation. IEEE Access 7, 102021–102038 (2019).  https://doi.org/10.1109/access.2019.2926040. http://dx.doi.org/10.1109/ACCESS.2019.2926040CrossRefGoogle Scholar
  13. 13.
    Fu, G.S., Levin-Schwartz, Y., Lin, Q.H., Zhang, D.: Machine learning for medical imaging. J. Healthcare Eng. (2019)Google Scholar
  14. 14.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  15. 15.
    Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2014)Google Scholar
  16. 16.
    Eykholt, K., et al.: Robust physical-world attacks on deep learning visual classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625–1634 (2018)Google Scholar
  17. 17.
    Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 (2016)
  18. 18.
    Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar
  19. 19.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10602-1_48CrossRefGoogle Scholar
  20. 20.
    Liu, W., et al.: SSD: Single shot multibox detector. In: European Conference on Computer Vision (2016)Google Scholar
  21. 21.
    Lu, J., Sibai, H., Fabry, E., Forsyth, D.: No need to worry about adversarial examples in object detection in autonomous vehicles. arXiv preprint arXiv:1707.03501 (2017)
  22. 22.
    Metzen, J.H., Genewein, T., Fischer, V., Bischoff, B.: On detecting adversarial perturbations (2017)Google Scholar
  23. 23.
    Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P.: Deepfool: a simple and accurate method to fool deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2574–2582 (2016)Google Scholar
  24. 24.
    Oord, A.v.d., Kalchbrenner, N., Kavukcuoglu, K.: Pixel recurrent neural networks, pp. 1747–1756 (2016)Google Scholar
  25. 25.
    Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506–519. ACM (2017)Google Scholar
  26. 26.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-cnn: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)Google Scholar
  27. 27.
    Samangouei, P., Kabkab, M., Chellappa, R.: Defense-gan: Protecting classifiers against adversarial attacks using generative models (2018)Google Scholar
  28. 28.
    Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 146–157. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59050-9_12CrossRefGoogle Scholar
  29. 29.
    Sharif, M., Bhagavatula, S., Bauer, L., Reiter, M.K.: Accessorize to a crime: real and stealthy attacks on state-of-the-art face recognition. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1528–1540. ACM (2016)Google Scholar
  30. 30.
    Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)
  31. 31.
    Song, Y., Kim, T., Nowozin, S., Ermon, S., Kushman, N.: Pixeldefend: leveraging generative models to understand and defend against adversarial examples (2018)Google Scholar
  32. 32.
    Szegedy, C., et al.: Intriguing properties of neural networks (2014)Google Scholar
  33. 33.
    Thys, S., Van Ranst, W., Goedemé, T.: Fooling automated surveillance cameras: adversarial patches to attack person detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.The MITRE CorporationMcLeanUSA

Personalised recommendations