Anti-bandit Neural Architecture Search for Model Defense

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12358)


Deep convolutional neural networks (DCNNs) have dominated as the best performers in machine learning, but can be challenged by adversarial attacks. In this paper, we defend against adversarial attacks using neural architecture search (NAS) which is based on a comprehensive search of denoising blocks, weight-free operations, Gabor filters and convolutions. The resulting anti-bandit NAS (ABanditNAS) incorporates a new operation evaluation measure and search process based on the lower and upper confidence bounds (LCB and UCB). Unlike the conventional bandit algorithm using UCB for evaluation only, we use UCB to abandon arms for search efficiency and LCB for a fair competition between arms. Extensive experiments demonstrate that ABanditNAS is about twice as fast as the state-of-the-art NAS method, while achieving an \(8.73\%\) improvement over prior arts on CIFAR-10 under PGD-7.


Neural architecture search (NAS) Bandit Adversarial defense 



Baochang Zhang is also with Shenzhen Academy of Aerospace Technology, Shenzhen, China, and he is the corresponding author. He is in part Supported by National Natural Science Foundation of China under Grant 61672079, Shenzhen Science and Technology Program (No.KQTD2016112515134654).


  1. 1.
    Athalye, A., Carlini, N., Wagner, D.: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In: ICML (2018)Google Scholar
  2. 2.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 235–256 (2002)Google Scholar
  3. 3.
    Bender, G., Kindermans, P.J., Zoph, B., Vasudevan, V., Le, Q.V.: Understanding and simplifying one-shot architecture search. In: ICML (2018)Google Scholar
  4. 4.
    Bengio, Y., Goodfellow, I., Courville, A.: Deep learning. Citeseer (2017)Google Scholar
  5. 5.
    Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: CVPR (2005)Google Scholar
  6. 6.
    Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (2017)Google Scholar
  7. 7.
    Cubuk, E.D., Zoph, B., Schoenholz, S.S., Le, Q.V.: Intriguing properties of adversarial examples. In: ICLR (2017)Google Scholar
  8. 8.
    Das, N., Shanbhogue, M., Chen, S., Hohman, F., Chen, L., Kounavis, M.E., Chau, D.H.: Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression (2017). arXivGoogle Scholar
  9. 9.
    DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout (2017). arXivGoogle Scholar
  10. 10.
    Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., Li, J.: Boosting adversarial attacks with momentum. In: CVPR (2018)Google Scholar
  11. 11.
    D.V. Vargas, S.Kotyan: Evolving robust neural architectures to defend from adversarial attacks (2019). arXivGoogle Scholar
  12. 12.
    Dziugaite, G.K., Ghahramani, Z., Roy, D.M.: A study of the effect of jpg compression on adversarial images (2016). arXivGoogle Scholar
  13. 13.
    Even-Dar, E., Mannor, S., Mansour, Y.: Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. J. Mach. Learn. Res. 7, 1079–1105 (2006)Google Scholar
  14. 14.
    Gabor, D.: Electrical engineers part iii: Radio and communication engineering. J. Inst. Electr. Eng. - Part III: Radio Commun. Eng. 1945–1948 93, 1 (1946)Google Scholar
  15. 15.
    Gabor, D.: Theory of communication. part 1: The analysis of information. J. Inst. Electr. Eng. Part III: Radio Commun. Eng. 93, 429–441 (1946)Google Scholar
  16. 16.
    Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2014). arXivGoogle Scholar
  17. 17.
    Gupta, P., Rahtu, E.: Ciidefence: Defeating adversarial attacks by fusing class-specific image inpainting and image denoising. In: ICCV (2019)Google Scholar
  18. 18.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  19. 19.
    Ilyas, A., Engstrom, L., Madry, A.: Prior convictions: Black-box adversarial attacks with bandits and priors. In: ICLR (2018)Google Scholar
  20. 20.
    Kocsis, L., Szepesvari, C.: Bandit based monte-carlo planning. In: Proceedings of the 17th European Conference on Machine Learning (2006)Google Scholar
  21. 21.
    Kurakin, A., Goodfellow, I., Bengio, S.: Adversarial examples in the physical world. In: ICLR (2016)Google Scholar
  22. 22.
    Liao, F., Liang, M., Dong, Y., Pang, T., Hu, X., Zhu, J.: Defense against adversarial attacks using high-level representation guided denoiser. In: CVPR (2018)Google Scholar
  23. 23.
    Liu, H., Simonyan, K., Yang, Y.: Darts: Differentiable architecture search. In: ICLR (2018)Google Scholar
  24. 24.
    Liu, Y., Chen, X., Liu, C., Song, D.: Delving into transferable adversarial examples and black-box attacks. In: ICLR (2016)Google Scholar
  25. 25.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)Google Scholar
  26. 26.
    Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: ICLR (2017)Google Scholar
  27. 27.
    Guo, M., Yang, Y., Xu, R., Liu, Z.: When NAS meets robustness: In search of robust architectures against adversarial attacks. In: CVPR (2020)Google Scholar
  28. 28.
    Na, T., Ko, J.H., Mukhopadhyay, S.: Cascade adversarial machine learning regularized with a unified embedding. In: ICLR (2017)Google Scholar
  29. 29.
    Dong, N., Xu, M., Liang, X., Jiang, Y., Dai, W., Xing, E.: Neural architecture search for adversarial medical image segmentation. In: MICCAI (2019)Google Scholar
  30. 30.
    Osadchy, M., Hernandez-Castro, J., Gibson, S., Dunkelman, O., Pérez-Cabo, D.: No bot expects the deepcaptcha! introducing immutable adversarial examples, with applications to captcha generation. IEEE Trans. Inf. Forensics Secur. 12, 2640–2653 (2017)Google Scholar
  31. 31.
    Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Efficient neural architecture search via parameter sharing. In: ICML (2018)Google Scholar
  32. 32.
    Pérez, J.C., Alfarra, M., Jeanneret, G., Bibi, A., Thabet, A.K., Ghanem, B., Arbeláez, P.: Robust gabor networks (2019). arXivGoogle Scholar
  33. 33.
    Shafahi, A., Najib, M., Ghiasi, M.A., Xu, Z., Dickerson, J., Studer, C., Davis, L.S., Taylor, G., Goldstein, T.: Adversarial training for free! In: NIPS (2019)Google Scholar
  34. 34.
    Silver, D., et al.: Mastering the game of go without human knowledge. In: Nature (2017)Google Scholar
  35. 35.
    Pouya, S., Maya, K., Rama, C.: Defense-GAN: Protecting classifiers against adversarial attacks using generative models. In: ICLR (2018)Google Scholar
  36. 36.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  37. 37.
    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks. In: ICLR (2013)Google Scholar
  38. 38.
    Wang, J., Zhang, H.: Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. In: ICCV (2019)Google Scholar
  39. 39.
    Wong, E., Rice, L., Kolter, J.Z.: Fast is better than free: Revisiting adversarial training. In: ICLR (2020)Google Scholar
  40. 40.
    Xie, C., Wu, Y., Maaten, L.V.D., Yuille, A.L., He, K.: Feature denoising for improving adversarial robustness. In: CVPR (2019)Google Scholar
  41. 41.
    Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G., Tian, Q., Xiong, H.: PC-DARTS: Partial channel connections for memory-efficient differentiable architecture search. In: ICLR (2019)Google Scholar
  42. 42.
    Yang, Y., Zhang, G., Katabi, D., Xu, Z.: ME-Net: Towards effective adversarial robustness with matrix estimation. In: ICML (2019)Google Scholar
  43. 43.
    Ying, C., Klein, A., Real, E., Christiansen, E., Murphy, K., Hutter, F.: Nas-bench-101: Towards reproducible neural architecture search. In: ICML (2019)Google Scholar
  44. 44.
    Zhang, C., Liu, A., Liu, X., Xu, Y., Yu, H., Ma, Y., Li, T.: Interpreting and improving adversarial robustness with neuron sensitivity (2019). arXivGoogle Scholar
  45. 45.
    Zheng, X., Ji, R., Tang, L., Wan, Y., Zhang, B., Wu, Y., Wu, Y., Shao, L.: Dynamic distribution pruning for efficient network architecture search (2019). arXivGoogle Scholar
  46. 46.
    Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. In: ICLR (2016)Google Scholar
  47. 47.
    Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: CVPR (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Beihang UniversityBeijingChina
  2. 2.University at BuffaloBuffaloUSA
  3. 3.Xiamen UniversityFujianChina

Personalised recommendations