Advertisement

Generating Adversarial Examples by Adversarial Networks for Semi-supervised Learning

  • Yun MaEmail author
  • Xudong Mao
  • Yangbin Chen
  • Qing Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11881)

Abstract

Semi-Supervised Learning (SSL) has exhibited strong effectiveness in boosting the performance of classification models with the aid of a large amount of unlabeled data. Recently, regularizing the classifier with the help of adversarial examples has proven effective for semi-supervised learning. Existing methods hypothesize that the adversarial examples are based on the pixel-wise perturbation of the original samples. However, other types of adversarial examples (e.g., with spatial transformation) should also be useful for improving the robustness of the classifier. In this paper, we propose a new generalized framework based on adversarial networks, which is able to generate various types of adversarial examples. Our model consists of two modules which are trained in an adversarial process: a generator mapping the original samples to adversarial examples which can fool the classifier, and a classifier that tries to classify the original samples and the adversarial examples consistently. We evaluate our model on several datasets, and the experimental results show that our model outperforms the state-of-the-art methods for semi-supervised learning. The experiments also demonstrate that our model can generate adversarial examples with various types of perturbation such as local spatial transformation, color transformation, and pixel-wise perturbation. Moreover, our model is also applicable to supervised learning, performing as a regularization term to improve the generalization performance of the classifier.

Keywords

Semi-supervised learning Adversarial networks Adversarial examples 

References

  1. 1.
    Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv:1701.07875 (2017)
  2. 2.
    Azulay, A., Weiss, Y.: Why do deep convolutional networks generalize so poorly to small image transformations? arXiv:1805.12177 (2018)
  3. 3.
    Bachman, P., Alsharif, O., Precup, D.: Learning with pseudo-ensembles. In: Proceedings of the Neural Information Processing Systems (NeurIPS), pp. 3365–3373 (2014)Google Scholar
  4. 4.
    Baluja, S., Fischer, I.: Learning to attack: adversarial transformation networks. In: Association for the Advancement of Artificial Intelligence (AAAI) (2018)Google Scholar
  5. 5.
    Cisse, M., Bojanowski, P., Grave, E., Dauphin, Y., Usunier, N.: Parseval networks: improving robustness to adversarial examples. In: International Conference on Machine Learning (ICML) (2017)Google Scholar
  6. 6.
    Collobert, R., Sinz, F.H., Weston, J., Bottou, L.: Large scale transductive SVMs. J. Mach. Learn. Res. (JMLR) (2006)Google Scholar
  7. 7.
    Dai, Z., Yang, Z., Yang, F., Cohen, W.W., Salakhutdinov, R.: Good semi-supervised learning that requires a bad GAN. In: Advances in Neural Information Processing Systems (NeurIPS) (2017)Google Scholar
  8. 8.
    Dumoulin, V., et al.: Adversarially learned inference. In: International Conference on Learning Representations (ICLR) (2017)Google Scholar
  9. 9.
    Gilmer, J., et al.: Adversarial spheres. arXiv:1801.02774 (2018)
  10. 10.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 2672–2680 (2014)Google Scholar
  11. 11.
    Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conference on Learning Representations (ICLR) (2015)Google Scholar
  12. 12.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: International Conference on Computer Vision (ICCV) (2017)Google Scholar
  13. 13.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  14. 14.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)Google Scholar
  15. 15.
    Kingma, D.P., Rezende, D.J., Mohamed, S., Welling, M.: Semi-supervised learning with deep generative models. In: Advances in Neural Information Processing Systems (NeurIPS) (2014)Google Scholar
  16. 16.
    Kumar, A., Sattigeri, P., Fletcher, T.: Semi-supervised learning with GANs: manifold invariance with improved inference. In: Proceedings of the Neural Information Processing Systems (NeurIPS), pp. 5534–5544 (2017)Google Scholar
  17. 17.
    Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. In: International Conference on Learning Representations (ICLR) (2017)Google Scholar
  18. 18.
    Li, C., Xu, K., Zhu, J., Zhang, B.: Triple generative adversarial nets. In: Advances in Neural Information Processing Systems (NeurIPS) (2017)Google Scholar
  19. 19.
    Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: International Conference on Computer Vision (ICCV) (2017)Google Scholar
  20. 20.
    Miyato, T., Maeda, S.I., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. (2018)Google Scholar
  21. 21.
    Paszke, A., et al.: Automatic differentiation in PyTorch. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) Workshop (2017)Google Scholar
  22. 22.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434 (2015)
  23. 23.
    Rasmus, A., Valpola, H., Honkala, M., Berglund, M., Raiko, T.: Semi-supervised learning with ladder networks. In: Advances in Neural Information Processing Systems (NeurIPS) (2015)Google Scholar
  24. 24.
    Sajjadi, M., Javanmardi, M., Tasdizen, T.: Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In: Advances in Neural Information Processing Systems (NeurIPS) (2016)Google Scholar
  25. 25.
    Salimans, T., et al.: Improved techniques for training GANs. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 2226–2234 (2016)Google Scholar
  26. 26.
    Springenberg, J.T.: Unsupervised and semi-supervised learning with categorical generative adversarial networks. In: International Conference on Learning Representations (ICLR) (2016)Google Scholar
  27. 27.
    Szegedy, C., et al.: Intriguing properties of neural networks. In: International Conference on Learning Representations (ICLR) (2014)Google Scholar
  28. 28.
    Xiao, C., Li, B., Zhu, J.Y., He, W., Liu, M., Song, D.: Generating adversarial examples with adversarial networks. In: International Joint Conference on Artificial Intelligence (IJCAI) (2018)Google Scholar
  29. 29.
    Xiao, C., Zhu, J.Y., Li, B., He, W., Liu, M., Song, D.: Spatially transformed adversarial examples. In: International Conference on Learning Representations (ICLR) (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceCity University of Hong KongHong KongChina
  2. 2.Department of ComputingThe Hong Kong Polytechnic UniversityHong KongChina

Personalised recommendations