Augmenting Image Classifiers Using Data Augmentation Generative Adversarial Networks

  • Antreas AntoniouEmail author
  • Amos StorkeyEmail author
  • Harrison EdwardsEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11141)


Effective training of neural networks requires much data. In the low-data regime, parameters are underdetermined, and learnt networks generalise poorly. Data Augmentation alleviates this by using existing data more effectively, but standard data augmentation produces only limited plausible alternative data. Given the potential to generate a much broader set of augmentations, we design and train a generative model to do data augmentation. The model, based on image conditional Generative Adversarial Networks, uses data from a source domain and learns to take a data item and augment it by generating other within-class data items. As this generative process does not depend on the classes themselves, it can be applied to novel unseen classes. We demonstrate that a Data Augmentation Generative Adversarial Network (DAGAN) augments classifiers well on Omniglot, EMNIST and VGG-Face.


Generative Adversarial Networks (GAN) Data Augmentation Unseen Classes Source Domain Conditional GANs 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported in by the EPSRC Centre for Doctoral Training in Data Science, funded by the UK Engineering and Physical Sciences Research Council and the University of Edinburgh as well as by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 732204 (Bonseyes) and by the Swiss State Secretariat for Education, Research and Innovation (SERI) under contract number 16.0159. The opinions expressed and arguments employed herein do not necessarily reflect the official views of these funding bodies.


  1. 1.
    Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv:1701.07875 (2017)
  2. 2.
    Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv:1607.06450 (2016)
  3. 3.
    Berthelot, D., Schumm, T., Metz, L.: BEGAN: boundary equilibrium generative adversarial networks. arXiv:1703.10717 (2017)
  4. 4.
    Bloice, M.D., Stocker, C., Holzinger, A.: Augmentor: an image augmentation library for machine learning. arXiv:1708.04680 (2017)
  5. 5.
    Choe, J., Park, S., Kim, K., Park, J.H., Kim, D., Shim, H.: Face generation for low-shot learning using generative adversarial networks. In: 2017 IEEE International Conference on Computer Vision Workshop (ICCVW). IEEE (2017)Google Scholar
  6. 6.
    Clark, C., Storkey, A.: Training deep convolutional networks to play Go. In: Proceedings of 32nd International Conference on Machine Learning (ICML2015) (2015). (arxiv 2014)Google Scholar
  7. 7.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: Computer Vision and Pattern Recognition. IEEE (2009)Google Scholar
  8. 8.
    Dixit, M., Kwitt, R., Niethammer, M., Vasconcelos, N.: AGA: attribute-guided augmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  9. 9.
    Fawzi, A., Samulowitz, H., Turaga, D., Frossard, P.: Adaptive data augmentation for image classification. In: 2013 International Conference on Image Processing (ICIP). IEEE (2016)Google Scholar
  10. 10.
    Foerster, J., Assael, Y.M., de Freitas, N., Whiteson, S.: Learning to communicate with deep multi-agent reinforcement learning (2016)Google Scholar
  11. 11.
    Goodfellow, I.J., et al.: Generative adversarial networks, June 2014Google Scholar
  12. 12.
    Gu, J., et al.: Recent advances in convolutional neural networks (2015)Google Scholar
  13. 13.
    Gu, S., Lillicrap, T., Sutskever, I., Levine, S.: Continuous deep q-learning with model-based acceleration. In: International Conference on Machine Learning (2016)Google Scholar
  14. 14.
    Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of wasserstein GANs. arXiv:1704.00028 (2017)
  15. 15.
    Hauberg, S., Freifeld, O., Larsen, A.B.L., Fisher, J., Hansen, L.: Dreaming more data: class-dependent distributions over diffeomorphisms for learned data augmentation. In: Artificial Intelligence and Statistics (2016)Google Scholar
  16. 16.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition, December 2015Google Scholar
  17. 17.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification (2015)Google Scholar
  18. 18.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). Scholar
  19. 19.
    Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Sig. Process. Mag. 29, 82–97 (2012)CrossRefGoogle Scholar
  20. 20.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors (2012)Google Scholar
  21. 21.
    Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. arXiv:1608.06993 (2016)
  22. 22.
    Ioffe, S.: Batch renormalization: towards reducing minibatch dependence in batch-normalized models (2017)Google Scholar
  23. 23.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015)Google Scholar
  24. 24.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks (2016)Google Scholar
  25. 25.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  26. 26.
    Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350, 1332–1338 (2015)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529 (2015)CrossRefGoogle Scholar
  28. 28.
    Nowlan, S.J., Hinton, G.E.: Simplifying neural networks by soft weight-sharing. Neural Comput. 4, 473–493 (1992). Scholar
  29. 29.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: Proceedings of ICLR 2016 (2015)Google Scholar
  30. 30.
    Ratner, A.J., Ehrenberg, H., Hussain, Z., Dunnmon, J., Ré, C.: Learning to compose domain-specific transformations for data augmentation. In: Advances in Neural Information Processing Systems, vol. 30 (2017)Google Scholar
  31. 31.
    Rosén, B.: Asymptotic theory for order sampling. J. Stat. Plan. Infer. 62, 135–158 (1997)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Salamon, J., Bello, J.P.: Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Sig. Process. Lett. 24, 279–283 (2017)CrossRefGoogle Scholar
  33. 33.
    Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Stat. Plan. Infer. 90, 227–244 (2000)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  35. 35.
    Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529, 484 (2016)CrossRefGoogle Scholar
  36. 36.
    Storkey, A.: When training and test sets are different: characterising learning transfer. In: Lawrence, C.S.S. (ed.) Dataset Shift in Machine Learning, Chap. 1. MIT Press (2009)Google Scholar
  37. 37.
    Storkey, A., Sugiyama, M.: Mixture regression for covariate shift. In: Advances in Neural Information Processing Systems (NIPS2006), vol. 19 (2007)Google Scholar
  38. 38.
    Sugiyama, M., Müller, K.R.: Input-dependent estimation of generalisation error under covariate shift. Stat. Decis. 23, 249–279 (2005)zbMATHGoogle Scholar
  39. 39.
    Takeki, A., Ikami, D., Irie, G., Aizawa, K.: Parallel grid pooling for data augmentation. arXiv:1803.11370 (2018)
  40. 40.
    Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: AAAI (2016)Google Scholar
  41. 41.
    Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems (2016)Google Scholar
  42. 42.
    Wang, Y., et al.: Tacotron: a fully end-to-end text-to-speech synthesis model. CoRR abs/1703.10135 (2017)Google Scholar
  43. 43.
    White, T.: Sampling generative networks, September 2016Google Scholar
  44. 44.
    Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv:1609.08144 (2016)
  45. 45.
    Xian, Y., Lorenz, T., Schiele, B., Akata, Z.: Feature generating networks for zero-shot learning. arXiv preprint arXiv:1712.00981 (2017)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.University of EdinburghEdinburghUK
  2. 2.Open AISan FranciscoUSA

Personalised recommendations