Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities

  • Guo-Jun QiEmail author


In this paper, we present the Lipschitz regularization theory and algorithms for a novel Loss-Sensitive Generative Adversarial Network (LS-GAN). Specifically, it trains a loss function to distinguish between real and fake samples by designated margins, while learning a generator alternately to produce realistic samples by minimizing their losses. The LS-GAN further regularizes its loss function with a Lipschitz regularity condition on the density of real data, yielding a regularized model that can better generalize to produce new data from a reasonable number of training examples than the classic GAN. We will further present a Generalized LS-GAN (GLS-GAN) and show it contains a large family of regularized GAN models, including both LS-GAN and Wasserstein GAN, as its special cases. Compared with the other GAN models, we will conduct experiments to show both LS-GAN and GLS-GAN exhibit competitive ability in generating new images in terms of the Minimum Reconstruction Error (MRE) assessed on a separate test set. We further extend the LS-GAN to a conditional form for supervised and semi-supervised learning problems, and demonstrate its outstanding performance on image classification tasks.



  1. Arjovsky, M., & Bottou, L. (2017). Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862
  2. Arjovsky, M., Chintala, S., & Bottou, L. (2017). Wasserstein GAN. arXiv preprint arXiv:1701.07875
  3. Arora, S., Ge, R., Liang, Y., Ma, T., & Zhang, Y. (2017). Generalization and equilibrium in generative adversarial nets (GANs). arXiv preprint arXiv:1703.00573
  4. Border, K. C. (1989). Fixed point theorems with applications to economics and game theory. Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  5. Carando, D., Fraiman, R., & Groisman, P. (2009). Nonparametric likelihood based estimation for a multivariate lipschitz density. Journal of Multivariate Analysis, 100(5), 981–992.MathSciNetCrossRefGoogle Scholar
  6. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016). Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in neural information processing systems (pp. 2172–2180).Google Scholar
  7. Coates, A., & Ng, A. Y. (2011). Selecting receptive fields in deep networks. In Advances in neural information processing systems (pp. 2528–2536).Google Scholar
  8. Denton, E. L., Chintala, S., Fergus, R., et al. (2015). Deep generative image models using a laplacian pyramid of adversarial networks. In Advances in neural information processing systems (pp. 1486–1494).Google Scholar
  9. Dosovitskiy, A., Fischer, P., Springenberg, J. T., Riedmiller, M., & Brox, T. (2015). Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 1734–1747.CrossRefGoogle Scholar
  10. Dosovitskiy, A., Tobias Springenberg, J., & Brox, T. (2015). Learning to generate chairs with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1538–1546).Google Scholar
  11. Dumoulin, V., Belghazi, I., Poole, B., Lamb, A., Arjovsky, M., Mastropietro, O., & Courville, A. (2016). Adversarially learned inference. arXiv preprint arXiv:1606.00704
  12. Edraki, M., Qi, & G. J. (2018). Generalized loss-sensitive adversarial learning with manifold margins. In Proceedings of the European conference on computer vision (ECCV) (pp. 87–102).Google Scholar
  13. Gatys, L. A., Ecker, A. S., & Bethge, M. (2015). A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576
  14. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672–2680).Google Scholar
  15. Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015). Draw: A recurrent neural network for image generation. arXiv preprint arXiv:1502.04623
  16. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017). Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028
  17. Hui, K. Y. (2013). Direct modeling of complex invariances for visual object features. In International conference on machine learning (pp. 352–360).Google Scholar
  18. Im, D. J., Kim, C. D., Jiang, H., & Memisevic, R. (2016). Generating images with recurrent adversarial networks. arXiv preprint arXiv:1602.05110
  19. Kingma, D., & Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  20. Kingma, D. P., Mohamed, S., Rezende, D. J., & Welling, M. (2014). Semi-supervised learning with deep generative models. In Advances in neural information processing systems (pp. 3581–3589).Google Scholar
  21. Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114
  22. Krizhevsky, A. (2009). Learning multiple layers of features from tiny images.Google Scholar
  23. Laine, S., & Aila, T. (2016). Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242
  24. Maaløe, L., Sønderby, C. K., Sønderby, S. K., & Winther, O. (2016). Auxiliary deep generative models. arXiv preprint arXiv:1602.05473
  25. Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784
  26. Miyato, T., Maeda, S.i., Koyama, M., Nakae, K., & Ishii, S. (2015). Distributional smoothing by virtual adversarial examples. arXiv preprint arXiv:1507.00677
  27. Nagarajan, V., & Kolter, J. Z. (2017). Gradient descent gan optimization is locally stable. In Advances in neural information processing systems (pp. 5585–5595).Google Scholar
  28. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., & Ng, A. Y. (2011). Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning 2011.
  29. Nowozin, S., Cseke, B., & Tomioka, R. (2016). f-GAN: Training generative neural samplers using variational divergence minimization. arXiv preprint arXiv:1606.00709
  30. Odena, A. (2016). Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv:1606.01583
  31. Qi, G. J., Zhang, L., Hu, H., Edraki, M., Wang, J., & Hua, X. S. (2018). Global versus localized generative adversarial nets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1517–1525).Google Scholar
  32. Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434
  33. Rasmus, A., Berglund, M., Honkala, M., Valpola, H., & Raiko, T. (2015). Semi-supervised learning with ladder networks. In Advances in Neural Information Processing Systems (pp. 3546–3554).Google Scholar
  34. Sajjadi, M., Javanmardi, M., & Tasdizen, T. (2016). Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In Advances in neural information processing systems (pp. 1163–1171).Google Scholar
  35. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., & Radford, A. (2016). Chen, X.: Improved techniques for training GANs. In Advances in neural information processing systems (pp. 2226–2234).Google Scholar
  36. Springenberg, J. T. (2015). Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv preprint arXiv:1511.06390
  37. Tarvainen, A., & Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in neural information processing systems (pp. 1195–1204).Google Scholar
  38. Valpola, H. (2015). From neural pca to deep unsupervised learning. In Advances in independent component analysis and learning machines (pp. 143–171). Academic Press.Google Scholar
  39. Wolpert, D. H. (1996). The lack of a priori distinctions between learning algorithms. Neural Computation, 8(7), 1341–1390.CrossRefGoogle Scholar
  40. Yadav, A., Shah, S., Xu, Z., Jacobs, D., & Goldstein, T. (2017). Stabilizing adversarial nets with prediction methods. arXiv preprint arXiv:1705.07364
  41. Zhao, J., Mathieu, M., Goroshin, R., & Lecun, Y. (2015). Stacked what-where auto-encoders. arXiv preprint arXiv:1506.02351
  42. Zhao, J., Mathieu, M., & LeCun, Y. (2016). Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126
  43. Zhao, Y., Jin, Z., Qi, G. J., Lu, H., & Hua, X. S. (2018). An adversarial approach to hard triplet generation. In Proceedings of the European conference on computer vision (ECCV) (pp. 501–517).Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Futurewei TechnologiesBellevueUSA

Personalised recommendations