Conditioned Generative Model via Latent Semantic Controlling for Learning Deep Representation of Data

  • Jin-Young Kim
  • Sung-Bae ChoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)


Learning representations of data is an important issue in machine learning. Though generative adversarial network has led to significant improvements in the data representations, it still has several problems such as unstable training, hidden manifold of data, and huge computational overhead. Moreover, most of GAN’s have a large size of manifold, resulting in poor scalability. In this paper, we propose a novel GAN to control the latent semantic representation, called LSC-GAN, which allows us to produce desired data and learns a representation of the data efficiently. Unlike the conventional GAN models with hidden distribution of latent space, we define the distributions explicitly in advance that are trained to generate the data based on the corresponding features by inputting the latent variables, which follow the distribution, into the generative model. As the larger scale of latent space caused by deploying various distributions makes training unstable, we need to separate the process of defining the distributions explicitly and operation of generation. We prove that a variational auto-encoder is proper for the former and modify a loss function of VAE to map the data into the corresponding pre-defined latent space. The decoder, which generates the data from the associated latent space, is used as the generator of the LSC-GAN. Several experiments on the CelebA dataset are conducted to verify the usefulness of the proposed method. Besides, our model achieves a high compression ratio that can hold about 24 pixels of information in each dimension of latent space.


Generative model Data representation Latent space Variational autoencoder Generative adversarial nets 



This work was supported by grant funded by 2019 IT promotion fund (Development of AI based Precision Medicine Emergency System) of the Korean government (Ministry of Science and ICT). J. Y. Kim has been supported by NRF (National Research Foundation of Korea) grant funded by the Korean government (NRF-2019-Fostering Core Leaders of the Future Basic Science Program/Global Ph.D. Fellowship Program).


  1. 1.
    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
  2. 2.
    Goodfellow, I., et al.: Generative adversarial nets. In: Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  3. 3.
    Kim, J.Y., Cho, S.B.: Electric energy consumption prediction by deep learning with state explainable autoencoder. Energies 12(4), 739–754 (2019)CrossRefGoogle Scholar
  4. 4.
    Yang, Z., Hu, Z., Salakgutdinov, R., Berg-Kirkpatrick, T.: Improved variational autoencoders for text modeling using dilated convolutions. In: International Conference on Machine Learning, pp. 3881–3890 (2017)Google Scholar
  5. 5.
    Kusner, M.J., Paige, B., Hernandez-Lobato, J.M.: Grammar variational autoencoder. In: International Conference on Machine Learning, pp. 1945–1954 (2017)Google Scholar
  6. 6.
    Denton, E.L., Chintala, S., Fergus, R.: Deep generative image models using a laplacian pyramid of adversarial networks. In: Neural Information Processing Systems, pp. 1486–1494 (2015)Google Scholar
  7. 7.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
  8. 8.
    Zhao, J., Mathieu, M., LeCun, Y.: Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126 (2016)
  9. 9.
    Kim, J.-Y., Cho, S.-B.: Detecting intrusive malware with a hybrid generative deep learning model. In: Yin, H., Camacho, D., Novais, P., Tallón-Ballesteros, Antonio J. (eds.) IDEAL 2018. LNCS, vol. 11314, pp. 499–507. Springer, Cham (2018). Scholar
  10. 10.
    Kim, J.Y., Bu, S.J., Cho, S.B.: Zero-day malware detection using transferred generative adversarial networks based on deep autoencoders. Inf. Sci. 460, 83–102 (2018)CrossRefGoogle Scholar
  11. 11.
    Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: Neural Information Processing Systems, pp. 3483–3491 (2015)Google Scholar
  12. 12.
    Van den Oord, A., Vinyals, O.: Neural discrete representation learning. In: Neural Information Processing Systems, pp. 6309–6318 (2017)Google Scholar
  13. 13.
    Larsen, A.B.L., Sonderby, S.K., Larochelle, H., Winther, O.: Autoencoding beyond pixels using a learned similarity mertic. arXiv preprint arXiv:1512.09300 (2015)
  14. 14.
    Chen, X., Duan, Y., Houthooft, R., Schuman, J., Sutskever, I., Abbeel, P.: InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In: Neural Information Processing Systems, pp. 2172–2180 (2016)Google Scholar
  15. 15.
    Nguyen, A., Clune, J., Bengio, Y., Dosovitskiy, A., Yosinski, J.: Plug & play generative networks: conditional iterative generative of images in latent space. Comput. Vis. Pattern Recogn. 2(5), 3510–3520 (2017)Google Scholar
  16. 16.
    Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: International Conference on Computer Vision, pp. 2813–2821 (2017)Google Scholar
  17. 17.
    Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: International Conference on Computer Vision, pp. 2730–2738 (2015)Google Scholar
  18. 18.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  19. 19.
    Kim, T., Cha, M., Kim, H., Lee, J., Kim, J.: Learning to discover cross-domain relations with generative adversarial networks. In: International Conference on Machine Learning, pp. 1857–1865 (2017)Google Scholar
  20. 20.
    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593 (2017)
  21. 21.
    Choi, Y., Choi, M., Kim, M., Ha, J., Kim, S., Choo, J.: StarGAN: unifired generative adversarial networks for multi-domain image-to-image translation. In: Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceYonsei UniversitySeoulSouth Korea

Personalised recommendations