Estimating the Success of Unsupervised Image to Image Translation

  • Sagie Benaim
  • Tomer Galanti
  • Lior Wolf
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11209)


While in supervised learning, the validation error is an unbiased estimator of the generalization (test) error and complexity-based generalization bounds are abundant, no such bounds exist for learning a mapping in an unsupervised way. As a result, when training GANs and specifically when using GANs for learning to map between domains in a completely unsupervised way, one is forced to select the hyperparameters and the stopping epoch by subjectively examining multiple options. We propose a novel bound for predicting the success of unsupervised cross domain mapping methods, which is motivated by the recently proposed Simplicity Principle. The bound can be applied both in expectation, for comparing hyperparameters and for selecting a stopping criterion, or per sample, in order to predict the success of a specific cross-domain translation. The utility of the bound is demonstrated in an extensive set of experiments employing multiple recent algorithms. Our code is available at


Unsupervised learning Generalization bounds Image to image translation GANs 



This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant ERC CoG 725974). The contribution of Sagie Benaim is part of Ph.D. thesis research conducted at Tel Aviv University.

Supplementary material

474210_1_En_14_MOESM1_ESM.pdf (386 kb)
Supplementary material 1 (pdf 385 KB)


  1. 1.
    Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017, pp. 214–223 (2017)Google Scholar
  2. 2.
    Benaim, S., Wolf, L.: One-sided unsupervised domain mapping. In: NIPS (2017)Google Scholar
  3. 3.
    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281–305 (2012)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Bojanowski, P., Joulin, A., Lopez-Paz, D., Szlam, A.: Optimizing the latent space of generative networks. arXiv preprint arXiv:1707.05776 (2017)
  5. 5.
    Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)Google Scholar
  6. 6.
    Dosovitskiy, A., Brox, T.: Generating images with perceptual similarity metrics based on deep networks. arXiv preprint arXiv:1602.02644 (2016)
  7. 7.
    Eggensperger, K., Feurer, M., Hutter, F., Bergstra, J., Snoek, J., Hoos, H.H.: Towards an empirical foundation for assessing Bayesian optimization of hyperparameters. In: NIPS workshop on Bayesian Optimization in Theory and Practice (2013)Google Scholar
  8. 8.
    Galanti, T., Benaim, S., Wolf, L.: Generalization bounds for unsupervised cross-domain mapping with WGANs. arXiv preprint arXiv:1807.08501 (2018)
  9. 9.
    Galanti, T., Wolf, L., Benaim, S.: The role of minimal complexity functions in unsupervised learning of semantic mappings. International Conference on Learning Representations (2018)Google Scholar
  10. 10.
    Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2030–2096 (2016)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Goodfellow, I., et al.: Generative adversarial nets. In: NIPS (2014)Google Scholar
  12. 12.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). Scholar
  13. 13.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR (2017)Google Scholar
  14. 14.
    Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual Losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). Scholar
  15. 15.
    Kim, T., Cha, M., Kim, H., Lee, J., Kim, J.: Learning to discover cross-domain relations with generative adversarial networks. arXiv preprint arXiv:1703.05192 (2017)
  16. 16.
    Li, L., Jamieson, K.G., DeSalvo, G., Rostamizadeh, A., Talwalkar, A.: Efficient hyperparameter optimization and infinitely many armed bandits. arXiv preprint arXiv:1603.06560 (2016)
  17. 17.
    Liu, M.Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. In: NIPS (2017)Google Scholar
  18. 18.
    Liu, M.Y., Tuzel, O.: Coupled generative adversarial networks. In: NIPS, pp. 469–477 (2016)Google Scholar
  19. 19.
    Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
  20. 20.
    Tyleček, R., Šára, R.: Spatial pattern templates for recognition of objects with regular structure. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 364–374. Springer, Heidelberg (2013). Scholar
  21. 21.
    Rakhlin, A., Panchenko, D., Mukherjee, S.: Probability and statistics risk bounds for mixture density estimation. ESAIM: Prob. Stat. 9 (2005)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., Lee, H.: Generative adversarial text to image synthesis. In: ICML (2016)Google Scholar
  23. 23.
    Seldin, Y., Tishby, N.: PAC-Bayesian generalization bound for density estimation with application to co-clustering. In: AISTATS (2009)Google Scholar
  24. 24.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical Byesian optimization of machine learning algorithms. In: NIPS (2012)Google Scholar
  25. 25.
    Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. In: International Conference on Learning Representations (ICLR) (2017)Google Scholar
  26. 26.
    Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-weka: combined selection and hyperparameter optimization of classification algorithms. In: KDD (2013)Google Scholar
  27. 27.
    Wolf, L., Taigman, Y., Polyak, A.: Unsupervised creation of parameterized avatars. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar
  28. 28.
    Xia, Y., et al.: Dual learning for machine translation. arXiv preprint arXiv:1611.00179 (2016)
  29. 29.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: ICCV (2015)Google Scholar
  30. 30.
    Yi, Z., Zhang, H., Tan, P., Gong, M.: DualGAN: unsupervised dual learning for image-to-image translation. arXiv preprint arXiv:1704.02510 (2017)
  31. 31.
    Yu, A., Grauman, K.: Fine-grained visual comparisons with local learning. In: CVPR (2014)Google Scholar
  32. 32.
    Zhu, J.-Y., Krähenbühl, P., Shechtman, E., Efros, A.A.: Generative visual manipulation on the natural image manifold. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 597–613. Springer, Cham (2016). Scholar
  33. 33.
    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593 (2017)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.The Blavatnik School of Computer ScienceTel Aviv UniversityTel AvivIsrael
  2. 2.Facebook AI ResearchTel AvivIsrael

Personalised recommendations