Adversarial Learning for Zero-Shot Domain Adaptation

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12366)


Zero-shot domain adaptation (ZSDA) is a category of domain adaptation problems where neither data sample nor label is available for parameter learning in the target domain. With the hypothesis that the shift between a given pair of domains is shared across tasks, we propose a new method for ZSDA by transferring domain shift from an irrelevant task (IrT) to the task of interest (ToI). Specifically, we first identify an IrT, where dual-domain samples are available, and capture the domain shift with a coupled generative adversarial networks (CoGAN) in this task. Then, we train a CoGAN for the ToI and restrict it to carry the same domain shift as the CoGAN for IrT does. In addition, we introduce a pair of co-training classifiers to regularize the training procedure of CoGAN in the ToI. The proposed method not only derives machine learning models for the non-available target-domain data, but also synthesizes the data themselves. We evaluate the proposed method on benchmark datasets and achieve the state-of-the-art performances.


Transfer learning Domain adaptation Zero-shot learning Coupled generative adversarial networks 



The authors wish to acknowledge the financial support from: (i) Natural Science Foundation China (NSFC) under the Grant no. 61620106008; (ii) Natural Science Foundation China (NSFC) under the Grant no. 61802266.


  1. 1.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  2. 2.
    Chen, Y., Lin, Y., Yang, M., Huang, J.: Crdoco: pixel-level domain transfer with cross-domain consistency. In: CVPR, pp. 1791–1800 (2019)Google Scholar
  3. 3.
    Cohen, G., Afshar, S., Tapson, J., van Schaik, A.: EMNIST: an extension of MNIST to handwritten letters. arXiv (2017)Google Scholar
  4. 4.
    Deng, J., Dong, W., Socher, R., Li, L., Li, K., Li, F.: Imagenet: a large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009)Google Scholar
  5. 5.
    Ding, Z., Fu, Y.: Deep domain generalization with structured low-rank constraint. IEEE Trans. Image Process. 27(1), 304–313 (2018)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: ICML, vol. 37, pp. 1180–1189 (2015)Google Scholar
  7. 7.
    Ghassami, A., Kiyavash, N., Huang, B., Zhang, K.: Multi-domain causal structure learning in linear systems. In: NeurIPS, pp. 6269–6279 (2018)Google Scholar
  8. 8.
    Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D.: Domain generalization for object recognition with multi-task autoencoders. In: ICCV (2015)Google Scholar
  9. 9.
    Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)Google Scholar
  10. 10.
    Grother, P., Hanaoka, K.: NIST special database 19 handprinted forms and characters database. In: National Institute of Standards and Technology (2016)Google Scholar
  11. 11.
    Haeusser, P., Frerix, T., Mordvintsev, A., Cremers, D.: Associative domain adaptation. In: ICCV, pp. 2784–2792 (2017)Google Scholar
  12. 12.
    Han, X., Kashif, R., Roland, V.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. CoRR abs/1708.07747 (2017)Google Scholar
  13. 13.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  14. 14.
    Khosla, A., Zhou, T., Malisiewicz, T., Efros, A.A., Torralba, A.: Undoing the damage of dataset bias. In: ECCV (2012)Google Scholar
  15. 15.
    Kodirov, E., Xiang, T., Fu, Z., Gong, S.: Unsupervised domain adaptation for zero-shot learning. In: ICCV (2015)Google Scholar
  16. 16.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1106–1114 (2012)Google Scholar
  17. 17.
    Kumagai, A., Iwata, T.: Zero-shot domain adaptation without domain semantic descriptors. CoRR abs/1807.02927 (2018)Google Scholar
  18. 18.
    Li, D., Yang, Y., Song, Y.Z., Hospedales, T.M.: Deeper, broader and artier domain generalization. In: ICCV (2017)Google Scholar
  19. 19.
    Li, D., Yang, Y., Song, Y.Z., Hospedales, T.M.: Learning to generalize: Meta-learning for domain generalization. In: AAAI (2018)Google Scholar
  20. 20.
    Li, Y., et al.: Deep domain generalization via conditional invariant adversarial networks. In: ECCV (2018)Google Scholar
  21. 21.
    Liu, M.Y., Tuzel, O.: Coupled generative adversarial networks. In: NIPS (2016)Google Scholar
  22. 22.
    Long, M., Zhu, H., Wang, J., Jordan, M.I.: Unsupervised domain adaptation with residual transfer networks. In: NIPS, pp. 136–144 (2016)Google Scholar
  23. 23.
    Lopez-Paz, D., Hernández-Lobato, J., Schölkopf, B.: Semi-supervised domain adaptation with non-parametric copulas. In: NIPS (2012)Google Scholar
  24. 24.
    Luo, Y., Zheng, L., Guan, T., Yu, J., Yang, Y.: Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In: CVPR (2019)Google Scholar
  25. 25.
    Lécun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE (1998)Google Scholar
  26. 26.
    Muandet, K., Balduzzi, D., Schölkopf, B.: Domain generalization via invariant feature representation. In: ICML (2013)Google Scholar
  27. 27.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE TKDE 22(10), 1345–1359 (2010)Google Scholar
  28. 28.
    Peng, K.C., Wu, Z., Ernst, J.: Zero-shot deep domain adaptation. In: ECCV (2018)Google Scholar
  29. 29.
    Pinheiro, P.O.: Unsupervised domain adaptation with similarity learning. In: CVPR (2018)Google Scholar
  30. 30.
    Saito, K., Ushiku, Y., Harada, T.: Asymmetric tri-training for unsupervised domain adaptation. In: ICML, pp. 2988–2997 (2017)Google Scholar
  31. 31.
    Saito, K., Watanabe, K., Ushiku, Y., Harada, T.: Maximum classifier discrepancy for unsupervised domain adaptation. In: CVPR, pp. 3723–3732 (2018)Google Scholar
  32. 32.
    Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: CVPR (2011)Google Scholar
  33. 33.
    Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: maximizing for domain invariance. Computer Science (2014)Google Scholar
  34. 34.
    Venkateswara, H., Eusebio, J., Chakraborty, S., Panchanathan, S.: Deep hashing network for unsupervised domain adaptation. In: CVPR, pp. 5385–5394 (2017)Google Scholar
  35. 35.
    Wang, J., Jiang, J.: An unsupervised deep learning framework via integrated optimization of representation learning and GMM-based modeling. In: ACCV, vol. 11361, pp. 249–265 (2018)Google Scholar
  36. 36.
    Wang, J., Jiang, J.: Conditional coupled generative adversarial networks for zero-shot domain adaptation. In: ICCV (2019)Google Scholar
  37. 37.
    Wang, J., Jiang, J.: SA-net: a deep spectral analysis network for image clustering. Neurocomputing 383, 10–23 (2020)CrossRefGoogle Scholar
  38. 38.
    Wang, J., Wang, G.: Hierarchical spatial sum-product networks for action recognition in still images. IEEE Trans. Circuits Syst. Video Techn. 28(1), 90–100 (2018)CrossRefGoogle Scholar
  39. 39.
    Wang, J., Wang, Z., Tao, D., See, S., Wang, G.: Learning common and specific features for RGB-D semantic segmentation with deconvolutional networks. In: ECCV, pp. 664–679 (2016)Google Scholar
  40. 40.
    Yan, H., Ding, Y., Li, P., Wang, Q., Xu, Y., Zuo, W.: Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation. In: IEEE, pp. 945–954 (2017)Google Scholar
  41. 41.
    Yang, Y., Hospedales, T.: Zero-shot domain adaptation via kernel regression on the grassmannian (2015).
  42. 42.
    Yao, T., Pan, Y., Ngo, C.W., Li, H., Tao, M.: Semi-supervised domain adaptation with subspace learning for visual recognition. In: CVPR (2015)Google Scholar
  43. 43.
    Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Invariance matters: Exemplar memory for domain adaptive person re-identification. In: CVPR (2019)Google Scholar
  44. 44.
    Zhu, P., Wang, H., Saligrama, V.: Learning classifiers for target domain with limited or no labels. In: ICML, pp. 7643–7653 (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Research Institute for Future Media Computing, College of Computer Science and Software Engineering, and Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)Shenzhen UniversityShenzhenChina

Personalised recommendations