Skip to main content
SpringerLink
Log in

Unsupervised domain adaptation with adversarial distribution adaptation network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Adversarial domain adaptation is a powerful approach to transfer the knowledge of the label-rich source domain to the label-scarce target domain by mitigating domain shifts across distributions. Existing domain adaptation methods align either the marginal distribution with a single-domain discriminator or conditional distributions with multiple-domain discriminators. However, aligning both marginal (global) and conditional (local) distributions should be considered for domain adaptation. This paper proposes a novel adversarial distribution adaptation network (ADAN) to jointly reduce both the global and local distribution discrepancies between different domains for learning domain-invariant representations. ADAN utilizes a single-domain discriminator to adapt the global distribution between two domains, and source decision boundaries to align the local distributions between sub-domains. Furthermore, we extend our ADAN as improved ADAN (iADAN), in which we utilize a feature norm term to regularize the task-specific features to improve model generalization. Extensive experimental results show that our method outperforms other state-of-the-art domain adaptation methods on Office-Home and ImageCLEF-DA datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availibility

Datasets are open source and can be obtained through website links or references.

Notes

  1. https://www.imageclef.org/2014/adaptation.

References

  1. Ben-David S, Blitzer J, Crammer K, Pereira F (2007) Analysis of representations for domain adaptation. In: Advances in neural information processing systems. pp 137–144

  2. Dai W, Yang Q, Xue GR, Yu Y (2007) Boosting for transfer learning. In: Proceedings of the 24th international conference on machine learning. ACM, pp 193–200

  3. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: International conference on machine learning. pp 647–655

  4. Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision. pp 2960–2967

  5. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2017) Domain-adversarial training of neural networks. In: Domain adaptation in computer vision applications. Springer, pp 189–209

  6. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on computer vision and pattern recognition. pp 2066–2073

  7. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems. pp 2672–2680

  8. Gui J, Sun Z, Wen Y, Tao D, Ye J (2020) A review on generative adversarial networks: algorithms, theory, and applications. arXiv preprint arXiv:2001.06937

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770–778

  10. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp 1097–1105

  11. Lee CY, Batra T, Baig MH, Ulbricht D (2019) Sliced wasserstein discrepancy for unsupervised domain adaptation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). pp 10277–10287

  12. Liu L, Zhang H, Xu X, Zhang Z, Yan S (2019) Collocating clothes with generative adversarial networks cosupervised by categories and attributes: a multidiscriminator framework. In: IEEE transactions on neural networks and learning systems

  13. Long M, Cao Y, Wang J, Jordan MI (2015) Learning transferable features with deep adaptation networks. In: Proceedings of the 32nd international conference on international conference on machine learning, . JMLR. org. vol 37, pp 97–105

  14. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. In: Advances in neural information processing systems, pp 1640–1650

  15. Long M, Wang J, Ding G, Pan SJ, Yu PS (2014) Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng 26:1076–1089

    Article  Google Scholar 

  16. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision. pp 2200–2207

  17. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: Proceedings of the 34th international conference on machine learning. JMLR. org, vol 70, pp 2208–2217

  18. Luo P, Zhuang F, Xiong H, Xiong Y, He Q (2008) Transfer learning from multiple source domains via consensus regularization. In: Proceedings of the 17th ACM conference on information and knowledge management. ACM, pp 103–112

  19. Luo Y, Zheng L, Guan T, Yu J, Yang Y (2019) Taking a closer look at domain shift: category-level adversaries for semantics consistent domain adaptation. 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). pp 2502–2511

  20. Maaten Lvd, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  21. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  22. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359

    Article  Google Scholar 

  23. Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. In: Thirty-second AAAI conference on artificial intelligence

  24. Saito K, Watanabe K, Ushiku Y, Harada T (2018) Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3723–3732

  25. Sun B, Saenko K (2015) Subspace distribution alignment for unsupervised domain adaptation. BMVC 4:24–1

    Google Scholar 

  26. Sun B, Saenko K (2016) Deep coral: correlation alignment for deep domain adaptation. In: European conference on computer vision. Springer, Berlin, pp 443–450

  27. Torralba A, Efros A.A et al (2011) Unbiased look at dataset bias. In: CVPR, vol 1, p 7. Citeseer

  28. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7167–7176

  29. Valiant LG (1984) A theory of the learnable. In: Proceedings of the sixteenth annual ACM symposium on theory of computing. ACM, pp 436–445

  30. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5018–5027

  31. Wang X, Jin Y, Long M, Wang J, Jordan MI (2019) Transferable normalization: towards improving transferability of deep neural networks. In: Advances in neural information processing systems, pp 1953–1963

  32. Xu R, Li G, Yang J, Lin L (2019) Larger norm more transferable: an adaptive feature norm approach for unsupervised domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 1426–1435

  33. Ye S, Wu K, Zhou M, Yang Y, Tan SH, Xu K, Song J, Bao C, Ma K (2020) Light-weight calibrator: a separable component for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13736–13745

  34. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in neural information processing systems, pp 3320–3328

  35. Yu C, Wang J, Chen Y, Huang M (2019) Transfer learning with dynamic adversarial adaptation network. In: 2019 IEEE international conference on data mining (ICDM). IEEE, pp 778–786

  36. Yu X, Liu T, Gong M, Zhang K, Batmanghelich K, Tao D (2020) Label-noise robust domain adaptation. In: ICML

  37. Zhang W, Ouyang W, Li W, Xu D (2018) Collaborative and adversarial network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3801–3809

  38. Zhuang F, Cheng X, Luo P, Pan SJ, He Q (2015) Supervised representation learning: transfer learning with deep autoencoders. In: Twenty-fourth international joint conference on artificial intelligence

Download references

Funding

This work was supported by the Fundamental Research Funds for the Central Universities under Grant 2019XD-A20.

Author information

Authors and Affiliations

  1. School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Qiang Zhou, Wen’an Zhou & Shirui Wang

  2. School of Automation, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Ying Xing

Authors
  1. Qiang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen’an Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shirui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen’an Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Q., Zhou, W., Wang, S. et al. Unsupervised domain adaptation with adversarial distribution adaptation network. Neural Comput & Applic 33, 7709–7721 (2021). https://doi.org/10.1007/s00521-020-05513-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05513-2

Keywords

Search

Navigation