Skip to main content
SpringerLink
Log in

Cross-Domain Error Minimization for Unsupervised Domain Adaptation

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12682))

Included in the following conference series:

Abstract

Unsupervised domain adaptation aims to transfer knowledge from a labeled source domain to an unlabeled target domain. Previous methods focus on learning domain-invariant features to decrease the discrepancy between the feature distributions as well as minimizing the source error and have made remarkable progress. However, a recently proposed theory reveals that such a strategy is not sufficient for a successful domain adaptation. It shows that besides a small source error, both the discrepancy between the feature distributions and the discrepancy between the labeling functions should be small across domains. The discrepancy between the labeling functions is essentially the cross-domain errors which are ignored by existing methods. To overcome this issue, in this paper, a novel method is proposed to integrate all the objectives into a unified optimization framework. Moreover, the incorrect pseudo labels widely used in previous methods can lead to error accumulation during learning. To alleviate this problem, the pseudo labels are obtained by utilizing structural information of the target domain besides source classifier and we propose a curriculum learning based strategy to select the target samples with more accurate pseudo-labels during training. Comprehensive experiments are conducted, and the results validate that our approach outperforms state-of-the-art methods.

Y. Chen and F. Cui—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://imageclef.org/2014/adaptation.

References

  1. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Vaughan, J.W.: A theory of learning from different domains. Mach. Learn. 151–175 (2009). https://doi.org/10.1007/s10994-009-5152-4

  2. Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: ICML 2009 (2009)

    Google Scholar 

  3. Cao, Y., Long, M., Wang, J.: Unsupervised domain adaptation with distribution matching machines. AAAI (2018)

    Google Scholar 

  4. Chen, C., Chen, Z., Jiang, B., Jin, X.: Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation. AAAI (2019)

    Google Scholar 

  5. Chen, C., et al.: Progressive feature alignment for unsupervised domain adaptation. In: CVPR, pp. 627–636 (2018)

    Google Scholar 

  6. Chen, Q., Du, Y., Tan, Z., Zhang, Y., Wang, C.: Unsupervised domain adaptation with joint domain-adversarial reconstruction networks. In: ECML/PKDD (2020)

    Google Scholar 

  7. Dai, W., Yang, Q., Xue, G.R., Yu, Y.: Boosting for transfer learning. In: ICML 2007 (2007)

    Google Scholar 

  8. Delany, S.J.: k-nearest neighbour classifiers (2007)

    Google Scholar 

  9. Donahue, J., et al.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: ICML (2014)

    Google Scholar 

  10. Evgeniou, T., Pontil, M.: Support vector machines: theory and applications. In: Paliouras, G., Karkaletsis, V., Spyropoulos, C.D. (eds.) ACAI 1999. LNCS (LNAI), vol. 2049, pp. 249–257. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44673-7_12

    Chapter  MATH  Google Scholar 

  11. Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adaptation. In: CVPR, pp. 2066–2073 (2012)

    Google Scholar 

  12. Graf, A.B.A., Bousquet, O., Rätsch, G., Schölkopf, B.: Prototype classification: insights from machine learning. Neural Comput. 21(1), 272–300 (2009)

    Article  MathSciNet  Google Scholar 

  13. Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset (2007)

    Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

    Google Scholar 

  15. Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 498–520 (1933)

    Article  Google Scholar 

  16. Jing, M., Li, J., Zhao, J., Lu, K.: Learning distribution-matched landmarks for unsupervised domain adaptation. In: DASFAA (2018)

    Google Scholar 

  17. Liang, J., He, R., Sun, Z., Tan, T.: Aggregating randomized clustering-promoting invariant projections for domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1027–1042 (2019)

    Article  Google Scholar 

  18. Liu, H., Long, M., Wang, J., Jordan, M.I.: Transferable adversarial training: a general approach to adapting deep classifiers. In: ICML (2019)

    Google Scholar 

  19. Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. In: ICML (2015)

    Google Scholar 

  20. Long, M., Cao, Z., Wang, J., Jordan, M.I.: Conditional adversarial domain adaptation. In: NeurIPS (2018)

    Google Scholar 

  21. Long, M., Wang, J., Ding, G., Pan, S.J., Yu, P.S.: Adaptation regularization: a general framework for transfer learning. IEEE TKDE 26, 1076–1089 (2014)

    Google Scholar 

  22. Long, M., Wang, J., Ding, G., Sun, J.G., Yu, P.S.: Transfer feature learning with joint distribution adaptation. In: ICCV, pp. 2200–2207 (2013)

    Google Scholar 

  23. Long, M., Zhu, H., Wang, J., Jordan, M.I.: Unsupervised domain adaptation with residual transfer networks. In: NIPS (2016)

    Google Scholar 

  24. Maaten, L.V.D., Hinton, G.E.: Visualizing data using T-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  25. Mansour, Y., Mohri, M., Rostamizadeh, A.: Domain adaptation: learning bounds and algorithms. In: NeurIPS (2009)

    Google Scholar 

  26. Mika, S., Rätsch, G., Weston, J., Scholkopf, B., Mullers, K.R.: Fisher discriminant analysis with kernels. In: Neural Networks for Signal Processing IX, pp. 41–48 (1999)

    Google Scholar 

  27. Pan, S.J., Tsang, I.W.H., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22, 199–210 (2011)

    Article  Google Scholar 

  28. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE TKDE 22(10), 1345–1359 (2010)

    Google Scholar 

  29. Pei, Z., Cao, Z., Long, M., Wang, J.: Multi-adversarial domain adaptation. AAAI (2018)

    Google Scholar 

  30. Raghu, M., Zhang, C., Kleinberg, J.M., Bengio, S.: Transfusion: understanding transfer learning for medical imaging. In: NeurIPS (2019)

    Google Scholar 

  31. Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 213–226. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_16

    Chapter  Google Scholar 

  32. Sankaranarayanan, S., Balaji, Y., Castillo, C.D., Chellappa, R.: Generate to adapt: aligning domains using generative adversarial networks. In: CVPR, pp. 8503–8512 (2018)

    Google Scholar 

  33. Smith, N., Gales, M.J.F.: Speech recognition using SVMs. In: NIPS (2001)

    Google Scholar 

  34. Sun, B., Feng, J., Saenko, K.: Return of frustratingly easy domain adaptation. AAAI (2015)

    Google Scholar 

  35. Sun, B., Saenko, K.: Deep CORAL: correlation alignment for deep domain adaptation. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 443–450. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49409-8_35

    Chapter  Google Scholar 

  36. Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: maximizing for domain invariance. arXiv:1412.3474 (2014)

  37. Wang, J., Chen, Y., Hao, S., Feng, W., Shen, Z.: Balanced distribution adaptation for transfer learning. In: ICDM, pp. 1129–1134 (2017)

    Google Scholar 

  38. Wang, J., Feng, W., Chen, Y., Yu, H., Huang, M., Yu, P.S.: Visual domain adaptation with manifold embedded distribution alignment. In: MM 2018 (2018)

    Google Scholar 

  39. Wang, Q., Breckon, T.P.: Unsupervised domain adaptation via structured prediction based selective pseudo-labeling. AAAI (2020)

    Google Scholar 

  40. Wang, Q., Bu, P., Breckon, T.P.: Unifying unsupervised domain adaptation and zero-shot visual recognition. In: IJCNN, pp. 1–8 (2019)

    Google Scholar 

  41. Wyner, A.J., Olson, M., Bleich, J., Mease, D.: Explaining the success of AdaBoost and random forests as interpolating classifiers. J. Mach. Learn. Res. 18, 48:1–48:33 (2017)

    MathSciNet  MATH  Google Scholar 

  42. Yang, L., Liang, X., Wang, T., Xing, E.: Real-to-virtual domain unification for end-to-end autonomous driving. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 553–570. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_33

    Chapter  Google Scholar 

  43. Zhang, J., Li, W., Ogunbona, P.: Joint geometrical and statistical alignment for visual domain adaptation. In: CVPR, pp. 5150–5158 (2017)

    Google Scholar 

  44. Zhang, W., Ouyang, W., Li, W., Xu, D.: Collaborative and adversarial network for unsupervised domain adaptation. In: CVPR, pp. 3801–3809 (2018)

    Google Scholar 

  45. Zhang, Y., Tang, H., Jia, K., Tan, M.: Domain-symmetric networks for adversarial domain adaptation. In: CVPR, pp. 5026–5035 (2019)

    Google Scholar 

  46. Zhang, Y., Liu, T., Long, M., Jordan, M.I.: Bridging theory and algorithm for domain adaptation. In: ICML (2019)

    Google Scholar 

  47. Zhao, H., des Combes, R.T., Zhang, K., Gordon, G.J.: On learning invariant representation for domain adaptation. In: ICML (2019)

    Google Scholar 

Download references

Acknowledgements

This paper is supported by the National Key Research and Development Program of China (Grant No. 2018YFB1403400), the National Natural Science Foundation of China (Grant No. 61876080), the Collaborative Innovation Center of Novel Software Technology and Industrialization at Nanjing University.

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology at Nanjing University, Nanjing University, Nanjing, 210023, China

    Yuntao Du, Yinghao Chen, Fengli Cui, Xiaowen Zhang & Chongjun Wang

Authors
  1. Yuntao Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinghao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fengli Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaowen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chongjun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chongjun Wang .

Editor information

Editors and Affiliations

  1. Aalborg University, Aalborg, Denmark

    Christian S. Jensen

  2. Singapore Management University, Singapore, Singapore

    Ee-Peng Lim

  3. Academia Sinica, Taipei, Taiwan

    De-Nian Yang

  4. The Pennsylvania State University, University Park, PA, USA

    Wang-Chien Lee

  5. National Chiao Tung University, Hsinchu, Taiwan

    Vincent S. Tseng

  6. Athens University of Economics and Business, Athens, Greece

    Vana Kalogeraki

  7. National Cheng Kung University, Tainan City, Taiwan

    Jen-Wei Huang

  8. National Tsing Hua University, Hsinchu, Taiwan

    Chih-Ya Shen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Du, Y., Chen, Y., Cui, F., Zhang, X., Wang, C. (2021). Cross-Domain Error Minimization for Unsupervised Domain Adaptation. In: Jensen, C.S., et al. Database Systems for Advanced Applications. DASFAA 2021. Lecture Notes in Computer Science(), vol 12682. Springer, Cham. https://doi.org/10.1007/978-3-030-73197-7_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-73197-7_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-73196-0

  • Online ISBN: 978-3-030-73197-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation