Advertisement

Improving Target Discriminability for Unsupervised Domain Adaptation

  • Fengmao Lv
  • Hao Chen
  • Jinzhao Wu
  • Linfeng Zhong
  • Xiaoyu Li
  • Guowu Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11305)

Abstract

In the recent years, unsupervised domain adaptation has become increasingly attractive, since it can effectively relieve the annotation burden of deep learning through transferring knowledge from a different but related source domain. Domain shift is the major problem in domain adaptation. Although the recently proposed feature alignment methods, which reduce the domain shifts through maximum mean discrepancy or adversarial training at intermediate layers of deep neural network, can obtain domain-invariant representations, these deep features are not necessarily discriminative for the target domain as no mechanism is explicitly enforced to achieve such a goal. In this paper, we propose to improve the classifier’s discriminative ability on the target domain through regularizing the entropies of the softmax predictions on the target data. We conduct our experiments on several standard adaptation benchmarks. The experiments demonstrate that our proposal can lead to significant performance improvement for unsupervised domain adaptation.

Keywords

Unsupervised domain adaptation Transfer learning Deep learning 

Notes

Acknowledgments

This paper is supported by the National Natural Science Foundation of China under grant No. 61572109, No. 11461006 and No. 61502082, and also the China Scholarship Council. Additionally, the authors would like to appreciate the anonymous reviewers for both the helpful and constructive comments.

References

  1. 1.
    Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Vaughan, J.W.: A theory of learning from different domains. Mach. Learn. 79(1), 151–175 (2010)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)CrossRefGoogle Scholar
  3. 3.
    Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation networks. In: Advances in Neural Information Processing Systems, pp. 343–351 (2016)Google Scholar
  4. 4.
    Donahue, J., et al.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655 (2014)Google Scholar
  5. 5.
    Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T.: Unsupervised visual domain adaptation using subspace alignment. In: International Conference on Computer Vision, pp. 2960–2967 (2013)Google Scholar
  6. 6.
    Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(59), 1–35 (2016)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D., Li, W.: Deep reconstruction-classification networks for unsupervised domain adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 597–613. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_36CrossRefGoogle Scholar
  8. 8.
    Glorot, X., Bordes, A., Bengio, Y.: Domain adaptation for large-scale sentiment classification: a deep learning approach. In: International Conference on Machine Learning, pp. 513–520 (2011)Google Scholar
  9. 9.
    Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2066–2073. IEEE (2012)Google Scholar
  10. 10.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  11. 11.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  12. 12.
    Krause, A., Perona, P., Gomes, R.G.: Discriminative clustering by regularized information maximization. In: Advances in Neural Information Processing Systems, pp. 775–783 (2010)Google Scholar
  13. 13.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  14. 14.
    Liu, M.Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. arXiv preprint arXiv:1703.00848 (2017)
  15. 15.
    Liu, M.Y., Tuzel, O.: Coupled generative adversarial networks. In: Advances in Neural Information Processing Systems, pp. 469–477 (2016)Google Scholar
  16. 16.
    Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791 (2015)
  17. 17.
    Long, M., Wang, J., Ding, G., Sun, J., Yu, P.S.: Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1410–1417 (2014)Google Scholar
  18. 18.
    Long, M., Zhu, H., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. arXiv preprint arXiv:1605.06636 (2016)
  19. 19.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)CrossRefGoogle Scholar
  20. 20.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  21. 21.
    Rozantsev, A., Salzmann, M., Fua, P.: Beyond sharing weights for deep domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. (2018)Google Scholar
  22. 22.
    Sener, O., Song, H.O., Saxena, A., Savarese, S.: Learning transferrable representations for unsupervised domain adaptation. In: Advances in Neural Information Processing Systems, pp. 2110–2118 (2016)Google Scholar
  23. 23.
    Springenberg, J.T.: Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv preprint arXiv:1511.06390 (2015)
  24. 24.
    Taigman, Y., Polyak, A., Wolf, L.: Unsupervised cross-domain image generation. arXiv preprint arXiv:1611.02200 (2016)
  25. 25.
    Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. arXiv preprint arXiv:1702.05464 (2017)
  26. 26.
    Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014)
  27. 27.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Fengmao Lv
    • 1
  • Hao Chen
    • 1
  • Jinzhao Wu
    • 2
  • Linfeng Zhong
    • 1
  • Xiaoyu Li
    • 3
  • Guowu Yang
    • 1
    • 2
  1. 1.Big Data Center, School of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengduPeople’s Republic of China
  2. 2.Guangxi Key Laboratory of Hybrid Computation and IC Design AnalysisGuangxi University for NationalitiesNanningPeople’s Republic of China
  3. 3.School of Information and Software EngineeringUniversity of Electronic Science and Technology of ChinaChengduPeople’s Republic of China

Personalised recommendations