Advertisement

Two-Phase Pseudo Label Densification for Self-training Based Domain Adaptation

Conference paper
  • 503 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12358)

Abstract

Recently, deep self-training approaches emerged as a powerful solution to the unsupervised domain adaptation. The self-training scheme involves iterative processing of target data; it generates target pseudo labels and retrains the network. However, since only the confident predictions are taken as pseudo labels, existing self-training approaches inevitably produce sparse pseudo labels in practice. We see this is critical because the resulting insufficient training-signals lead to a sub-optimal, error-prone model. In order to tackle this problem, we propose a novel Two-phase Pseudo Label Densification framework, referred to as TPLD. In the first phase, we use sliding window voting to propagate the confident predictions, utilizing intrinsic spatial-correlations in the images. In the second phase, we perform a confidence-based easy-hard classification. For the easy samples, we now employ their full pseudo-labels. For the hard ones, we instead adopt adversarial learning to enforce hard-to-easy feature alignment. To ease the training process and avoid noisy predictions, we introduce the bootstrapping mechanism to the original self-training loss. We show the proposed TPLD can be easily integrated into existing self-training based approaches and improves the performance significantly. Combined with the recently proposed CRST self-training framework, we achieve new state-of-the-art results on two standard UDA benchmarks.

Keywords

Unsupervised domain adaptataion Self-training 

Notes

Acknowledgement

This research is supported by the National Cancer Center(NCC).

Supplementary material

504454_1_En_32_MOESM1_ESM.pdf (1.3 mb)
Supplementary material 1 (pdf 1335 KB)

References

  1. 1.
    Atapour-Abarghouei, A., Breckon, T.P.: Real-time monocular depth estimation using synthetic data with domain adaptation via image style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2800–2810 (2018)Google Scholar
  2. 2.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell., June 2016.  https://doi.org/10.1109/TPAMI.2017.2699184
  3. 3.
    Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation, June 2017Google Scholar
  4. 4.
    Chen, Y., Li, W., Sakaridis, C., Dai, D., Van Gool, L.: Domain adaptive faster R-CNN for object detection in the wild. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 3339–3348 (2018)Google Scholar
  5. 5.
    Chen, Y.C., Lin, Y.Y., Yang, M.H., Huang, J.B.: Crdoco: pixel-level domain transfer with cross-domain consistency. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), June 2019Google Scholar
  6. 6.
    Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  7. 7.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: Imagenet: a large-scale hierarchical image database, pp. 248–255, June 2009.  https://doi.org/10.1109/CVPR.2009.5206848
  8. 8.
    Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T.: Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of International Conference on Computer Vision (ICCV), pp. 2960–2967 (2013)Google Scholar
  9. 9.
    Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409.7495 (2014)
  10. 10.
    Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D., Li, W.: Deep reconstruction-classification networks for unsupervised domain adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 597–613. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_36CrossRefGoogle Scholar
  11. 11.
    Golemo, F., Taiga, A.A., Courville, A., Oudeyer, P.Y.: Sim-to-real transfer with neural-augmented robot simulation. In: Billard, A., Dragan, A., Peters, J., Morimoto, J. (eds.) Proceedings of the 2nd Conference on Robot Learning. Proceedings of Machine Learning Research, vol. 87, pp. 817–828. PMLR, 29–31 October 2018. http://proceedings.mlr.press/v87/golemo18a.html
  12. 12.
    Gong, B., Shi, Y., Sha, F., Grauman, K.: Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 2066–2073. IEEE (2012)Google Scholar
  13. 13.
    Gopalan, R., Li, R., Chellappa, R.: Domain adaptation for object recognition: an unsupervised approach. In: Proceedings of International Conference on Computer Vision (ICCV), pp. 999–1006. IEEE (2011)Google Scholar
  14. 14.
    Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization, pp. 529–536 (2005)Google Scholar
  15. 15.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, June 2016.  https://doi.org/10.1109/CVPR.2016.90
  16. 16.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  17. 17.
    Hoffman, J., et al.: CyCADA: cycle-consistent adversarial domain adaptation. In: Proceedings of International Conference on Machine Learning (ICML), pp. 1989–1998 (2018)Google Scholar
  18. 18.
    Hoffman, J., Wang, D., Yu, F., Darrell, T.: FCNS in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649 (2016)
  19. 19.
    Hong, W., Wang, Z., Yang, M., Yuan, J.: Conditional generative adversarial network for structured domain adaptation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), June 2018Google Scholar
  20. 20.
    Kulis, B., Saenko, K., Darrell, T.: What you saw is not what you get: domain adaptation using asymmetric kernel transforms. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 1785–1792. IEEE (2011)Google Scholar
  21. 21.
    Li, D., Yang, Y., Song, Y.Z., Hospedales, T.M.: Deeper, broader and artier domain generalization. In: Proceedings of International Conference on Computer Vision (ICCV), pp. 5542–5550 (2017)Google Scholar
  22. 22.
    Li, W., Xu, Z., Xu, D., Dai, D., Van Gool, L.: Domain generalization and adaptation using low rank exemplar SVMs. In: IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 40, 1114–1127. IEEE (2017)Google Scholar
  23. 23.
    Long, M., Cao, Y., Cao, Z., Wang, J., Jordan, M.I.: Transferable representation learning with deep adaptation networks. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 41, 3071–3085 (2019).  https://doi.org/10.1109/TPAMI.2018.2868685CrossRefGoogle Scholar
  24. 24.
    Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791 (2015)
  25. 25.
    Luo, Y., Zheng, L., Guan, T., Yu, J., Yang, Y.: Taking a closer look at domain shift: category-level adversaries for semantics consistent domain adaptation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  26. 26.
    van der Maaten, L., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 11, 2579–2605 (2008)zbMATHGoogle Scholar
  27. 27.
    Motiian, S., Piccirilli, M., Adjeroh, D.A., Doretto, G.: Unified deep supervised domain adaptation and generalization. In: Proceedings of International Conference on Computer Vision (ICCV), pp. 5715–5725 (2017)Google Scholar
  28. 28.
    Murez, Z., Kolouri, S., Kriegman, D., Ramamoorthi, R., Kim, K.: Image to image translation for domain adaptation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 4500–4509, June 2018.  https://doi.org/10.1109/CVPR.2018.00473
  29. 29.
    Panareda Busto, P., Gall, J.: Open set domain adaptation. In: Proceedings of International Conference on Computer Vision (ICCV), pp. 754–763 (2017)Google Scholar
  30. 30.
    Reed, S., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., Rabinovich, A.: Training deep neural networks on noisy labels with bootstrapping, December 2014Google Scholar
  31. 31.
    Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: ground truth from computer games. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 102–118. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46475-6_7CrossRefGoogle Scholar
  32. 32.
    Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 3234–3243 (2016)Google Scholar
  33. 33.
    Sener, O., Song, H.O., Saxena, A., Savarese, S.: Learning transferrable representations for unsupervised domain adaptation. pp. 2110–2118 (2016)Google Scholar
  34. 34.
    Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: Proc. of Computer Vision and Pattern Recognition (CVPR), pp. 2107–2116 (2017)Google Scholar
  35. 35.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv 1409.1556, September 2014Google Scholar
  36. 36.
    Tsai, Y.H., Hung, W.C., Schulter, S., Sohn, K., Yang, M.H., Chandraker, M.: Learning to adapt structured output space for semantic segmentation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 7472–7481 (2018)Google Scholar
  37. 37.
    Vu, T.H., Jain, H., Bucher, M., Cord, M., Pérez, P.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 2517–2526 (2019)Google Scholar
  38. 38.
    Zhu, X.: Semi-supervised learning tutorial. In: Proceedings of International Conference on Machine Learning (ICML) (2007)Google Scholar
  39. 39.
    Zou, Y., Yu, Z., Kumar, B.V., Wang, J.: Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In: Proceedings of European Conf. on Computer Vision (ECCV), pp. 289–305 (2018)Google Scholar
  40. 40.
    Zou, Y., Yu, Z., Liu, X., Kumar, B.V., Wang, J.: Confidence regularized self-training. In: Proceedings of International Conference on Computer Vision (ICCV), October 2019Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.KAISTDaejeonSouth Korea

Personalised recommendations