Skip to main content
SpringerLink
Log in

DecoupleNet: Decoupled Network for Domain Adaptive Semantic Segmentation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13693))

Included in the following conference series:

Abstract

Unsupervised domain adaptation in semantic segmentation alleviates the reliance on expensive pixel-wise annotation. It uses a labeled source domain dataset as well as unlabeled target domain images to learn a segmentation network. In this paper, we observe two main issues of existing domain-invariant learning framework. (1) Being distracted by the feature distribution alignment, the network cannot focus on the segmentation task. (2) Fitting source domain data well would compromise the target domain performance. To address these issues, we propose DecoupleNet to alleviate source domain overfitting and let the final model focus more on the segmentation task. Also, we put forward Self-Discrimination (SD) and introduce an auxiliary classifier to learn more discriminative target domain features with pseudo labels. Finally, we propose Online Enhanced Self-Training (OEST) to contextually enhance the quality of pseudo labels in an online manner. Experiments show our method outperforms existing state-of-the-art methods. Extensive ablation studies verify the effectiveness of each component. Code is available at https://github.com/dvlab-research/DecoupleNet.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Nie, Z., Lin, Y., Yan, M., Cao, Y., Ning, S.: An adversarial training method for improving model robustness in unsupervised domain adaptation. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, S.-Y. (eds.) KSEM 2021. LNCS (LNAI), vol. 12817, pp. 3–13. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-82153-1_1

    Chapter  Google Scholar 

  2. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. Trans. Pattern Aanal. Mach. Intell. (2017)

    Google Scholar 

  3. Chen, C., et al.: Progressive feature alignment for unsupervised domain adaptation. In: CVPR (2019)

    Google Scholar 

  4. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.: LDeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. Trans. Pattern Anal. Mach. Intell. (2017)

    Google Scholar 

  5. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.: LDeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. Trans. Pattern Anal. Mach. Intell. (2018)

    Google Scholar 

  6. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. In: ICML(2020)

    Google Scholar 

  7. Chen, Y.H., Chen, W.Y., Chen, Y.T., Tsai, B.C., Frank Wang, Y.C., Sun, M.: No more discrimination: Cross city adaptation of road scene segmenters. In: ICCV (2017)

    Google Scholar 

  8. Cheng, Y., Wei, F., Bao, J., Chen, D., Wen, F., Zhang, W.: Dual path learning for domain adaptation of semantic segmentation. In: ICCV (2021)

    Google Scholar 

  9. Cicek, S., Soatto, S.: Unsupervised domain adaptation via regularized conditional alignment. In: ICCV (2019)

    Google Scholar 

  10. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)

    Google Scholar 

  11. Cui, S., Wang, S., Zhuo, J., Su, C., Huang, Q., Tian, Q.: Gradually vanishing bridge for adversarial domain adaptation. In: CVPR (2020)

    Google Scholar 

  12. Deng, Z., Luo, Y., Zhu, J.: Cluster alignment with a teacher for unsupervised domain adaptation. In: ICCV (2019)

    Google Scholar 

  13. Donahue, J., et al.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: ICML (2014)

    Google Scholar 

  14. Du, L., et al.: SSF-DAN: separated semantic feature based domain adaptation network for semantic segmentation. In: ICCV (2019)

    Google Scholar 

  15. Fu, J., et al.: Dual attention network for scene segmentation. In: CVPR (2019)

    Google Scholar 

  16. Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: ICML (2015)

    Google Scholar 

  17. Gong, R., Li, W., Chen, Y., Gool, L.V.: Dlow: domain flow for adaptation and generalization. In: CVPR (2019)

    Google Scholar 

  18. Goodfellow, I.J., et al.: Generative adversarial networks. arXiv:1406.2661 (2014)

  19. Gopalan, R., Li, R., Chellappa, R.: Domain adaptation for object recognition: an unsupervised approach. In: ICCV (2011)

    Google Scholar 

  20. Guizilini, V., Li, J., Ambrus, R., Gaidon, A.: Geometric unsupervised domain adaptation for semantic segmentation. In: ICCV (2021)

    Google Scholar 

  21. Guo, X., Yang, C., Li, B., Yuan, Y.: MetaCorrection: domain-aware meta loss correction for unsupervised domain adaptation in semantic segmentation. In: CVPR (2021)

    Google Scholar 

  22. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

    Google Scholar 

  23. Hoffman, J., et al.: CYCADA: cycle-consistent adversarial domain adaptation. In: ICML (2018)

    Google Scholar 

  24. Hong, W., Wang, Z., Yang, M., Yuan, J.: Conditional generative adversarial network for structured domain adaptation. In: CVPR (2018)

    Google Scholar 

  25. Huang, J., Lu, S., Guan, D., Zhang, X.: Contextual-relation consistent domain adaptation for semantic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12360, pp. 705–722. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58555-6_42

    Chapter  Google Scholar 

  26. Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: CCNet: crisis-cross attention for semantic segmentation. In: ICCV (2019)

    Google Scholar 

  27. Kang, G., Wei, Y., Yang, Y., Zhuang, Y., Hauptmann, A.: Pixel-level cycle association: A new perspective for domain adaptive semantic segmentation. In: NeurIPS (2020)

    Google Scholar 

  28. Kang, G., Zheng, L., Yan, Y., Yang, Y.: Deep adversarial attention alignment for unsupervised domain adaptation: the benefit of target expectation maximization. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 420–436. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01252-6_25

    Chapter  Google Scholar 

  29. Kim, M., Byun, H.: Learning texture invariant representation for domain adaptation of semantic segmentation. In: CVPR (2020)

    Google Scholar 

  30. Kurmi, V.K., Kumar, S., Namboodiri, V.P.: Attending to discriminative certainty for domain adaptation. In: CVPR (2019)

    Google Scholar 

  31. Lai, X., et al.: Semi-supervised semantic segmentation with directional context-aware consistency. In: CVPR (2021)

    Google Scholar 

  32. Lee, S., Kim, D., Kim, N., Jeong, S.G.: Drop to adapt: Learning discriminative features for unsupervised domain adaptation. In: ICCV (2019)

    Google Scholar 

  33. Li, S., et al.: Semantic concentration for domain adaptation. In: ICCV (2021)

    Google Scholar 

  34. Li, Y., Wang, N., Shi, J., Liu, J., Hou, X.: Revisiting batch normalization for practical domain adaptation. arXiv:1603.04779 (2016)

  35. Li, Y., Yuan, L., Vasconcelos, N.: Bidirectional learning for domain adaptation of semantic segmentation. In: CVPR (2019)

    Google Scholar 

  36. Liu, H., Long, M., Wang, J., Jordan, M.: Transferable adversarial training: a general approach to adapting deep classifiers. In: ICML (2019)

    Google Scholar 

  37. Liu, M.Y., Tuzel, O.: Coupled generative adversarial networks. arXiv:1606.07536 (2016)

  38. Liu, W., Rabinovich, A., Berg, A.C.: ParseNet: looking wider to see better. arXiv (2015)

    Google Scholar 

  39. Liu, W., Ferstl, D., Schulter, S., Zebedin, L., Fua, P., Leistner, C.: Domain adaptation for semantic segmentation via patch-wise contrastive learning. arXiv:2104.11056 (2021)

  40. Liu, X., et al.: Adversarial unsupervised domain adaptation with conditional and label shift: Infer, align and iterate. In: ICCV (2021)

    Google Scholar 

  41. Liu, X., Li, S., Ge, Y., Ye, P., You, J., Lu, J.: Recursively conditional gaussian for ordinal unsupervised domain adaptation. In: ICCV (2021)

    Google Scholar 

  42. Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable features with deep adaptation networks. In: ICML (2015)

    Google Scholar 

  43. Long, M., Zhu, H., Wang, J., Jordan, M.I.: Unsupervised domain adaptation with residual transfer networks. arXiv:1602.04433 (2016)

  44. Long, M., Zhu, H., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. In: ICML (2017)

    Google Scholar 

  45. Luo, Y., Liu, P., Guan, T., Yu, J., Yang, Y.: Significance-aware information bottleneck for domain adaptive semantic segmentation. In: ICCV (2019)

    Google Scholar 

  46. Luo, Y., Zheng, L., Guan, T., Yu, J., Yang, Y.: Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In: CVPR (2019)

    Google Scholar 

  47. Ma, X., Zhang, T., Xu, C.: GCAN: graph convolutional adversarial network for unsupervised domain adaptation. In: CVPR (2019)

    Google Scholar 

  48. Mancini, M., Porzi, L., Bulo, S.R., Caputo, B., Ricci, E.: Boosting domain adaptation by discovering latent domains. In: CVPR (2018)

    Google Scholar 

  49. Mao, X., Li, Q., Xie, H., Lau, R.Y.K., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: ICCV (2017)

    Google Scholar 

  50. Maria Carlucci, F., Porzi, L., Caputo, B., Ricci, E., Rota Bulo, S.: Autodial: Automatic domain alignment layers. In: ICCV (2017)

    Google Scholar 

  51. Mei, K., Zhu, C., Zou, J., Zhang, S.: Instance adaptive self-training for unsupervised domain adaptation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12371, pp. 415–430. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58574-7_25

    Chapter  Google Scholar 

  52. Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: ICCV (2015)

    Google Scholar 

  53. Pandey, P., Tyagi, A.K., Ambekar, S., Prathosh, A.P.: Unsupervised domain adaptation for semantic segmentation of NIR images through generative latent search. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12351, pp. 413–429. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58539-6_25

    Chapter  Google Scholar 

  54. Peng, X., Li, Y., Saenko, K.: Domain2Vec: domain embedding for unsupervised domain adaptation. arXiv:2007.09257 (2020)

  55. Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: ground truth from computer games. In: ECCV (2016)

    Google Scholar 

  56. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  57. Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The SYNTHIA dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: CVPR (2016)

    Google Scholar 

  58. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. Trans. Pattern Anal. Mach. Intell. (2017)

    Google Scholar 

  59. Shin, I., Woo, S., Pan, F., Kweon, I.S.: Two-phase pseudo label densification for self-training based domain adaptation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12358, pp. 532–548. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58601-0_32

    Chapter  Google Scholar 

  60. Sohn, K., Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In: NeurIPS (2020)

    Google Scholar 

  61. Sun, B., Saenko, K.: Deep CORAL: correlation alignment for deep domain adaptation. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 443–450. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49409-8_35

    Chapter  Google Scholar 

  62. Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: CVPR (2011)

    Google Scholar 

  63. Truong, T.D., Duong, C.N., Le, N., Phung, S.L., Rainwater, C., Luu, K.: BiMaL: Bijective maximum likelihood approach to domain adaptation in semantic scene segmentation. In: ICCV (2021)

    Google Scholar 

  64. Tsai, Y.H., Hung, W.C., Schulter, S., Sohn, K., Yang, M.H., Chandraker, M.: Learning to adapt structured output space for semantic segmentation. In: CVPR (2018)

    Google Scholar 

  65. Tsai, Y.H., Sohn, K., Schulter, S., Chandraker, M.: Domain adaptation for structured output via discriminative patch representations. In: ICCV (2019)

    Google Scholar 

  66. Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. In: CVPR (2017)

    Google Scholar 

  67. Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: Maximizing for domain invariance. arXiv:1412.3474 (2014)

  68. Volpi, R., Morerio, P., Savarese, S., Murino, V.: Adversarial feature augmentation for unsupervised domain adaptation. In: CVPR (2018)

    Google Scholar 

  69. Vu, T.H., Jain, H., Bucher, M., Cord, M., Pérez, P.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: CVPR (2019)

    Google Scholar 

  70. Wang, H., Shen, T., Zhang, W., Duan, L.-Y., Mei, T.: Classes matter: a fine-grained adversarial approach to cross-domain semantic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12359, pp. 642–659. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58568-6_38

    Chapter  Google Scholar 

  71. Wang, Q., Dai, D., Hoyer, L., Van Gool, L., Fink, O.: Domain adaptive semantic segmentation with self-supervised depth estimation. In: ICCV (2021)

    Google Scholar 

  72. Wei, G., Lan, C., Zeng, W., Chen, Z.: MetaAlign: coordinating domain alignment and classification for unsupervised domain adaptation. In: CVPR (2021)

    Google Scholar 

  73. Woong-Gi, C., Tackgeun, Y., Seonguk, S., Suha, K., Bohyung, H.: Domain-specific batch normalization for unsupervised domain adaptation. In: CVPR (2019)

    Google Scholar 

  74. Xie, S., Zheng, Z., Chen, L., Chen, C.: Learning semantic representations for unsupervised domain adaptation. In: ICML (2018)

    Google Scholar 

  75. Xu, M., Zhang, J., Ni, B., Li, T., Wang, C., Tian, Q., Zhang, W.: Adversarial domain adaptation with domain mixup. In: AAAI (2020)

    Google Scholar 

  76. Yang, J., et al.: An adversarial perturbation oriented domain adaptation approach for semantic segmentation. In: AAAI (2020)

    Google Scholar 

  77. Yang, J., An, W., Wang, S., Zhu, X., Yan, C., Huang, J.: Label-driven reconstruction for domain adaptation in semantic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12372, pp. 480–498. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58583-9_29

    Chapter  Google Scholar 

  78. Yang, J., et al.: Exploring robustness of unsupervised domain adaptation in semantic segmentation. In: ICCV (2021)

    Google Scholar 

  79. Yang, M., Yu, K., Zhang, C., Li, Z., Yang, K.: DenseASPP for semantic segmentation in street scenes. In: CVPR (2018)

    Google Scholar 

  80. Yang, Y., Soatto, S.: FDA: Fourier domain adaptation for semantic segmentation. In: CVPR (2020)

    Google Scholar 

  81. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: ICLR (2016)

    Google Scholar 

  82. Zhang, F., Koltun, V., Torr, P., Ranftl, R., Richter, S.R.: Unsupervised contrastive domain adaptation for semantic segmentation. arXiv:2204.08399 (2022)

  83. Zhang, P., Zhang, B., Zhang, T., Chen, D., Wang, Y., Wen, F.: Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation. In: CVPR (2021)

    Google Scholar 

  84. Zhang, Q., Zhang, J., Liu, W., Tao, D.: Category anchor-guided unsupervised domain adaptation for semantic segmentation. In: NeurIPS (2019)

    Google Scholar 

  85. Zhang, Y., Tang, H., Jia, K., Tan, M.: Domain-symmetric networks for adversarial domain adaptation. In: CVPR (2019)

    Google Scholar 

  86. Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: ICNet for real-time semantic segmentation on high-resolution images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 418–434. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_25

    Chapter  Google Scholar 

  87. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: CVPR (2017)

    Google Scholar 

  88. Zhao, H., Zhang, Y., Liu, S., Shi, J., Loy, C.C., Lin, D., Jia, J.: PSANet: point-wise spatial attention network for scene parsing. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11213, pp. 270–286. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01240-3_17

    Chapter  Google Scholar 

  89. Zhu, Z., Xu, M., Bai, S., Huang, T., Bai, X.: Asymmetric non-local neural networks for semantic segmentation. In: ICCV (2019)

    Google Scholar 

  90. Zou, Y., Yu, Z., Vijaya Kumar, B.V.K., Wang, J.: Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 297–313. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_18

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CUHK, Ma Liu Shui, Hong Kong

    Xin Lai, Zhuotao Tian, Xiaogang Xu, Liwei Wang & Jiaya Jia

  2. SmartMore, Shenzhen, China

    Shu Liu & Jiaya Jia

  3. HKUST(GZ), Guangzhou, China

    Yingcong Chen

  4. HKUST, Clear Water Bay, Hong Kong

    Yingcong Chen

  5. HKUST(GZ)-SmartMore Joint Lab, Clear Water Bay, Hong Kong

    Yingcong Chen

  6. HKU, Pok Fu Lam, Hong Kong

    Hengshuang Zhao

  7. MIT, Cambridge, USA

    Hengshuang Zhao

Authors
  1. Xin Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhuotao Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaogang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yingcong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hengshuang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Liwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jiaya Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingcong Chen .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 10528 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lai, X. et al. (2022). DecoupleNet: Decoupled Network for Domain Adaptive Semantic Segmentation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13693. Springer, Cham. https://doi.org/10.1007/978-3-031-19827-4_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19827-4_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19826-7

  • Online ISBN: 978-3-031-19827-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation