Abstract
Unsupervised domain adaptation(UDA) for semantic segmentation aims to learn from labeled synthetic data to segment the unlabeled real data. Many recent methods use generative networks to acquire real-like images for mitigating domain shift. However, these methods only ensure global style consistency between two domains and fail to impose pixel-wise constraint which is also referred to as local content consistency. To address the above problem, we propose a global and local consistency network to reduce the domain gap in unsupervised domain adaptation for semantic segmentation. To this end, we first constrain global style consistency through a generative adversarial network to acquire real-like latent domain images. Then we enhance local content consistency based on pixel-wise entropy minimization. Experimental results show that our method has superiority over other competitive methods on GTA5 \(\rightarrow \) Cityscapes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Borse, S., Wang, Y., Zhang, Y., Porikli, F.: Inverseform: A loss function for structured boundary-aware segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5901–5911 (2021)
Cardace, A., Ramirez, P.Z., Salti, S., Di Stefano, L.: Shallow features guide unsupervised domain adaptation for semantic segmentation at class boundaries. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1160–1170 (2022)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Analysis Mach. Intell. 40(4), 834–848 (2017)
Chen, M., Xue, H., Cai, D.: Domain adaptation for semantic segmentation with maximum squares loss. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2090–2099 (2019)
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213–3223 (2016)
Fang, Y., Deng, W., Du, J., Hu, J.: Identity-aware cyclegan for face photo-sketch synthesis and recognition. Pattern Recogn. 102, 107249 (2020)
Gong, R., Li, W., Chen, Y., Gool, L.V.: Dlow: domain flow for adaptation and generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2477–2486 (2019)
Hoffman, J., et al.: Cycada: cycle-consistent adversarial domain adaptation. In: International Conference on Machine Learning, pp. 1989–1998. PMLR (2018)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Li, Q., Du, J., Song, F., Wang, C., Liu, H., Lu, C.: Region-based multi-focus image fusion using the local spatial frequency. In: 2013 25th Chinese Control and Decision Conference (CCDC), pp. 3792–3796. IEEE (2013)
Liu, Y., Deng, J., Gao, X., Li, W., Duan, L.: Bapa-net: boundary adaptation and prototype alignment for cross-domain semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8801–8811 (2021)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Luo, Y., Zheng, L., Guan, T., Yu, J., Yang, Y.: Taking a closer look at domain shift: category-level adversaries for semantics consistent domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2507–2516 (2019)
Piva, F.J., Dubbelman, G.: Exploiting image translations via ensemble self-supervised learning for unsupervised domain adaptation. arXiv preprint arXiv:2107.06235 (2021)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: ground truth from computer games. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 102–118. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_7
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3234–3243 (2016)
Shannon, C.E.: A mathematical theory of communication. Bell Syst. Technol. J 27(3), 379–423 (1948)
Tsai, Y.H., Hung, W.C., Schulter, S., Sohn, K., Yang, M.H., Chandraker, M.: Learning to adapt structured output space for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7472–7481 (2018)
Vu, T.H., Jain, H., Bucher, M., Cord, M., Pérez, P.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2517–2526 (2019)
Wang, C., et al.: Active boundary loss for semantic segmentation. arXiv preprint arXiv:2102.02696 (2021)
Wang, Z., Li, Y., Wang, S.: Noisy boundaries: lemon or lemonade for semi-supervised instance segmentation? arXiv preprint arXiv:2203.13427 (2022)
Xu, L., Du, J., Li, Q.: Image fusion based on nonsubsampled contourlet transform and saliency-motivated pulse coupled neural networks. Math. Prob. Eng. 2013 (2013)
Yang, Y., Lao, D., Sundaramoorthi, G., Soatto, S.: Phase consistent ecological domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9011–9020 (2020)
Yang, Y., Soatto, S.: FDA: fourier domain adaptation for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4085–4095 (2020)
Yin, Z., Liang, K., Ma, Z., Guo, J.: Duplex contextual relation network for polyp segmentation. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2022)
Zhang, P., Zhang, B., Zhang, T., Chen, D., Wang, Y., Wen, F.: Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12414–12424 (2021)
Zhang, X., Zhang, H., Lu, J., Shao, L., Yang, J.: Target-targeted domain adaptation for unsupervised semantic segmentation. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 13560–13566. IEEE (2021)
Zhu, C., Zhang, X., Li, Y., Qiu, L., Han, K., Han, X.: Sharpcontour: a contour-based boundary refinement approach for efficient and accurate instance segmentation. arXiv preprint arXiv:2203.13312 (2022)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Acknowledgements
This work was supported in part by National Natural Science Foundation of China (NSFC) No. 61922015, 62106022, U19B2036, 62225601, and in part by Beijing Natural Science Foundation Project No. Z200002.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shan, X., Yin, Z., Gao, J., Liang, K., Ma, Z., Guo, J. (2022). Unsupervised Domain Adaptation for Semantic Segmentation with Global and Local Consistency. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13604. Springer, Cham. https://doi.org/10.1007/978-3-031-20497-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-20497-5_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20496-8
Online ISBN: 978-3-031-20497-5
eBook Packages: Computer ScienceComputer Science (R0)