Abstract
Ideally, visual learning algorithms should be generalizable, for dealing with any unseen domain shift when deployed in a new target environment; and data-efficient, for reducing development costs by using as little labels as possible. To this end, we study semi-supervised domain generalization (SSDG), which aims to learn a domain-generalizable model using multi-source, partially-labeled training data. We design two benchmarks that cover state-of-the-art methods developed in two related fields, i.e., domain generalization (DG) and semi-supervised learning (SSL). We find that the DG methods, which by design are unable to handle unlabeled data, perform poorly with limited labels in SSDG; the SSL methods, especially FixMatch, obtain much better results but are still far away from the basic vanilla model trained using full labels. We propose StyleMatch, a simple approach that extends FixMatch with a couple of new ingredients tailored for SSDG: (1) stochastic modeling for reducing overfitting in scarce labels, and (2) multi-view consistency learning for enhancing domain generalization. Despite the concise designs, StyleMatch achieves significant improvements in SSDG. We hope our approach and the comprehensive benchmarks can pave the way for future research on generalizable and data-efficient learning systems. The source code is released at https://github.com/KaiyangZhou/ssdg-benchmark.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
All data used in this research are publicly available and can be accessed through the following link: https://github.com/KaiyangZhou/ssdg-benchmark.
Notes
It is also worth noting that in the SSL literature, models are often trained from scratch while in SSDG, we follow the common practice used in the DG literature, i.e., using the pretrain-then-finetune paradigm, allowing the model to be initialized with pretrained weights and focusing on downstream DG performance.
Domain labels are assumed to be available.
To clarify, here pseudo-labels mean all estimated labels in a minibatch before being filtered by the threshold. Therefore, the pseudo-labeling accuracy measures the classification accuracy of the current minibatch.
The results are comparable because the test data and neural network architecture are the same.
References
Abuduweili, A., Li, X., Shi, H., Xu, C. Z., & Dou, D. (2021). Adaptive consistency regularization for semi-supervised transfer learning. In CVPR
Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019). Invariant risk minimization. arXiv preprint arXiv:1907.02893
Balaji, Y., Sankaranarayanan, S., & Chellappa, R. (2018). Metareg: Towards domain generalization using meta-regularization. In NeurIPS
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., & Raffel, C. A. (2019). Mixmatch: A holistic approach to semi-supervised learning. In NeurIPS
Blanchard, G., Lee, G., & Scott, C. (2011). Generalizing from several related classification tasks to a new unlabeled sample. In NeurIPS
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight uncertainty in neural network. In ICML
Carlucci, F. M., D’Innocente, A., Bucci, S., Caputo, B., & Tommasi, T. (2019). Domain generalization by solving jigsaw puzzles. In CVPR
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) A simple framework for contrastive learning of visual representations. In ICML
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020) Randaugment: Practical data augmentation with no separate search. In CVPR-W
DeVries, T., & Taylor, G. W. (2017). Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552
Dou, Q., Castro, D. C., Kamnitsas, K., & Glocker, B. (2019). Domain generalization via model-agnostic learning of semantic features. In NeurIPS
Dubey, A., Ramanathan, V., Pentland, A., & Mahajan, D. (2021). Adaptive methods for real-world domain generalization. In CVPR
Gal, Y., & Ghahramani, Z. (2016). Bayesian convolutional neural networks with bernoulli approximate variational inference. In ICLR-W
Ghifary, M., Balduzzi, D., Kleijn, W. B., & Zhang, M. (2017). Scatter component analysis: A unified framework for domain adaptation and domain generalization. In TPAMI
Grandvalet, Y., & Bengio, Y. (2004). Semi-supervised learning by entropy minimization. In NeurIPS
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In CVPR
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In CVPR
Hoffman, J., Tzeng, E., Park, T., Zhu, J. Y., Isola, P., Saenko, K., Efros, A., & Darrell, T. (2018). Cycada: Cycle-consistent adversarial domain adaptation. In ICML
Huang, X., & Belongie, S. (2017). Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV
Huang, Z., Wang, H., Xing, E. P., & Huang, D. (2020). Self-challenging improves cross-domain generalization. In ECCV
Kendall, A., Badrinarayanan, V., & Cipolla, R. (2017). Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. In BMVC
Kingma, D. P., & Welling, M. (2014). Auto-encoding variational bayes. In ICLR
Lee, D. H. (2013). Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML-W
Li, D., Yang, Y., Song, Y. Z., & Hospedales, T. M. (2017). Deeper, broader and artier domain generalization. In ICCV
Li, D., Yang, Y., Song, Y. Z., & Hospedales, T. M. (2018a). Learning to generalize: Meta-learning for domain generalization. In AAAI
Li, H., Jialin Pan, S., Wang, S., & Kot, A. C. (2018b) Domain generalization with adversarial feature learning. In CVPR
Li, Y., Tiana, X., Gong, M., Liu, Y., Liu, T., Zhang, K., Tao, D. (2018c). Deep domain generalization via conditional invariant adversarial networks. In ECCV
Li, Y., Yang, Y., Zhou, W., & Hospedales, T. (2019). Feature-critic networks for heterogeneous domain generalization. In ICML
Liu, Z., Miao, Z., Pan, X., Zhan, X., Lin, D., Yu, S. X., & Gong, B. (2020). Open compound domain adaptation. In CVPR
Lu, Z., Yang, Y., Zhu, X., Liu, C., Song, Y. Z., & Xiang, T. (2020). Stochastic classifiers for unsupervised domain adaptation. In CVPR
Miyato, T., Maeda, S. I., Koyama, M., & Ishii S. (2018). Virtual adversarial training: a regularization method for supervised and semi-supervised learning. In TPAMI
Qiao, F., & Peng, X. (2021). Uncertainty-guided model generalization to unseen domains. In CVPR
Qiao, F., Zhao, L., & Peng, X. (2020). Learning to learn single domain generalization. In CVPR
Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019). Do imagenet classifiers generalize to imagenet? In ICML
Shankar, S., Piratla, V., Chakrabarti, S., Chaudhuri, S., Jyothi, P., Sarawagi, S. (2018). Generalizing across domains via cross-gradient training. In ICLR
Shu, Y., Cao, Z., Wang, C., Wang, J., & Long, M. (2021). Open domain generalization with domain-augmented meta-learning. In CVPR
Snell, J., Swersky, K., Zemel, R. (2017). Prototypical networks for few-shot learning. In NeurIPS
Sohn, K., Berthelot, D., Li, C. L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., & Raffel, C. (2020). Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. In JMLR
Tarvainen, A., & Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NeurIPS
Tzeng, E., Hoffman, J., Darrell, T., Saenko, K. (2015) Simultaneous deep transfer across domains and tasks. In ICCV
Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017). Deep hashing network for unsupervised domain adaptation. In CVPR
Volpi, R., & Murino, V. (2019). Addressing model vulnerability to distributional shifts over image transformation sets. In ICCV
Volpi, R., Namkoong, H., Sener, O., Duchi, J., Murino, V., & Savarese, S. (2018). Generalizing to unseen domains via adversarial data augmentation. In NeurIPS
Wang, Q., Li, W., & Gool, L. V. (2019). Semi-supervised learning by augmented distribution alignment. In ICCV
Wang, S., Yu, L., Li, C., Fu, C. W., & Heng, P. A. (2020). Learning from extrinsic and intrinsic supervisions for domain generalization. In ECCV
Xie, Q., Dai, Z., Hovy, E., Luong, M. T., Le, Q. V. (2020a). Unsupervised data augmentation for consistency training. In NeurIPS
Xie, Q., Luong, M. T., Hovy, E., & Le, Q. V. (2020b). Self-training with noisy student improves imagenet classification. In CVPR
Xu, Z., Liu, D., Yang, J., Raffel, C., & Niethammer, M. (2021). Robust and generalizable visual representation learning via random convolutions. In ICLR
Yu, T., Li, D., Yang, Y., Hospedales, T. M., & Xiang, T. (2019). Robust person re-identification by modelling feature uncertainty. In ICCV
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2017). Understanding deep learning requires rethinking generalization. In ICLR
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2004). Learning with local and global consistency. In NeurIPS
Zhou, K., Yang, Y., Hospedales, T., Xiang, T. (2020a). Learning to generate novel domains for domain generalization. In ECCV
Zhou, K., Yang, Y., Hospedales, T.M., & Xiang, T. (2020b). Deep domain-adversarial image generation for domain generalisation. In AAAI
Zhou, K., Yang, Y., Qiao, Y., & Xiang T. (2020c). Domain adaptive ensemble learning. arXiv preprint arXiv:2003.07325
Zhou, K., Yang, Y., Qiao, Y., & Xiang, T. (2021a). Domain generalization with mixstyle. In ICLR
Zhou, K., Yang, Y., Qiao, Y., & Xiang, T. (2021b). Mixstyle neural networks for domain generalization and adaptation. arXiv preprint arXiv:2107.02053
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022). Domain generalization: a survey. In TPAMI
Zhou, Z. H. (2012). Ensemble methods: foundations and algorithms. Chapman and Hall/CRC
Acknowledgements
This study is supported by the Ministry of Education, Singapore, under its MOE AcRF Tier 2 (MOE-T2EP20221- 0012), NTU NAP, and under the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhou, K., Loy, C.C. & Liu, Z. Semi-Supervised Domain Generalization with Stochastic StyleMatch. Int J Comput Vis 131, 2377–2387 (2023). https://doi.org/10.1007/s11263-023-01821-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-023-01821-x