Iterative Discriminative Domain Adaptation

Abstract

A popular formulation of domain adaptation (DA) is to simultaneously minimize the source risk and the cross-domain discrepancy between the source domain $\mathcal {D}_s$ and target domain $\mathcal {D}_t$. However, this is believed to be suboptimal since the shared feature, which is indistinguishable by a domain classifier, could be far from optimum for the purpose of classification. In this paper, we propose an iterative DA framework for directly optimizing the classification error, which provides DA solutions to both unsupervised and semi-supervised scenarios. Instead of directly attacking $\mathcal {D}_s \rightarrow \mathcal {D}_t$, we employ an iterative self-training approach of $\mathcal {D}_s + \mathcal {D}_t^{l-1} \rightarrow \mathcal {D}_t^l$ for progressively-labelling of $\mathcal {D}_t$ with the aim of $\lim _{l\rightarrow \infty }\mathcal {D}_t^l \approx \mathcal {D}_t$. For unsupervised DA, it performs comparable to the state-of-the-art DA methods. In particular, it performs the best among various unsupervised DA methods for the very difficult task MNIST $\rightarrow$ SVHN. By employing a few labeled samples in the target domain, we show that it can achieve significantly improved performance. For MNIST $\rightarrow$ SVHN, the use of 60 labeled samples from SVHN is able to improve the accuracy margin about $+10\%$ over the state-or-the-art unsupervised DA method. For a comparison with semi-supervised learning methods, it achieves the accuracy margin about $+30\%$ over Mean Teacher with 60 labeled samples in SVHN.