Online Meta-learning for Multi-source and Semi-supervised Domain Adaptation

Abstract

Domain adaptation (DA) is the topical problem of adapting models from labelled source datasets so that they perform well on target datasets where only unlabelled or partially labelled data is available. Many methods have been proposed to address this problem through different ways to minimise the domain shift between source and target datasets. In this paper we take an orthogonal perspective and propose a framework to further enhance performance by meta-learning the initial conditions of existing DA algorithms. This is challenging compared to the more widely considered setting of few-shot meta-learning, due to the length of the computation graph involved. Therefore we propose an online shortest-path meta-learning framework that is both computationally tractable and practically effective for improving DA performance. We present variants for both multi-source unsupervised domain adaptation (MSDA), and semi-supervised domain adaptation (SSDA). Importantly, our approach is agnostic to the base adaptation algorithm, and can be applied to improve many techniques. Experimentally, we demonstrate improvements on classic (DANN) and recent (MCD and MME) techniques for MSDA and SSDA, and ultimately achieve state of the art results on several DA benchmarks including the largest scale DomainNet.

Appendices

A Short-Path Gradient Descent

Optimizing Eq. 3 naively by Algorithm 1 would be costly and ineffective. It is costly because in the case of domain adaptation (unlike for example, few-shot learning [12], the inner loop requires many iterations). So back-propagating through the whole optimization path to update the initial $\varTheta$ in the outer loop will produce multiple high-order gradients. For example, if the inner loop applies j iterations, we will have

$$\begin{aligned} \begin{aligned} \varTheta _{}^{(1)}&= \varTheta _{}^{} - \alpha \nabla _{\varTheta _{}^{(0)}} \mathcal {L}_{\text {uda}}(.) \\ \dots \\ \varTheta _{}^{(j)}&= \varTheta _{}^{(j-1)} - \alpha \nabla _{\varTheta _{}^{(j-1)}}\mathcal {L}_{\text {uda}}(.) \end{aligned} \end{aligned}$$

(13)

then the outer loop will update the initial condition as

$$\begin{aligned} \begin{aligned} \varTheta ^* = \varTheta - \alpha \overbrace{\nabla _{\varTheta }\mathcal {L}_{\text {sup}}(\varTheta _{}^{(j)} , \mathcal {D}_{\text {val}})}^{\text {Meta Gradient}} \end{aligned} \end{aligned}$$

(14)

where higher-order gradient will be required for all items $\scriptstyle \nabla _{\varTheta _{}^{(0)}}\mathcal {L}_{\text {uda}}(.), \dots ,\nabla _{\varTheta _{}^{(j-1)}}\mathcal {L}_{\text {uda}}(.)$ in the update of Eq. 14.

One intuitive way of eliminating higher-order gradients for computing Eq. 14 is making $\nabla _{\varTheta _{}^{(0)}}\mathcal {L}_{\text {uda}}(.),\dots ,\nabla _{\varTheta _{}^{(j-1)}}\mathcal {L}_{\text {uda}}(.)$ constant during the optimization. Then, Eq. 14 is equivalent to

$$\begin{aligned} \begin{aligned} \varTheta ^* = \varTheta - \alpha \overbrace{\nabla _{\varTheta ^{(j)}}\mathcal {L}_{\text {sup}}(\varTheta ^{(j)} , \mathcal {D}_\text {val})}^{\text {First-order Meta Gradient}} \end{aligned} \end{aligned}$$

(15)

However, in order to compute Eq. 15, one still needs to store the optimization path of Eq. 13 in memory and back-propagate through it to optimize $\varTheta$, which requires high computational load. Therefore, we propose a practical solution an iterative meta-learning algorithm to iteratively optimize the model parameters during training.

Shortest Path Optimization.  To obtain the meta gradient in Eq. 15 in a more efficient way, we propose a more scalable and efficient meta-learning method using shortest-path gradient (S-P.G.)  [36]. Before the optimization of Eq. 13, we copy the parameters $\varTheta$ as $\tilde{\varTheta }^{(0)}$ and use $\tilde{\varTheta }^{(0)}$ in the inner-level algorithm.

$$\begin{aligned} \begin{aligned} \tilde{\varTheta }^{(j)}= {\left\{ \begin{array}{ll} \tilde{\varTheta }^{(0)} - \alpha \nabla _{\varTheta ^{(0)}}\mathcal {L}_{\text {uda}}(\tilde{\varTheta }^{(0)}, \mathcal {D}_\text {tr}), \\ \dots \\ \tilde{\varTheta }^{(j-1)} -\alpha \nabla _{\varTheta ^{(0)}}\mathcal {L}_{\text {uda}}(\tilde{\varTheta }^{(j-1)}, \mathcal {D}_\text {tr}) \end{array}\right. } \end{aligned} \end{aligned}$$

(16)

then, after finishing the optimization in Eq. 16, we can get the shortest-path gradient between two items $\tilde{\varTheta _i}^{(j)}$ and $\varTheta _i$.

$$\begin{aligned} \begin{aligned} \nabla _{\varTheta }^{\text {short}} = \varTheta - \tilde{\varTheta }^{(j)} \end{aligned} \end{aligned}$$

(17)

Different from Eq. 15, we use this shortest-path gradient $\nabla _{\varTheta }^{\text {short}}$ and initial parameter $\varTheta$ to compute $\mathcal {L}_{\text {sup}}(.)$ as,

$$\begin{aligned} \begin{aligned} \mathcal {L}_{\text {sup}}(\varTheta _i - \nabla _{\varTheta _i}^{\text {short}} , \mathcal {D}_{\text {val}}) \end{aligned} \end{aligned}$$

(18)

Then, one-step meta update of Eq. 18 will be,

$$\begin{aligned} \begin{aligned} \varTheta _i^*&= \varTheta _i - \alpha \nabla _{\varTheta _i}\mathcal {L}_{\text {sup}}(\varTheta _i - \nabla _{\varTheta _i}^{\text {short}} , \mathcal {D}_{\text {val}}) \\&=\varTheta _i - \alpha \nabla _{\varTheta _i-\nabla _{\varTheta _i}^{\text {short}}}\mathcal {L}_{\text {sup}}(\varTheta _i - \nabla _{\varTheta _i}^{\text {short}} , \mathcal {D}_{\text {val}}) \\&= \varTheta _i - \alpha \nabla _{\tilde{\varTheta _i}^{(j)}}\mathcal {L}_{\text {sup}}(\tilde{\varTheta _i}^{(j)} , \mathcal {D}_{\text {val}}) \end{aligned} \end{aligned}$$

(19)

Effectiveness: We can see that one update of Eq. 19 corresponds to that of Eq. 15, which proves that using shortest-path optimization has the equivalent effectiveness to the first-order meta optimization. Scalability/Efficiency: The computation memory of the first-order meta-learning increases linearly with the inner-loop update steps, which is constrained by the total GPU memory. However, for the shortest-path optimization, storing the optimization graph is no longer necessary, which makes it scalable and efficient. We also experimentally evaluate that one step shortest-path optimization is 7x faster than one-step first-order meta optimization in our setting. The overall algorithm flow is shown in Algorithm 2.

B Additional Illustrative Schematics

To better explain the contrast between our online meta-learning domain adaptation approach with the sequential meta-learning approach, we add a schematic illustration in Figure 4. The main difference between sequential and online meta-learning approaches is how do we distribute the meta and DA updates. Sequential meta-learning approach performs meta updates and DA updates sequentially. And online meta-learning conducts the alternative meta and DA updates throughout the whole training procedure.

Fig. 4.

Illustrative schematics of sequential and online meta domain adaptation. Left: Optimization paths of different approaches on domain adaptation loss (shading). (Solid line) Vanilla gradient descent on a DA objective from a fixed start point. (Multi-segment line) Online meta-learning iterates meta and gradient descent updates. (Two segment line) Sequential meta-learning provides an alternative approximation: update initial condition, then perform gradient descent. Right: (Top) Sequential meta-learning performs meta updates and DA updates sequentially. (Bottom) Online meta-learning alternates between meta-optimization and domain adaptation.

C Additional Experiments

Visualization of the Learned Features.  We visualize the learned features of MCD and Meta-MCD on PACS when sketch is the target domain as shown in Fig. 5. We can see that both MCD and Meta-MCD can learn discriminative features. However, the features learned by Meta-MCD is more separable than vanilla MCD. This explains why our Meta-MCD performs better than the vanilla MCD method.

Effect of Varying  S. Our online meta-learning method has iteration hyper-parameters S and J. We fix $J=1$ throughout, and analyze the effect of varying S here using the DomainNet MSDA experiment with ResNet-18. The result in Table 7 shows that MetaDA is rather insensitive to this hyperparameter.

Varying the Number of Source Domains in MSDA.  For multi-source DA, the performance of both Meta-DA and the baselines is expected to drop with fewer sources (same for SSDA if fewer labeled target domain points). To disentangle the impact of the number of sources for baseline vs Meta-DA we compare MSDA by Meta-MCD on PACS with 2 vs 3 sources. The results for Meta-MCD vs vanilla MCD are 82.30% vs 80.07% (two source, gap 2.23%) and 87.24% vs 84.79% (three source, gap 2.45%). Meta-DA margin is similar with reduction of domains. Most difference is accounted for by the impact on the base DA algorithm.

Fig. 5.

t-SNE  [32] visualization of learned MCD (left) and Meta-MCD (right) features on PACS (sketch as target domain). Different colors indicate different categories.

Table 7. MetaDA is insensitive to the update ratio hyperparameter S – Results for MSDA ResNet-18 performance on DomainNet.

Table 8. Test accuracy on PACS. * our run.

Table 9. Test accuracy on Digit-Five.

Other Base DA Methods.  Besides the base DA methods evaluated in the main paper (DANN, MCD and MME), our method is applicable to any base domain adaptation method. We use the published code of JiGen⁵ and M$^3$SDA⁶, and further apply our Meta-DA on the existing code. The results are shown in Table 8 and 9. From the results, we can see that our Meta-JiGen and Meta-M$^3$SDA-$\beta$ improves over the base methods by 3.42% and 1.2% accuracy respectively, which confirms our Meta-DA’s generality. The reason we excluded these from the main results is that: (i) Re-running JiGen’s published code on our compute environment failed to replicate their published numbers. (ii) M$^3$SDA as a base algorithm is very slow to run comprehensive experiments on. Nevertheless, these results provide further evidence that Meta-DA can be a useful module going forward to plug in and improve future new base DA methods as well as those evaluated here.

Table 10. Test accuracy of MCD on PACS (sketch) with different initialization.

Initialization Dependence of Domain Adaptation.  One may not think of domain adaptation as being sensitive to initial condition, but given the lack of target domain supervision to guide learning, different initialization can lead to a significant difference in accuracy. To illustrate this we re-ran MCD-based DA on PACS with sketch target using different initializations. From the results in Table 10, we can see that both different classic initialization heuristics, and simple perturbation of a given initial condition with noise can lead to significant differences in final performance. This confirms that studying methods for tuning initialization provide a valid research direction for advancing DA performance.