Gaussian process decentralized data fusion meets transfer learning in large-scale distributed cooperative perception

Abstract

This paper presents novel Gaussian process decentralized data fusion algorithms exploiting the notion of agent-centric support sets for distributed cooperative perception of large-scale environmental phenomena. To overcome the limitations of scale in existing works, our proposed algorithms allow every mobile sensing agent to utilize a different support set and dynamically switch to another during execution for encapsulating its own data into a local summary that, perhaps surprisingly, can still be assimilated with the other agents’ local summaries (i.e., based on their current support sets) into a globally consistent summary to be used for predicting the phenomenon. To achieve this, we propose a novel transfer learning mechanism for a team of agents capable of sharing and transferring information encapsulated in a summary based on a support set to that utilizing a different support set with some loss that can be theoretically bounded and analyzed. To alleviate the issue of information loss accumulating over multiple instances of transfer learning, we propose a new information sharing mechanism to be incorporated into our algorithms in order to achieve memory-efficient lazy transfer learning. Empirical evaluation on three real-world datasets for up to 128 agents show that our algorithms outperform the state-of-the-art methods.

Appendices

Appendix A: Gaussian predictive distribution computed by the GP-$\hbox {DDF}^+$ algorithm

Definition 5

(GP-$\hbox {DDF}^+$) Given a common support set $\mathcal {S}\subset \mathcal {X}$ known to all N agents, global summary $(\dot{\nu }_{\mathcal {S}},\dot{\varPsi }_{\mathcal {S}\mathcal {S}})$, local summary $(\nu _{\mathcal {S}|\mathcal {D}_i},\varPsi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})$, and a column vector $y_{\mathcal {D}_i}$ of realized measurements for observed locations $\mathcal {D}_i$, the GP-$\hbox {DDF}^+$ algorithm run by each agent i computes a Gaussian predictive distribution $\mathcal {N}(\overline{\mu }_{x}, \overline{\sigma }^2_{x})$ of the measurement for any unobserved location $x \in \mathcal {X}{\setminus }\mathcal {D}$ where

$$\begin{aligned} \begin{aligned} \overline{\mu }_{x}&\triangleq \displaystyle \mu _{x}+\left( \gamma _{x\mathcal {S}}^i\dot{\varPsi }^{-1}_{\mathcal {S}\mathcal {S}}\dot{\nu }_{\mathcal {S}} -\varSigma _{x\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\nu _{\mathcal {S}|\mathcal {D}_i}\right) +{\nu }_{x|\mathcal {D}_i},\\ \overline{\sigma }^2_{x}&\triangleq \displaystyle \sigma _{xx} - \Big (\gamma _{x\mathcal {S}}^i\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}x}-\varSigma _{x\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varPsi _{\mathcal {S}x|\mathcal {D}_i}\\&\quad \displaystyle -\gamma _{x\mathcal {S}}^i\dot{\varPsi }_{\mathcal {S}\mathcal {S}}^{-1}\gamma _{\mathcal {S}x}^i \Big )-\varPsi _{xx|\mathcal {D}_i}, \end{aligned} \end{aligned}$$

(10)

$\gamma _{x\mathcal {S}}^i \triangleq \displaystyle \varSigma _{x\mathcal {S}}+\varSigma _{x\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varPsi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}-\varPsi _{x\mathcal {S}|\mathcal {D}_i}\ ,$ and $\gamma _{\mathcal {S}x}^i \triangleq \gamma _{x\mathcal {S}}^{i\top }.$

The Gaussian predictive distribution (10) computed by the GP-$\hbox {DDF}^+$ algorithm is observed to exploit the local and global summaries (i.e., terms within brackets) as well as the data local to agent i (i.e., ${\nu }_{x|\mathcal {D}_i}$ and $\varPsi _{xx|\mathcal {D}_i}$ terms).

Appendix B: Proof of Proposition 1

$$\begin{aligned} \begin{aligned}&\omega _{\mathcal {S}'|\mathcal {D}_i}\\&\quad =\displaystyle \varSigma _{\mathcal {S}'\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}(y_{\mathcal {D}_i}-\mu _{\mathcal {D}_i})\\&\quad = \displaystyle \varSigma _{\mathcal {S}'\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}(y_{\mathcal {D}_i}-\mu _{\mathcal {D}_i})\\&\quad =\displaystyle \varSigma _{\mathcal {S}'\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\omega _{\mathcal {S}|\mathcal {D}_i} \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \begin{aligned}&\varPhi _{\mathcal {S}'\mathcal {S}'|\mathcal {D}_i}\\&\quad =\displaystyle \varSigma _{\mathcal {S}'\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}'}\\&\quad = \displaystyle \varSigma _{\mathcal {S}'\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {S}'}\\&\quad =\displaystyle \varSigma _{\mathcal {S}'\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {S}'} \end{aligned} \end{aligned}$$

where the second equalities above follow from the assumption that $\mathcal {S}'$ and $\mathcal {D}_i$ are conditionally independent given $\mathcal {S}$ (i.e., $\varSigma _{\mathcal {S}'\mathcal {D}_i|\mathcal {S}}= \varSigma _{\mathcal {S}'\mathcal {D}_i} - \varSigma _{\mathcal {S}'\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i} =\underline{0}$).

Appendix C: Proof of Proposition 2

$$\begin{aligned} \begin{aligned}&\varPsi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i|\mathcal {S}}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\\&\quad =\displaystyle \varSigma _{\mathcal {S}\mathcal {D}_i}(\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}+\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1})\varSigma _{\mathcal {D}_i\mathcal {S}}\\&\quad =\displaystyle \varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\\&\qquad \displaystyle +\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}+\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}})^{-1}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}+\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(I+(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1}(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}+\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1}\varSigma _{\mathcal {S}\mathcal {S}} \end{aligned} \end{aligned}$$

where the second equality follows from the matrix inverse lemma on $\varSigma _{\mathcal {D}_i\mathcal {D}_i|\mathcal {S}}^{-1} = (\varSigma _{\mathcal {D}_i\mathcal {D}_i}-\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i})^{-1} =\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}+\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1} \varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}$. As a result,

$$\begin{aligned}&\varPsi _{SS|\mathcal {D}_i}^{-1}\\&\quad =\varSigma _{SS}^{-1}(\varSigma _{SS} -\varPhi _{SS|\mathcal {D}_i})\varPhi _{SS|\mathcal {D}_i}^{-1} =\varPhi _{SS|\mathcal {D}_i}^{-1} - \varSigma _{SS}^{-1}.\\&\nu _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i|\mathcal {S}}^{-1}y_{\mathcal {D}_i}\\&\quad =\displaystyle \varSigma _{\mathcal {S}\mathcal {D}_i}(\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}+\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1})y_{\mathcal {D}_i}\\&\quad =\displaystyle \varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}y_{\mathcal {D}_i}+\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}y_{\mathcal {D}_i}\\&\quad =\displaystyle \omega _{\mathcal {S}|\mathcal {D}_i}+\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}})^{-1}\omega _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \omega _{\mathcal {S}|\mathcal {D}_i}+\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1}\omega _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}+(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1})\omega _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1}\\&\qquad \displaystyle ((\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}+I)\omega _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}(\varSigma _{\mathcal {S}\mathcal {S}}-\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i})^{-1}\varSigma _{\mathcal {S}\mathcal {S}}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\omega _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle (\varSigma _{\mathcal {S}\mathcal {S}}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}-I)^{-1}\varSigma _{\mathcal {S}\mathcal {S}}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\omega _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle (\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}-\varSigma _{\mathcal {S}\mathcal {S}}^{-1})^{-1}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\omega _{\mathcal {S}|\mathcal {D}_i}\\&\quad =\displaystyle \varPsi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}\varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\omega _{\mathcal {S}|\mathcal {D}_i} \end{aligned}$$

where the second equality follows from the matrix inverse lemma on $\varSigma _{\mathcal {D}_i\mathcal {D}_i|\mathcal {S}}^{-1} =\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}+\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}\varSigma _{\mathcal {D}_i\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}\varSigma _{\mathcal {D}_i\mathcal {D}_i}^{-1}$. As a result, $\varPsi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\nu _{\mathcal {S}|\mathcal {D}_i} = \varPhi _{\mathcal {S}\mathcal {S}|\mathcal {D}_i}^{-1}\omega _{\mathcal {S}|\mathcal {D}_i}$. So, (8) follows.

Appendix D: Proof of Theorem 1

The following lemma is necessary for deriving our main result here:

Lemma 1

Define $\sigma _{xx'}$ using a squared exponential covariance function. Then, every covariance component $\sigma _{xx'}$ in $\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}}$, $\varSigma _{\mathcal {S}\mathcal {S}}$, $\varSigma _{\mathcal {S}'_{t}\mathcal {S}}$, and $\varSigma _{\mathcal {D}_{it'}\mathcal {S}}$ satisfies $(\sigma _{xx'}-\sigma _{ss'})^2\le {3e^{-1}\sigma _s^4}(\Vert \varLambda ^{-1}(x-s)\Vert ^2 + \Vert \varLambda ^{-1}(x'-s')\Vert ^2)$ for all $x,x',s,s'\in \mathcal {X}$.

Proof

Since every covariance component $\sigma _{xx'}$ in $\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}}$, $\varSigma _{\mathcal {S}\mathcal {S}}$, $\varSigma _{\mathcal {S}'_{t}\mathcal {S}}$, and $\varSigma _{\mathcal {D}_{it'}\mathcal {S}}$ does not involve the noise variance $\sigma ^2_n$, it follows from (1) that

$$\begin{aligned} \begin{aligned} \sigma _{xx'}&=\displaystyle \sigma _s^2\exp \left( -\left\| \frac{{\varLambda }^{-1}({x} - {x}')}{\sqrt{2}}\right\| ^2\right) \\&=\displaystyle \sigma _s^2 k\left( \left\| \frac{{\varLambda }^{-1}({x} - {x}')}{\sqrt{2}}\right\| \right) \end{aligned} \end{aligned}$$

where $k(a)\triangleq \exp (-a^2)$. Then,

$$\begin{aligned} \begin{aligned}&\displaystyle (\sigma _{xx'}-\sigma _{ss'})^2\\&\quad \displaystyle = \sigma _s^4 \left\{ k\left( \left\| \frac{{\varLambda }^{-1}({x} - {x}')}{\sqrt{2}}\right\| \right) - k\left( \left\| \frac{{\varLambda }^{-1}({s} - {s}')}{\sqrt{2}}\right\| \right) \right\} ^2\\&\quad \displaystyle = 0.5\sigma _s^4 k'(\xi )^2(\Vert \varLambda ^{-1}({x} - {x}')\Vert - \Vert \varLambda ^{-1}({s} - {s}')\Vert )^2\\&\quad \displaystyle \le e^{-1}\sigma _s^4 (\Vert \varLambda ^{-1}(x-s)\Vert + \Vert \varLambda ^{-1}(x'-s')\Vert )^2\\&\quad \displaystyle \le e^{-1}\sigma _s^4(\Vert \varLambda ^{-1}(x-s)\Vert + \Vert \varLambda ^{-1}(x'-s')\Vert )^2\\&\quad \displaystyle \le {3e^{-1}\sigma _s^4}(\Vert \varLambda ^{-1}(x-s)\Vert ^2 + \Vert \varLambda ^{-1}(x'-s')\Vert ^2) \end{aligned} \end{aligned}$$

where the second equality is due to mean value theorem such that $k'(\xi )$ is the first-order derivative of k evaluated at some $\xi \in (\Vert \varLambda ^{-1}(s-s')\Vert /\sqrt{2},\Vert \varLambda ^{-1}(x-x')\Vert /\sqrt{2})$ without loss of generality, the first inequality follows from the fact that $k'(a)$ is maximized at $a=-1/\sqrt{2}$ and hence $k'(\xi )\le k'(-1/\sqrt{2})=\sqrt{2/e}$, and the second inequality is due to triangle inequality (i.e., $\Vert \varLambda ^{-1}(x-x')\Vert \le \Vert \varLambda ^{-1}(x-s)\Vert +\Vert \varLambda ^{-1}(s-s')\Vert +\Vert \varLambda ^{-1}(s'-x')\Vert$). $\square$

Supposing each subset $\mathcal {D}_{is}$ ($\mathcal {S}'_s$) contains T ($T'$) locations,¹⁰ select one location from each subset to form a new subset $\mathcal {D}_{it'}\triangleq \{x_{it's} \}_{s\in \mathcal {S}}$ ($\mathcal {S}'_t\triangleq \{x'_{ts} \}_{s\in \mathcal {S}}$) of $|\mathcal {S}|$ locations for $t'=1$ ($t=1$) and repeat this for $t'=2,\ldots ,T$ ($t=2,\ldots ,T'$). Then, $\mathcal {D}_{i}=\bigcup ^T_{t'=1}\mathcal {D}_{it'}$ and $\mathcal {S}'=\bigcup ^{T'}_{t=1}\mathcal {S}'_{t}$. It follows that $\varSigma _{\mathcal {S}'\mathcal {S}} = [\varSigma _{\mathcal {S}'_{t}\mathcal {S}}]_{t=1,\ldots ,T'}$, $\varSigma _{\mathcal {S}\mathcal {D}_{i}} = [\varSigma _{\mathcal {S}\mathcal {D}_{it'}}]_{t'=1,\ldots ,T}$, and $\varSigma _{\mathcal {S}'\mathcal {D}_{i}} = [\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}}]_{t=1,\ldots ,T',t'=1,\ldots ,T}$.

Using the definition of Frobenius norm followed by the subadditivity of a square root function,

$$\begin{aligned} \begin{aligned}&\displaystyle ||\varSigma _{\mathcal {S}'\mathcal {D}_i} - \varSigma _{\mathcal {S}'\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}||_F\\&\quad = \displaystyle ||\varSigma _{\mathcal {S}'\mathcal {D}_i|\mathcal {S}}||_F\\&\quad = \displaystyle \sqrt{\sum ^{T'}_{t=1} \sum ^{T}_{t'=1} ||\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}|\mathcal {S}}||^2_F}\\&\quad \le \displaystyle \sum ^{T'}_{t=1} \sum ^{T}_{t'=1} ||\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}|\mathcal {S}}||_F. \end{aligned} \end{aligned}$$

(11)

Let $A_{\mathcal {S}'_{t}\mathcal {D}_{it'}}\triangleq \varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}}-\varSigma _{\mathcal {S}\mathcal {S}}$, $B_{\mathcal {S}'_t\mathcal {S}}\triangleq \varSigma _{\mathcal {S}'_t\mathcal {S}}-\varSigma _{\mathcal {S}\mathcal {S}}$, and $C_{\mathcal {D}_{it'}\mathcal {S}}\triangleq \varSigma _{\mathcal {D}_{it'}\mathcal {S}}-\varSigma _{\mathcal {S}\mathcal {S}}$. Then,

$$\begin{aligned} \begin{aligned}&\displaystyle ||\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}|\mathcal {S}}||_F\\&\quad = \displaystyle ||\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}} - \varSigma _{\mathcal {S}'_t\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_{it'}}||_F\\&\quad = \displaystyle || \varSigma _{\mathcal {S}\mathcal {S}}+A_{\mathcal {S}'_{t}\mathcal {D}_{it'}} \\&\qquad -\displaystyle (\varSigma _{\mathcal {S}\mathcal {S}}+B_{\mathcal {S}'_t\mathcal {S}})\varSigma _{\mathcal {S}\mathcal {S}}^{-1}(\varSigma _{\mathcal {S}\mathcal {S}}+C_{\mathcal {D}_{it'}\mathcal {S}})^{\top }||_F\\&\quad = \displaystyle || \varSigma _{\mathcal {S}\mathcal {S}}+A_{\mathcal {S}'_{t}\mathcal {D}_{it'}} -\varSigma _{\mathcal {S}\mathcal {S}}^{\top }-C_{\mathcal {D}_{it'}\mathcal {S}}^{\top } -B_{\mathcal {S}'_t\mathcal {S}}\\&\qquad -\displaystyle B_{\mathcal {S}'_t\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}C_{\mathcal {D}_{it'}\mathcal {S}}^{\top }||_F\\&\quad \le \displaystyle || A_{\mathcal {S}'_{t}\mathcal {D}_{it'}}||_F +||B_{\mathcal {S}'_t\mathcal {S}}||_F +||C_{\mathcal {D}_{it'}\mathcal {S}}||_F \\&\quad \quad +||B_{\mathcal {S}'_t\mathcal {S}}||_F ||C_{\mathcal {D}_{it'}\mathcal {S}}||_F ||\varSigma _{\mathcal {S}\mathcal {S}}^{-1}||_F \end{aligned} \end{aligned}$$

(12)

where the inequality is due to the subadditivity and submultiplicativity of the matrix norm.

Let $\epsilon _{\mathcal {S}'_{t}}\triangleq (1/|\mathcal {S}|)\sum _{x\in \mathcal {S}'_{t}}||\varLambda ^{-1}(x-c(x))||^2$ and $\epsilon _{\mathcal {D}_{it'}}\triangleq (1/|\mathcal {S}|)\sum _{x\in \mathcal {D}_{it'}}||\varLambda ^{-1}(x-c(x))||^2$. Then,

$$\begin{aligned} \begin{aligned}&\displaystyle || A_{\mathcal {S}'_{t}\mathcal {D}_{it'}}||^2_F\\&\quad = \displaystyle ||\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}}-\varSigma _{\mathcal {S}\mathcal {S}}||^2_F\\&\quad = \displaystyle \sum _{s,s'\in \mathcal {S}} (\sigma _{x'_{ts}x_{it's'}}-\sigma _{ss'})^2\\&\quad \le 3e^{-1}\sigma ^4_s \displaystyle \sum _{s,s'\in \mathcal {S}}\left( ||\varLambda ^{-1}(x'_{ts}-s)||^2 + ||\varLambda ^{-1}(x_{it's'}-s')||^2\right) \\&\quad = 3e^{-1}\sigma ^4_s|\mathcal {S}|\displaystyle \Bigg (\sum _{s\in \mathcal {S}}||\varLambda ^{-1}(x'_{ts}-s)||^2 \\&\qquad \displaystyle +\sum _{s'\in \mathcal {S}}||\varLambda ^{-1}(x_{it's'}-s')||^2\Bigg )\\&\quad =3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle (\epsilon _{\mathcal {S}'_{t}} + \epsilon _{\mathcal {D}_{it'}}) \end{aligned} \end{aligned}$$

(13)

since $\epsilon _{\mathcal {S}'_{t}}=(1/|\mathcal {S}|)\sum _{s\in \mathcal {S}}||\varLambda ^{-1}(x'_{ts}-s)||^2$ and $\epsilon _{\mathcal {D}_{it'}}=(1/|\mathcal {S}|)$$\sum _{s'\in \mathcal {S}}||\varLambda ^{-1}(x_{it's'}-s')||^2$. The inequality is due to Lemma 1.

$$\begin{aligned} \begin{aligned}&\displaystyle || B_{\mathcal {S}'_{t}\mathcal {S}}||^2_F\\&\quad = \displaystyle ||\varSigma _{\mathcal {S}'_{t}\mathcal {S}}-\varSigma _{\mathcal {S}\mathcal {S}}||^2_F\\&\quad = \displaystyle \sum _{s,s'\in \mathcal {S}} (\sigma _{x'_{ts}s'}-\sigma _{ss'})^2\\&\quad \le 3e^{-1}\sigma ^4_s\displaystyle \sum _{s,s'\in \mathcal {S}}\left( ||\varLambda ^{-1}(x'_{ts}-s)||^2 + ||\varLambda ^{-1}(s'-s')||^2\right) \\&\quad = 3e^{-1}\sigma ^4_s|\mathcal {S}|\displaystyle \sum _{s\in \mathcal {S}}||\varLambda ^{-1}(x'_{ts}-s)||^2\\&\quad =3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle \epsilon _{\mathcal {S}'_{t}} \end{aligned} \end{aligned}$$

(14)

such that the inequality is due to Lemma 1.

$$\begin{aligned} \begin{aligned}&\displaystyle || C_{\mathcal {D}_{it'}\mathcal {S}}||^2_F\\&\quad = \displaystyle ||\varSigma _{\mathcal {D}_{it'}\mathcal {S}}-\varSigma _{\mathcal {S}\mathcal {S}}||^2_F\\&\quad = \displaystyle \sum _{s,s'\in \mathcal {S}} (\sigma _{x_{it's}s'}-\sigma _{ss'})^2\\&\quad \le 3e^{-1}\sigma ^4_s\displaystyle \sum _{s,s'\in \mathcal {S}}\left( ||\varLambda ^{-1}(x_{it's}-s)||^2 + ||\varLambda ^{-1}(s'-s')||^2\right) \\&\quad = 3e^{-1}\sigma ^4_s|\mathcal {S}|\displaystyle \sum _{s\in \mathcal {S}}||\varLambda ^{-1}(x_{it's}-s)||^2\\&\quad =3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle \epsilon _{\mathcal {D}_{it'}} \end{aligned} \end{aligned}$$

(15)

such that the inequality is due to Lemma 1.

By substituting (13), (14), and (15) into (12),

$$\begin{aligned} \begin{aligned}&\displaystyle ||\varSigma _{\mathcal {S}'_{t}\mathcal {D}_{it'}|\mathcal {S}}||_F\\&\quad \le \displaystyle \sqrt{3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle (\epsilon _{\mathcal {S}'_{t}} + \epsilon _{\mathcal {D}_{it'}})} +\sqrt{3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle \epsilon _{\mathcal {S}'_{t}}}\\&\qquad \displaystyle +\sqrt{3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle \epsilon _{\mathcal {D}_{it'}}}\\&\qquad +\displaystyle \sqrt{3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle \epsilon _{\mathcal {S}'_{t}}} \sqrt{3e^{-1}\sigma ^4_s|\mathcal {S}|^2\displaystyle \epsilon _{\mathcal {D}_{it'}}} ||\varSigma _{\mathcal {S}\mathcal {S}}^{-1}||_F\\&\quad =\displaystyle \sqrt{3/e}\sigma ^2_s|\mathcal {S}|\Big (\sqrt{\epsilon _{\mathcal {S}'_{t}} + \epsilon _{\mathcal {D}_{it'}}}+\sqrt{\epsilon _{\mathcal {S}'_{t}}}+\sqrt{\epsilon _{\mathcal {D}_{it'}}} \\&\qquad \displaystyle +\sigma ^2_s||\varSigma _{\mathcal {S}\mathcal {S}}^{-1}||_F|\mathcal {S}|\sqrt{3\epsilon _{\mathcal {S}'_{t}}\epsilon _{\mathcal {D}_{it'}}/e}\Big ). \end{aligned} \end{aligned}$$

(16)

By substituting (16) into (11),

$$\begin{aligned} \begin{aligned}&\displaystyle ||\varSigma _{\mathcal {S}'\mathcal {D}_i} - \varSigma _{\mathcal {S}'\mathcal {S}}\varSigma _{\mathcal {S}\mathcal {S}}^{-1}\varSigma _{\mathcal {S}\mathcal {D}_i}||_F\\&\quad \le \displaystyle \sqrt{3/e}\sigma ^2_s|\mathcal {S}|\sum ^{T'}_{t=1} \sum ^{T}_{t'=1} \Big (\sqrt{\epsilon _{\mathcal {S}'_{t}} + \epsilon _{\mathcal {D}_{it'}}}+\sqrt{\epsilon _{\mathcal {S}'_{t}}}+\sqrt{\epsilon _{\mathcal {D}_{it'}}} \\&\qquad \displaystyle +\sigma ^2_s||\varSigma _{\mathcal {S}\mathcal {S}}^{-1}||_F|\mathcal {S}|\sqrt{3\epsilon _{\mathcal {S}'_{t}}\epsilon _{\mathcal {D}_{it'}}/e}\Big )\\&\quad \le \displaystyle \sqrt{3/e}\sigma ^2_s|\mathcal {S}| \Bigg (\sqrt{TT'\sum ^{T'}_{t=1} \sum ^{T}_{t'=1} (\epsilon _{\mathcal {S}'_{t}} + \epsilon _{\mathcal {D}_{it'}})} \\&\qquad \displaystyle +\sqrt{TT'\sum ^{T'}_{t=1} \sum ^{T}_{t'=1} \epsilon _{\mathcal {S}'_{t}}} + \sqrt{TT'\sum ^{T'}_{t=1} \sum ^{T}_{t'=1} \epsilon _{\mathcal {D}_{it'}}} \\&\qquad \displaystyle +\sigma ^2_s||\varSigma _{\mathcal {S}\mathcal {S}}^{-1}||_F|\mathcal {S}|\sqrt{TT'(3/e)\sum ^{T'}_{t=1} \sum ^{T}_{t'=1} \epsilon _{\mathcal {S}'_{t}}\epsilon _{\mathcal {D}_{it'}}}\Bigg )\\&\quad = \displaystyle \sqrt{3/e}\sigma ^2_s|\mathcal {S}|\Bigg (\sqrt{TT'\left( T\sum ^{T'}_{t=1} \epsilon _{\mathcal {S}'_{t}} + T'\sum ^{T}_{t'=1}\epsilon _{\mathcal {D}_{it'}}\right) } \\&\qquad \displaystyle +\sqrt{T^2T'\sum ^{T'}_{t=1} \epsilon _{\mathcal {S}'_{t}}} + \sqrt{TT'^2 \sum ^{T}_{t'=1} \epsilon _{\mathcal {D}_{it'}}} \\&\qquad \displaystyle +\sigma ^2_s||\varSigma _{\mathcal {S}\mathcal {S}}^{-1}||_F|\mathcal {S}|\sqrt{TT'(3/e)\sum ^{T'}_{t=1} \epsilon _{\mathcal {S}'_{t}}\sum ^{T}_{t'=1} \epsilon _{\mathcal {D}_{it'}}}\Bigg )\\&\quad = \displaystyle \sqrt{3/e}\sigma ^2_s|\mathcal {S}|TT' \Big (\sqrt{\epsilon _{\mathcal {S}'} + \epsilon _{\mathcal {D}_{i}}} +\sqrt{\epsilon _{\mathcal {S}'}}+\sqrt{\epsilon _{\mathcal {D}_{i}}} \\&\qquad \displaystyle +\sigma ^2_s||\varSigma _{\mathcal {S}\mathcal {S}}^{-1}||_F|\mathcal {S}|\sqrt{3\epsilon _{\mathcal {S}'}\epsilon _{\mathcal {D}_{i}}/e}\Big ) \end{aligned} \end{aligned}$$

such that the second inequality follows from

$$\begin{aligned} \sum ^T_{t=1}\sqrt{a_t}\le \sqrt{T\sum ^T_{t=1}a_t} \end{aligned}$$

which can be obtained by applying Jensen’s inequality to the concave square root function. The last equality is due to $\epsilon _{\mathcal {S}'} = (1/T')\sum ^{T'}_{t=1}\epsilon _{\mathcal {S}'_t}$ and $\epsilon _{\mathcal {D}_{i}} = (1/T)\sum ^{T}_{t'=1}\epsilon _{\mathcal {D}_{it'}}$.

Appendix E: Hyperparameter learning

The hyperparameters of our GP-DDF-ASS and GP-$\hbox {DDF}^+$-ASS algorithms are learned by maximizing the sum of log-marginal likelihoods $\sum _{\mathcal {S}} \log p(y_\mathcal {D}|\mathcal {S})$ over the support set $\mathcal {S}$ of every different local area via gradient ascent with respect to a common set of signal variance, noise variance, and length-scale hyperparameters (Sect. 2) where, as derived in Quiñonero-Candela and Rasmussen (2005),

$$\begin{aligned} \log p(y_\mathcal {D}|\mathcal {S})= & {} -0.5 (\log |\varXi _{\mathcal {D}\mathcal {D}|\mathcal {S}}|+y^{\top }_\mathcal {D}\varXi _{\mathcal {D}\mathcal {D}|\mathcal {S}}^{-1}y_\mathcal {D} \\&+ |\mathcal {D}|\log (2\pi )) \end{aligned}$$

such that $\varXi _{\mathcal {D}\mathcal {D}|\mathcal {S}}\triangleq \varPhi _{\mathcal {D}\mathcal {D}|\mathcal {S}}+\text {blockdiag}[\varSigma _{\mathcal {D}\mathcal {D}|\mathcal {S}}]+\sigma ^2_n I$. Note that these learned hyperparameters of our GP-DDF-ASS and GP-$\hbox {DDF}^+$-ASS algorithms correspond to the case where our proposed lazy transfer learning mechanism incurs minimal information loss, as explained in Sect. 4.2.