Skip to main content

Accelerating Metropolis-within-Gibbs sampler with localized computations of differential equations

Abstract

Inverse problem is ubiquitous in science and engineering, and Bayesian methodologies are often used to infer the underlying parameters. For high-dimensional temporal-spatial models, classical Markov chain Monte Carlo methods are often slow to converge, and it is necessary to apply Metropolis-within-Gibbs (MwG) sampling on parameter blocks. However, the computation cost of each MwG iteration is typically \(O(n^2)\), where n is the model dimension. This can be too expensive in practice. This paper introduces a new reduced computation method to bring down the computation cost to O(n), for the inverse initial value problem of a stochastic differential equation (SDE) with local interactions. The key observation is that each MwG proposal is only different from the original iterate at one parameter block, and this difference will only propagate within a local domain in the SDE computations. Therefore, we can approximate the global SDE computation with a surrogate updated only within the local domain for reduced computation cost. Both theoretically and numerically, we show that the approximation errors can be controlled by the local domain size. We discuss how to implement the local computation scheme using Euler–Maruyama and fourth-order Runge–Kutta methods. We numerically demonstrate the performance of the proposed method with the Lorenz 96 model and a linear stochastic flow model.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  • Andrieu, C., Roberts, G.O.: The pseudo-marginal approach for efficient Monte Carlo computations. Ann. Stat. 37, 697–725 (2009)

    MathSciNet  Article  Google Scholar 

  • Andrieu, C., Vihola, M.: Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms. Ann. Appl. Probab. 25(2), 1030 (2015)

    MathSciNet  Article  Google Scholar 

  • Asch, M., Bocquet, M., Nodet, M.: Data Assimilation: Methods, Algorithms, and Applications. Society for Industrial and Applied Mathematics. SIAM, Philadelphia (2016)

    Book  Google Scholar 

  • Chadan, K., Sabatier, P., Newton, R.: Inverse Problems in Quantum Scattering Theory. Springer, Berlin (1977)

    Book  Google Scholar 

  • Chen, N., Majda, A.J., Tong, X.T.: Spatial localization for nonlinear dynamical stochastic models for excitable media. Chin. Ann. Math. Ser. B 40(6), 891–924 (2019)

    MathSciNet  Article  Google Scholar 

  • Cotter, S., Dashti, M., Robinson, J., Stuart, A.: Bayesian inverse problems for functions and applications to fluid mechanics. Inverse Prob. 25, 115008 (2009)

    MathSciNet  Article  Google Scholar 

  • Crisan, D., Rozovskii, B. (eds.): The Oxford Handbook of Nonlinear Filtering. Oxford University Press, Oxford (2011)

    MATH  Google Scholar 

  • Fertig, E.J., Harlim, J., Hunt, B.R.: A comparative study of 4D-VAR and a 4D Ensemble Kalman Filter: perfect model simulations with Lorenz-96. Tellus 59A, 96–100 (2007)

    Article  Google Scholar 

  • Gelfand, A., Smith, A.: Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85, 398–409 (1990)

    MathSciNet  Article  Google Scholar 

  • Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)

    Article  Google Scholar 

  • Gilks, W., Richardson, S., Spiegelhalter, D.: Markov Chain Monte Carlo in Practice. Chapmann & Hall, Boca Raton (1996)

    MATH  Google Scholar 

  • Hastings, W.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970)

    MathSciNet  Article  Google Scholar 

  • Kaipio, J., Somersalo, E.: Statistical and Computational Inverse Problems, vol. 160. Springer, Berlin (2005)

    MATH  Google Scholar 

  • Law, K., Sanz-Alonso, D., Shukla, A., Stuart, A.: Filter accuracy for the Lorenz 96 model: fixed versus adaptive observation operators. Physica D 325, 1–13 (2016)

    MathSciNet  Article  Google Scholar 

  • Liu, J.: Monte Carlo strategies in Scientific Computing. Springer, Berlin (2003)

    Google Scholar 

  • Lorenz, E.N.: Predictability: a problem partly solved. In: ECMWF Seminar on predictability pp. 1–18 (1995)

  • Majda, A., Harim, J.: Filtering Complex Turbulent Systems. Cambridge University Press, Cambridge (2012)

    Book  Google Scholar 

  • Meinhold, R.J., Singpurwalla, N.D.: Understanding the Kalman filter. Am. Stat. 37(2), 123–127 (1983)

    Google Scholar 

  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

    Article  Google Scholar 

  • Morzfeld, M., Tong, X., Marzouk, Y.: Localization for MCMC: sampling high-dimensional posterior distributions with local structure. J. Comput. Phys. 380, 1–28 (2019)

    MathSciNet  Article  Google Scholar 

  • Ott, E., Hunt, B.R., Szunyogh, I., Zimin, A.V., Kostelich, E.J., Corazza, M., Kalnay, E., Patil, D.J., Yorke, J.A.: A local ensemble Kalman filter for atmospheric data assimilation. Tellus 56A, 415–428 (2004)

    Article  Google Scholar 

  • Richter, M.: Inverse Problem: Basics, Theory and Applications in Geophysics. Birkhauser, Basel (2016)

    Book  Google Scholar 

  • Rosenthal, J.S., Roberts, G.O.: Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. Ser. B 60, 255–268 (1998)

    MathSciNet  Article  Google Scholar 

  • Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451–559 (2010)

    MathSciNet  Article  Google Scholar 

  • Tong, X.: Performance analysis of local ensemble Kalman filter. J. Nonlinear Sci. 28, 1397–1442 (2018)

    MathSciNet  Article  Google Scholar 

  • Wunsch, C.: The Ocean Circulation Inverse Problem. Cambridge University Press, Cambridge (1996)

    Book  Google Scholar 

Download references

Acknowledgements

This work is supported by Singapore MOE-AcRF Grant R-146-000-258-114 and R-146-000-292-114.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qiang Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Auxiliary lemmas and their proofs

Lemma 1

An \(n \times n\) matrix A is circulant tridiagonal if \(A_{j,i} = 0\) for all pairs of (ji) such that \(d(j,i) > 1\) (recall \(d(j,i) = \min \{ |j-i|, |j+n-i|, |i+n-j| \}\)). For such type of matrix, we have

$$\begin{aligned} |(\mathrm{{e}}^{A})_{j,i}| \le \mathrm{{e}}^{-C\cdot d(j,i)}\cdot \mathrm{{e}}^{(\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)\rho }, \quad \forall \ j,i=1,2,\ldots ,n, \end{aligned}$$

where \(\rho = \max \limits _{j,i = 1,2,\ldots ,n} |A_{j,i}|\), C is any fixed positive constant.

Remark 1

If A is further a tridiagonal matrix, we have

$$\begin{aligned} |(\mathrm{{e}}^{A})_{j,i}| \le \mathrm{{e}}^{-C\cdot |j-i|}\cdot \mathrm{{e}}^{(\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)\rho }, \quad \forall \ j,i=1,2,\ldots ,n. \end{aligned}$$

Proof

Firstly, by using mathematical induction, we prove that with any positive constant C,

$$\begin{aligned} \begin{aligned}&|(A^{m})_{j,i}| \\&\le (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)^{m}\cdot \rho ^m\cdot \mathrm{{e}}^{-C\cdot d(j,i)}, \ \text {for} \ m=0,1,\ldots . \end{aligned} \end{aligned}$$
(31)

The conclusion is true for \(m=0\) since \(A^{0} = {\mathbf {I}}_n\) and \(\mathrm{{e}}^{-C\cdot d(j,i)} > 0\) with \(\mathrm{{e}}^{-C\cdot d(j,j)} = 1\). If the result is true for \(m=k\), namely we have \(|(A^{k})_{j,i}| \le (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)^{k}\cdot \rho ^k\cdot \mathrm{{e}}^{-C\cdot d(j,i)}\). Then, for \(m=k+1\), we observe that

$$\begin{aligned} \begin{aligned}&(A^{k+1})_{j,i} \\&= (A^{k})_{j,i-1}A_{i-1,i} + (A^{k})_{j,i}A_{i,i} + (A^{k})_{j,i+1}A_{i+1,i}, \end{aligned} \end{aligned}$$
(32)

with \((A^{k})_{j,0} = (A^{k})_{j,n}\), \((A^{k})_{j, n+1} = (A^{k})_{j,1}\), \(A_{0,i} = A_{n,i}\), and \(A_{n+1, i} = A_{1,i}\). Since \(|A_{j,i}| \le \rho \), together with (31) for \(m=k\), (32) gives us

$$\begin{aligned} \begin{aligned}&|(A^{k+1})_{j,i}| \\&\le | (A^{k})_{j,i-1}\Vert A_{i-1,i} | + |(A^{k})_{j,i}\Vert A_{i,i}| + |(A^{k})_{j,i+1}\Vert A_{i+1,i} | \\&\le (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)^{k}\cdot \rho ^k\cdot \mathrm{{e}}^{-C\cdot d(j,i-1)} \cdot \rho \\&\qquad + (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)^{k}\cdot \rho ^k\cdot \mathrm{{e}}^{-C\cdot d(j,i)} \cdot \rho \\&\qquad + (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)^{k}\cdot \rho ^k\cdot \mathrm{{e}}^{-C\cdot d(j,i+1)} \cdot \rho . \end{aligned} \end{aligned}$$
(33)

We claim that

$$\begin{aligned} \begin{aligned}&\mathrm{{e}}^{-C\cdot d(j,i-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot d(j,i+1)} \\&\le (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)\mathrm{{e}}^{-C\cdot d(j,i)}, \end{aligned} \end{aligned}$$
(34)

which will be proved later. Combining it with (33), we obtain

$$\begin{aligned} |(A^{k+1})_{j,i}| \le (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)^{k+1}\cdot \rho ^{k+1}\cdot \mathrm{{e}}^{-C\cdot d(j,i)}, \end{aligned}$$
(35)

this finishes the proof of (31). Claim (34) can be proved by the following case-by-case discussion. For the case \(n=2k\) with \(k\in {\mathbb {N}}^{+}\), we see \(d(j,i) \le k\) from the definition of d(ji). If \(d(j,i) < k\), we have

$$\begin{aligned}&\mathrm{{e}}^{-C\cdot d(j,i-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot d(j,i+1)} \\&= \mathrm{{e}}^{-C\cdot (d(j,i)+1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot (d(j,i)-1)} \\&= (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)\mathrm{{e}}^{-C\cdot d(j,i)}. \end{aligned}$$

Otherwise, if \(d(j,i) = k\), we have

$$\begin{aligned}&\mathrm{{e}}^{-C\cdot d(j,i-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot d(j,i+1)} \\&= \mathrm{{e}}^{-C\cdot (d(j,i)-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot (d(j,i)-1)} \\&\le (\mathrm{{e}}^{-C}+\mathrm{{e}}^{-C}+1)\mathrm{{e}}^{-C\cdot d(j,i)}. \end{aligned}$$

For \(n=2k-1\) with \(k\in {\mathbb {N}}^{+}\), we observe \(d(j,i) \le k-1\). If \(d(j,i) < k-1\), we have

$$\begin{aligned}&\mathrm{{e}}^{-C\cdot d(j,i-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot d(j,i+1)} \\&= \mathrm{{e}}^{-C\cdot (d(j,i)-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot (d(j,i)+1)} \\&= (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)\mathrm{{e}}^{-C\cdot d(j,i)}. \end{aligned}$$

Otherwise, if \(d(j,i) =k-1\), we have

$$\begin{aligned}&\mathrm{{e}}^{-C\cdot d(j,i-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot d(j,i+1)} \\&= \mathrm{{e}}^{-C\cdot (d(j,i)-1)} + \mathrm{{e}}^{-C\cdot d(j,i)} + \mathrm{{e}}^{-C\cdot d(j,i)} \\&\le (\mathrm{{e}}^{-C}+1+1)\mathrm{{e}}^{-C\cdot d(j,i)}. \end{aligned}$$

For all the cases discussed above, we always have (34).

With (31) proved, we have

$$\begin{aligned}&|(\mathrm{{e}}^{A})_{j,i}| = \Big | \sum _{m=0}^{\infty } \frac{1}{m!}\cdot (A^m)_{j,i} \Big | \le \sum _{m=0}^{\infty } \frac{1}{m!}\cdot |(A^m)_{j,i}| \\&\le \sum _{m=0}^{\infty } \frac{1}{m!}\cdot (\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)^{m}\cdot \rho ^m\cdot \mathrm{{e}}^{-C\cdot d(j,i)} \\&= \mathrm{{e}}^{(\mathrm{{e}}^{C}+\mathrm{{e}}^{-C}+1)\rho } \cdot \mathrm{{e}}^{-C\cdot d(j,i)}. \end{aligned}$$

The proof is complete. \(\square \)

Lemma 2

(Gronwall’s inequality) Given an \(n \times n\) matrix M whose entries are positive, and if the two \(n \times 1\) vectors \({\mathbf {x}}(t)\), \({\mathbf {b}}(t)\) satisfy

$$\begin{aligned} M\cdot {\mathbf {x}}(t) + {\mathbf {b}}(t) \succeq \dfrac{\mathrm{{d}}{\mathbf {x}}(t)}{\mathrm{{d}}t}, \ \text {for} \ t \in [0, T], \end{aligned}$$
(36)

(recall \(\succeq \) is entry-wise inequality, as defined in Sect. 1.4), then

$$\begin{aligned} \mathrm{{e}}^{MT}\cdot {\mathbf {x}}(0) + \int _0^T \mathrm{{e}}^{M(T-t)} \cdot \mathbf{b} (t) \mathrm{{d}}t\succeq {\mathbf {x}}(T). \end{aligned}$$
(37)

Proof

Consider an n-dimensional column vector \({\mathbf {y}}(t)\) defined as

$$\begin{aligned} \frac{\mathrm{{d}}{\mathbf {y}}(t)}{\mathrm{{d}}t} = M\cdot {\mathbf {y}}(t) + {\mathbf {b}}(t), \ \text {for} \ t \in [0, T], \end{aligned}$$

with initial value \({\mathbf {y}}(0) = {\mathbf {x}}(0) + \epsilon \cdot {\mathbf {1}}_{n \times 1}\), where \(\epsilon \) is a positive constant. Its solution can be written as

$$\begin{aligned} {\mathbf {y}}(T) = \mathrm{{e}}^{MT}\cdot ( {\mathbf {x}}(0) + \epsilon \cdot {\mathbf {1}}_{n\times 1} ) + \int _0^T \mathrm{{e}}^{M(T-t)} \cdot \mathbf{b} (t) \mathrm{{d}}t. \end{aligned}$$

Define \({\mathbf {z}}(t) = {\mathbf {y}}(t) - {\mathbf {x}}(t)\) for \(t \in [0,T]\), then \({\mathbf {z}}(0) = \epsilon \cdot {\mathbf {1}}_{n \times 1} \succeq {\mathbf {0}}_{n \times 1}\). And according to (36), we have

$$\begin{aligned} \frac{\mathrm{{d}}{\mathbf {z}}(t)}{\mathrm{{d}}t}&= \frac{\mathrm{{d}}{\mathbf {y}}(t)}{\mathrm{{d}}t} - \frac{\mathrm{{d}}{\mathbf {x}}(t)}{\mathrm{{d}}t} \\&= M\cdot {\mathbf {y}}(t) + {\mathbf {b}}(t) - \frac{\mathrm{{d}}{\mathbf {x}}(t)}{\mathrm{{d}}t} \\&\succeq M \cdot ({\mathbf {y}}(t) - {\mathbf {x}}(t)) = M \cdot {\mathbf {z}}(t). \end{aligned}$$

We define \(\tau = \inf \{ t: \text {there exists at least one index}~\)i\(, \text {such} \text {that} \ {\mathbf {z}}_{i}(t) < 0. \}\), and note that \(\tau > 0\) by continuity of \({\mathbf {z}}(t)\). According to the definition, we have \({\mathbf {z}}(t) \succeq {\mathbf {0}}_{n \times 1}\) for \(t \in [0,\tau )\). Together with that all the entries of M are positive, we have \(\frac{\mathrm{{d}}{\mathbf {z}}(t)}{\mathrm{{d}}t} \succeq M \cdot {\mathbf {z}}(t) \succeq {\mathbf {0}}_{n \times 1}\) for \(t \in [0, \tau )\). It implies that \({\mathbf {z}}(t)\) is entry-wisely nondecreasing in \([0, \tau )\), thus \({\mathbf {z}}(t) \succeq {\mathbf {z}}(0) = \epsilon \cdot {\mathbf {1}}_{n \times 1}\) for \(t \in [0, \tau )\), namely

$$\begin{aligned} \begin{aligned}&\mathrm{e}^{Mt}\cdot ( {\mathbf {x}}(0) + \epsilon \cdot {\mathbf {1}}_{n\times 1} ) + \int _0^t \mathrm{{e}}^{M(t-s)} \cdot \mathbf{b} (s) {\mathrm{d}}s - \mathbf{x}(t) \\&\quad \succeq \epsilon \cdot {\mathbf {1}}_{n \times 1}. \end{aligned} \end{aligned}$$
(38)

We show that \(\tau > T\), which can be proved by contradiction. Suppose that \(\tau \le T\). From the definition of \(\tau \) and the previous analyses, we know that \({\mathbf {z}}(t) \succeq \epsilon \cdot {\mathbf {1}}_{n \times 1}\) when \(t \in [0,\tau )\), and there exists at least one index, say i, such that \({\mathbf {z}}_{i}(\tau ) < 0\). Thus, we have \(\lim \limits _{t \rightarrow \tau ^{-}} {\mathbf {z}}_{i}(t) - {\mathbf {z}}_{i}(\tau ) \ge \epsilon > 0\). This result contradicts the continuity of \({\mathbf {z}}(t)\) for \(t \in [0,T]\) and gives us the result \(\tau > T\).

Conclusion (37) is obtained by taking \(\epsilon \rightarrow 0\) in (38), together with \(\tau > T\). \(\square \)

Lemma 3

For two \(n \times n\) matrices M and N whose entries are positive, if

$$\begin{aligned} M \preceq N, \end{aligned}$$
(39)

we have

$$\begin{aligned} \mathrm{{e}}^{M} \preceq \mathrm{{e}}^{N}. \end{aligned}$$
(40)

Proof

By mathematical induction, we firstly prove

$$\begin{aligned} M^{k} \preceq N^{k}, \quad \text {for} \ k=0,\ldots ,\infty . \end{aligned}$$
(41)

The result is true for \(k=0\), since \(M^{0} = N^{0} = {\mathbf {I}}_n\). If the result is true for \(k=K\), namely we have

$$\begin{aligned} M^{K} \preceq N^{K}. \end{aligned}$$
(42)

Then for \(k= K+1\), since

$$\begin{aligned} (M^{K+1})_{i,j}&= \sum _{s=1}^{n}(M^{K})_{i,s}M_{s,j}, \\ (N^{K+1})_{i,j}&= \sum _{s=1}^{n}(N^{K})_{i,s}N_{s,j}, \ \ \text {for} \ i,j = 1,\ldots ,n, \end{aligned}$$

the result \((M^{K+1})_{i,j} \le (N^{K+1})_{i,j}\) follows from (39) and (42).

According to the definition of matrix exponential, we have

$$\begin{aligned} \mathrm{{e}}^{M} - \mathrm{{e}}^{N} = \sum _{k=0}^{+\infty } \dfrac{1}{k!} (M^{k} - N^{k}), \end{aligned}$$

conclusion (40) naturally follows from (41). \(\square \)

Lemma 4

(Volterra’s inequality) Given an \(n \times n\) matrix M whose entries are positive, if the two \(n \times 1\) vectors \({\mathbf {x}}(t)\), \({\mathbf {b}}(t)\) satisfy

$$\begin{aligned} {\mathbf {x}}(t) \preceq {\mathbf {b}}(t) + \int _{0}^{t}M\cdot {\mathbf {x}}(s) \mathrm{{d}}s, \ \text {for} \ t \in [0, T], \end{aligned}$$
(43)

we have

$$\begin{aligned} {\mathbf {x}}(T) \preceq {\mathbf {b}}(T) + \int _{0}^{T} \mathrm{{e}}^{M(T-t)}\cdot M \cdot {\mathbf {b}}(t)\mathrm{{d}}t. \end{aligned}$$
(44)

Proof

We define \({\mathbf {y}}(t) = \int _{0}^{t} M\cdot {\mathbf {x}}(s)\mathrm{{d}}s\) for \(t \in [0, T]\), then (43) gives us

$$\begin{aligned} {\mathbf {x}}(t) \preceq {\mathbf {b}}(t) + {\mathbf {y}}(t), \ \text {for} \ t \in [0,T]. \end{aligned}$$
(45)

And we observe that

$$\begin{aligned} \dfrac{\mathrm{{d}}{\mathbf {y}}(t)}{\mathrm{{d}}t} = M\cdot {\mathbf {x}}(t) \preceq M\cdot {\mathbf {b}}(t) + M\cdot {\mathbf {y}}(t), \end{aligned}$$

with \({\mathbf {y}}(0) = 0\). According to Lemma 2, we have

$$\begin{aligned} {\mathbf {y}}(T) \preceq \int _{0}^{T} \mathrm{{e}}^{M(T-t)}\cdot M \cdot {\mathbf {b}}(t)\mathrm{{d}}t. \end{aligned}$$
(46)

Conclusion (44) follows from (45) with \(t=T\) and (46). \(\square \)

Main lemmas and their proofs

Lemma 5

Under Assumption 1, with \(\mathbf {x}^{o }(t), \mathbf {x}^{p }(t)\) for \(t \in [0,T]\) defined in Sect. 3.1, we have, for \(j=1,\ldots ,m\),

$$\begin{aligned} \begin{aligned}&{\mathbf {E}}[\Vert \mathbf {x}^\mathrm{{o}}_j(t) - \mathbf {x}_j^\mathrm{{p}}(t)\Vert ^2] \\&\le \Vert \mathbf {x}^{o }_{i_\star }-\mathbf {x}^{p }_{i_\star } \Vert ^2\cdot \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) t} \cdot \mathrm{{e}}^{-C_d\cdot d(j,i_{\star })}, \end{aligned} \end{aligned}$$
(47)

where \(C_d\) is any given positive constant, \(C_1({\mathbf {f}}, \varvec{\sigma })\) is defined as (11).

Proof

Recall that \(\mathbf {x}^{\text {o}}_j(t)\), \(\mathbf {x}_j^{\text {p}}(t)\) are solutions of

$$\begin{aligned}&\mathrm{{d}}{\mathbf {x}}_j(t) = {\mathbf {f}}_j(t, {\mathbf {x}}_{j-1}(t), {\mathbf {x}}_j(t),{\mathbf {x}}_{j+1}(t) )\mathrm{{d}}t \\&\qquad \qquad + \varvec{\sigma }_j(t,{\mathbf {x}}_j(t))\mathrm{{d}}{\mathbf {W}}_j(t), \ \text {for} \ j=1,\ldots ,m,\\&{\mathbf {x}}_{0}(t) = {\mathbf {x}}_{m}(t), \ {\mathbf {x}}_{m+1}(t) = {\mathbf {x}}_{1}(t), \ t \in [0,T], \end{aligned}$$

with corresponding initial values \(\mathbf {x}^{\text {o}}_j(0)\), \(\mathbf {x}_j^{\text {p}}(0)\), which only differ at the \(i_{\star }\)th entry. Thus, we have

$$\begin{aligned} \mathrm{{d}}({\mathbf {x}}^{\text {o}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t)) = \Delta _{{\mathbf {f}}_j}(t) \mathrm{{d}}t + \Delta _{\varvec{\sigma }_{j}}(t)\mathrm{{d}}{\mathbf {W}}_j(t), \end{aligned}$$

with

$$\begin{aligned}&\Delta _{{\mathbf {f}}_j}(t) = {\mathbf {f}}_j(t, {\mathbf {x}}^{\text {o}}_{j-1}(t), {\mathbf {x}}^{\text {o}}_j(t),{\mathbf {x}}^{\text {o}}_{j+1}(t) ) \\&\qquad \qquad - {\mathbf {f}}_j(t, {\mathbf {x}}^{\text {p}}_{j-1}(t), {\mathbf {x}}^{\text {p}}_j(t),{\mathbf {x}}^{\text {p}}_{j+1}(t) ),\\&\Delta _{\varvec{\sigma }_{j}}(t) = (\varvec{\sigma }_j(t,{\mathbf {x}}^{\text {o}}_j(t))- \varvec{\sigma }_j(t,{\mathbf {x}}^{\text {p}}_j(t))). \end{aligned}$$

According to It\(\hat{\text {o}}\)’s formula,

$$\begin{aligned} \begin{aligned}&\mathrm{{d}}\Vert {\mathbf {x}}^{\text {o}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t)\Vert ^2\\&= (2({\mathbf {x}}^{\text {o}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t))^{T}\Delta _{{\mathbf {f}}_j}(t) + \Vert \Delta _{\varvec{\sigma }_{j}}(t)\Vert ^2)\mathrm{{d}}t \\&\quad +2({\mathbf {x}}^{\text {o}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t))^{T}\Delta _{\varvec{\sigma }_{j}}(t) \mathrm{{d}}{\mathbf {W}}_j(t). \end{aligned} \end{aligned}$$
(48)

Its solution can be written as

$$\begin{aligned}&\Vert {\mathbf {x}}^{\text {o}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t)\Vert ^2 - \Vert {\mathbf {x}}^{\text {o}}_j(0) - {\mathbf {x}}^{\text {p}}_j(0)\Vert ^2 \\&= \int _{0}^{t}(2({\mathbf {x}}^{\text {o}}_j(s) - {\mathbf {x}}^{\text {p}}_j(s))^{T}\Delta _{{\mathbf {f}}_j}(s) + \Vert \Delta _{\varvec{\sigma }_{j}}(s)\Vert ^2)\mathrm{{d}}s\\&\quad + \int _{0}^{t}2({\mathbf {x}}^{\text {o}}_j(s) - {\mathbf {x}}^{\text {p}}_j(s))^{T}\Delta _{\varvec{\sigma }_{j}}(s) \mathrm{{d}}{\mathbf {W}}_j(s). \end{aligned}$$

After taking expectation with respect to both sides of above formula, and according to Assumption 1 and Cauchy–Schwartz inequality, we have

$$\begin{aligned} \begin{aligned}&{\mathbf {E}}[\Vert {\mathbf {x}}^{\text {o}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t)\Vert ^2] - \Vert {\mathbf {x}}^{\text {o}}_j(0) - {\mathbf {x}}^{\text {p}}_j(0)\Vert ^2 \\&= \int _{0}^{t}(2({\mathbf {x}}^{\text {o}}_j(s) - {\mathbf {x}}^{\text {p}}_j(s))^T\Delta _{{\mathbf {f}}_j}(s) + \Vert \Delta _{\varvec{\sigma }_{j}}(s)\Vert ^2)\mathrm{{d}}s\\&\le \int _{0}^{t}{\mathbf {E}}[\Vert {\mathbf {x}}^{\text {o}}_j(s) - {\mathbf {x}}^{\text {p}}_j(s)\Vert ^2]\mathrm{{d}}s \\&\quad + \int _{0}^{t}{\mathbf {E}}[\Vert \Delta _{{\mathbf {f}}_j}(s)\Vert ^2 + \Vert \Delta _{\varvec{\sigma }_{j}}(s)\Vert ^2]\mathrm{{d}}s\\&\le C_{{\mathbf {f}}}\int _{0}^{t}{\mathbf {E}}[\Vert {\mathbf {x}}^{\text {o}}_{j-1}(s) - {\mathbf {x}}^{\text {p}}_{j-1}(s)\Vert ^2]\mathrm{{d}}s \\&\quad + (C_{{\mathbf {f}}} +C_{\varvec{\sigma }} + 1) \int _{0}^{t}{\mathbf {E}}[\Vert {\mathbf {x}}^{\text {o}}_j(s) - {\mathbf {x}}^{\text {p}}_j(s)\Vert ^2]\mathrm{{d}}s\\&\quad + C_{{\mathbf {f}}} \int _{0}^{t}{\mathbf {E}}[\Vert {\mathbf {x}}^{\text {o}}_{j+1}(s) - {\mathbf {x}}^{\text {p}}_{j+1}(s)\Vert ^2]\mathrm{{d}}s. \end{aligned} \end{aligned}$$
(49)

Equivalently, we can write (49) into the following vector form

$$\begin{aligned} \varvec{\Delta }(t) \preceq \int _{0}^{t}M\cdot \varvec{\Delta }(s)\mathrm{{d}}s + \varvec{\Delta }(0), \end{aligned}$$

where \(\varvec{\Delta }(t)\) is an \(m \times 1\) vector whose jth entry is \({\mathbf {E}}[\Vert \mathbf {x}^{\text {o}}_j(t) - \mathbf {x}^{\text {p}}_j(t)\Vert ^2] \); M is an \(m \times m\) matrix with

$$\begin{aligned}&M_{1,1} =C_{\varvec{\sigma }} + C_{{\mathbf {f}}}+1, M_{1,2} = M_{1,n} =C_{{\mathbf {f}}}, \\&M_{j,j} =C_{\varvec{\sigma }}+ C_{{\mathbf {f}}}+1, M_{j,j+1} = M_{j,j-1} = C_{{\mathbf {f}}},\\&\qquad \text {for} \ j=2,\ldots ,m-1,\\&M_{m,m} =C_{\varvec{\sigma }}+ C_{{\mathbf {f}}}+1, M_{m,1} = M_{m,m-1} = C_{{\mathbf {f}}}, \end{aligned}$$

and other entries are 0; moreover, we have \(\varvec{\Delta }_{i_{\star }}(0) = \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2\) and other entries of \(\varvec{\Delta }(0)\) are 0. According to Lemma 4, we have

$$\begin{aligned} \varvec{\Delta }(t) \preceq \varvec{\Delta }(0) + \int _{0}^{t} \mathrm{{e}}^{M(t-s)}\cdot M \cdot \varvec{\Delta }(0)\mathrm{{d}}s. \end{aligned}$$

We observe that for the m-dimensional column vector \((M \cdot \varvec{\Delta }(0))\), its \(i_{\star }-1, i_{\star }, i_{\star }+1\)th entries are \(C_{{\mathbf {f}}}\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2, (C_{{\mathbf {f}}} + C_{\varvec{\sigma }}+1)\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 , C_{{\mathbf {f}}}\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \), respectively, while other entries are 0. Together with Lemma 1 for \(C=C_d\), \(\max \limits _{j,i = 1,2,\ldots ,m} |M_{j,i}| = (C_{{\mathbf {f}}}+C_{\varvec{\sigma }}+1)\), (34) and the definition of \(C({\mathbf {f}}, \varvec{\sigma })= (\mathrm{{e}}^{C_d}+\mathrm{{e}}^{-C_d}+1)(C_{{\mathbf {f}}}+C_{\varvec{\sigma }}+1)\), we have

$$\begin{aligned}&\varvec{\Delta }_j(t) \\&\le \varvec{\Delta }_j(0) + \int _{0}^{t}|(\mathrm{{e}}^{M(t-s)})_{j,i_{\star }-1}|\cdot (M \cdot \varvec{\Delta }(0) )_{i_{\star }-1,1}\mathrm{{d}}s \\&\quad + \int _{0}^{t}|(\mathrm{{e}}^{M(t-s)})_{j,i_{\star }}|\cdot (M \cdot \varvec{\Delta }(0))_{i_{\star },1}\mathrm{{d}}s \\&\quad + \int _{0}^{t}|(\mathrm{{e}}^{M(t-s)})_{j,i_{\star }+1}|\cdot (M\cdot \varvec{\Delta }(0))_{i_{\star }+1,1}\mathrm{{d}}s \\&\le \varvec{\Delta }_j(0) + (C_{{\mathbf {f}}}+ C_{\varvec{\sigma }}+1)\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \int _{0}^{t} \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (t-s) }\mathrm{{d}}s \\&\quad \cdot (\mathrm{{e}}^{-C_d\cdot d(j,i_{\star }-1)} + \mathrm{{e}}^{-C_d\cdot d(j,i_{\star })} + \mathrm{{e}}^{-C_d\cdot d(j,i_{\star }+1)}) \\&\le \varvec{\Delta }_j(0) + \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \cdot (\mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })t }-1) \mathrm{{e}}^{-C_d\cdot d(j,i_{\star })} \\&\le \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2\cdot \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })\cdot t } \cdot \mathrm{{e}}^{-C_d\cdot d(j,i_{\star })}, \end{aligned}$$

where the last inequality is derived by considering two cases: when \(j=i_{\star }\), \( \varvec{\Delta }_j(0) = \Vert \mathbf {x}^\mathrm{{o}}_{i_\star }-\mathbf {x}^\mathrm{{p}}_{i_\star } \Vert ^2\) and \(d(j,i_{\star }) = 0\), the result is true; when \(j \ne i_{\star }\), \( \varvec{\Delta }_j(0) = 0\), the result can also be verified. Recall that \(\varvec{\Delta }_j(t) = {\mathbf {E}}[\Vert \mathbf {x}^{\text {o}}_j(t) - \mathbf {x}^{\text {p}}_j(t)\Vert ^2] \), the proof is complete. \(\square \)

Lemma 6

Under Assumption 1, with \(\mathbf {x}^{o }(t)\), \(\mathbf {x}^{l }(t)\), \(\mathbf {x}^{p }(t)\) defined for \(t \in [0,T]\) in Sects. 3.1 and 3.2, we have, for \(j=1,\ldots ,m\),

$$\begin{aligned} \begin{aligned}&{\mathbf {E}}[ \Vert \mathbf{x}^\mathrm{l}_{ j}(t) - \mathbf {x}_{ j}^\mathrm{p}(t)\Vert ^2 ]\\&\le C_2({\mathbf {f}}, \varvec{\sigma }) \Vert \mathbf {x}^{o }_{i_\star }-\mathbf {x}^{p }_{i_\star } \Vert ^2 \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma }) t} \mathrm{{e}}^{-C_d\cdot (L+1)}, \end{aligned} \end{aligned}$$
(50)

where \(C_d\) is any given positive constant, \(C_1({\mathbf {f}}, \varvec{\sigma })\), \(C_2({\mathbf {f}}, \varvec{\sigma })\) are defined in (11) and (14).

Proof

For simplicity, we consider \(L+2\le i_{\star } \le n-L-1\), this does not sacrifice any generality because we can always rotate the index to make it true. So \(B_{i_{\star }} = \{ i_{\star }-L, \ldots , i_{\star }+L\}\).

Suppose \(j \in B^c_{i_{\star }}\), since \(\mathbf {x}^{\text {l}}_j(t) = \mathbf {x}^{\text {o}}_j(t)\) and \(d(j,i_{\star })\ge L+1\), (50) follows from the conclusion of Lemma 5.

Suppose \(j \in B_{i_{\star }}\), we observe that both \(\mathbf {x}_j^{\text {l}}(t)\) and \(\mathbf {x}_j^{\text {p}}(t)\) follow the evolutionary system

$$\begin{aligned}&\mathrm{{d}}\mathbf {x}_j(t) ={\mathbf {f}}_{j}(t,\mathbf {x}_{j-1}(t), \mathbf {x}_j(t),\mathbf {x}_{j+1}(t) )\mathrm{{d}}t \\&\qquad \qquad + \varvec{\sigma }_{j}(t,\mathbf {x}_j(t))\mathrm{{d}}{\mathbf {W}}_j(t), \ t \in [0,T], \end{aligned}$$

with initial value \((\mathbf{x}^{\text {o}}_{i_{\star }-L}, \ldots ,\mathbf{x}^{\text {o}}_{i_{\star }-1}, \mathbf{x}^{\text {p}}_{i_{\star }}, \mathbf{x}^{\text {o}}_{i_{\star }+1},\ldots , \mathbf{x}^{\text {o}}_{i_{\star }+L} )^{T}\). But when \(j\in B_{i_{\star }}^{c}\), it is restricted that \(\mathbf {x}_{j}^{\text {l}}(t) = \mathbf {x}_{j}^{\text {o}}(t)\), where \(\mathbf {x}^{\text {o}}(t)\) is the solution of (4) with initial value \((\mathbf {x}_1^{\text {o}}, \ldots , \mathbf {x}_m^{\text {o}})^{T}\). Recall that \(\mathbf {x}^{\text {p}}(t)\) also solves (4), but with a locally perturbed initial value, which can be written as \((\mathbf {x}_1^{\text {o}}, \ldots , \mathbf {x}_{i_{\star -1}}^{\text {o}}, \mathbf {x}_{i_\star }^{\text {p}}, \mathbf {x}_{i_{\star }+1}^{\text {o}} , \ldots , \mathbf {x}_m^{\text {o}})^{T}\). Thus, for \(i_{\star }-L \le j \le i_{\star }+L\), we have

$$\begin{aligned} \mathrm{{d}}({\mathbf {x}}^{\text {l}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t)) = \Delta _{{\mathbf {f}}_j}(t) \mathrm{{d}}t + \Delta _{\varvec{\sigma }_{j}}(t)\mathrm{{d}}{\mathbf {W}}_j(t), \end{aligned}$$

with

$$\begin{aligned}&\Delta _{{\mathbf {f}}_j}(t) = {\mathbf {f}}_j(t, {\mathbf {x}}^{\text {l}}_{j-1}(t), {\mathbf {x}}^{\text {l}}_j(t),{\mathbf {x}}^{\text {l}}_{j+1}(t) ) \\&\qquad \qquad - {\mathbf {f}}_j(t, {\mathbf {x}}^{\text {p}}_{j-1}(t), {\mathbf {x}}^{\text {p}}_j(t),{\mathbf {x}}^{\text {p}}_{j+1}(t) ),\\&\Delta _{\varvec{\sigma }_{j}}(t) = (\varvec{\sigma }_j(t,{\mathbf {x}}^{\text {l}}_j(t))- \varvec{\sigma }_j(t,{\mathbf {x}}^{\text {p}}_j(t))). \end{aligned}$$

According to It\(\hat{\text {o}}\)’s formula,

$$\begin{aligned} \begin{aligned}&\mathrm{{d}}\Vert {\mathbf {x}}^{\text {l}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t)\Vert ^2\\&= (2({\mathbf {x}}^{\text {l}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t))^{T}\Delta _{{\mathbf {f}}_j}(t) + \Vert \Delta _{\varvec{\sigma }_{j}}(t)\Vert ^2)\mathrm{{d}}t \\&\quad +2({\mathbf {x}}^{\text {l}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t))^{T}\Delta _{\varvec{\sigma }_{j}}(t) \mathrm{{d}}{\mathbf {W}}_j(t). \end{aligned} \end{aligned}$$
(51)

Result (51) is similar to (48), solving it and according to (49), we obtain

$$\begin{aligned} \begin{aligned}&{\mathbf {E}}[({\mathbf {x}}^{\text {l}}_j(t) - {\mathbf {x}}^{\text {p}}_j(t))^2] - ({\mathbf {x}}^{\text {l}}_j(0) - {\mathbf {x}}^{\text {p}}_j(0))^2 \\&\le C_{{\mathbf {f}}}\int _{0}^{t}{\mathbf {E}}[({\mathbf {x}}^{\text {l}}_{j-1}(s) - {\mathbf {x}}^{\text {p}}_{j-1}(s))^2]\mathrm{{d}}s \\&\quad + (C_{{\mathbf {f}}} +C_{\varvec{\sigma }} + 1) \int _{0}^{t}{\mathbf {E}}[({\mathbf {x}}^{\text {l}}_j(s) - {\mathbf {x}}^{\text {p}}_j(s))^2]\mathrm{{d}}s \\&\quad + C_{{\mathbf {f}}} \int _{0}^{t}{\mathbf {E}}[({\mathbf {x}}^{\text {l}}_{j+1}(s) - {\mathbf {x}}^{\text {p}}_{j+1}(s))^2]\mathrm{{d}}s. \end{aligned} \end{aligned}$$
(52)

We define an \((2L+1)\times 1\) vector \(\varvec{\Delta }(t) = \big ({\mathbf {E}}[\Vert \mathbf {x}^{\text {l}}_{i_{\star }-L}(t) - \mathbf {x}_{i_{\star }-L}^{\text {p}}(t)\Vert ^2], \ldots , {\mathbf {E}}[\Vert \mathbf {x}^{\text {l}}_{i_{\star }+L}(t) - \mathbf {x}_{i_{\star }+L}^{\text {p}}(t)\Vert ^2] \big )^{T}\), whose jth element \(\varvec{\Delta }_j(t)= {\mathbf {E}}[\Vert \mathbf {x}^{\text {l}}_{i_{\star }-L+j-1}(t) - \mathbf {x}_{i_{\star }-L+j-1}^{\text {p}}(t)\Vert ^2]\), and observe that \(\varvec{\Delta }(0) = {\mathbf {0}}_{(2L+1)\times 1}\). Since \(\mathbf {x}^{\text {l}}_{i_{\star }-L-1}(t) = \mathbf {x}^{\text {o}}_{i_{\star }-L-1}(t)\) and \(\mathbf {x}^{\text {l}}_{i_{\star }+L+1}(t) = \mathbf {x}^{\text {o}}_{i_{\star }+L+1}(t)\), we can write (52) as the following vector form

$$\begin{aligned} \varvec{\Delta }(t) \preceq \int _{0}^{t}M \cdot \varvec{\Delta }(s) \mathrm{{d}}s + \varvec{\Delta }(t), \end{aligned}$$
(53)

where M is a \((2L+1) \times (2L+1)\) tridiagonal matrix with

$$\begin{aligned}&M_{1,1} = C_{{\mathbf {f}}}+C_{\varvec{\sigma }}+1, \ M_{1,2} =C_{{\mathbf {f}}}, \\&M_{j,j} =C_{{\mathbf {f}}}+C_{\varvec{\sigma }}+1, \ M_{j,j+1} =M_{j,j-1} =C_{{\mathbf {f}}},\\&\qquad \ \text {for} \ 2 \le j \le 2L \\&M_{2L+1,2L} = C_{{\mathbf {f}}}, \ M_{2L+1,2L+1} = C_{{\mathbf {f}}}+C_{\varvec{\sigma }}+ 1, \end{aligned}$$

and \(\varvec{\Delta }(t)\) is an \((2L+1)\)-dimensional column vector with \(\varvec{\Delta }_{1}(t) =C_{{\mathbf {f}}}\int _{0}^{t} {\mathbf {E}}[\Vert \mathbf {x}^{\text {o}}_{i_{\star }-L-1}(s) - \mathbf {x}_{i_{\star }-L-1}^{\text {p}}(s)\Vert ^2]\mathrm{{d}}s\), \(\varvec{\Delta }_{2L+1}(t) = C_{{\mathbf {f}}}\int _{0}^{t}{\mathbf {E}}[\Vert \mathbf {x}^{\text {o}}_{i_{\star }+L+1}(s) - \mathbf {x}_{i_{\star }+L+1}^{\text {p}}(s)\Vert ^2]\mathrm{{d}}s\), while other entries are 0. According to Lemma 5, we have

$$\begin{aligned} \begin{aligned}&\max _{j=1,\ldots ,2L+1}{\{|\varvec{\Delta }_{j}(t)|\}} \\&\quad \le C_{{\mathbf {f}}} \int _{0}^{t}\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })\cdot s} \mathrm{{e}}^{-C_d(L+1)}\mathrm{{d}}s \\&\quad \le \dfrac{C_{{\mathbf {f}}}}{C_1({\mathbf {f}}, \varvec{\sigma })}\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })\cdot t} \mathrm{{e}}^{-C_d(L+1)}. \end{aligned} \end{aligned}$$
(54)

By Lemma 4, under (53), we have

$$\begin{aligned} \varvec{\Delta }(t) \preceq \varvec{\Delta }(t) + \int _{0}^{t} \mathrm{{e}}^{M(t-s)}\cdot M \cdot \varvec{\Delta }(s)\mathrm{{d}}s. \end{aligned}$$
(55)

For the \((2L+1)\times 1\) vector \((M \cdot \varvec{\Delta }(t))\), we observe

$$\begin{aligned}&(M \cdot \varvec{\Delta }(t))_{1} = (C_{{\mathbf {f}}} + C_{\varvec{\sigma }}+1)\varvec{\Delta }_{1}(t) , (M \cdot \varvec{\Delta }(t))_{2} =C_{{\mathbf {f}}}\varvec{\Delta }_{1}(t),\\&(M \cdot \varvec{\Delta }(t))_{2L} = C_{{\mathbf {f}}}\varvec{\Delta }_{2L+1}(t) , \\&(M \cdot \varvec{\Delta }(t))_{2L+1} =(C_{{\mathbf {f}}} + C_{\varvec{\sigma }}+1)\varvec{\Delta }_{2L+1}(t), \end{aligned}$$

while other entries are 0. Together with (54), we have

$$\begin{aligned} \begin{aligned}&\max _{j=1,\ldots ,2L+1}{\{|(M \cdot \varvec{\Delta }(t))_{j,1}|\}} \\&\quad \le \dfrac{C_{{\mathbf {f}}}}{C_1({\mathbf {f}}, \varvec{\sigma })}(C_{{\mathbf {f}}} + C_{\varvec{\sigma }}+1)\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })\cdot t} \mathrm{{e}}^{-C_d(L+1)}. \end{aligned} \end{aligned}$$
(56)

Since \(\max \limits _{j,i=1,\ldots ,2L+1} M_{j,i} = (C_{{\mathbf {f}}} + C_{\varvec{\sigma }}+1)\), according to Lemma 3 and the conclusion of Lemma 1 with \(C=C_d\), we have

$$\begin{aligned} \begin{aligned} |(\mathrm{{e}}^{M(t-s)})_{j,i}| \le |(\mathrm{{e}}^{Mt})_{j,i}| \le \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })t}. \end{aligned} \end{aligned}$$
(57)

From (55), together with (54), (56) and (57), we obtain

$$\begin{aligned}&\varvec{\Delta }_j(t) \\&\quad \le |\varvec{\Delta }_j(t)| + \sum _{k=1,2,2L,2L+1 }\int _{0}^{t}|(\mathrm{{e}}^{M(t-s)})_{j,k}| |(M\cdot \varvec{\Delta }(s))_{k,1}|\mathrm{{d}}s\\&\quad \le \max _{j=1,\ldots ,2L+1}{\{|\varvec{\Delta }_{j}(t)|\}} \\&\qquad + 4\mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })t}\int _{0}^{t}\max _{j=1,\ldots ,2L+1}{\{|(M \cdot \varvec{\Delta }(s))_{j,1}|\}}\mathrm{{d}}s \\&\quad \le \dfrac{C_{{\mathbf {f}}}}{C_1({\mathbf {f}}, \varvec{\sigma })}\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })\cdot t} \mathrm{{e}}^{-C_d(L+1)} \\&\qquad \cdot (1+ 4\int _{0}^{t}(C_{{\mathbf {f}}} + C_{\varvec{\sigma }}+1) \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })\cdot s}\mathrm{{d}}s ) \\&\quad \le \dfrac{C_{{\mathbf {f}}}}{C_1({\mathbf {f}}, \varvec{\sigma })}\Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })\cdot t} \mathrm{{e}}^{-C_d(L+1)} \\&\qquad \cdot (1+\dfrac{4(\mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })t}-1)}{\mathrm{{e}}^{C_d}+\mathrm{{e}}^{-C_d}+1}) \\&\quad \le \dfrac{2C_{{\mathbf {f}}}}{C_1({\mathbf {f}}, \varvec{\sigma })} \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma })\cdot t} \mathrm{{e}}^{-C_d(L+1)}, \end{aligned}$$

where the last inequality is derived by using \(\mathrm{{e}}^{C_d}+\mathrm{{e}}^{-C_d}+1 \ge 2\). Recall the definition of \(\varvec{\Delta }_j(t)\), the proof is complete. \(\square \)

Lemma 7

Under the same settings in Lemma 6, for any given \(\epsilon >0\), if the local domain radius L satisfies

$$\begin{aligned} L \ge \dfrac{\log {\left( \dfrac{\epsilon }{C_2({\mathbf {f}}, \varvec{\sigma }) \Vert \mathbf {x}^{o }_{i_\star }-\mathbf {x}^{p }_{i_\star }\Vert ^2} \right) }}{-C_d} + \dfrac{2C_1({\mathbf {f}}, \varvec{\sigma })}{C_d}\cdot T, \end{aligned}$$

then \({\mathbf {E}}[ \Vert {\mathbf {x}}^\mathrm{{l}}_j(t) - \mathbf {x}_j^\mathrm{{p}}(t)\Vert ^2 ]\le \epsilon \) for all \(t\le T\) and \(j=1,\ldots ,m\).

Proof

According to Lemma 6, it is equivalent for us to solve

$$\begin{aligned} C_2({\mathbf {f}}, \varvec{\sigma }) \cdot \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star } \Vert ^2 \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma })\cdot t} \mathrm{{e}}^{-C_d(L+1)} \le \epsilon \end{aligned}$$

for \(t \in [0,T]\), which can be obtained by solving

$$\begin{aligned}&\mathrm{{e}}^{-C_d(L+1)} \le \dfrac{\epsilon }{C_2({\mathbf {f}}, \varvec{\sigma })\cdot \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star }\Vert ^2 \cdot \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma })\cdot T } }. \end{aligned}$$

After taking log for both sides, we obtain that the above result is true if

$$\begin{aligned} L \ge \dfrac{\log {\left( \dfrac{\epsilon }{C_2({\mathbf {f}}, \varvec{\sigma }) \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star }\Vert ^2} \right) }}{-C_d} + \dfrac{2C_1({\mathbf {f}}, \varvec{\sigma })}{C_d}\cdot T. \end{aligned}$$

The proof is finished. \(\square \)

Lemma 8

Under Assumption 1, with \(\tilde{\mathbf {x}}^{o }_j(ih)\), \(\tilde{\mathbf {x}}^{p }_j(ih)\), for \(j=1,\ldots ,m\) and \(i=0,\ldots ,T/h\), defined in Sect. 3.3, we have

$$\begin{aligned} \begin{aligned}&\Vert \tilde{\mathbf {x}}^{o }_j(ih) - \tilde{\mathbf {x}}^{p }_j(ih) \Vert ^2\\&\quad \le \mathrm{{e}}^{-C_d\cdot d(j,i_{\star })}\cdot \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma })( 1+h)ih} \Vert \mathbf {x}^{o }_{i_\star }-\mathbf {x}^{p }_{i_\star }\Vert ^2, \end{aligned} \end{aligned}$$
(58)

where \(C_d\) is any given positive constant, \(C_1({\mathbf {f}}, \varvec{\sigma })\) is defined in (11).

Proof

Since \(\tilde{{\mathbf {x}}}^{\text {o}}_{j}(ih)\) are obtained by iterating

$$\begin{aligned} \begin{aligned} \tilde{{\mathbf {x}}}^{\text {o}}_j((i+1)h)&= \tilde{{\mathbf {x}}}^{\text {o}}_j(ih) +\varvec{\sigma }_j(ih,\tilde{{\mathbf {x}}}^{\text {o}}_j(ih))\sqrt{h}W_{i,j} \\&\quad + {\mathbf {f}}_j(ih,\tilde{{\mathbf {x}}}^{\text {o}}_{j-1}(ih), \tilde{{\mathbf {x}}}^{\text {o}}_j(ih),\tilde{{\mathbf {x}}}^{\text {o}}_{j+1}(ih) )h, \end{aligned} \end{aligned}$$
(59)

with initial value \(\tilde{{\mathbf {x}}}^{\text {o}}_{j}(0)={\mathbf {x}}^{\text {o}}_{j}\). Comparing it with (16), we have

$$\begin{aligned} \begin{aligned}&\tilde{{\mathbf {x}}}^{\text {o}}_j((i+1)h) - \tilde{{\mathbf {x}}}^{\text {p}}_j((i+1)h)\\&\quad = \tilde{{\mathbf {x}}}^{\text {o}}_j(ih) - \tilde{{\mathbf {x}}}^{\text {p}}_j(ih) +{\tilde{\Delta }}_{\varvec{\sigma }_j} (ih) \sqrt{h}W_{i,j} + {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih) h, \end{aligned} \end{aligned}$$
(60)

with

$$\begin{aligned} {\tilde{\Delta }}_{\varvec{\sigma }_j} (ih)&= \varvec{\sigma }_j(ih,\tilde{{\mathbf {x}}}^{\text {o}}_j(ih)) - \varvec{\sigma }_j(ih,\tilde{{\mathbf {x}}}^{\text {p}}_j(ih)),\\ {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih)&= {\mathbf {f}}_j(ih,\tilde{{\mathbf {x}}}^{\text {o}}_{j-1}(ih), \tilde{{\mathbf {x}}}^{\text {o}}_j(ih),\tilde{{\mathbf {x}}}^{\text {o}}_{j+1}(ih) ) \\&\quad - {\mathbf {f}}_j(ih,\tilde{{\mathbf {x}}}^{\text {p}}_{j-1}(ih), \tilde{{\mathbf {x}}}^{\text {p}}_j(ih),\tilde{{\mathbf {x}}}^{\text {p}}_{j+1}(ih) ). \end{aligned}$$

Since \(\tilde{\mathbf {x}}_j^{\text {o}}(ih)\) and \(\tilde{\mathbf {x}}_j^{\text {p}}(ih)\) are independent of \(W_{i,j}\), together with Assumption 1, and Cauchy–Schwartz inequality, from (60), we have

$$\begin{aligned} \begin{aligned}&{\mathbf {E}}[\Vert \tilde{\mathbf {x}}_j^{\text {o}}((i+1)h) - \tilde{\mathbf {x}}_j^{\text {p}}((i+1)h)\Vert ^2] \\&\quad = {\mathbf {E}}[\Vert \tilde{\mathbf {x}}_j^{\text {o}}(ih) - \tilde{\mathbf {x}}_j^{\text {p}}(ih) \Vert ^2] + {\mathbf {E}}[\Vert {\tilde{\Delta }}_{\varvec{\sigma }_j} (ih) \sqrt{h}W_{i,j} \Vert ^2] \\&\qquad + {\mathbf {E}}[\Vert {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih) h\Vert ^2] + 2{\mathbf {E}}[(\tilde{\mathbf {x}}_j^{\text {o}}(ih) - \tilde{\mathbf {x}}_j^{\text {p}}(ih))^{T} {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih) h] \\&\quad \le {\mathbf {E}}[\Vert \tilde{\mathbf {x}}_j^{\text {o}}(ih) - \tilde{\mathbf {x}}_j^{\text {p}}(ih) \Vert ^2] + {\mathbf {E}}[\Vert {\tilde{\Delta }}_{\varvec{\sigma }_j} (ih)\Vert ^2] h\\&\qquad + {\mathbf {E}}[\Vert {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih) \Vert ^2] h^2 \\&\qquad + ( {\mathbf {E}}[\Vert \tilde{\mathbf {x}}_j^{\text {o}}(ih) - \tilde{\mathbf {x}}_j^{\text {p}}(ih) \Vert ^2] + {\mathbf {E}}[\Vert {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih) \Vert ^2] )h \\&\quad \le (C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2 ){\mathbf {E}}[(\tilde{\mathbf {x}}^{\text {o}}_{j-1}(ih) - \tilde{\mathbf {x}}^{\text {p}}_{j-1}(ih) )^2] +\\&\qquad (1+(C_{\varvec{\sigma }} + C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2) {\mathbf {E}}[(\tilde{\mathbf {x}}_j^{\text {o}}(ih) - \tilde{\mathbf {x}}_j^{\text {p}}(ih) )^2] \\&\qquad +(C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2 ) {\mathbf {E}}[\Vert \tilde{\mathbf {x}}^{\text {o}}_{j+1}(ih) - \tilde{\mathbf {x}}^{\text {p}}_{j+1}(ih) \Vert ^2]. \end{aligned} \end{aligned}$$
(61)

Equivalently, we have

$$\begin{aligned} \varvec{\Delta }((i+1)h) \preceq ({\mathbf {I}}_{m} + M) \cdot \varvec{\Delta }(ih), \end{aligned}$$
(62)

where \(\varvec{\Delta }(ih)\) is defined to be an \(m\times 1\) vector with its jth entry \(\varvec{\Delta }_j(ih) = {\mathbf {E}}[\Vert \tilde{\mathbf {x}}_j^{\text {o}}(ih) - \tilde{\mathbf {x}}_j^{\text {p}}(ih)\Vert ^2] \) for \(i=0,\ldots ,T/h\); M is an \(m \times m\) matrix with

$$\begin{aligned}&M_{1,1} =(C_{\varvec{\sigma }} + C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2, M_{1,2} =C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, \\&M_{1,m} =C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, \\&M_{j,j-1} =C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, M_{j,j} =(C_{\varvec{\sigma }}+ C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2, \\&M_{j,j+1} = C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, \quad \text {for} \ j=2,\ldots ,m-1,\\&M_{m,1} = C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2,M_{m,m-1} = C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, \\&M_{m,m} =(C_{\varvec{\sigma }}+ C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2, \end{aligned}$$

and other entries are 0. After iterating (62) for i times, we have

$$\begin{aligned} \varvec{\Delta }(ih) \preceq ({\mathbf {I}}_{m} + M)^{i} \cdot \varvec{\Delta }(0), \end{aligned}$$
(63)

and for \(\varvec{\Delta }(0)\), we know \(\varvec{\Delta }_{i_{\star }}(0) = \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star }\Vert ^2\) and other entries are 0. We observe

$$\begin{aligned} {\mathbf {I}}_{m}+M \preceq \mathrm{{e}}^{M}, \end{aligned}$$
(64)

together with Lemma 1, \(\max \limits _{j,k = 1,2,\ldots ,m} |(iM)_{j,k}| = ((C_{\varvec{\sigma }}+C_{{\mathbf {f}}}+1) + C_{{\mathbf {f}}}h)ih\) and (41), we have, for \(j=1,\ldots ,m\),

$$\begin{aligned} \begin{aligned} |(({\mathbf {I}}_{m}+ M)^{i})_{j,i_{\star }}|&\le |((\mathrm{{e}}^{M})^i)_{j,i_{\star }}| = |(\mathrm{{e}}^{iM})_{j,i_{\star }}| \\&\le \mathrm{{e}}^{-C_d\cdot d(j,i_{\star })}\cdot \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)ih}. \end{aligned} \end{aligned}$$
(65)

Substituting (65) into (63) results in (58). \(\square \)

Lemma 9

Under Assumption 1, with \(\tilde{\mathbf {x}}^{o }_j(ih)\)\(\tilde{\mathbf {x}}^{l }_j(ih)\), \(\tilde{\mathbf {x}}^{p }_j(ih)\), for \(j=1,\ldots ,m\) and \(i=0,\ldots ,T/h\), defined in Sect. 3.3, we have

$$\begin{aligned} \begin{aligned}&{\mathbf {E}}[\Vert \tilde{\mathbf {x}}^\mathrm{{l}}_j(ih) - \tilde{\mathbf {x}}_j^\mathrm{{p}}(ih)\Vert ^2] \\&\le C_2({\mathbf {f}}, \varvec{\sigma }) \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)ih} \mathrm{{e}}^{-C_d(L+1)} \Vert \mathbf {x}^{o }_{i_\star }-\mathbf {x}^{p }_{i_\star }\Vert ^2, \end{aligned} \end{aligned}$$
(66)

where \(C_d\) is any given positive constant, \(C_1({\mathbf {f}}, \varvec{\sigma })\) and \(C_2({\mathbf {f}}, \varvec{\sigma })\) are defined in (11) and (14).

Proof

Suppose \(j \in B^c_{i_{\star }}\), since \(d(j,i_{\star })\ge L+1\), and \(\tilde{\mathbf {x}}^{\text {l}}_j(ih) = \tilde{\mathbf {x}}^{\text {o}}_j(ih) \), the result follows from Lemma 8.

Suppose \(j \in B_{i_{\star }} \), comparing (16) and (18), we have

$$\begin{aligned} \begin{aligned}&\tilde{{\mathbf {x}}}^{\text {l}}_j((i+1)h) - \tilde{{\mathbf {x}}}^{\text {p}}_j((i+1)h)\\&\quad = \tilde{{\mathbf {x}}}^{\text {l}}_j(ih) - \tilde{{\mathbf {x}}}^{\text {p}}_j(ih) +{\tilde{\Delta }}_{\varvec{\sigma }_j} (ih) \sqrt{h}W_{i,j} + {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih) h, \end{aligned} \end{aligned}$$
(67)

with

$$\begin{aligned} {\tilde{\Delta }}_{\varvec{\sigma }_j} (ih)&= \varvec{\sigma }_j(ih,\tilde{{\mathbf {x}}}^{\text {l}}_j(ih)) - \varvec{\sigma }_j(ih,\tilde{{\mathbf {x}}}^{\text {p}}_j(ih)),\\ {\tilde{\Delta }}_{{\mathbf {f}}_j} (ih)&= {\mathbf {f}}_j(ih,\tilde{{\mathbf {x}}}^{\text {l}}_{j-1}(ih), \tilde{{\mathbf {x}}}^{\text {l}}_j(ih),\tilde{{\mathbf {x}}}^{\text {l}}_{j+1}(ih) ) \\&\quad - {\mathbf {f}}_j(ih,\tilde{{\mathbf {x}}}^{\text {p}}_{j-1}(ih), \tilde{{\mathbf {x}}}^{\text {p}}_j(ih),\tilde{{\mathbf {x}}}^{\text {p}}_{j+1}(ih) ). \end{aligned}$$

And it is required that \(\tilde{\mathbf {x}}_{k}^{\text {l}}(ih) = \tilde{\mathbf {x}}_{k}^{\text {o}}(ih)\) for \(k\in B_{i_{\star }}^{c}\) during the evolution of \(\tilde{{\mathbf {x}}}^{\text {l}}(ih)\). Since (67) is similar to (60), according to (61), we have

$$\begin{aligned}&{\mathbf {E}}[\Vert \tilde{\mathbf {x}}_j^{\text {l}}((i+1)h) - \tilde{\mathbf {x}}_j^{\text {p}}((i+1)h)\Vert ^2] \\&\le (C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2 ){\mathbf {E}}[(\tilde{\mathbf {x}}^{\text {l}}_{j-1}(ih) - \tilde{\mathbf {x}}^{\text {p}}_{j-1}(ih) )^2] \\&\quad + (1+(C_{\varvec{\sigma }} + C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2) {\mathbf {E}}[(\tilde{\mathbf {x}}_j^{\text {l}}(ih) - \tilde{\mathbf {x}}_j^{\text {p}}(ih) )^2] \\&\quad +(C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2 ) {\mathbf {E}}[\Vert \tilde{\mathbf {x}}^{\text {l}}_{j+1}(ih) - \tilde{\mathbf {x}}^{\text {p}}_{j+1}(ih) \Vert ^2]. \end{aligned}$$

Namely, for the \((2L+1) \times 1\) vector \(\varvec{\Delta }(ih)\) whose jth entry \(\varvec{\Delta }_j(ih) = {\mathbf {E}}[ \Vert \tilde{\mathbf {x}}^{\text {l}}_{i_{\star }-L+j-1}(ih) - \tilde{\mathbf {x}}^{\text {p}}_{i_{\star }-L+j-1}(ih)\Vert ^2]\) for \(j=1,\ldots ,2L+1\), we have

$$\begin{aligned} \varvec{\Delta }((i+1)h) \preceq ({\mathbf {I}}_{(2L+1)} + M) \cdot \varvec{\Delta }(ih) + \varvec{\Delta }(ih), \end{aligned}$$
(68)

where M is a \((2L+1)\) by \((2L+1)\) tridiagonal matrix with

$$\begin{aligned}&M_{1,1} = (C_{\varvec{\sigma }} + C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2, M_{1,2} =C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, \\&M_{j,j-1} =C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, M_{j,j} =(C_{\varvec{\sigma }} + C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2, \\&M_{j,j+1} = C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, \qquad \text {for} \ 2 \le j \le 2L \\&M_{2L+1,2L} = C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2, \\&M_{2L+1,2L+1} = (C_{\varvec{\sigma }} + C_{{\mathbf {f}}}+1)h +C_{{\mathbf {f}}}h^2, \end{aligned}$$

and \(\varvec{\Delta }(ih)\) is a \((2L+1)\)-dimensional vector with

$$\begin{aligned} \mathbf {{\delta }}_{1}(ih) =(C_{{\mathbf {f}}}+C_{{\mathbf {f}}}h)h {\mathbf {E}}[\Vert \tilde{\mathbf {x}}^{\text {o}}_{i_{\star }-L-1}(ih) - \tilde{\mathbf {x}}^{\text {p}}_{i_{\star }-L-1}(ih)\Vert ^2],\\ \varvec{\Delta }_{2L+1}(ih) = (C_{{\mathbf {f}}}+C_{{\mathbf {f}}}h)h{\mathbf {E}}[\Vert \tilde{\mathbf {x}}^{\text {o}}_{i_{\star }+L+1}(ih) - \tilde{\mathbf {x}}_{i_{\star }+L+1}^{\text {p}}(ih)\Vert ^2] \end{aligned}$$

and other entries are 0. After iterating (68) for i times, we obtain

$$\begin{aligned} \begin{aligned} \varvec{\Delta }(ih)&\preceq \sum _{k=0}^{i-1}({\mathbf {I}}_{2L+1} + M)^{i-1-k} \cdot \varvec{\Delta }(kh) \\&\quad + ({\mathbf {I}}_{2L+1} + M)^{i}\cdot \varvec{\Delta }(0), \end{aligned} \end{aligned}$$
(69)

and we see \(\varvec{\Delta }(0) = {\mathbf {0}}_{(2L+1) \times 1}\) from the definition of \(\tilde{\mathbf {x}}^{\text {l}}_j(ih)\), \(\tilde{\mathbf {x}}^{\text {p}}_j(ih)\). According to (64), (41), together with Lemma 1, \(\max \limits _{j,l = 1,2,\ldots ,2L+1} |(iM)_{j,k}| = ((C_{\varvec{\sigma }}+C_{{\mathbf {f}}}+1) + C_{{\mathbf {f}}}h)ih\), for \(l=1,\ldots ,2L+1\), we have

$$\begin{aligned} \begin{aligned}&|(({\mathbf {I}}_{2L+1}+ M)^{i-1-k})_{j,l}| \\&\le |((\mathrm{{e}}^{M})^{(i-1-k)})_{j,l}| = |(\mathrm{{e}}^{(i-1-k)M})_{j,l}| \\&\le \mathrm{{e}}^{-C_d\cdot d(j,l)} \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)(i-1-k)h} \le \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)(i-1-k)h}. \end{aligned} \end{aligned}$$
(70)

According to Lemma (8), we have

$$\begin{aligned} \begin{aligned}&\max \{ |\varvec{\Delta }_{1}(kh)| , |\varvec{\Delta }_{2L+1}(kh)| \} \\&\le (C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2) \mathrm{{e}}^{-C_d(L+1)} \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)kh} \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star }\Vert ^2. \end{aligned} \end{aligned}$$
(71)

By applying Taylor’s theorem, we have

$$\begin{aligned} \begin{aligned} \sum _{k=0}^{i-1} \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)kh}&= \dfrac{ \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)ih}-1}{\mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)h}-1}\\&\le \dfrac{ \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)ih}-1}{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)h}. \end{aligned} \end{aligned}$$
(72)

Substituting (70), (71), (72) into (69), we have, for \(j=1,\ldots ,2L+1\),

$$\begin{aligned}&\varvec{\Delta }_j(ih) \\&\quad \le \sum _{k=0}^{i-1}(({\mathbf {I}}_{(2L+1)} + M)^{i-1-k})_{j,1} \cdot \varvec{\Delta }_1(kh) \\&\qquad + \sum _{k=0}^{i-1}(({\mathbf {I}}_{(2L+1)} + M)^{i-1-k})_{j,2L+1} \cdot \varvec{\Delta }_{2L+1}(kh)\\&\quad \le 2\cdot \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)ih}\cdot \mathrm{{e}}^{-C_d(L+1)} \cdot \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star }\Vert ^2\\&\quad \quad \cdot (C_{{\mathbf {f}}}h+C_{{\mathbf {f}}}h^2) \cdot \sum _{k=0}^{i-1} \mathrm{{e}}^{C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)kh} \\&\quad \le \dfrac{2C_{{\mathbf {f}}}}{C_1({\mathbf {f}}, \varvec{\sigma })} \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)ih} \mathrm{{e}}^{-C_d(L+1)} \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star }\Vert ^2. \end{aligned}$$

Recall the definition of \(\varvec{\Delta }_j(ih)\), the proof is complete. \(\square \)

Lemma 10

Under the same settings in Lemma 9, given any positive constant \(\epsilon \), if only

$$\begin{aligned} L \ge \dfrac{\log {\left( \dfrac{\epsilon }{C_2({\mathbf {f}}, \varvec{\sigma }) \Vert {\mathbf {x}}^{o }_{i_{\star }} - {\mathbf {x}}^{p }_{i_{\star }}\Vert ^2} \right) }}{-C_d}+\dfrac{2C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)}{C_d}T, \end{aligned}$$

we have \({\mathbf {E}}[\Vert {\tilde{\mathbf {x}}}^{l }_j(ih) - \tilde{\mathbf {x}}_j^{p }(ih)\Vert ^2] \le \epsilon \) for \(j=1,\ldots ,m\) and \(i=0,\ldots ,T/h\).

Proof

According to Lemma 9, we only need to solve

$$\begin{aligned} C_2({\mathbf {f}}, \varvec{\sigma }) \cdot \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)ih} \cdot \mathrm{{e}}^{-C_d(L+1)} \cdot \Vert \mathbf {x}^{\text {o}}_{i_\star }-\mathbf {x}^{\text {p}}_{i_\star }\Vert ^2\le \epsilon , \end{aligned}$$

for \(i=0,\ldots ,T/h\), which can be obtained by solving

$$\begin{aligned} \mathrm{{e}}^{-C_d(L+1)} \le \dfrac{\epsilon }{C_2({\mathbf {f}}, \varvec{\sigma })\cdot \mathrm{{e}}^{2C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)T} \cdot \Vert {\mathbf {x}}^{\text {o}}_{i_{\star }} - {\mathbf {x}}^{\text {p}}_{i_{\star }}\Vert ^2}. \end{aligned}$$

After taking log for both sides, we have

$$\begin{aligned} L \ge \dfrac{\log {\left( \dfrac{\epsilon }{C_2({\mathbf {f}}, \varvec{\sigma })\Vert {\mathbf {x}}^{\text {o}}_{i_{\star }} - {\mathbf {x}}^{\text {p}}_{i_{\star }}\Vert ^2} \right) }}{-C_d}+\dfrac{2C_1({\mathbf {f}}, \varvec{\sigma }) (1+h)}{C_d}T. \end{aligned}$$

The proof is complete. \(\square \)

Proofs of propositions and theorems

The proof of Proposition 1:

It is a direct result of Lemma 5. \(\square \)

The proof of Theorem 1:

The conclusions are direct results of Lemma 6, 7. \(\square \)

The proof of Theorem 2:

The conclusions are direct results of Lemma 9, 10. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Tong, X.T. Accelerating Metropolis-within-Gibbs sampler with localized computations of differential equations. Stat Comput 30, 1037–1056 (2020). https://doi.org/10.1007/s11222-020-09934-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-020-09934-w

Keywords

  • Bayesian inverse problem
  • High-dimensional data
  • Metropolis-within-Gibbs sampling
  • Banded covariance
  • Computation efficiency