Skip to main content

Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration

Abstract

A novel probabilistic numerical method for quantifying the uncertainty induced by the time integration of ordinary differential equations (ODEs) is introduced. Departing from the classical strategy to randomise ODE solvers by adding a random forcing term, we show that a probability measure over the numerical solution of ODEs can be obtained by introducing suitable random time steps in a classical time integrator. This intrinsic randomisation allows for the conservation of geometric properties of the underlying deterministic integrator such as mass conservation, symplecticity or conservation of first integrals. Weak and mean square convergence analysis is derived. We also analyse the convergence of the Monte Carlo estimator for the proposed random time step method and show that the measure obtained with repeated sampling converges in the mean square sense independently of the number of samples. Numerical examples including chaotic Hamiltonian systems, chemical reactions and Bayesian inferential problems illustrate the accuracy, robustness and versatility of our probabilistic numerical method.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  • Abdulle, A.: Fourth order Chebyshev methods with recurrence relation. SIAM J. Sci. Comput. 23, 2041–2054 (2002)

    MathSciNet  Article  Google Scholar 

  • Abdulle, A., Medovikov, A.A.: Second order Chebyshev methods based on orthogonal polynomials. Numer. Math. 90, 1–18 (2001)

    MathSciNet  Article  Google Scholar 

  • Andrieu, C., Roberts, G.O.: The pseudo-marginal approach for efficient Monte Carlo computations. Ann. Stat. 37, 697–725 (2009)

    MathSciNet  Article  Google Scholar 

  • Benettin, G., Giorgilli, A.: On the Hamiltonian interpolation of near to the identity symplectic mappings with application to symplectic integration algorithms. J. Stat. Phys. 74, 1117–1143 (1994)

    MathSciNet  Article  Google Scholar 

  • Chkrebtii, O.A., Campbell, D.A., Calderhead, B., Girolami, M.A.: Bayesian solution uncertainty quantification for differential equations. Bayesian Anal. 11, 1239–1267 (2016)

    MathSciNet  Article  Google Scholar 

  • Cockayne, J., Oates, C.J., Sullivan, T.J., Girolami, M.: Probabilistic numerical methods for PDE-constrained Bayesian inverse problems. In: AIP Conference Proceedings, vol 1853, p. 060001 (2017)

  • Cockayne, J., Oates, C.J., Sullivan, T.J., Girolami, M.: Bayesian probabilistic numerical methods. SIAM Rev. 61, 756–789 (2019)

    MathSciNet  Article  Google Scholar 

  • Conrad, P.R., Girolami, M., Särkkä, S., Stuart, A., Zygalakis, K.: Statistical analysis of differential equations: introducing probability measures on numerical solutions. Stat. Comput. 27, 1065–1082 (2017)

    MathSciNet  Article  Google Scholar 

  • Dashti, M., Stuart, A.M.: The Bayesian approach to inverse problems. In: Handbook of Uncertainty Quantification. Springer, pp. 1–118 (2016)

  • Hairer, E.: Variable time step integration with symplectic methods. Appl. Numer. Math. 25, 219–227 (1997)

    MathSciNet  Article  Google Scholar 

  • Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II. Stiff and Differential-Algebraic Problems. Springer, Berlin (1996)

    MATH  Google Scholar 

  • Hairer, E., Nørsett, S.P., Wanner, G.: Solving Ordinary Differential Equations I. Nonstiff Problems, vol. 8, Springer Verlag Series in Computational Mathematics, Berlin (1993)

  • Hairer, E., Lubich, C., Wanner, G.: Geometric Numerical Integration. Structure-Preserving Algorithms for Ordinary Differential Equations, Springer Series in Computational Mathematics, vol. 31, 2nd edn. Springer, Berlin (2006)

    MATH  Google Scholar 

  • Hénon, M., Heiles, C.: The applicability of the third integral of motion: some numerical experiments. Astron. J. 69, 73–79 (1964)

    MathSciNet  Article  Google Scholar 

  • Kersting, H., Hennig, P.: Active uncertainty calibration in Bayesian ODE solvers. In: Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI 2016), pp. 309–318. AUAI Press (2016)

  • Lie, H.C., Sullivan, T.J., Teckentrup, A.L.: Random forward models and log-likelihoods in Bayesian inverse problems. SIAM/ASA J. Uncertain. Quantif. 6, 1600–1629 (2018)

    MathSciNet  Article  Google Scholar 

  • Lie, H.C., Stuart, A.M., Sullivan, T.J.: Strong convergence rates of probabilistic integrators for ordinary differential equations. Stat. Comput. 29, 1265–1283 (2019)

    MathSciNet  Article  Google Scholar 

  • Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20, 130–141 (1963)

    MathSciNet  Article  Google Scholar 

  • Milstein, G.N., Tretyakov, M.V.: Stochastic Numerics for Mathematical Physics, Scientific Computing. Springer, Berlin (2004)

    Book  Google Scholar 

  • Milstein, G.N., Tretyakov, M.V.: Numerical integration of stochastic differential equations with nonglobally Lipschitz coefficients. SIAM J. Numer. Anal. 43, 1139–1154 (2005)

    MathSciNet  Article  Google Scholar 

  • Oates, C.J., Sullivan, T.J.: A modern retrospective on probabilistic numerics. Stat. Comput. 29, 1335–1351 (2019)

    MathSciNet  Article  Google Scholar 

  • Olsen, L.F.: An enzyme reaction with a strange attractor. Phys. Lett. A 94, 454–457 (1983)

    Article  Google Scholar 

  • Pavliotis, G.A.: Stochastic Processes and Applications, vol. 60 of Texts in Applied Mathematics. Diffusion processes, the Fokker–Planck and Langevin Equations. Springer, New York (2014)

    Book  Google Scholar 

  • Pavliotis, G.A., Stuart, A.M.: Multiscale Methods: Averaging and Homogenization. Texts in Applied Mathematics, vol. 53. Springer, New York (2008)

    MATH  Google Scholar 

  • Schober, M., Duvenaud, D., Hennig, P.: Probabilistic ODE solvers with Runge–Kutta means. In: Advances in Neural Information Processing Systems 27. Curran Associates, Inc., pp. 739–747 (2014)

  • Skeel, R.D., Gear, C.W.: Does variable step size ruin a symplectic integrator? Physica 60, 311–313 (1992)

    MathSciNet  MATH  Google Scholar 

  • Störmer, C.: Sur les trajectoires des corpuscules électrisés. Arch. sci. phys. nat. Genève 24, 5–18, 113–158, 221–247 (1907)

  • Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451–559 (2010)

    MathSciNet  Article  Google Scholar 

  • Sullivan, T.J.: Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors. Inverse Probl. Imaging 11, 857–874 (2017)

    MathSciNet  Article  Google Scholar 

  • van der Houwen, P.J., Sommeijer, B.P.: On the internal stability of explicit, \(m\)-stage Runge–Kutta methods for large \(m\)-values. Z. Angew. Math. Mech. 60, 479–485 (1980)

    MathSciNet  Article  Google Scholar 

  • Verlet, L.: Computer “experiments” on classical fluids. I. Thermodynamical properties of Lennard–Jones molecules. Phys. Rev. 159, 98–103 (1967)

    Article  Google Scholar 

Download references

Acknowledgements

We thank the two anonymous reviewers whose comments helped improve and clarify this manuscript. This work is partially supported by the Swiss National Science Foundation, Grant No. 200020 172710.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Assyr Abdulle.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

A modified stochastic differential equation

In Remark 5, we claim the existence of a modified stochastic differential equation (SDE) whose solution is well approximated by the RTS-RK method. Let us denote by \({\widetilde{f}}\) the function defining the modified equation corresponding to the numerical flow \(\Psi _h\) truncated after l terms, i.e.

$$\begin{aligned} {\widetilde{f}}(y) = f(y) + h^q f_{q+1}(y) + h^{q+1} f_{q+2}(y) + \cdots + h^l f_{l+1}(y). \end{aligned}$$

Details about the construction of such a function can be found in Sect. 7.2. In particular, analyticity of the function f is needed for a rigorous backward error analysis to hold. Therefore, we will refer in this section to Assumption 5 (see Sect. 7.2). For the additive noise method presented in Conrad et al. (2017), the authors consider the SDE

$$\begin{aligned} \,\mathrm {d}Y = {\widetilde{f}}(Y) \,\mathrm {d}t + \sqrt{Q h^{2p}} \,\mathrm {d}W, \end{aligned}$$
(51)

where W is a d-dimensional standard Brownian motion. It is possible to show (Conrad et al. 2017, Theorem 2.4) that the solution of (51) satisfies

$$\begin{aligned} {|}{{\mathbb {E}}}\big (\Phi (Y_N) - \Phi (Y(T)) \mid Y_0 = y \big ){|} \le Ch^{2p}, \end{aligned}$$

where \(T = Nh\) and \(Y_N\) is the numerical solution given by the additive noise method after N steps. Here, we present a similar construction for the RTS-RK method. In particular, let us consider the modified SDE

$$\begin{aligned} \,\mathrm {d}{\widetilde{Y}}= & {} \Big ({\widetilde{f}}({\widetilde{Y}}) + \frac{1}{2}Ch^{2p}\partial _{tt}\Psi _h({\widetilde{Y}})\Big ) \,\mathrm {d}t \nonumber \\&\quad +\, \sqrt{Ch^{2p}\partial _t \Psi _h({\widetilde{Y}})\partial _t \Psi _h({\widetilde{Y}})^\top } \,\mathrm {d}W, \end{aligned}$$
(52)

where C is given in Assumption 1(iii). Let us denote by \({\widetilde{{\mathcal {L}}}}\) the generator of (52), which can be written explicitly as

$$\begin{aligned} {\widetilde{{\mathcal {L}}}} = \Big ({\widetilde{f}} + \frac{1}{2}Ch^{2p}\partial _{tt}\Psi _h\Big ) \cdot \nabla + \frac{1}{2}Ch^{2p}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2, \end{aligned}$$

and, adopting the semi-group notation, it satisfies

$$\begin{aligned} {{\mathbb {E}}}\big (\Phi ({\widetilde{Y}}(h))\mid {\widetilde{Y}}(0) = y\big ) = \hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (y). \end{aligned}$$

In the following lemma, we consider the error over one step between the numerical solution given by the RTS-RK method and the solution of (52) in the weak sense. The proof is inspired by the calculations presented in Conrad et al. (2017, Section 2.4).

Lemma 9

Under the assumptions of Lemma 1 and if Assumption 5 holds, then

$$\begin{aligned} {|}{{\mathbb {E}}}\big (\Phi (Y_1)- \Phi ({\widetilde{Y}}(h))\mid Y_0 = y\big ){|} \le C h^{2p+1}, \end{aligned}$$

where C is a positive constant independent of h and of y, \({\widetilde{Y}}\) is the solution of (52) and \(Y_1\) is the numerical solution given by the RTS-RK method after one step.

Proof

Let us consider the modified ODE

$$\begin{aligned} {\widehat{y}}'(t) = {\widetilde{f}}({\widehat{y}}), \end{aligned}$$
(53)

and denote its flow as \({\widehat{\varphi }}_t\). The generator \({\widehat{{\mathcal {L}}}} = {\widetilde{f}} \cdot \nabla \) satisfies, adopting the semi-group notation,

$$\begin{aligned} \Phi ({\widehat{\varphi }}_h(y)) = \hbox {e}^{h{\widehat{{\mathcal {L}}}}}\Phi (y). \end{aligned}$$

We can now compute the distance between the solution to (52) and (53) as

$$\begin{aligned} \begin{aligned}&\hbox {e}^{h {\widetilde{{\mathcal {L}}}}}\Phi (y)- \hbox {e}^{h {\widehat{{\mathcal {L}}}}}\Phi (y) = \hbox {e}^{h{\widetilde{f}} \cdot \nabla }\\&\quad \Big (\hbox {e}^{\frac{1}{2}Ch^{{2p+1}}\partial _{tt}\Psi _h \cdot \nabla + \frac{1}{2}Ch^{{2p+1}}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2} - I\Big ) \Phi (y)\\&\quad = \big (1 + {\mathcal {O}}(h)\big )\Big (\frac{1}{2}Ch^{{2p+1}}\partial _{tt}\Psi _h \cdot \nabla + \frac{1}{2}Ch^{{2p+1}}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2 \\&\qquad + {\mathcal {O}}\big (h^{{4p+1}}\big )\Big )\Phi (y) \\&\quad = \frac{1}{2}Ch^{2p+1}\partial _{tt}\Psi _h \cdot \nabla \Phi (y) + \frac{1}{2}Ch^{2p+1}\partial _t \Psi _h \partial _t \Psi _h^\top : \nabla ^2 \Phi (y) \\&\qquad + {\mathcal {O}}\big (h^{{4p+1}}\big ). \end{aligned} \end{aligned}$$

Let us recall that equation (10) gives

$$\begin{aligned} \begin{aligned}&\hbox {e}^{h{\mathcal {L}}_h}\Phi (y) - \Phi (\Psi _h(y)) = \frac{1}{2} Ch^{2p+1}\partial _{tt}\Psi _h(y) \cdot \nabla \Phi (y)\\&\quad +\frac{1}{2}Ch^{2p+1}\partial _t \Psi _h(y) \partial _t \Psi _h(y)^\top :\nabla ^2\Phi (y) + {\mathcal {O}}(h^{2p+1}), \end{aligned} \end{aligned}$$

which implies that

$$\begin{aligned} \hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (y)- \hbox {e}^{h{\mathcal {L}}_h}\Phi (y) = \hbox {e}^{h{\widehat{{\mathcal {L}}}}}\Phi (y) - \Phi (\Psi _h(y)) + {\mathcal {O}}(h^{2p+1}). \end{aligned}$$

Now, the theory of backward error analysis (see Sect. 7.2 or e.g. Hairer et al. 2006, Chapter IX) guarantees that

$$\begin{aligned} \hbox {e}^{h{\widehat{{\mathcal {L}}}}}\Phi (y) - \Phi (\Psi _h(y)) = {\mathcal {O}}(h^{q+l+2}). \end{aligned}$$

Choosing \(l = 2p - q - 1\), we have therefore

$$\begin{aligned} \hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (y)- \hbox {e}^{h{\mathcal {L}}_h}\Phi (y) = {\mathcal {O}}(h^{2p+1}), \end{aligned}$$

which is the desired result. \(\square \)

The error can be then propagated to final time as in Theorem 1, as presented in the following theorem.

Theorem 7

Under the assumptions of Lemma 9 and Theorem 1, and if there exists a constant \(L > 0\) independent of h such that for all \(\Phi \in {\mathcal {C}}^\infty _b({\mathbb {R}}^d, {\mathbb {R}})\)

$$\begin{aligned} \sup _{u\in {\mathbb {R}}^d} {|}\hbox {e}^{h{\widetilde{{\mathcal {L}}}}}\Phi (u){|} \le (1 + Lh)\sup _{u\in {\mathbb {R}}^d}{|}\Phi (u){|}, \end{aligned}$$

then it holds

$$\begin{aligned} {|}{{\mathbb {E}}}\big (\Phi (Y_N)- \Phi ({\widetilde{Y}}(T))\mid Y_0 = y\big ){|} \le Ch^{2p}, \end{aligned}$$

where \(T = Nh\) and C is a positive constant independent of h and of y, \({\widetilde{Y}}\) is the solution of (52) and \(Y_N\) is the numerical solution given by the RTS-RK method after N steps.

Proof

The proof follows by replacing \({\mathcal {L}}\) with \(\widetilde{{\mathcal {L}}}\) and Lemma 1 with Lemma 9 in the proof of Theorem 1. \(\square \)

Proof of Lemma 6

In the following, we denote by \(\llbracket a, b \rrbracket \) the interval \(\llbracket a, b \rrbracket = [a, b]\) if \(a < b\) and \(\llbracket a, b \rrbracket = [b, a]\) if \(a \ge b\). Let us first consider \(r \ge 2\) and the function \(\gamma _r(x) = x^r \hbox {e}^{-r\kappa /x}\), whose first derivative is given by

$$\begin{aligned} \gamma _r'(x) = rx^{r-2}(x + \kappa ) \hbox {e}^{-r\kappa /x}. \end{aligned}$$

Under Assumption 6, we have that \(H_j \le Mh\) almost surely, and hence for any \(t \in \llbracket h, H_j\rrbracket \)

$$\begin{aligned} {|}\gamma _r'(t){|} \le r (Mh)^{r-2} (Mh + \kappa )\hbox {e}^{-r\kappa /(Mh)}, \end{aligned}$$

where we exploited that \(\hbox {e}^{-r\kappa /x}\) is a growing function of x. The fundamental theorem of calculus gives

$$\begin{aligned} \begin{aligned} {|}\gamma _r(H_j){|}&= \Big |\gamma _r(h) + \int _{h}^{H_j} \gamma _r'(t) \,\mathrm {d}t \,\Big |\\&\le \gamma _r(h) + r (Mh)^{r-2} (Mh + \kappa )\hbox {e}^{-r\kappa /(Mh)} {|}H_j - h{|}, \quad \text {almost surely}. \end{aligned} \end{aligned}$$

Taking expectation on both sides and since by (33) it holds \({|}\eta _j{|}^r \le C\gamma _r(H_j)\), we obtain

$$\begin{aligned} {{\mathbb {E}}}{|}\eta _j{|}^r \le C\left( \gamma _r(h) + r M^{r-2} h^{p+r-3/2} (Mh + \kappa )\hbox {e}^{-r\kappa /(Mh)}\right) , \end{aligned}$$

which proves the desired inequality. This is because Assumption 6 and Assumption 1(ii) imply that \(M \ge 1\), and because Mh can be bounded by M. Let us now consider \(r = 1\). In this case, we have for \(t \in \llbracket h, H_j\rrbracket \)

$$\begin{aligned} {|}\gamma _1'(t){|} \le (mh)^{-1} (Mh + \kappa )\hbox {e}^{-\kappa /(Mh)}, \quad \text {almost surely.} \end{aligned}$$

Hence, we apply the same reasoning as above and obtain almost surely

$$\begin{aligned} {|}\gamma _1(H_j){|} \le \gamma _1(h) + (mh)^{-1} (Mh + \kappa ) \hbox {e}^{-\kappa /(Mh)} {|}H_j - h{|}, \end{aligned}$$

which implies the desired result by proceeding as above. \(\square \)

Proof of Lemma 7

We first expand the square as

$$\begin{aligned} \begin{aligned}&\left( \sum _{j=0}^{n-1} \left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \right) ^2 = \sum _{j=0}^{n-1} \left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) ^2 \\&\quad + 2 \sum _{j=1}^{n-1} \sum _{i=0}^{j-1}\left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \left( \sum _{k=q}^{N-1} a_{ik} + b_i\right) . \end{aligned} \end{aligned}$$
(54)

Then, we expand the square in the first sum and obtain

$$\begin{aligned} \begin{aligned}&\left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) ^2 = \left( \sum _{k=q}^{N-1} a_{jk}\right) ^2 + b_j^2 + 2 b_j \sum _{k=q}^{N-1} a_{jk}\\&\qquad = \sum _{k=q}^{N-1} a_{jk}^2 + 2\sum _{k=q+1}^{N-1}\sum _{l=q}^{k-1} a_{jk} a_{jl} + b_j^2 + 2 b_j \sum _{k=q}^{N-1} a_{jk}\\&\qquad = a_{jq}^2 + \sum _{k=q+1}^{N-1} a_{jk}^2 + 2\sum _{k=q+1}^{N-1}\sum _{l=q}^{k-1} a_{jk} a_{jl} + b_j^2 + 2 b_j \sum _{k=q}^{N-1} a_{jk}. \end{aligned} \end{aligned}$$
(55)

We then rewrite the term appearing in the double sum in (54) as

$$\begin{aligned} \begin{aligned}&\left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \left( \sum _{k=q}^{N-1} a_{ik} + b_i\right) = a_{jq}a_{iq} + \sum _{k=q}^{N-1}\sum _{\begin{array}{c} l=q \\ l+k > 2q \end{array}}^{N-1} a_{jk}a_{il} \\&\qquad + b_j \sum _{k=q}^{N-1} a_{ik} + b_i \sum _{k=q}^{N-1} a_{jk} + b_i b_j. \end{aligned} \end{aligned}$$
(56)

Substituting expressions (55) and (56) in (54), we finally get

$$\begin{aligned}&\left( \sum _{j=0}^{n-1} \left( \sum _{k=q}^{N-1} a_{jk} + b_j\right) \right) ^2 = \sum _{j=0}^{n-1} a_{jq}^2\\&\qquad + 2 \sum _{j=1}^{n-1} \sum _{i=0}^{j-1} a_{jq}a_{iq} + R(a) + S(a, b), \end{aligned}$$

where the remainder R(a) can be written as \(R = R_1 + R_2 + R_3\) where

$$\begin{aligned} \begin{aligned}&R_1(a) = \sum _{j=0}^{n-1}\sum _{k=q+1}^{N-1} a_{jk}^2, \\&R_2(a) = 2 \sum _{j=0}^{n-1}\sum _{k=q+1}^{N-1}\sum _{l=q}^{k-1} a_{jk} a_{jl},\\&R_3(a) = 2 \sum _{j=1}^{n-1} \sum _{i=0}^{j-1} \sum _{k=q}^{N-1}\sum _{\begin{array}{c} l=q \\ l+k > 2q \end{array}}^{N-1} a_{jk}a_{il}, \end{aligned} \end{aligned}$$

and the remainder S(ab) can be written as \(S = S_1 + S_2 + S_3 + S_4\) where

$$\begin{aligned}&S_1(a, b) = \sum _{j=0}^{n-1} b_j^2, \\&S_2(a, b) = 2\sum _{j=1}^{n-1} \sum _{i=0}^{j-1} b_i b_j, \\&S_3(a, b) = 2\sum _{j=1}^{n-1} \sum _{k=q}^{N-1} b_j a_{jk}, \\&S_4(a, b) = 2\sum _{j=1}^{n-1}\sum _{i=0}^{n-1}\left( b_j \sum _{k=q}^{N-1} a_{ik} + b_i \sum _{k=q}^{N-1} a_{jk}\right) .\\ \end{aligned}$$

which proves the desired result. \(\square \)

Proof of Lemma 8

In the following, all the constants are independent of h and n, but can depend on N and q. Moreover, since \(h < 1\), we often apply \(h^r \le h^s\) for \(r \ge s\). We first notice that, under Assumption 2 and Assumption 5, we get for all \(j = 0, \ldots , n-1\) and \(k = q, \ldots , N-1\)

$$\begin{aligned} \begin{aligned} {|}\Delta _{j,k}{|}&= {|}Q_{k+1}(Y_j) - Q_{k+1}(Y_{j+1}){|}\\&\le C {{\Vert }\Psi _0(Y_j) - \Psi _{H_j}(Y_j){\Vert }}\\&\le C_\Delta {|}H_j{|}, \end{aligned} \end{aligned}$$
(57)

almost surely and where \(C_\Delta \) is independent of h. Above, we exploited that \(Q_{k+1}\) is Lipschitz continuous for all \(k=q, \ldots , N+1\) due to Assumption 5. Let us now consider \(R(\Delta )\). Due to (57) and to Assumption 6, we have

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}(H_j^k - h^k)^2 \Delta _{j,k}^2&\le {C_\Delta ^2}{{\mathbb {E}}}(H_j^{k+1} - H_j h^k)^2\\&= {C_\Delta ^2}\big (h^{2(k+1)} + C_{2(k+1)}h^{2p+2(k+1)-1} \\&+ h^{2(k+1)} + C_2 h^{2p+2k+1} \\&\quad - 2h^{2k+2} - 2C_{k+2}h^{2p+2k+1}\big )\\&= {C_\Delta ^2}\big ((C_{2(k+1)} + C_2 - 2C_{k+2})h^{2p+2k+1}\big )\\&\le C h^{2p+2k+1}, \end{aligned} \end{aligned}$$
(58)

where \(C > 0\) is a positive constant. Now, since \(k \ge q+1\), we get

$$\begin{aligned} {{\mathbb {E}}}(H_j^k - h^k)^2 \Delta _{j,k}^2 \le C h^{2(p+q+1)}. \end{aligned}$$

Hence, for \(R_1(\Delta )\) there exists a constant \({\widetilde{C}}_1\) such that

$$\begin{aligned} {{\mathbb {E}}}R_1(\Delta ) \le {\widetilde{C}}_1 n h^{2(p+q+1)}. \end{aligned}$$

We now proceed to the second remainder \(R_2(\Delta )\). Applying the Cauchy–Schwarz inequality and (58), we get

$$\begin{aligned} \begin{aligned}&{{\mathbb {E}}}\left( (H_j^k - h^k)\Delta _{j,k}(H_j^l - h^l) \Delta _{j,l}\right) \le \Big ({{\mathbb {E}}}\big ((H_j^k - h^k)^2\Delta _{j,k}^2\Big )^{1/2}\\&\qquad \Big ({{\mathbb {E}}}\big ((H_j^l - h^l)^2\Delta _{j,l}^2\Big )^{1/2}\\&\quad \le C h^{2p+k+l+1}, \end{aligned} \end{aligned}$$

where \(C > 0\) is a positive constant. Now, since in the definition of \(R_2(a)\) in (57) we have \(k \ge q+1\) and \(l \ge q\), we have here \(k+l \ge 2q+1\). Therefore, there exists a constant \({\widetilde{C}}_2\) such that

$$\begin{aligned} {{\mathbb {E}}}R_2(\Delta ) \le {\widetilde{C}}_2 n h^{2(p+q+1)}. \end{aligned}$$

We now consider the term \(R_3(\Delta )\). Since \(H_i\) and \(H_j\) are independent for \(i \ne j\), we have

$$\begin{aligned} \begin{aligned}&{{\mathbb {E}}}\big ((H_j^k - h^k)\Delta _{j,k}(H_i^l - h^l) \Delta _{i,l}\big ) \\&\quad = {{\mathbb {E}}}(H_j^k - h^k)\Delta _{j,k} {{\mathbb {E}}}(H_i^l - h^l) \Delta _{i,l}. \end{aligned} \end{aligned}$$

Computing the two factors singularly, we have due to (57) and to Assumption 6

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}(H_j^k - h^k)\Delta _{j,k}&\le C_\Delta {{\mathbb {E}}}(H_j^{k+1} - H_jh^k) \\&= C_\Delta C_{k+1} h^{2p+k}, \end{aligned} \end{aligned}$$
(59)

and analogously for \({{\mathbb {E}}}(H_i^l - h^l) \Delta _{i,l}\). Then, since \(k+l \ge 2q+1\)

$$\begin{aligned}&{{\mathbb {E}}}\big ((H_j^k - h^k)\Delta _{j,k}(H_i^l - h^l) \Delta _{i,l}\big )\nonumber \\&\quad \le C_\Delta ^2 C_{k+1} C_{l+1} h^{2(2p+q+1/2)}. \end{aligned}$$
(60)

Hence, we have for a constant \({\widetilde{C}}_3 > 0\)

$$\begin{aligned} {{\mathbb {E}}}R_3(\Delta ) \le {\widetilde{C}}_3 n^2 h^{2(2p+q+1/2)}. \end{aligned}$$

Finally, replacing \(t_n = nh\), we can write for a constant \(C > 0\)

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}R(\Delta )&\le ({\widetilde{C}}_1 + {\widetilde{C}}_2) n h^{2(p+q+1)} + {\widetilde{C}}_3 n^2 h^{2(2p+q+1/2)}\\&= ({\widetilde{C}}_1 + {\widetilde{C}}_2) t_n h^{2(p + q + 1/2)} + {\widetilde{C}}_3 t_n^2 h^{2(2p + q - 1/2)}. \end{aligned} \end{aligned}$$

Let us now consider \(S(\Delta , \eta )\). First, we notice that under the assumption \(p \ge 3/2\) we have for any \(r \ge 1\), \(\min \{r, p+r-3/2\} = r\), and therefore Lemma 6 simplifies to

$$\begin{aligned} {{\mathbb {E}}}{|}\eta _j{|}^r \le Ch^r \hbox {e}^{-r\kappa /(Mh)}. \end{aligned}$$

We first consider \(S_1(\Delta , \eta )\). Applying Lemma 6 with \(r = 2\), we obtain for a constant \({\widehat{C}}_1 > 0\)

$$\begin{aligned} {{\mathbb {E}}}S_1(\Delta , \eta ) \le {\widehat{C}}_1 n h^2 \hbox {e}^{-2\kappa /(Mh)}. \end{aligned}$$

For the second term \(S_2(\Delta , \eta )\), we have by (33) that \({|}\eta _i{|}\le CH^i \hbox {e}^{-\kappa /H_i}\) and \(\eta _j\le CH^j \hbox {e}^{-\kappa /H_j}\) almost surely. These two bounds are independent for \(i \ne j\), and therefore, applying Lemma 6 with \(r = 1\), we have for a constant \({\widehat{C}}_2 > 0\)

$$\begin{aligned} {{\mathbb {E}}}S_2(\Delta , \eta ) \le {\widehat{C}}_2 n^2 h^2 \hbox {e}^{-2\kappa /(Mh)}. \end{aligned}$$

We now consider the third remainder \(S_3(\Delta , \eta )\). Applying the Cauchy–Schwarz inequality, we obtain

$$\begin{aligned} {{\mathbb {E}}}\eta _j (H_j^k - h^k) \Delta _{j,k} \le ({{\mathbb {E}}}\eta _j^2)^{1/2} ({{\mathbb {E}}}(H_j^k - h^k)^2 \Delta _{j,k}^2)^{1/2}. \end{aligned}$$

Applying Lemma 6 with \(r = 2\) to the first factor and (58) to the second, we get

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}\eta _j (H_j^k - h^k) \Delta _{j,k}&\le C h \hbox {e}^{-\kappa /(Mh)}h^{p+k+1/2}\\&= C h^{p+k+3/2}\hbox {e}^{-\kappa /(Mh)} \end{aligned} \end{aligned}$$

Now, since \(k \ge q\), we have for a constant \({\widehat{C}}_3 > 0\)

$$\begin{aligned} {{\mathbb {E}}}S_3(\Delta , \eta ) \le {\widehat{C}}_3 n h^{p+q+3/2}\hbox {e}^{-\kappa /(Mh)}. \end{aligned}$$

Finally, we consider the last term \(S_4(\Delta , \eta )\). Since by (33) it holds \({|}\eta _j{|} \le CH_j \hbox {e}^{-\kappa /H_j}\) almost surely, and this bound is independent of \(H_i\) for \(i \ne j\), applying (59) and Lemma 6 we have

$$\begin{aligned} \begin{aligned} {{\mathbb {E}}}\eta _j(H_i^k - h^k)\Delta _{i,k}&= {{\mathbb {E}}}\eta _j {{\mathbb {E}}}(H_i^k - h^k)\Delta _{i,k}\\&\le Ch \hbox {e}^{-\kappa /(Mh)}h^{2p+k}, \end{aligned} \end{aligned}$$

which, since \(k \ge q\), implies that there exists a constant \({\widehat{C}}_4 > 0\) such that

$$\begin{aligned} {{\mathbb {E}}}S_4(\Delta , \eta ) \le {\widehat{C}}_4 n^2 h^{2p+q+1}\hbox {e}^{-\kappa /(Mh)}. \end{aligned}$$

Finally, replacing \(t_n = nh\), we can write

$$\begin{aligned} \begin{aligned}&{{\mathbb {E}}}S(\Delta , \eta ) \le ({\widehat{C}}_1 n h^2 + {\widehat{C}}_2 n^2 h^2) \hbox {e}^{-2\kappa /(Mh)} \\&\qquad + ({\widehat{C}}_3 n h^{p+q+3/2} + {\widehat{C}}_4 n^2 h^{2p+q+1})\hbox {e}^{-\kappa /(Mh)}\\&\quad = ({\widehat{C}}_1 t_n h + {\widehat{C}}_2 t_n^2) \hbox {e}^{-2\kappa /(Mh)} \\&\qquad + ({\widehat{C}}_3 t_n h^{p+q+1/2} + {\widehat{C}}_4 t_n^2 h^{2p+q-1})\hbox {e}^{-\kappa /(Mh)}, \end{aligned} \end{aligned}$$

which completes the proof. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Abdulle, A., Garegnani, G. Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration. Stat Comput 30, 907–932 (2020). https://doi.org/10.1007/s11222-020-09926-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-020-09926-w

Keywords

  • Probabilistic methods for ODEs
  • Random time steps
  • Uncertainty quantification
  • Chaotic systems
  • Geometric integration
  • Inverse problems

Mathematics Subject Classification

  • 65C30
  • 65F15
  • 65L09