Statistics and Computing

, Volume 29, Issue 6, pp 1285–1295 | Cite as

Adaptive step-size selection for state-space probabilistic differential equation solvers

  • Oksana A. ChkrebtiiEmail author
  • David A. Campbell


When models are defined implicitly by systems of differential equations with no closed-form solution, small local errors in finite-dimensional solution approximations can propagate into deviations from the true underlying model trajectory. Some recent perspectives in quantifying this uncertainty are based on Bayesian probability modeling: a prior is defined over the unknown solution and updated by conditioning on interrogations of the forward model. Improvement in accuracy via grid refinement must be considered in order for such Bayesian numerical methods to compete with state-of-the-art numerical techniques. We review the principles of Bayesian statistical design and apply these to develop an adaptive probabilistic method to sequentially select time-steps for state-space probabilistic ODE solvers. We investigate the behavior of local error under the adaptive scheme which underlies numerical variable step-size methods. Numerical experiments are used to illustrate the performance of such an adaptive scheme, showing improved accuracy over uniform designs in terms of local error.


Uncertainty quantification Differential equations Statistical design Numerical methods Data assimilation 



This material was developed, in part, at the Prob Num 2018 workshop hosted by the Lloyd’s Register Foundation programme on Data-Centric Engineering at the Alan Turing Institute, UK, and supported by the National Science Foundation, USA. This material was based upon work partially supported by the National Science Foundation under Grant DMS-1127914 to the Statistical and Applied Mathematical Sciences Institute (SAMSI) and the Isaac Newton Institute for Mathematical Sciences’ program on Uncertainty Quantification for Complex Systems: theory and methodologies. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the above-named funding bodies and research institutions.


  1. Alexanderian, A., Gloor, P.J., Ghattas, O.: On Bayesian A- and D-optimal experimental designs in infinite dimensions. Bayesian Anal. 11(3), 671 (2016). MathSciNetCrossRefzbMATHGoogle Scholar
  2. Bernardo, J.M., Smith, A.F.M.: Bayesian Theory. Wiley, New York (1994)CrossRefGoogle Scholar
  3. Butcher, J.: Numerical Methods for Ordinary Differential Equations. Wiley, Hoboken (2008). CrossRefzbMATHGoogle Scholar
  4. Chaloner, K., Verdinelli, I.: Bayesian experimental design: a review. Stat. Sci. 10, 273 (1995). MathSciNetCrossRefzbMATHGoogle Scholar
  5. Chkrebtii, O., Campbell, D., Calderhead, B., Girolami, M.: Bayesian solution uncertainty quantification for differential equations. Bayesian Anal. 11(4), 1239 (2016a). MathSciNetCrossRefzbMATHGoogle Scholar
  6. Chkrebtii, O., Campbell, D., Calderhead, B., Girolami, M.: Rejoinder. Bayesian Anal. 4(11), 1295 (2016b). CrossRefzbMATHGoogle Scholar
  7. Cockayne, J., Oates, C., Sullivan, T., Girolami, M.: Bayesian probabilistic numerical methods (2017). arXiv:1702.03673
  8. Diaconis, P.: Bayesian Numerical Analysis, pp. 163–175. Springer, New York (1988)zbMATHGoogle Scholar
  9. Hennig, P., Hauberg, S.: Probabilistic solutions to differential equations and their application to Riemannian statistics. In: Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 33 (2014)Google Scholar
  10. Higdon, D., Gattiker, J., Lawrence, E., Jackson, C., Tobis, M., Pratola, M., Habib, S., Heitmann, K., Price, S.: Computer model calibration using the ensemble Kalman filter. Technometrics 55(4), 488 (2013). MathSciNetCrossRefGoogle Scholar
  11. Jazwinski, A.: Stochastic Processes and Filtering Theory. Mathematics in Science and Engineering. Elsevier Science, Amsterdam (1970). CrossRefzbMATHGoogle Scholar
  12. Kassam, A., Trefethen, L.: Fourth-order time-stepping for stiff PDEs. SIAM J. Sci. Comput. 26(4), 1214 (2005). MathSciNetCrossRefzbMATHGoogle Scholar
  13. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22, 49 (1951). MathSciNetCrossRefzbMATHGoogle Scholar
  14. Kuramoto, Y., Tsuzuki, T.: Persistent propagation of concentration waves in dissipative media far from thermal equilibrium. Prog. Theor. Phys. 55, 356 (1976). CrossRefGoogle Scholar
  15. Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20, 130 (1963). MathSciNetCrossRefzbMATHGoogle Scholar
  16. O’Hagan, A.O.: Some Bayesian numerical analysis. Bayesian Stat. 4, 345 (1992)MathSciNetGoogle Scholar
  17. Paciorek, C.J., Schervish, M.J.: In: Thrun, S., Saul, L.K., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16, pp. 273–280. MIT Press, Cambridge (2004)Google Scholar
  18. Paciorek, C., Schervish, M.: Spatial modelling using a new class of nonstationary covariance functions. Environmetrics 17(5), 483 (2006). MathSciNetCrossRefGoogle Scholar
  19. Schober, M., Duvenaud, D.K., Hennig, P.: In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 739–747. Curran Associates, Inc. Red Hook (2014)Google Scholar
  20. Sebastiani, P., Wynn, H.: Maximum entropy sampling and optimal Bayesian experimental design. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 62(1), 145 (2000). MathSciNetCrossRefzbMATHGoogle Scholar
  21. Shampine, L.F.: Error estimation and control for ODEs. J. Sci. Comput. 25(1), 3 (2005). MathSciNetCrossRefzbMATHGoogle Scholar
  22. Skilling, J.: Bayesian Solution of Ordinary Differential Equations, pp. 23–37. Kluwer Academic Publishers, Seattle (1991)zbMATHGoogle Scholar
  23. Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451 (2010). MathSciNetCrossRefzbMATHGoogle Scholar
  24. Wang, J., Cockayne, J., Oates, C.: On the Bayesian solution of differential equations (2018). arXiv:1805.07109
  25. Wikle, C.K., Berliner, L.M.: A Bayesian tutorial for data assimilation. Physica D 230(1), 1 (2007). MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of StatisticsThe Ohio State UniversityColumbusUSA
  2. 2.School of Mathematics and StatisticsCarleton UniversityOttawaCanada

Personalised recommendations