Skip to main content
SpringerLink
Log in

Ergodicity and Accuracy of Optimal Particle Filters for Bayesian Data Assimilation

  • Published:
Chinese Annals of Mathematics, Series B Aims and scope Submit manuscript

Abstract

Data assimilation refers to the methodology of combining dynamical models and observed data with the objective of improving state estimation. Most data assimilation algorithms are viewed as approximations of the Bayesian posterior (filtering distribution) on the signal given the observations. Some of these approximations are controlled, such as particle filters which may be refined to produce the true filtering distribution in the large particle number limit, and some are uncontrolled, such as ensemble Kalman filter methods which do not recover the true filtering distribution in the large ensemble limit. Other data assimilation algorithms, such as cycled 3DVAR methods, may be thought of as controlled estimators of the state, in the small observational noise scenario, but are also uncontrolled in general in relation to the true filtering distribution. For particle filters and ensemble Kalman filters it is of practical importance to understand how and why data assimilation methods can be effective when used with a fixed small number of particles, since for many large-scale applications it is not practical to deploy algorithms close to the large particle limit asymptotic. In this paper, the authors address this question for particle filters and, in particular, study their accuracy (in the small noise limit) and ergodicity (for noisy signal and observation) without appealing to the large particle number limit. The authors first overview the accuracy and minorization properties for the true filtering distribution, working in the setting of conditional Gaussianity for the dynamics-observation model. They then show that these properties are inherited by optimal particle filters for any fixed number of particles, and use the minorization to establish ergodicity of the filters. For completeness we also prove large particle number consistency results for the optimal particle filters, by writing the update equations for the underlying distributions as recursions. In addition to looking at the optimal particle filter with standard resampling, they derive all the above results for (what they term) the Gaussianized optimal particle filter and show that the theoretical properties are favorable for this method, when compared to the standard optimal particle filter.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Akashi, H. and Kumamoto, H., Random sampling approach to state estimation in switching environments, Automatica, 13(4), 1977, 429–434.

    Article  MATH  Google Scholar 

  2. Atar, R. and Zeitouni, O., Exponential stability for nonlinear filtering, Annales de l’IHP Probabilitiés et statistiques, 33, 1997, 697–725,.

    MathSciNet  MATH  Google Scholar 

  3. Bauer, P., Thorpe, A. and Brunet, G., The quiet revolution of numerical weather prediction, Nature, 525(7567), 2015, 47–55.

    Article  Google Scholar 

  4. Blömker, D., Law, K., Stuart, A. M. and Zygalakis, K. C., Accuracy and stability of the continuous-time 3DVAR filter for the Navier-Stokes equation, Nonlinearity, 26(8), 2013, 2193.

    Article  MathSciNet  MATH  Google Scholar 

  5. Budhiraja, A. and Ocone, D., Exponential stability of discrete-time filters for bounded observation noise, Systems & Control Letters, 30(4), 1997, 185–193.

    Article  MathSciNet  MATH  Google Scholar 

  6. Burgers, G., Leeuwen, P. J. V. and Evensen, G. Analysis scheme in the ensemble Kalman filter, Monthly Weather Review, 126(6), 1998, 1719–1724.

    Article  Google Scholar 

  7. Chorin, A. J. and Tu, X. M., Implicit sampling for particle filters, Proceedings of the National Academy of Sciences, 106(41), 2009, 17249–17254.

    Article  Google Scholar 

  8. Crisan, D. and Heine, K., Stability of the discrete time filter in terms of the tails of noise distributions, Journal of the London Mathematical Society, 78(2), 2008, 441–458.

    Article  MathSciNet  MATH  Google Scholar 

  9. Moral, P. D., Kurtzmann, A. and Tugaut, J., On the stability and the uniform propagation of chaos of extended ensemble Kalman-Bucy filters, 2016, arXiv:1606.08256.

    MATH  Google Scholar 

  10. Douc, R., Fort, G., Moulines, E. and Priouret, P., Forgetting the initial distribution for hidden Markov models, Stochastic Processes and Their Applications, 119(4), 2009, 1235–1256.

    Article  MathSciNet  MATH  Google Scholar 

  11. Douc, R., Moulines, E. and Olsson, J., Optimality of the auxiliary particle filter, Probability and Mathematical Statistics, 29(1), 2009, 1–28.

    MathSciNet  MATH  Google Scholar 

  12. Doucet, A., Freitas, N. D. and Gordon, N., An introduction to sequential Monte Carlo methods, Sequential Monte Carlo Methods in Practice, Springer-Verlag, New York, 2010.

    MATH  Google Scholar 

  13. Doucet, A., Godsill, S. and Andrieu, C., On sequential Monte Carlo sampling methods for Bayesian filtering, Statistics and Computing, 10(3), 2000, 197–208.

    Article  Google Scholar 

  14. Evensen, G., Using the extended Kalman filter with a multilayer quasi-geostrophic ocean model, Journal of Geophysical Research: Oceans, 97(C11), 1992, 17905–17924.

    Article  Google Scholar 

  15. Evensen, G., The ensemble Kalman filter: Theoretical formulation and practical implementation, Ocean Dynamics, 53(4), 2003, 343–367.

    Article  Google Scholar 

  16. Ghil, M. and Malanotte-Rizzoli, P., Data assimilation in meteorology and oceanography, Advances in Geophysics, 33, 1991, 141–266.

    Article  Google Scholar 

  17. Handel, R. V., The stability of conditional Markov processes and Markov chains in random environments, The Annals of Probability, 37(5), 2009, 1876–1925.

    Article  MathSciNet  MATH  Google Scholar 

  18. Harlim, J. and Majda, A. J., Catastrophic filter divergence in filtering nonlinear dissipative systems, Communications in Mathematical Sciences, 8(1), 2010, 27–43.

    Article  MathSciNet  MATH  Google Scholar 

  19. Houtekamer, P. L. and Mitchell, H. L., Ensemble Kalman filtering, Quarterly Journal of the Royal Meteorological Society, 131(613), 2005, 3269–3289.

    Article  Google Scholar 

  20. Johansen, A. M. and Doucet, A., A note on auxiliary particle filters, Statistics & Probability Letters, 78(12), 2008, 1498–1504.

    MathSciNet  MATH  Google Scholar 

  21. Kelly, D., Majda, A. J. and Tong, X. T., Concrete ensemble Kalman filters with rigorous catas-trophic filter divergence, Proceedings of the National Academy of Sciences, 112(34), 2015, 10589–10594.

    Article  Google Scholar 

  22. Kelly, D., Vanden-Eijnden, E. and Weare, J., Personal Communications, 2016.

    Google Scholar 

  23. Kelly, D. T. B., Law, K. J. H. and Stuart, A. M., Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time, Nonlinearity, 27(10), 2014, 2579.

    Article  MathSciNet  MATH  Google Scholar 

  24. Kleptsyna, M. L. and Veretennikov, A. Y., On discrete time ergodic filters with wrong initial data, Probability Theory and Related Fields, 141(3-4), 2008, 411–444.

    Article  MathSciNet  MATH  Google Scholar 

  25. Kunita, H., Asymptotic behavior of the nonlinear filtering errors of Markov processes, Journal of Multivariate Analysis, 1(4), 1971, 365–393.

    Article  MathSciNet  MATH  Google Scholar 

  26. Law, K., Stuart, A. and Zygalakis, K., Data Assimilation, Springer-Verlag, New York, 2015.

    Book  MATH  Google Scholar 

  27. Law, K. J. H., Sanz-Alonso, D., Shukla, A. and Stuart, A. M., Filter accuracy for the Lorenz 96 model: fixed versus adaptive observation operators, Physica D: Nonlinear Phenomena, 325, 2016, 1–13.

    Article  MathSciNet  MATH  Google Scholar 

  28. Law, K. J. H., Shukla, A. and Stuart, A. M., Analysis of the 3DVAR filter for the partially observed Lorenz ’63 model, Discrete and Continuous Dynamical Systems A, 34, 2014, 1061–1078.

    Article  MathSciNet  MATH  Google Scholar 

  29. Law, K. J. H., Tembine, H. and Tempone, R., Deterministic mean-field ensemble Kalman filtering, 2014, arXiv:1409.0628.

    MATH  Google Scholar 

  30. Leeuwen, P. J. V., Particle filtering in geophysical systems, Monthly Weather Review, 137(12), 2009, 4089–4114.

    Article  Google Scholar 

  31. Leeuwen, P. J. V., Nonlinear data assimilation in geosciences: An extremely efficient particle filter, Quarterly Journal of the Royal Meteorological Society, 136(653), 2010, 1991–1999.

    Article  Google Scholar 

  32. Liu, J. S. and Chen, R., Blind deconvolution via sequential imputations, Journal of the American Statistical Association, 90(430), 1995, 567–576.

    Article  MathSciNet  MATH  Google Scholar 

  33. Lorenc, A. C., Analysis methods for numerical weather prediction, Quarterly Journal of the Royal Meteorological Society, 112(474), 1986, 1177–1194.

    Article  Google Scholar 

  34. Majda, A. J. and Harlim, J., Filtering Complex Turbulent Systems, Cambridge University Press, Cambridge, 2012.

    Book  MATH  Google Scholar 

  35. Majda, A. J. and Tong, X. T., Robustness and accuracy of finite ensemble Kalman filters in large dimensions, 2016, arXiv:1606.09321.

    Google Scholar 

  36. Moodey, A. J. F., Lawless, A. S., Potthast, R. W. E. and Leeuwen, P. J. V., Nonlinear error dynamics for cycled data assimilation methods, Inverse Problems, 29(2), 2013, 025002.

    Article  MathSciNet  MATH  Google Scholar 

  37. Pitt, M. K. and Shephard, N., Filtering via simulation: Auxiliary particle filters, Journal of the American Statistical Association, 94(446), 1999, 590–599.

    Article  MathSciNet  MATH  Google Scholar 

  38. Poyiadjis, G., Doucet, A. and Singh, S. S., Particle approximations of the score and observed information matrix in state space models with application to parameter estimation, Biometrika, 98(1), 2011, 65–80.

    Article  MathSciNet  MATH  Google Scholar 

  39. Rebeschini, P. and Handel, R. V., Can local particle filters beat the curse of dimensionality? The Annals of Applied Probability, 25(5), 2015, 2809–2866.

    Article  MathSciNet  MATH  Google Scholar 

  40. Sanz-Alonso, D. and Stuart, A. M., Long-time asymptotics of the filtering distribution for partially observed chaotic dynamical systems, SIAM/ASA Journal on Uncertainty Quantification, 3(1), 2015, 1200–1220.

    Article  MathSciNet  MATH  Google Scholar 

  41. Sedigh-Sarvestani, M., Albers, D. J., and Gluckman, B. J., Data assimilation of glucose dynamics for use in the intensive care unit, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2012, 5437–5440.

    Google Scholar 

  42. Sedigh-Sarvestani, M., Schiff, S. J. and Gluckman, B. J., Reconstructing mammalian sleep dynamics with data assimilation, PLoS Comput Biol, 8(11), 2012, e1002788.

    Article  MathSciNet  Google Scholar 

  43. Snyder, C., Particle filters, the “optimal” proposal and high-dimensional systems, Proceedings of the ECMWF Seminar on Data Assimilation for Atmosphere and Ocean, 2011.

    Google Scholar 

  44. Snyder, C., Bengtsson, T. and Morzfeld, M., Performance bounds for particle filters using the optimal proposal, Monthly Weather Review, 143(11), 2015, 4750–4761.

    Article  Google Scholar 

  45. Tadic, V. Z. B. and Doucet, A., Asymptotic properties of recursive maximum likelihood estimation in non-linear state-space models, 2018, arXiv:1806.09571.

    Google Scholar 

  46. Thrun, S., Particle filters in robotics, Proceedings of the Eighteenth Conference on Uncertainty in artificial Intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2002, 511–518.

    Google Scholar 

  47. Tong, X. T. and Handel, R. V., Ergodicity and stability of the conditional distributions of nondegenerate Markov chains, Annals of Applied Probability, 22(4), 2012, 1495–1540.

    Article  MathSciNet  MATH  Google Scholar 

  48. Tong, X. T., Majda, A. J. and Kelly, D., Nonlinear stability of the ensemble Kalman filter with adaptive covariance inflation, Communications in Mathematical Sciences, 14(5), 2016, 1283–1313.

    Article  MathSciNet  MATH  Google Scholar 

  49. Tong, X. T., Majda, A. J. and Kelly, D., Nonlinear stability and ergodicity of ensemble based Kalman filters, Nonlinearity, 29(2), 2016, 657.

    Article  MathSciNet  MATH  Google Scholar 

  50. Work, D. B., Blandin, S., Tossavainen, O. P. and et al., A traffic model for velocity data assimilation, Applied Mathematics Research Express, 2010(1), 2010, 1–35.

    MathSciNet  MATH  Google Scholar 

  51. Zaritskii, V. S., Svetnik, V. B. and Šimelevišc, L. I., Monte-Carlo technique in problems of optimal information processing, Avtomatika i Telemekhanika, 36(12), 1975, 95–103.

    MathSciNet  Google Scholar 

Download references

Acknowledgements

DK was supported as an NYU-Courant instructor. The authors are grateful to an anonymous referee for comments on an earlier version of this paper which have led to a correction of the presentation in Subsection 2.2, and pointers to relevant literature that were previously omitted; they are also grateful to Jonathan Mattingly for discussions relating to ergodicity of nonlinear Markov processes, and to Arnaud Doucet and Adam Johansen for helpful pointers to the literature. The authors also highlight, on the occasion of his 70th birthday, the inspirational work of Andy Majda, in applied mathematics in general, and in the area of data assimilation and filtering (the subject of this paper) in particular.

Author information

Authors and Affiliations

  1. Courant Institute of Mathematical Sciences, New York University, New York, USA

    David Kelly

  2. The Voleon Group, 2150 Dwight Way, Berkeley, CA, 94704, USA

    David Kelly

  3. Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, 91125, USA

    Andrew M. Stuart

Authors
  1. David Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew M. Stuart
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to David Kelly or Andrew M. Stuart.

Additional information

This work was supported by EPSRC (EQUIPS Program Grant), DARPA (contract W911NF-15-2-0121) and ONR (award N00014-17-1-2079).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kelly, D., Stuart, A.M. Ergodicity and Accuracy of Optimal Particle Filters for Bayesian Data Assimilation. Chin. Ann. Math. Ser. B 40, 811–842 (2019). https://doi.org/10.1007/s11401-019-0161-5

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11401-019-0161-5

Keywords

2000 MR Subject Classification

Search

Navigation