Skip to main content
Log in

Abstract

There are many computational tasks, in which it is necessary to sample a given probability density function (or pdf for short), i.e., to use a computer to construct a sequence of independent random vectors x i (i = 1, 2, …), whose histogram converges to the given pdf. This can be difficult because the sample space can be huge, and more importantly, because the portion of the space, where the density is significant, can be very small, so that one may miss it by an ill-designed sampling scheme. Indeed, Markovchain Monte Carlo, the most widely used sampling scheme, can be thought of as a search algorithm, where one starts at an arbitrary point and one advances step-by-step towards the high probability region of the space. This can be expensive, in particular because one is typically interested in independent samples, while the chain has a memory. The authors present an alternative, in which samples are found by solving an algebraic equation with a random right-hand side rather than by following a chain; each sample is independent of the previous samples. The construction in the context of numerical integration is explained, and then it is applied to data assimilation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Doucet, A., de Freitas, N. and Gordon, N., Sequential Monte Carlo Methods in Practice, Springer-Verlag, New York, 2001.

    MATH  Google Scholar 

  2. Chorin, A. J. and Hald, O. H. Stochastic Tools in Mathematics and Science, 2nd edition, Springer-Verlag, New York, 2009.

    Book  MATH  Google Scholar 

  3. Morzfeld, M., Tu, X., Atkins, E. and Chorin, A. J., A random map implementation of implicit filters, J. Comput. Phys., 231, 2012, 2049–2066.

    Article  MathSciNet  MATH  Google Scholar 

  4. Morzfeld, M. and Chorin, A. J., Implicit particle filtering for models with partial noise, and an application to geomagnetic data assimilation, Nonlin. Processes Geophys., 19, 2012, 365–382.

    Article  Google Scholar 

  5. Kloeden, P. E. and Platen, E., Numerical Solution of Stochastic Differential Equations, 3rd edition, Springer-Verlag, New York, 1999.

    Google Scholar 

  6. Chorin, A. J. and Tu, X., Implicit sampling for particle filters, Proc. Nat. Acad. Sc. USA, 106, 2009, 17249–17254.

    Article  Google Scholar 

  7. Chorin, A. J., Morzfeld, M. and Tu, X., Implicit particle filters for data assimilation, Commun. Appl. Math. Comput. Sci., 5(2), 2010, 221–240.

    Article  MathSciNet  MATH  Google Scholar 

  8. Arulampalam, M. S., Maskell, S., Gordon, N. and Clapp, T., A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking, IEEE Trans. Signal Process, 10, 2002, 197–208.

    Google Scholar 

  9. Bickel, P., Li, B. and Bengtsson, T., Sharp failure rates for the bootstrap particle filter in high dimensions, Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, 2008, 318–329.

  10. Snyder, C. C., Bengtsson, T., Bickel, P. and Anderson, J., Obstacles to high-dimensional particle filtering, Mon. Wea. Rev., 136, 2008, 4629–4640.

    Article  Google Scholar 

  11. Gordon, N. J., Salmon, D. J. and Smith, A. F. M., Novel approach to nonlinear/non-Gaussian Bayesian state estimation, IEEE Proceedings F on Radar and Signal Processing, 140, 1993, 107–113.

    Article  Google Scholar 

  12. Doucet, A., Godsill, S. and Andrieu, C., On sequential Monte Carlo sampling methods for Bayesian filtering, Statistics and Computing, 50, 2000, 174–188.

    Google Scholar 

  13. Del Moral, P., Feynman-Kac Formulae, Springer-Verlag, New York, 2004.

    Book  MATH  Google Scholar 

  14. Del Moral, P., Measure-valued processes and interacting particle systems. Application to nonlinear filtering problems, Annals of Applied Probability, 8(2), 1998, 438–495.

    Article  MathSciNet  MATH  Google Scholar 

  15. Zaritskii, V. S. and Shimelevich, L. I., Monte Carlo technique in problems of optimal data processing, Automation and Remote Control, 12, 1975, 95–103.

    MathSciNet  Google Scholar 

  16. Kalman, R. E., A new approach to linear filtering and prediction theory, Trans. ASME, Ser. D, 82, 1960, 35–48.

    Article  Google Scholar 

  17. Kalman, R. E. and Bucy, R. S., New results in linear filtering and prediction theory, Trans. ASME, Ser. D, 83, 1961, 95–108.

    Article  MathSciNet  Google Scholar 

  18. Evensen, G., Data Assimilation, Springer-Verlag, New York, 2007.

    MATH  Google Scholar 

  19. Zakai, M., On the optimal filtering of diffusion processes, Zeit. Wahrsch., 11, 1969, 230–243.

    Article  MathSciNet  MATH  Google Scholar 

  20. Talagrand, O. and Courtier, P., Variational assimilation of meteorological observations with the adjoint vorticity equation. I: Theory, Q. J. R. Meteorol. Soc., 113, 1987, 1311–1328.

    Article  Google Scholar 

  21. Bennet, A. F., Leslie, L. M., Hagelberg, C. R. and Powers, P. E., A cyclone prediction using a barotropic model initialized by a general inverse method, Mon. Weather Rev., 121, 1993, 1714–1728.

    Article  Google Scholar 

  22. Courtier, P., Thepaut, J. N. and Hollingsworth, A., A strategy for operational implementation of 4D-var, using an incremental appoach, Q. J. R. Meteorol. Soc., 120, 1994, 1367–1387.

    Article  Google Scholar 

  23. Courtier, P., Dual formulation of four-dimensional variational assimilation, Q. J. R. Meteorol. Soc., 123, 1997, 2449–2461.

    Article  Google Scholar 

  24. Talagrand, O., Assimilation of observations, an introduction, J. R. Meteorol. Soc. of Japan, 75(1), 1997, 191–209.

    MathSciNet  Google Scholar 

  25. Tremolet, Y., Accounting for an imperfect model in 4D-var, Q. J. R. Meteorol. Soc., 621(132), 2006, 2483–2504.

    Article  Google Scholar 

  26. Atkins, E., Morzfeld, M. and Chorin, A. J., Implicit particle methods and their connection to variational data assimilation, Mon. Weather Rev., in press.

  27. Kuramoto, Y. and Tsuzuki, T., On the formation of dissipative structures in reaction-diffusion systems, Progr. Theoret. Phys., 54, 1975, 687–699.

    Article  Google Scholar 

  28. Sivashinsky, G., Nonlinear analysis of hydrodynamic instability in laminar flames, Part I, Derivation of basic equations, Acta Astronaut., 4, 1977, 1177–1206.

    Article  MathSciNet  MATH  Google Scholar 

  29. Chorin, A. J. and Krause, P., Dimensional reduction for a Bayesian filter, PNAS, 101, 2004, 15013–15017.

    Article  MathSciNet  MATH  Google Scholar 

  30. Jardak, M., Navon, I. M. and Zupanski, M., Comparison of sequential data assimilation methods for the Kuramoto-Sivashinsky equation, Int. J. Numer. Methods Fluids, 62, 2009, 374–402.

    MathSciNet  Google Scholar 

  31. Lord, G. J. and Rougemont, J., A numerical scheme for stochastic PDEs with Gevrey regularity, IMA Journal of Numerical Analysis, 24, 2004, 587–604.

    Article  MathSciNet  MATH  Google Scholar 

  32. Jentzen, A. and Kloeden, P. E., Overcoming the order barrier in the numerical approximation of stochastic partial differential equations with additive space-time noise, Proc. R. Soc. A, 465, 2009, 649–667.

    Article  MathSciNet  MATH  Google Scholar 

  33. Fletcher, R., Practical Methods of Optimization, Wiley, New York, 1987.

    MATH  Google Scholar 

  34. Nocedal, J. and Wright, S. T., Numerical Optimization, 2nd edition, Springer-Verlag, New York, 2006.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alexandre J. Chorin.

Additional information

In honor of the scientific heritage of Jacques-Louis Lions

Project supported by the Director, Office of Science, Computational and Technology Research, U. S. Department of Energy (No.DE-AC02-05CH11231) and the National Science Foundation (Nos.DMS-0705910, OCE-0934298).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chorin, A.J., Morzfeld, M. & Tu, X. Implicit Sampling, with Application to Data Assimilation. Chin. Ann. Math. Ser. B 34, 89–98 (2013). https://doi.org/10.1007/s11401-012-0757-5

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11401-012-0757-5

Keywords

2000 MR Subject Classification

Navigation