Skip to main content
SpringerLink
Log in

Ensemble Kalman Filter: Current Status and Potential

  • Chapter
  • First Online:
Data Assimilation

Abstract

In this chapter we give an introduction to different types of Ensemble Kalman filter, describe the Local Ensemble Transform Kalman Filter (LETKF) as a representative prototype of these methods, and several examples of how advanced properties and applications that have been developed and explored for 4D-Var (four-dimensional variational assimilation) can be adapted to the LETKF without requiring an adjoint model. Although the Ensemble Kalman filter is less mature than 4D-Var (Kalnay 2003), its simplicity and its competitive performance with respect to 4D-Var suggest that it may become the method of choice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Strictly speaking the combinations are not linear since the weights depend on the forecasts (Nerger et al. 2005).

  2. 2.

    Takemasa Miyoshi (personal communication) has pointed out that Jazwinski (1970) proposed the same “outer loop” algorithm for Extended Kalman filter (see footnote on page 276).

References

  • Anderson, J.L., 2001. An ensemble adjustment Kalman filter for data assimilation. Mon. Weather Rev., 129, 2884–2903.

    Article  Google Scholar 

  • Andersson, E., M. Fisher, E. Hólm, L. Isaksen, G. Radnoti and Y. Trémolet, 2005. Will the 4D-Var approach be defeated by nonlinearity? ECMWF Technical Memorandum, No. 479.

    Google Scholar 

  • Baek, S.-J., B.R. Hunt, E. Kalnay, E. Ott and I. Szunyogh, 2006. Local ensemble Kalman filtering in the presence of model bias. Tellus A, 58, 293–306.

    Article  Google Scholar 

  • Barker, D.M., 2008. How 4DVar can benefit from or contribute to EnKF (a 4DVar perspective). Available from http://4dvarenkf.cima.fcen.uba.ar/Download /Session_8/4DVar_EnKF_Barker.pdf.

  • Bishop, C.H., B.J. Etherton and S.J. Majumdar, 2001. Adaptive sampling with ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Weather Rev., 129, 420–436.

    Article  Google Scholar 

  • Buehner, M., C. Charente, B. He, et al., 2008. Intercomparison of 4-D Var and EnKF systems for operational deterministic NWP. Available from http://4dvarenkf.cima.fcen.uba.ar/Download /Session_7/Intercomparison_4D-Var_EnKF_Buehner.pdf.

  • Burgers, G., P.J. van Leeuwen and G. Evensen, 1998. On the analysis scheme in the ensemble Kalman filter. Mon. Weather Rev., 126, 1719–1724.

    Article  Google Scholar 

  • Cardinali, C., S. Pezzulli and E. Andersson, 2004. Influence-matrix diagnostic of a data assimilation system. Q. J. R. Meteorol. Soc., 130, 2767–2786.

    Article  Google Scholar 

  • Caya, A., J. Sun and C. Snyder, 2005. A comparison between the 4D-VAR and the ensemble Kalman filter techniques for radar data assimilation. Mon. Weather Rev., 133, 3081–3094.

    Article  Google Scholar 

  • Courtier, P. and O. Talagrand, 1990. Variational assimilation of meteorological observations with the direct and adjoint shallow water equations. Tellus, 42A, 531–549.

    Article  Google Scholar 

  • Danforth, C.M. and E. Kalnay, 2008. Using singular value decomposition to parameterize state dependent model errors. J. Atmos. Sci., 65, 1467–1478.

    Article  Google Scholar 

  • Danforth, C.M., E. Kalnay and T. Miyoshi, 2006. Estimating and correcting global weather model error. Mon. Weather Rev., 134, 281–299.

    Google Scholar 

  • Dee, D.P. and A.M. da Silva, 1998. Data assimilation in the presence of forecast bias. Q. J. R. Meteorol. Soc., 124, 269–295.

    Article  Google Scholar 

  • Desroziers, G., L. Berre, B. Chapnik and P. Poli, 2005. Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc., 131, 3385–3396.

    Google Scholar 

  • Evensen, G., 1994. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10,143–10,162.

    Article  Google Scholar 

  • Evensen, G., 2003. The ensemble Kalman filter: Theoretical formulation and practical implementation. Ocean Dyn., 53, 343–367.

    Article  Google Scholar 

  • Evensen, G. and P.J. van Leeuwen, 1996. Assimilation of Geosat altimeter data for the Agulhas current using the ensemble Kalman filter with a quasi-geostrophic model. Mon. Weather Rev., 124, 85–96.

    Article  Google Scholar 

  • Fisher, M., M. Leutbecher and G. Kelly, 2005. On the equivalence between Kalman smoothing and weak-constraint four-dimensional variational data assimilation. Q. J. R. Meteorol. Soc., 131, 3235–3246.

    Article  Google Scholar 

  • Gaspari, G. and S.E. Cohn, 1999. Construction of correlation functions in two and three dimensions. Q. J. R. Meteorol. Soc., 125, 723–757.

    Article  Google Scholar 

  • Ghil, M. and P. Malanotte-Rizzoli, 1991. Data assimilation in meteorology and oceanography. Adv. Geophys., 33, 141–266.

    Article  Google Scholar 

  • Greybush, S., E. Kalnay, T. Miyoshi, K. Ide and B. Hunt, 2009. EnKF localization techniques and balance. Presented at the WMO 5th International Symposium on Data Assimilation. Melbourne, Australia, 6–9 October 2009, submitted to Mon. Weather Rev.,Available at http://www.weatherchaos.umd.edu/papers/Greybush_Melbourne2009.ppt.

  • Gustafsson, N., 2007. Response to the discussion on “4-D-Var or EnKF?”. Tellus A, 59, 778–780.

    Article  Google Scholar 

  • Hamill, T.M., J.S. Whitaker and C. Snyder, 2001. Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Weather Rev., 129, 2776–2790.

    Article  Google Scholar 

  • Harlim, J. and B.R. Hunt, 2007a. A non-Gaussian ensemble filter for assimilating infrequent noisy observations. Tellus A, 59, 225–237.

    Article  Google Scholar 

  • Harlim, J. and B.R. Hunt, 2007b. Four-dimensional local ensemble transform Kalman filter: Variational formulation and numerical experiments with a global circulation model. Tellus A, 59, 731–748.

    Article  Google Scholar 

  • Houtekamer, P.L. and H.L. Mitchell, 1998. Data assimilation using an ensemble Kalman filter technique. Mon. Weather Rev., 126, 796–811.

    Article  Google Scholar 

  • Houtekamer, P.L. and H.L. Mitchell, 2001. A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Weather Rev., 129, 123–137.

    Article  Google Scholar 

  • Houtekamer, P.L., H.L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek and B. Hansen, 2005. Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Weather Rev., 133, 604–620.

    Article  Google Scholar 

  • Hunt, B.R., 2005. An efficient implementation of the local ensemble Kalman filter. Available at http://arxiv.org./abs/physics/0511236.

    Google Scholar 

  • Hunt, B.R., E. Kalnay, E.J. Kostelich, et al., 2004. Four-dimensional ensemble Kalman filtering. Tellus, 56A, 273–277.

    Article  Google Scholar 

  • Hunt, B.R., E.J. Kostelich and I. Szunyogh, 2007. Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112–126.

    Article  Google Scholar 

  • Ide, K., P. Courtier, M. Ghil and A. Lorenc, 1997. Unified notation for data assimilation: Operational, sequential and variational. J. Meteorol. Soc. Jpn., 75, 181–189.

    Google Scholar 

  • Järvinen, H., E. Andersson and F. Bouttier, 1999. Variational assimilation of time sequences of surface observations with serially correlated errors. Tellus, 51A, 469–488.

    Google Scholar 

  • Jazwinski, A.H., 1970. Stochastic Processes and Filtering Theory. Academic Press, NY, 376 pp.

    Google Scholar 

  • Kalman, R.E., 1960. A new approach to linear filtering and prediction problems. J. Basic Eng., 82, 35–45.

    Article  Google Scholar 

  • Kalnay, E., 2003. Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, Cambridge, UK, 341 pp.

    Google Scholar 

  • Kalnay, E., H. Li, T. Miyoshi, S.-C. Yang and J. Ballabrera-Poy, 2007a. 4D-Var or ensemble Kalman filter? Tellus A, 59, 758–773.

    Article  Google Scholar 

  • Kalnay, E., H. Li, T. Miyoshi, S.-C. Yang and J. Ballabrera-Poy, 2007b. Response to the discussion on “4D-Var or EnKF?” by Nils Gustaffson. Tellus A, 59, 778–780.

    Article  Google Scholar 

  • Kalnay, E. and S.-C. Yang, 2008. Accelerating the spin-up in EnKF. Arxiv: physics:Nonlinear/0.806.0180v1.

    Google Scholar 

  • Keppenne, C.L., 2000. Data assimilation into a primitive-equation model with a parallel ensemble Kalman filter. Mon. Weather Rev., 128, 1971–1981.

    Article  Google Scholar 

  • Keppenne, C. and H. Rienecker, 2002. Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model. Mon. Weather Rev., 130, 2951–2965.

    Article  Google Scholar 

  • Langland, R.H. and N.L. Baker, 2004. Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56A, 189–201.

    Article  Google Scholar 

  • Li, H., 2007. Local ensemble transform Kalman filter with realistic observations. Ph. D. thesis. Available at http://hdl.handle.net/1903/7317.

  • Li, H., E. Kalnay and T. Miyoshi, 2009a. Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Q. J. R. Meteorol. Soc., 135, 523–533.

    Article  Google Scholar 

  • Li, H., E. Kalnay, T. Miyoshi and C.M. Danforth, 2009b. Accounting for model errors in ensemble data assimilation. Mon. Weather Rev., 137, 3407–3419.

    Article  Google Scholar 

  • Liu, J. and E. Kalnay, 2008. Estimating observation impact without adjoint model in an ensemble Kalman filter. Q. J. R. Meteorol. Soc., 134, 1327–1335.

    Article  Google Scholar 

  • Liu, J., E. Kalnay, T. Miyoshi and C. Cardinali, 2009. Analysis sensitivity calculation in an ensemble Kalman filter. Q. J. R. Meteorol. Soc., 135, 523–533.

    Article  Google Scholar 

  • Lorenc, A.C., 1986. Analysis methods for numerical weather prediction. Q. J .R. Meteorol. Soc., 112, 1177–1194.

    Article  Google Scholar 

  • Lorenc, A.C., 2003. The potential of the ensemble Kalman filter for NWP – a comparison with 4D-Var. Q. J. R. Meteorol. Soc., 129, 3183–3203.

    Article  Google Scholar 

  • Lorenz, E., 1963. Deterministic non-periodic flow. J. Atmos. Sci., 20, 130–141.

    Article  Google Scholar 

  • Mitchell, H.L., P.L. Houtekamer and G. Pellerin, 2002. Ensemble size, balance, and model-error representation in an ensemble Kalman filter. Mon. Weather Rev., 130, 2791–2808.

    Article  Google Scholar 

  • Miyoshi, T., 2005. Ensemble Kalman Filter Experiments with a Primitive-Equation Global Model. Doctoral dissertation, University of Maryland, College Park, 197 pp. Available at https://drum.umd.edu/dspace/handle/1903/3046.

    Google Scholar 

  • Miyoshi, T. and S. Yamane, 2007. Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution. Mon. Weather Rev., 135, 3841–3861.

    Article  Google Scholar 

  • Molteni, F., 2003. Atmospheric simulations using a GCM with simplified physical parameterizations. I: Model climatology and variability in multi-decadal experiments. Clim. Dyn., 20, 175–191.

    Google Scholar 

  • Nerger, L., W. Hiller and J. Scroeter, 2005. A comparison of error subspace Kalman filters. Tellus, 57A, 715–735.

    Article  Google Scholar 

  • Nutter, P., M. Xue and D. Stensrud, 2004. Application of lateral boundary condition perturbations to help restore dispersion in limited-area ensemble forecasts. Mon. Weather Rev., 132, 2378–2390.

    Article  Google Scholar 

  • Ott, E., B.R. Hunt, I. Szunyogh, A.V. Zimin, E.J. Kostelich, M. Corazza, E. Kalnay, D.J. Patil and J.A. Yorke, 2004. A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415–428.

    Article  Google Scholar 

  • Pham, D.T., 2001. Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Weather Rev., 129, 1194–1207.

    Article  Google Scholar 

  • Pires, C., R. Vautard and O. Talagrand, 1996. On extending the limits of variational assimilation in chaotic systems. Tellus, 48A, 96–121.

    Article  Google Scholar 

  • Rabier, F., H. Järvinen, E. Klinker, J.-F. Mahfouf and A. Simmons, 2000. The ECMWF operational implementation of four-dimensional variational physics. Q. J. R. Meteorol. Soc., 126, 1143–1170.

    Article  Google Scholar 

  • Radakovich, J.D., P.R. Houser, A.M. da Silva and M.G. Bosilovich, 2001. Results from global land-surface data assimilation methods. Proceeding of the 5th Symposium on Integrated Observing Systems, 14–19 January 2001, Albuquerque, NM, pp 132–134.

    Google Scholar 

  • Reichle, R.H., W.T. Crow and C.L. Keppenne, 2008. An adaptive ensemble Kalman filter for soil moisture data assimilation. Water Resour. Res., 44, W03423, doi:10.1029/2007WR006357.

    Google Scholar 

  • Rotunno, R. and J.W. Bao, 1996. A case study of cyclogenesis using a model hierarchy. Mon. Weather Rev., 124, 1051–1066.

    Article  Google Scholar 

  • Szunyogh, I., E. Kostelich, G. Gyarmati, E. Kalnay, B.R. Hunt, E. Ott, E. Satterfield and J.A. Yorke, 2008. A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A, 113–130.

    Google Scholar 

  • Talagrand, O. and P. Courtier, 1987. Variational assimilation of meteorological observations with the adjoint vorticity equation I: theory. Q. J. R. Meteorol. Soc., 113, 1311–1328.

    Article  Google Scholar 

  • Thépaut, J.-N. and P. Courtier, 1991. Four-dimensional data assimilation using the adjoint of a multilevel primitive equation model. Q. J. R. Meteorol. Soc., 117, 1225–1254.

    Article  Google Scholar 

  • Tippett, M.K., J.L. Anderson, C.H. Bishop, T.M. Hamill and J.S. Whitaker, 2003. Ensemble square root filters. Mon. Weather Rev., 131, 1485–1490.

    Article  Google Scholar 

  • Torn, R.D. and G.J. Hakim, 2008. Performance characteristics of a Pseudo-Operational Ensemble Kalman Filter. Mon. Weather Review, 136, 3947–3963.

    Google Scholar 

  • Torn, R.D., G.J. Hakim and C. Snyder, 2006. Boundary conditions for limited-area ensemble Kalman filters. Mon. Weather Rev., 134, 2490–2502.

    Article  Google Scholar 

  • Trémolet, Y., 2007. Model-error estimation in 4D-Var. Q. J. R. Meteorol. Soc., 133, 1267–1280.

    Article  Google Scholar 

  • Wang, X., C.H. Bishop and S.J. Julier, 2004. Which is better, an ensemble of positive-negative pairs or a centered spherical simplex ensemble? Mon. Weather Rev., 132, 1590–1605.

    Article  Google Scholar 

  • Whitaker, J.S., G.P. Compo, X. Wei and T.M. Hamill, 2004. Reanalysis without radiosondes using ensemble data assimilation. Mon. Weather Rev., 132, 1190–1200.

    Article  Google Scholar 

  • Whitaker, J.S. and T.M. Hamill, 2002. Ensemble data assimilation without perturbed observations. Mon. Weather Rev., 130, 1913–1924.

    Article  Google Scholar 

  • Whitaker, J.S., T.M. Hamill, X. Wei, Y. Song and Z. Toth, 2008. Ensemble data assimilation with the NCEP global forecast system. Mon. Weather Rev., 136, 463–482.

    Article  Google Scholar 

  • Yang, S.-C., M. Corazza, A. Carrassi, E. Kalnay and T. Miyoshi, 2009a. Comparison of ensemble-based and variational-based data assimilation schemes in a quasi-geostrophic model. Mon. Weather Rev., 137, 693–709.

    Article  Google Scholar 

  • Yang, S.-C., E. Kalnay, B. Hunt and N. Bowler, 2009b. Weight interpolation for efficient data assimilation with the local ensemble transform Kalman filter. Q. J. R. Meteorol. Soc., 135, 251–262.

    Article  Google Scholar 

  • Zhu, Y. and R. Gelaro, 2008. Observation sensitivity calculations using the adjoint of the gridpoint statistical interpolation (GSI) analysis system. Mon. Weather Rev., 136, 335–351.

    Article  Google Scholar 

  • Zupanski, M., 2005. Maximum likelihood ensemble filter: Theoretical aspects. Mon. Weather Rev., 133, 1710–1726.

    Article  Google Scholar 

  • Zupanski, M., S.J. Fletcher, I.M. Navon, et al., 2006. Initiation of ensemble data assimilation. Tellus, 58A, 159–170.

    Article  Google Scholar 

Download references

Acknowledgments

I want to thank the members of the Chaos-Weather group at the University of Maryland, and in particular to Profs. Shu-Chih Yang, Brian Hunt, Kayo Ide, Eric Kostelich, Ed Ott, Istvan Szunyogh, and Jim Yorke. My deepest gratitude is to my former students at the University of Maryland, Drs. Matteo Corazza, Chris Danforth, Hong Li, Junjie Liu, Takemasa Miyoshi, Malaquías Peña, Shu-Chih Yang, Ji-Sun Kang, Matt Hoffman, and present students Steve Penny, Steve Greybush, Tamara Singleton, Javier Amezcua and others, whose creative research allowed us to learn together. Interactions with the thriving Ensemble Kalman Filter community members, especially Ross Hoffman, Jeff Whitaker, Craig Bishop, Kayo Ide, Joaquim Ballabrera, Jidong Gao, Zoltan Toth, Milija Zupanski, Tom Hamill, Herschel Mitchell, Peter Houtekamer, Chris Snyder, Fuqing Zhang and others, as well as with Michael Ghil, Arlindo da Silva, Jim Carton, Dick Dee, and Wayman Baker, have been a source of inspiration. Richard Ménard, Ross Hoffman, Kayo Ide, Lars Nerger and William Lahoz made important suggestions that improved the review and my own understanding of the subject.

Author information

Authors and Affiliations

  1. University of Maryland, College Park, MD, 20742-2425, USA

    Eugenia Kalnay

Authors
  1. Eugenia Kalnay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eugenia Kalnay .

Editor information

Editors and Affiliations

  1. Dept. Meteorology, University of Reading, Earley Gate, Reading, RG66BB, United Kingdom

    William Lahoz

  2. Arapahoe Avenue 373, Boulder, 80302, Colorado, USA

    Boris Khattatov

  3. Atmospheric Science & Technology, Directorate, Environment Canada, Trans-Canada Highway 2121, Dorval, H9P 1J3, Québec, Canada

    Richard Menard

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kalnay, E. (2010). Ensemble Kalman Filter: Current Status and Potential. In: Lahoz, W., Khattatov, B., Menard, R. (eds) Data Assimilation. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74703-1_4

Download citation

Publish with us

Policies and ethics

Search

Navigation