Advertisement

Ensemble Kalman Filter: Current Status and Potential

  • Eugenia KalnayEmail author
Chapter

Abstract

In this chapter we give an introduction to different types of Ensemble Kalman filter, describe the Local Ensemble Transform Kalman Filter (LETKF) as a representative prototype of these methods, and several examples of how advanced properties and applications that have been developed and explored for 4D-Var (four-dimensional variational assimilation) can be adapted to the LETKF without requiring an adjoint model. Although the Ensemble Kalman filter is less mature than 4D-Var (Kalnay 2003), its simplicity and its competitive performance with respect to 4D-Var suggest that it may become the method of choice.

Keywords

Kalman Filter Data Assimilation Ensemble Member Outer Loop Ensemble Forecast 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

I want to thank the members of the Chaos-Weather group at the University of Maryland, and in particular to Profs. Shu-Chih Yang, Brian Hunt, Kayo Ide, Eric Kostelich, Ed Ott, Istvan Szunyogh, and Jim Yorke. My deepest gratitude is to my former students at the University of Maryland, Drs. Matteo Corazza, Chris Danforth, Hong Li, Junjie Liu, Takemasa Miyoshi, Malaquías Peña, Shu-Chih Yang, Ji-Sun Kang, Matt Hoffman, and present students Steve Penny, Steve Greybush, Tamara Singleton, Javier Amezcua and others, whose creative research allowed us to learn together. Interactions with the thriving Ensemble Kalman Filter community members, especially Ross Hoffman, Jeff Whitaker, Craig Bishop, Kayo Ide, Joaquim Ballabrera, Jidong Gao, Zoltan Toth, Milija Zupanski, Tom Hamill, Herschel Mitchell, Peter Houtekamer, Chris Snyder, Fuqing Zhang and others, as well as with Michael Ghil, Arlindo da Silva, Jim Carton, Dick Dee, and Wayman Baker, have been a source of inspiration. Richard Ménard, Ross Hoffman, Kayo Ide, Lars Nerger and William Lahoz made important suggestions that improved the review and my own understanding of the subject.

References

  1. Anderson, J.L., 2001. An ensemble adjustment Kalman filter for data assimilation. Mon. Weather Rev., 129, 2884–2903.CrossRefGoogle Scholar
  2. Andersson, E., M. Fisher, E. Hólm, L. Isaksen, G. Radnoti and Y. Trémolet, 2005. Will the 4D-Var approach be defeated by nonlinearity? ECMWF Technical Memorandum, No. 479.Google Scholar
  3. Baek, S.-J., B.R. Hunt, E. Kalnay, E. Ott and I. Szunyogh, 2006. Local ensemble Kalman filtering in the presence of model bias. Tellus A, 58, 293–306.CrossRefGoogle Scholar
  4. Barker, D.M., 2008. How 4DVar can benefit from or contribute to EnKF (a 4DVar perspective). Available from http://4dvarenkf.cima.fcen.uba.ar/Download /Session_8/4DVar_EnKF_Barker.pdf.
  5. Bishop, C.H., B.J. Etherton and S.J. Majumdar, 2001. Adaptive sampling with ensemble transform Kalman filter. Part I: Theoretical aspects. Mon. Weather Rev., 129, 420–436.CrossRefGoogle Scholar
  6. Buehner, M., C. Charente, B. He, et al., 2008. Intercomparison of 4-D Var and EnKF systems for operational deterministic NWP. Available from http://4dvarenkf.cima.fcen.uba.ar/Download /Session_7/Intercomparison_4D-Var_EnKF_Buehner.pdf.
  7. Burgers, G., P.J. van Leeuwen and G. Evensen, 1998. On the analysis scheme in the ensemble Kalman filter. Mon. Weather Rev., 126, 1719–1724.CrossRefGoogle Scholar
  8. Cardinali, C., S. Pezzulli and E. Andersson, 2004. Influence-matrix diagnostic of a data assimilation system. Q. J. R. Meteorol. Soc., 130, 2767–2786.CrossRefGoogle Scholar
  9. Caya, A., J. Sun and C. Snyder, 2005. A comparison between the 4D-VAR and the ensemble Kalman filter techniques for radar data assimilation. Mon. Weather Rev., 133, 3081–3094.CrossRefGoogle Scholar
  10. Courtier, P. and O. Talagrand, 1990. Variational assimilation of meteorological observations with the direct and adjoint shallow water equations. Tellus, 42A, 531–549.CrossRefGoogle Scholar
  11. Danforth, C.M. and E. Kalnay, 2008. Using singular value decomposition to parameterize state dependent model errors. J. Atmos. Sci., 65, 1467–1478.CrossRefGoogle Scholar
  12. Danforth, C.M., E. Kalnay and T. Miyoshi, 2006. Estimating and correcting global weather model error. Mon. Weather Rev., 134, 281–299.Google Scholar
  13. Dee, D.P. and A.M. da Silva, 1998. Data assimilation in the presence of forecast bias. Q. J. R. Meteorol. Soc., 124, 269–295.CrossRefGoogle Scholar
  14. Desroziers, G., L. Berre, B. Chapnik and P. Poli, 2005. Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc., 131, 3385–3396.Google Scholar
  15. Evensen, G., 1994. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10,143–10,162.CrossRefGoogle Scholar
  16. Evensen, G., 2003. The ensemble Kalman filter: Theoretical formulation and practical implementation. Ocean Dyn., 53, 343–367.CrossRefGoogle Scholar
  17. Evensen, G. and P.J. van Leeuwen, 1996. Assimilation of Geosat altimeter data for the Agulhas current using the ensemble Kalman filter with a quasi-geostrophic model. Mon. Weather Rev., 124, 85–96.CrossRefGoogle Scholar
  18. Fisher, M., M. Leutbecher and G. Kelly, 2005. On the equivalence between Kalman smoothing and weak-constraint four-dimensional variational data assimilation. Q. J. R. Meteorol. Soc., 131, 3235–3246.CrossRefGoogle Scholar
  19. Gaspari, G. and S.E. Cohn, 1999. Construction of correlation functions in two and three dimensions. Q. J. R. Meteorol. Soc., 125, 723–757.CrossRefGoogle Scholar
  20. Ghil, M. and P. Malanotte-Rizzoli, 1991. Data assimilation in meteorology and oceanography. Adv. Geophys., 33, 141–266.CrossRefGoogle Scholar
  21. Greybush, S., E. Kalnay, T. Miyoshi, K. Ide and B. Hunt, 2009. EnKF localization techniques and balance. Presented at the WMO 5th International Symposium on Data Assimilation. Melbourne, Australia, 6–9 October 2009, submitted to Mon. Weather Rev.,Available at http://www.weatherchaos.umd.edu/papers/Greybush_Melbourne2009.ppt.
  22. Gustafsson, N., 2007. Response to the discussion on “4-D-Var or EnKF?”. Tellus A, 59, 778–780.CrossRefGoogle Scholar
  23. Hamill, T.M., J.S. Whitaker and C. Snyder, 2001. Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Weather Rev., 129, 2776–2790.CrossRefGoogle Scholar
  24. Harlim, J. and B.R. Hunt, 2007a. A non-Gaussian ensemble filter for assimilating infrequent noisy observations. Tellus A, 59, 225–237.CrossRefGoogle Scholar
  25. Harlim, J. and B.R. Hunt, 2007b. Four-dimensional local ensemble transform Kalman filter: Variational formulation and numerical experiments with a global circulation model. Tellus A, 59, 731–748.CrossRefGoogle Scholar
  26. Houtekamer, P.L. and H.L. Mitchell, 1998. Data assimilation using an ensemble Kalman filter technique. Mon. Weather Rev., 126, 796–811.CrossRefGoogle Scholar
  27. Houtekamer, P.L. and H.L. Mitchell, 2001. A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Weather Rev., 129, 123–137.CrossRefGoogle Scholar
  28. Houtekamer, P.L., H.L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek and B. Hansen, 2005. Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Weather Rev., 133, 604–620.CrossRefGoogle Scholar
  29. Hunt, B.R., 2005. An efficient implementation of the local ensemble Kalman filter. Available at http://arxiv.org./abs/physics/0511236.Google Scholar
  30. Hunt, B.R., E. Kalnay, E.J. Kostelich, et al., 2004. Four-dimensional ensemble Kalman filtering. Tellus, 56A, 273–277.CrossRefGoogle Scholar
  31. Hunt, B.R., E.J. Kostelich and I. Szunyogh, 2007. Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D, 230, 112–126.CrossRefGoogle Scholar
  32. Ide, K., P. Courtier, M. Ghil and A. Lorenc, 1997. Unified notation for data assimilation: Operational, sequential and variational. J. Meteorol. Soc. Jpn., 75, 181–189.Google Scholar
  33. Järvinen, H., E. Andersson and F. Bouttier, 1999. Variational assimilation of time sequences of surface observations with serially correlated errors. Tellus, 51A, 469–488.Google Scholar
  34. Jazwinski, A.H., 1970. Stochastic Processes and Filtering Theory. Academic Press, NY, 376 pp.Google Scholar
  35. Kalman, R.E., 1960. A new approach to linear filtering and prediction problems. J. Basic Eng., 82, 35–45.CrossRefGoogle Scholar
  36. Kalnay, E., 2003. Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, Cambridge, UK, 341 pp.Google Scholar
  37. Kalnay, E., H. Li, T. Miyoshi, S.-C. Yang and J. Ballabrera-Poy, 2007a. 4D-Var or ensemble Kalman filter? Tellus A, 59, 758–773.CrossRefGoogle Scholar
  38. Kalnay, E., H. Li, T. Miyoshi, S.-C. Yang and J. Ballabrera-Poy, 2007b. Response to the discussion on “4D-Var or EnKF?” by Nils Gustaffson. Tellus A, 59, 778–780.CrossRefGoogle Scholar
  39. Kalnay, E. and S.-C. Yang, 2008. Accelerating the spin-up in EnKF. Arxiv: physics:Nonlinear/0.806.0180v1.Google Scholar
  40. Keppenne, C.L., 2000. Data assimilation into a primitive-equation model with a parallel ensemble Kalman filter. Mon. Weather Rev., 128, 1971–1981.CrossRefGoogle Scholar
  41. Keppenne, C. and H. Rienecker, 2002. Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model. Mon. Weather Rev., 130, 2951–2965.CrossRefGoogle Scholar
  42. Langland, R.H. and N.L. Baker, 2004. Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56A, 189–201.CrossRefGoogle Scholar
  43. Li, H., 2007. Local ensemble transform Kalman filter with realistic observations. Ph. D. thesis. Available at http://hdl.handle.net/1903/7317.
  44. Li, H., E. Kalnay and T. Miyoshi, 2009a. Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Q. J. R. Meteorol. Soc., 135, 523–533.CrossRefGoogle Scholar
  45. Li, H., E. Kalnay, T. Miyoshi and C.M. Danforth, 2009b. Accounting for model errors in ensemble data assimilation. Mon. Weather Rev., 137, 3407–3419.CrossRefGoogle Scholar
  46. Liu, J. and E. Kalnay, 2008. Estimating observation impact without adjoint model in an ensemble Kalman filter. Q. J. R. Meteorol. Soc., 134, 1327–1335.CrossRefGoogle Scholar
  47. Liu, J., E. Kalnay, T. Miyoshi and C. Cardinali, 2009. Analysis sensitivity calculation in an ensemble Kalman filter. Q. J. R. Meteorol. Soc., 135, 523–533.CrossRefGoogle Scholar
  48. Lorenc, A.C., 1986. Analysis methods for numerical weather prediction. Q. J .R. Meteorol. Soc., 112, 1177–1194.CrossRefGoogle Scholar
  49. Lorenc, A.C., 2003. The potential of the ensemble Kalman filter for NWP – a comparison with 4D-Var. Q. J. R. Meteorol. Soc., 129, 3183–3203.CrossRefGoogle Scholar
  50. Lorenz, E., 1963. Deterministic non-periodic flow. J. Atmos. Sci., 20, 130–141.CrossRefGoogle Scholar
  51. Mitchell, H.L., P.L. Houtekamer and G. Pellerin, 2002. Ensemble size, balance, and model-error representation in an ensemble Kalman filter. Mon. Weather Rev., 130, 2791–2808.CrossRefGoogle Scholar
  52. Miyoshi, T., 2005. Ensemble Kalman Filter Experiments with a Primitive-Equation Global Model. Doctoral dissertation, University of Maryland, College Park, 197 pp. Available at https://drum.umd.edu/dspace/handle/1903/3046.Google Scholar
  53. Miyoshi, T. and S. Yamane, 2007. Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution. Mon. Weather Rev., 135, 3841–3861.CrossRefGoogle Scholar
  54. Molteni, F., 2003. Atmospheric simulations using a GCM with simplified physical parameterizations. I: Model climatology and variability in multi-decadal experiments. Clim. Dyn., 20, 175–191.Google Scholar
  55. Nerger, L., W. Hiller and J. Scroeter, 2005. A comparison of error subspace Kalman filters. Tellus, 57A, 715–735.CrossRefGoogle Scholar
  56. Nutter, P., M. Xue and D. Stensrud, 2004. Application of lateral boundary condition perturbations to help restore dispersion in limited-area ensemble forecasts. Mon. Weather Rev., 132, 2378–2390.CrossRefGoogle Scholar
  57. Ott, E., B.R. Hunt, I. Szunyogh, A.V. Zimin, E.J. Kostelich, M. Corazza, E. Kalnay, D.J. Patil and J.A. Yorke, 2004. A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415–428.CrossRefGoogle Scholar
  58. Pham, D.T., 2001. Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Weather Rev., 129, 1194–1207.CrossRefGoogle Scholar
  59. Pires, C., R. Vautard and O. Talagrand, 1996. On extending the limits of variational assimilation in chaotic systems. Tellus, 48A, 96–121.CrossRefGoogle Scholar
  60. Rabier, F., H. Järvinen, E. Klinker, J.-F. Mahfouf and A. Simmons, 2000. The ECMWF operational implementation of four-dimensional variational physics. Q. J. R. Meteorol. Soc., 126, 1143–1170.CrossRefGoogle Scholar
  61. Radakovich, J.D., P.R. Houser, A.M. da Silva and M.G. Bosilovich, 2001. Results from global land-surface data assimilation methods. Proceeding of the 5th Symposium on Integrated Observing Systems, 14–19 January 2001, Albuquerque, NM, pp 132–134.Google Scholar
  62. Reichle, R.H., W.T. Crow and C.L. Keppenne, 2008. An adaptive ensemble Kalman filter for soil moisture data assimilation. Water Resour. Res., 44, W03423, doi:10.1029/2007WR006357.Google Scholar
  63. Rotunno, R. and J.W. Bao, 1996. A case study of cyclogenesis using a model hierarchy. Mon. Weather Rev., 124, 1051–1066.CrossRefGoogle Scholar
  64. Szunyogh, I., E. Kostelich, G. Gyarmati, E. Kalnay, B.R. Hunt, E. Ott, E. Satterfield and J.A. Yorke, 2008. A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus, 60A, 113–130.Google Scholar
  65. Talagrand, O. and P. Courtier, 1987. Variational assimilation of meteorological observations with the adjoint vorticity equation I: theory. Q. J. R. Meteorol. Soc., 113, 1311–1328.CrossRefGoogle Scholar
  66. Thépaut, J.-N. and P. Courtier, 1991. Four-dimensional data assimilation using the adjoint of a multilevel primitive equation model. Q. J. R. Meteorol. Soc., 117, 1225–1254.CrossRefGoogle Scholar
  67. Tippett, M.K., J.L. Anderson, C.H. Bishop, T.M. Hamill and J.S. Whitaker, 2003. Ensemble square root filters. Mon. Weather Rev., 131, 1485–1490.CrossRefGoogle Scholar
  68. Torn, R.D. and G.J. Hakim, 2008. Performance characteristics of a Pseudo-Operational Ensemble Kalman Filter. Mon. Weather Review, 136, 3947–3963.Google Scholar
  69. Torn, R.D., G.J. Hakim and C. Snyder, 2006. Boundary conditions for limited-area ensemble Kalman filters. Mon. Weather Rev., 134, 2490–2502.CrossRefGoogle Scholar
  70. Trémolet, Y., 2007. Model-error estimation in 4D-Var. Q. J. R. Meteorol. Soc., 133, 1267–1280.CrossRefGoogle Scholar
  71. Wang, X., C.H. Bishop and S.J. Julier, 2004. Which is better, an ensemble of positive-negative pairs or a centered spherical simplex ensemble? Mon. Weather Rev., 132, 1590–1605.CrossRefGoogle Scholar
  72. Whitaker, J.S., G.P. Compo, X. Wei and T.M. Hamill, 2004. Reanalysis without radiosondes using ensemble data assimilation. Mon. Weather Rev., 132, 1190–1200.CrossRefGoogle Scholar
  73. Whitaker, J.S. and T.M. Hamill, 2002. Ensemble data assimilation without perturbed observations. Mon. Weather Rev., 130, 1913–1924.CrossRefGoogle Scholar
  74. Whitaker, J.S., T.M. Hamill, X. Wei, Y. Song and Z. Toth, 2008. Ensemble data assimilation with the NCEP global forecast system. Mon. Weather Rev., 136, 463–482.CrossRefGoogle Scholar
  75. Yang, S.-C., M. Corazza, A. Carrassi, E. Kalnay and T. Miyoshi, 2009a. Comparison of ensemble-based and variational-based data assimilation schemes in a quasi-geostrophic model. Mon. Weather Rev., 137, 693–709.CrossRefGoogle Scholar
  76. Yang, S.-C., E. Kalnay, B. Hunt and N. Bowler, 2009b. Weight interpolation for efficient data assimilation with the local ensemble transform Kalman filter. Q. J. R. Meteorol. Soc., 135, 251–262.CrossRefGoogle Scholar
  77. Zhu, Y. and R. Gelaro, 2008. Observation sensitivity calculations using the adjoint of the gridpoint statistical interpolation (GSI) analysis system. Mon. Weather Rev., 136, 335–351.CrossRefGoogle Scholar
  78. Zupanski, M., 2005. Maximum likelihood ensemble filter: Theoretical aspects. Mon. Weather Rev., 133, 1710–1726.CrossRefGoogle Scholar
  79. Zupanski, M., S.J. Fletcher, I.M. Navon, et al., 2006. Initiation of ensemble data assimilation. Tellus, 58A, 159–170.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.University of MarylandCollege ParkUSA

Personalised recommendations