Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Background error covariance iterative updating with invariant observation measures for data assimilation

  • 67 Accesses

Abstract

In order to leverage the information embedded in the background state and observations, covariance matrices modelling is a pivotal point in data assimilation algorithms. These matrices are often estimated from an ensemble of observations or forecast differences. Nevertheless, for many industrial applications the modelling still remains empirical based on some form of expertise and physical constraints enforcement in the absence of historical observations or predictions. We have developed two novel robust adaptive assimilation methods named Covariance Updating iTerativE and Partially Updating BLUE. These two non-parametric methods are based on different optimization objectives, both capable of sequentially adapting background error covariance matrices in order to improve assimilation results under the assumption of a good knowledge of the observation error covariances . We have compared these two methods with the standard approach using a misspecified background matrix in a shallow water twin experiments framework with a linear observation operator. Numerical experiments have shown that the proposed methods bear a real advantage both in terms of posterior error correlation identification and assimilation accuracy.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. 1.

    Here, by the term “exact”, we refer to the covariance truly corresponding to the remaining errors present in the analysed state, no matter the level of optimality of the chosen assimilation scheme.

References

  1. Argaud J-P, Bouriquet B, Courtois M, Le Roux J-C (2016) Reconstruction by data assimilation of the inner temperature field from outer measurements in a thick pipe. In: Pressure vessels and piping conference, British Columbia, Canada, July 17–21. ASME

  2. Asch M, Bocquet M, Maëlle N (2016) Data assimilation: methods, algorithms, and applications. Fundamentals of algorithms. SIAM, Philadelphia

  3. Bishop CH (2019) Data assimilation strategies for state dependent observation error variances. Q J R Meteorol Soc 145:217–227

  4. Bouttier F, Courtier P (1999) Data assimilation, concepts and methods. Meteorological training course lecture series. ECMWF, European Center for Medium range Weather Forecast, Reading, UK

  5. Carrassi A, Bocquet M, Bertino L, Evensen G (2018) Data assimilation in the geosciences: an overview of methods, issues, and perspectives. Wiley Interdiscip Rev Clim Change 9(5):e535

  6. Chabot V, Berre L, Desroziers G (2017) Diagnosis and normalization of gridpoint background-error variances induced by a block-diagonal wavelet covariance matrix. Q J R Meteorol Soc 143(704):1268–1279

  7. Chapnik B, Desroziers G, Rabier F, Talagrand O (2006) Diagnosis and tuning of observational error in a quasi-operational data assimilation setting. Q J R Meteorol Soc 132(615):543–565

  8. Cherian A, Sra S, Banerjee A, Papanikolopoulos N (2011) Efficient similarity search for covariance matrices via the Jensen–Bregman LogDet divergence. In: 2011 international conference on computer vision. IEEE, pp 2399–2406

  9. Cioaca A, Sandu A (2014) Low-rank approximations for computing observation impact in 4d-var data assimilation. Comput Math Appl 67(12):2112–2126

  10. Clayton AM, Lorenc AC, Barker DM (2012) Operational implementation of a hybrid ensemble/4D-VAR global data assimilation system at the MET office. Q J R Meteorol Soc 139(675):1445–1461

  11. Courtier P, Andersson E, Heckley W, Vasiljevic D, Hamrud M, Hollingsworth A, Rabier F, Fisher M, Pailleux J (1998) The ECMWF implementation of three-dimensional variational assimilation (3D-VAR). I: formulation. Q J R Meteorol Soc 124(550):1783–1807

  12. Desroziers G, Ivanov S (2001) Diagnosis and adaptive tuning of observation-error parameters in a variational assimilation. Q J R Meteorol Soc 127(574):1433–1452

  13. Desroziers G, Berre L, Chapnik B, Poli P (2005) Diagnosis of observation, background and analysis-error statistics in observation space. Q J R Meteorol Soc 131(613):3385–3396

  14. Dreano D, Tandeo P, Pulido M, Ait-El-Fquih B, Chonavel T, Hoteit I (2017) Estimating model error covariances in nonlinear state-space models using Kalman smoothing and the expectation–maximisation algorithm. Q J R Meteorol Soc 143(705):1877–1885

  15. Eyre JR, Hilton FI (2013) Sensitivity of analysis error covariance to the mis-specification of background error covariance. Q J R Meteorol Soc 139(671):524–533

  16. Fisher M (2003) Background error covariance modelling. In: Seminar on recent developments in data assimilation for atmosphere and ocean, Shinfield Park, Reading, 8–12 September. ECMWF

  17. Fisher M, Leutbecher M, Kelly GA (2005) On the equivalence between Kalman smoothing and weak-constraint four-dimensional variational data assimilation. Q J R Meteorol Soc 131(613):3235–3246

  18. Fitt A, Norbury J, Ockendon H, Eddie Wilson R (2010) Progress in industrial mathematics at ECMI 2008. In: Proceedings of the 15th European conference on mathematics for industry, vol 15, London, UK, June 30–July 4, 2008

  19. Gaspari G, Cohn ES (1999) Construction of correlation functions in two and three dimensions. Q J R Meteorol Soc 125(554):723–757

  20. Goeury C, Ponçot A, Argaud J-P, Zaoui F, Ata R, Audouin Y (2017) Optimal calibration of TELEMAC-2D models based on a data assimilation algorithm. In: the 14th TELEMAC-MASCARET user conference, 17–20 October 2017, Graz University of Technology, Graz, Austria

  21. Hodges J, Reich B (2010) Adding spatially-correlated errors can mess up the fixed effect you love. Am Stat 64(4):325–334

  22. Hollingsworth A, Lönnberg P (1989) The verification of objective analyses: diagnostics of analysis system performance. Meteorol Atmos Phys 40:3–27

  23. Hunt B, Kostelich E, Szunyogh I (2005) Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Phys D 230(1):112–126

  24. Ishibashi T (2015) Tensor formulation of ensemble-based background error covariance matrix factorization. Mon Weather Rev 143(12):4963–4973

  25. Janjić T, Bormann N, Bocquet M, Carton JA, Cohn SE, Dance SL, Losa SN, Nichols NK, Potthast R, Waller JA, Weston P (2018) On the representation error in data assimilation. Q J R Meteorol Soc 144(713):1257–1278

  26. Kalnay E, Yang S-C (2010) Accelerating the spin-up of Ensemble Kalman Filtering. Q J R Meteorol Soc 136(651):1644–1651

  27. Leisenring M, Moradkhani H (2011) Snow water equivalent prediction using Bayesian data assimilation methods. Stoch Environ Res Risk Assess 25(2):253–270

  28. Li W, Sankarasubramanian A, Ranjithan RS, Sinha T (2016) Role of multimodel combination and data assimilation in improving streamflow prediction over multiple time scales. Stoch Environ Res Risk Assess 30(8):2255–2269

  29. Liu C, Xue M (2016) Relationships among four-dimensional hybrid ensemble-variational data assimilation algorithms with full and approximate ensemble covariance localization. Mon Weather Rev 144(2):591–606

  30. Lucor D, Le Maître OP (2018) Cardiovascular modeling with adapted parametric inference. ESAIM: ProcS 62:91–107

  31. Mirouze I (2010) Régularisation de problèmes inverses à l’aide de l’équation de diffusion, avec application à l’assimilation variationnelle de données océaniques. Ph.D. thesis, Université de Toulouse

  32. Parrish DF, Derber JC (1992) The National Meteorological Center’s spectral statistical-interpolation analysis system. Mon Weather Rev 120(8):1747–1763

  33. Pennec X, Fillard P, Ayache N (2006) A Riemannian framework for tensor computing. Int J Comput Vis 66(1):41–66

  34. Ponçot A, Argaud J-P, Bouriquet B, Erhard P, Gratton S, Thual O (2013) Variational assimilation for xenon dynamical forecasts in neutronic using advanced background error covariance matrix. Ann Nucl Energy 60:39–50 10

  35. Rabier F (2005) Overview of global data assimilation developments in numerical weather-prediction centres. Q J R Meteorol Soc 131(613):3215–3233

  36. Rochoux M, Collin A, Zhang C, Trouvé A, Lucor D, Moireau P (2018) Front shape similarity measure for shape-oriented sensitivity analysis and data assimilation for Eikonal equation. ESAIM: ProcS 63:258–279

  37. Saint-Venant AB (1871) Théorie du mouvement non permanent des eaux, avec application aux crues des rivières et a l’introduction de marées dans leurs lits. Comptes rendus de l’Académie des Sciences 73:147–154

  38. Sénégas J, Wackernagel H, Rosenthal W, Wolf T (2001) Error covariance modeling in sequential data assimilation. Stoch Environ Res Risk Assess 15(1):65–86

  39. Singh K, Jardak M, Sandu A, Bowman K, Lee M, Jones D (2011) Construction of non-diagonal background error covariance matrices for global chemical data assimilation. Geosci Model Dev 4(2):299–316

  40. Sinsbeck M, Tartakovsky D (2015) Impact of data assimilation on cost-accuracy tradeoff in multifidelity models. SIAM/ASA J Uncertain Quantif 3(1):954–968

  41. Sørensen JVT, Madsen H (2004) Data assimilation in hydrodynamic modelling: on the treatment of non-linearity and bias. Stoch Environ Res Risk Assess 18(4):228–244

  42. Stewart LM, Dance SL, Nichols NK (2013) Data assimilation with correlated observation errors: experiments with a 1-D shallow water model. Tellus A: Dyn Meteorol Oceanogr 65(1):19546

  43. Talagrand O (1999) A posteriori evaluation and verification of analysis and assimilation algorithms. In: Workshop on diagnosis of data assimilation systems, Shinfield Park, Reading, ECMWF, pp 17–28

  44. Waller JA, Simonin D, Dance SL, Nichols NK, Ballard SP (2016) Diagnosing observation error correlations for doppler radar radial winds in the Met office UKV model using observation-minus-background and observation-minus-analysis statistics. Mon Weather Rev 144(10):3533–3551

  45. Weaver AT, Mirouze I (2013) On the diffusion equation and its application to isotropic and anisotropic correlation modelling in variational assimilation. Q J R Meteorol Soc 139(670):242–260

  46. Xu M, Yuan B, Wang L, Zhang L (2017) Data assimilation for Fukushima nuclear accident assessments. In: International conference on nuclear engineering, vol 4, Shanghai, China. ASME

Download references

Author information

Correspondence to Didier Lucor.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cheng, S., Argaud, J., Iooss, B. et al. Background error covariance iterative updating with invariant observation measures for data assimilation. Stoch Environ Res Risk Assess 33, 2033–2051 (2019). https://doi.org/10.1007/s00477-019-01743-6

Download citation

Keywords

  • Data assimilation
  • Covariance matrices
  • Iterative methods
  • Correlation structure
  • Iterative learning