Advertisement

Fundamentals of Data Assimilation and Theoretical Advances

  • Hamid Moradkhani
  • Grey S. Nearing
  • Peyman Abbaszadeh
  • Sahani Pathiraja
Reference work entry

Abstract

Hydrometeorological predictions are not perfect as models often suffer either from inadequate conceptualization of underlying physics or non-uniqueness of model parameters or inaccurate initialization. During the past two decades, Data Assimilation (DA) has received increased prominence among researchers and practitioners as an effective and reliable method to integrate the hydrometeorological observations from in situ measure and remotely-sensed sensors into predictive models for enhancing the forecast skills while taking into account all sources of uncertainties. The successful application of DA in different disciplines has resulted in an ever-increasing publications. This chapter provides a progressive essay covering fundamental and theoretical underpinnings of DA techniques and their applications in a variety of scientific fields. More detailed examples of applications are presented in following chapters in this section.

Keywords

Hydrometeorological predictions Uncertainty Data Assimilation (DA) 

References

  1. P. Abbaszadeh, H. Moradkhani, H. Yan, Enhancing hydrologic data assimilation by evolutionary Particle Filter and Markov Chain Monte Carlo method. Adv. Water Resour. 111, 192–204 (2018).  https://doi.org/10.1016/j.advwatres.2017.11.011CrossRefGoogle Scholar
  2. K.M. Andreadis, D.P. Lettenmaier, Assimilating remotely sensed snow observations into a macroscale hydrology model. Adv. Water Resour. 29, 872–886 (2006)CrossRefGoogle Scholar
  3. J.D. Annan, J.C. Hargreaves, N.R. Edwards, R. Marsh, Parameter estimation in an intermediate complexity Earth system model using an ensemble Kalman filter. Ocean Model. 8(1), 135–154 (2005)CrossRefGoogle Scholar
  4. M.S. Arulampalam, S. Maskell, N. Gordon, T. Clapp, A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)CrossRefGoogle Scholar
  5. D.M. Barker, W. Huang, Y.-R. Guo, A.J. Bourgeois, Q.N. Xiao, A three-dimensional variational data assimilation system for MM5: implementation and initial results. Mon. Weather Rev. 132(4), 897–914 (2004)CrossRefGoogle Scholar
  6. T. Bengtsson, P. Bickel, B. Li, Curse of dimensionality revisited: the collapse of importance sampling in very large scale systems, in IMS Collections: Probability and Statistics: Essays in Honor of David A. Freedman,, vol. 2, ed. by D. Nolan, T. Speed (Institute of Mathematical Statistics, Beachwood), pp. 316–334 (2008)Google Scholar
  7. N. Bulygina, H. Gupta, Estimating the uncertain mathematical structure of a water balance model via Bayesian data assimilation. Water Resour. Res. 45(12), W00B13 (2009).  https://doi.org/10.1029/2007WR006749CrossRefGoogle Scholar
  8. N. Bulygina, H. Gupta, How Bayesian data assimilation can be used to estimate the mathematical structure of a model. Stoch. Environ. Res. Risk Assess. 24(6), 925 (2010).  https://doi.org/10.1007/s00477-00010-00387-yCrossRefGoogle Scholar
  9. N. Bulygina, H. Gupta, Correcting the mathematical structure of a hydrological model via Bayesian data assimilation. Water Resour. Res. 47(5), W05514 (2011).  https://doi.org/10.1029/2010WR009614CrossRefGoogle Scholar
  10. M.P. Clark, D.E. Rupp, R.A. Woods, X. Zheng, R.P. Ibbitt, A.G. Slater, J. Schmidt, M.J. Uddstrom, Hydrological data assimilation with the ensemble Kalman filter: use of streamflow observations to update states in a distributed hydrological model. Adv. Water Resour. 31, 1309 (2008)CrossRefGoogle Scholar
  11. W.T. Crow, E.F. Wood, The assimilation of remotely sensed soil brightness temperature imagery into a land surface model using ensemble Kalman filtering: a case study based on ESTAR measurements during SGP97. Adv. Water Resour. 26(2), 137–149 (2003)CrossRefGoogle Scholar
  12. G.J.M. De Lannoy, R.H. Reichle, P.R. Houser, V.R.N. Pauwels, N.E.C. Verhoest, Correcting for forecast bias in soil moisture assimilation with the ensemble Kalman filter. Water Resour. Res. 43, W09410 (2007).  https://doi.org/10.1029/2006WR00544CrossRefGoogle Scholar
  13. G.J.M. De Lannoy, R.H. Reichle, K.R. Arsenault, P.R. Houser, S. Kumar, N.E.C. Verhoest, V. Pauwels, Multiscale assimilation of advanced microwave scanning radiometer–EOS snow water equivalent and moderate resolution imaging spectroradiometer snow cover fraction observations in northern Colorado. Water Resour. Res. 48, W01522 (2012).  https://doi.org/10.1029/2011WR010588
  14. P. De Rosnay, M. Drusch, D. Vasiljevic, G. Balsamo, C. Albergel, L. Isaksen, A simplified Extended Kalman Filter for the global operational soil moisture analysis at ECMWF. Q. J. R. Meteorol. Soc. 139(674), 1199–1213 (2013).  https://doi.org/10.1002/qj.2023CrossRefGoogle Scholar
  15. C. DeChant, H. Moradkhani, Improving the characterization of initial condition for ensemble streamflow prediction using data assimilation. Hydrol. Earth Syst. Sci. 15, 3399–3410 (2011a).  https://doi.org/10.5194/hess-15-3399CrossRefGoogle Scholar
  16. C. DeChant, H. Moradkhani, Radiance data assimilation for operational snow and streamflow forecasting. Adv. Water Resour. 34(3), 351–364 (2011b)CrossRefGoogle Scholar
  17. C.M. DeChant, H. Moradkhani, Examining the effectiveness and robustness of sequential data assimilation methods for quantification of uncertainty in hydrologic forecasting. Water Resour. Res. 48(4), W04518 (2012)CrossRefGoogle Scholar
  18. C.M. DeChant, H. Moradkhani, Toward a reliable prediction of seasonal forecast uncertainty: addressing model and initial condition uncertainty with ensemble data assimilation and sequential Bayesian combination. J. Hydrol. 519, 2967–2977 (2014a).  https://doi.org/10.1016/j.jhydrol.2014.05.045. Special issue on Ensemble Forecasting and data assimilationCrossRefGoogle Scholar
  19. C.M. DeChant, H. Moradkhani, Hydrologic prediction and uncertainty quantification, in Handbook of Engineering Hydrology, Modeling, Climate Change and Variability (CRC Press, Taylor & Francis Group, Boca Raton, 2014b), pp. 387–414CrossRefGoogle Scholar
  20. D.P. Dee et al., The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137(656), 553–597 (2011)CrossRefGoogle Scholar
  21. R. Douc, O. Cappe, Comparison of resampling schemes for particle filtering, paper presented at image and signal processing and analysis, 2005. ISPA 2005, in Proceedings of the 4th International Symposium on, 15–17 Sept 2005 (2005)Google Scholar
  22. M. Durand, S.A. Margulis, Effects of uncertainty magnitude and accuracy on assimilation of multiscale measurements for snowpack characterization. J. Geophys. Res. 113(D2), D02105 (2008)Google Scholar
  23. R.M. Errico, What is an adjoint model? Bull. Am. Meteorol. Soc. 78(11), 2577–2591 (1997)CrossRefGoogle Scholar
  24. G. Evensen, The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn. 53(4), 343–367 (2003)CrossRefGoogle Scholar
  25. Z. Ghahramani, S.T. Roweis, Learning nonlinear dynamical systems using an EM algorithm. Adv. Neural Inf. Process. Syst. 11, 431–437 (1999)Google Scholar
  26. M.E. Gharamti, J. Tjiputra, I. Bethke, A. Samuelsen, I. Skjelvan, M. Bentsen, L. Bertino, Ensemble data assimilation for ocean biogeochemical state and parameter estimation at different sites. Ocean Model. 112, 65–89 (2017)CrossRefGoogle Scholar
  27. R. Giering, Tangent Linear and Adjoint Model Compiler, Users Manual (Center for Global Change Sciences, Department of Earth, Atmospheric, and Planetary Science. MIT, Cambridge, 1997)Google Scholar
  28. N. Gordon, D. Salmond, A. Smith, Novel approach to nonlinear/non-Gaussian Bayesian state estimation. Proc. Inst. Elect. Eng. F. 140(2), 107–113 (1993)CrossRefGoogle Scholar
  29. P. Guingla, D. Antonio, R. De Keyser, G. De Lannoy, L. Giustarini, P. Matgen, V. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012)CrossRefGoogle Scholar
  30. C.M. Hoppe, H. Elbern, J. Schwinger, A variational data assimilation system for soil–atmosphere flux estimates for the Community Land Model (CLM3. 5). Geosci. Model Dev. 7(3), 1025–1036 (2014)CrossRefGoogle Scholar
  31. T. Hou, F. Kong, X. Chen, H. Lei, Impact of 3DVAR data assimilation on the prediction of heavy rainfall over Southern China. Adv. Meteorol. 2013, 1 (2013)CrossRefGoogle Scholar
  32. P.L. Houtekamer, H.L. Mitchell, Data assimilation using an ensemble Kalman filter technique. Mon. Weather Rev. 126(3), 796–811 (1998)CrossRefGoogle Scholar
  33. R.E. Kalman, A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(Series D), 35–45 (1960).  https://doi.org/10.1115/1111.3662552CrossRefGoogle Scholar
  34. S. Kumar, C. Peters-Lidard, D. Mocko, R. Reichle, Y. Liu, K. Arsenault, Y. Xia, M. Ek, G. Riggs, B. Livneh, M Cosh, Assimilation of remotely sensed soil moisture and snow depth retrievals for drought estimation. J. Hydrometeorol. 15, 2446–2469 (2014).  https://doi.org/10.1175/JHM-D-13-0132.1CrossRefGoogle Scholar
  35. H. Lee, D.J. Seo, Y. Liu, V. Koren, P. McKee, R. Corby, Variational assimilation of streamflow into operational distributed hydrologic models: effect of spatiotemporal scale of adjustment. Hydrol. Earth Syst. Sci. 16(7), 2233–2251 (2012)CrossRefGoogle Scholar
  36. M. Leisenring, H. Moradkhani, Snow water equivalent prediction using Bayesian data assimilation methods. Stoch. Environ. Res. Risk Assess. 25(2), 253–270 (2011)CrossRefGoogle Scholar
  37. M. Leisenring, H. Moradkhani, Analyzing the uncertainty of suspended sediment load prediction using sequential Monte Carlo methods. J. Hydrol. 468–469, 268–282 (2012).  https://doi.org/10.1016/j.jhydrol.2012.08.049CrossRefGoogle Scholar
  38. Y. Liu, A.H. Weerts, M. Clark, H.J. Hendricks Franssen, S. Kumar, H. Moradkhani, D.J. Seo, D. Schwanenberg, P. Smith, A.I.J.M. van Dijk, N. van Velzen, M. He, H. Lee, S.J. Noh, O. Rakovec, P. Restrepo, Toward advancing data assimilation in operational hydrologic forecasting and water resources management: current status, challenges, and emerging opportunities. Hydrol. Earth Syst. Sci. 16, 3863–3887 (2012)CrossRefGoogle Scholar
  39. A.C. Lorenc, The potential of the ensemble Kalman filter for NWP – a comparison with 4D-Var. Q. J. R. Meteorol. Soc. 129(595), 3183–3203 (2003)CrossRefGoogle Scholar
  40. P. Matgen, R. Hostache, G. Schumann, L. Pfister, L. Hoffmann, H.H.G. Savenije, Towards an automated SAR-based flood monitoring system, Lessons learned from two case studies. Phys. Chem. Earth. 36(7–8), 241–252 (2011).  https://doi.org/10.1016/j.pce.2010.12.009CrossRefGoogle Scholar
  41. C.L. Meng, Z.L. Li, X. Zhan, J.C. Shi, C. Y. Liu, Land surface temperature data assimilation and its impact on evapotranspiration estimates from the Common Land Model. Water Resour. Res. 45, W02421 (2009).  https://doi.org/10.1029/2008WR006971
  42. C. Montzka, H. Moradkhani, L. Weihermüller, H.J. Hendricks Franssen, M. Canty, H. Vereecken, Hydraulic parameter estimation by remotely-sensed top soil moisture observations with the particle filter. J. Hydrol. 399(3–4), 410–421 (2011).  https://doi.org/10.1016/j.jhydrol.2011.01.020CrossRefGoogle Scholar
  43. C. Montzka, J. Grant, H. Moradkhani, H.J. Hendricks Franssen, L. Weihermüller, M. Drusch, H. Vereecken, Estimation of radiative transfer parameters from L-Band passive microwave brightness temperatures using data assimilation. Vadose Zone Hydrol. Special Issue of Remote Sensing. (2013).  https://doi.org/10.2136/vzj2012.0040CrossRefGoogle Scholar
  44. H. Moradkhani, S. Sorooshian, H.V. Gupta, P.R. Houser, Dual state–parameter estimation of hydrological models using ensemble Kalman filter. Adv. Water Resour. 28(2), 135–147 (2005a)CrossRefGoogle Scholar
  45. H. Moradkhani, K.L. Hsu, H. Gupta, S. Sorooshian, Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter. Water Resour. Res. 41, W05012 (2005b)CrossRefGoogle Scholar
  46. H. Moradkhani, C.M. DeChant, S. Sorooshian, Evolution of ensemble data assimilation for uncertainty quantification using the Particle Filter-Markov Chain Monte Carlo method. Water Resour. Res. 48, W12520 (2012).  https://doi.org/10.1029/2012WR012144CrossRefGoogle Scholar
  47. G.S. Nearing, H.V. Gupta, The quantity and quality of information in hydrologic models. Water Resour. Res. 51(1), 524–538 (2015)CrossRefGoogle Scholar
  48. G.S. Nearing, H.V. Gupta, W.T. Crow, Information loss in approximately bayesian estimation techniques: a comparison of generative and discriminative approaches to estimating agricultural productivity. J. Hydrol. 507, 163–173 (2013)CrossRefGoogle Scholar
  49. S.J. Noh, Y. Tachikawa, M. Shiiba, S. Kim, Applying sequential Monte Carlo methods into a distributed hydrologic model: lagged particle filtering approach with regularization. Hydrol. Earth Syst. Sci. 15(10), 3237 (2011)CrossRefGoogle Scholar
  50. S. Park, J.P. Hwang, E. Kim, H. Kang, A new evolutionary particle filter for the prevention of sample impoverishment. IEEE Trans. Signal Process. 13(4), 801–809 (2009)Google Scholar
  51. M. Parrish, H. Moradkhani, C.M. DeChant, Towards reduction of model uncertainty: integration of Bayesian model averaging and data assimilation. Water Resour. Res. 48, W03519 (2012).  https://doi.org/10.1029/2011WR011116CrossRefGoogle Scholar
  52. S. Pathiraja, L. Marshall, A. Sharma, H. Moradkhani, Detecting non-stationary hydrologic model parameters in a paired catchment system using data assimilation. Adv. Water Resour. 94, 103–119 (2016a).  https://doi.org/10.1016/j.advwatres.2016.04.021CrossRefGoogle Scholar
  53. S. Pathiraja, L. Marshall, A. Sharma, H. Moradkhani, Hydrologic modeling in dynamic catchments: a data assimilation approach. Water Resour. Res. (2016b).  https://doi.org/10.1002/2015WR017192CrossRefGoogle Scholar
  54. S. Pathiraja, D. Anghileri, P. Burlando, A. Sharma, L. Marshall, H. Moradkhani, Time varying parameter models for catchments with land use change: the importance of model structure. Hydrol. Earth Syst. Sci. Discuss. (2017).  https://doi.org/10.5194/hess-2017-382
  55. S. Pathiraja, H. Moradkhani, L. Marshall, A. Sharma, G. Geenens, Data driven model uncertainty estimation in data assimilation. Water Resour. Res. (2018a).  https://doi.org/10.1002/2018WR022627CrossRefGoogle Scholar
  56. S. Pathiraja, D. Anghileri, P. Burlando, A. Sharma, L. Marshall, H. Moradkhani, Insights on the impact of systematic model errors on data assimilation performance in changing catchments. Adv. Water Resour. (2018b).  https://doi.org/10.1016/j.advwatres.2017.12.006CrossRefGoogle Scholar
  57. D.A. Plaza, R. De Keyser, G.J.M. De Lannoy, L. Giustarini, P. Matgen, V.R.N. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012)CrossRefGoogle Scholar
  58. R.H. Reichle, D. Entekhabi, D.B. McLaughlin, Downscaling of radio brightness measurements for soil moisture estimation: a four-dimensional variational data assimilation approach. Water Resour. Res. 37(9), 2353–2364 (2001)CrossRefGoogle Scholar
  59. R.H. Reichle, D.B. McLaughlin, D. Entekhabi, Hydrologic data assimilation with the ensemble Kalman filter. Mon. Weather Rev. 130(1), 103–114 (2002)CrossRefGoogle Scholar
  60. J. Ruiz, M. Pulido, Parameter estimation using ensemble-based data assimilation in the presence of model error. Mon. Weather Rev. 143(5), 1568–1582 (2015)CrossRefGoogle Scholar
  61. P. Salamon, L. Feyen, Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter. J. Hydrol. 376(3), 428–442 (2009)CrossRefGoogle Scholar
  62. J. Samuel, P. Coulibaly, G. Dumedah, H. Moradkhani, Assessing model state variation in hydrologic data assimilation. J. Hydrol. 513, 127–141 (2014).  https://doi.org/10.1016/j.jhydrol.2014.03.048CrossRefGoogle Scholar
  63. D.-J. Seo, V. Koren, N. Cajina, Real-time variational assimilation of hydrologic and hydrometeorological data into operational hydrologic forecasting. J. Hydrometeorol. 4(3), 627–641 (2003)CrossRefGoogle Scholar
  64. D.J. Seo, Y. Liu, H. Moradkhani, A. Weerts, Ensemble prediction and data assimilation for operational hydrology. J. Hydrol. 519, 2661–2662 (2014).  https://doi.org/10.1016/j.jhydrol.2014.11.035CrossRefGoogle Scholar
  65. A.G. Slater, M.P. Clark, Snow data assimilation via an ensemble Kalman filter. J. Hydrometeorol. 7, 478 (2005)CrossRefGoogle Scholar
  66. P.J. Smith, G.D. Thornhill, S.L. Dance, A.S. Lawless, D.C. Mason, N.K. Nichols, Data assimilation for state and parameter estimation: application to morphodynamic modelling. Q. J. R. Meteorol. Soc. 139(671), 314–327 (2013)CrossRefGoogle Scholar
  67. C. Snyder, T. Bengtsson, P. Bickel, J. Anderson, Obstacles to high-dimensional particle filtering. Mon. Weather Rev. 136(12), 4629 (2008)CrossRefGoogle Scholar
  68. J.A. Vrugt, C.G.H. Diks, H.V. Gupta, W. Bouten, J.M. Verstraten, Improved treatment of uncertainty in hydrologic modeling: combining the strengths of global optimization and data assimilation. Water Resour. Res. 41(1), W01017 (2005).  https://doi.org/10.1029/2004WR003059CrossRefGoogle Scholar
  69. A.H. Weerts, G.Y.H. El Serafy, Particle filtering and ensemble Kalman filtering for state updating with hydrological conceptual rainfall-runoff models. Water Resour. Res. 42, W09403 (2006).  https://doi.org/10.1029/2005WR004093
  70. J.S. Whitaker, T.M. Hamill, Ensemble data assimilation without perturbed observations. Monthly Weather Rev. 130(7), 1913–1924 (2002).  https://doi.org/10.1175/1520-0493(2002)130<1913:EDAWPO>2.0.CO;2CrossRefGoogle Scholar
  71. R.D. Wilkinson, M. Vrettas, D. Cornford, J.E. Oakley, Quantifying simulator discrepancy in discrete-time dynamical simulators. J. Agric. Biol. Environ. Stat. 16(4), 554–570 (2011)CrossRefGoogle Scholar
  72. H. Yan, H. Moradkhani, Combined assimilation of streamflow and satellite soil moisture with the particle filter and geostatistical modeling. Adv. Water Resour. 94, 364–378 (2016).  https://doi.org/10.1016/j.advwatres.2016.06.002CrossRefGoogle Scholar
  73. H. Yan, C.M. DeChant, H. Moradkhani, Improving soil moisture profile prediction with the Particle Filter-Markov Chain Monte Carlo method. IEEE Trans. Geosci. Remote Sens. (2015).  https://doi.org/10.1109/TGRS.2015.2432067CrossRefGoogle Scholar
  74. H. Yan, H. Moradkhani, M. Zarekarizi, A probabilistic drought forecasting framework: a combined dynamical and statistical approach. J. Hydrol. 548, 291–304 (2017).  https://doi.org/10.1016/j.jhydrol.2017.03.004CrossRefGoogle Scholar
  75. S. Yin, X. Zhu, Intelligent particle filter and its application to fault detection of nonlinear systems. IEEE Trans. Ind. Electron. 62(6), 3852–3861 (2015)Google Scholar
  76. D.a. Županski, F. Mesinger, Four-dimensional variational assimilation of precipitation data. Mon. Weather Rev. 123(4), 1112–1127 (1995)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Hamid Moradkhani
    • 1
  • Grey S. Nearing
    • 2
  • Peyman Abbaszadeh
    • 1
  • Sahani Pathiraja
    • 3
  1. 1.Department of Civil, Construction and Environmental EngineeringThe University of AlabamaTuscaloosaUSA
  2. 2.Department of Geological SciencesUniversity of AlabamaTuscaloosaUSA
  3. 3.Institute for Mathematics, University of PotsdamPotsdamGermany

Section editors and affiliations

  • Hamid Moradkhani
    • 1
  • Albrecht Weerts
    • 2
  1. 1.Department of Civil & Environmental EngineeringPortland State UniversityPortlandUSA
  2. 2.Inland Water SystemsDeltaresThe Netherlands

Personalised recommendations