Skip to main content
SpringerLink
Log in

Fundamentals of Data Assimilation and Theoretical Advances

  • Living reference work entry
  • First Online:
Handbook of Hydrometeorological Ensemble Forecasting

Abstract

Hydrometeorological predictions are not perfect as models often suffer either from inadequate conceptualization of underlying physics or non-uniqueness of model parameters or inaccurate initialization. During the past two decades, Data Assimilation (DA) has received increased prominence among researchers and practitioners as an effective and reliable method to integrate the hydrometeorological observations from in situ measure and remotely-sensed sensors into predictive models for enhancing the forecast skills while taking into account all sources of uncertainties. The successful application of DA in different disciplines has resulted in an ever-increasing publications. This chapter provides a progressive essay covering fundamental and theoretical underpinnings of DA techniques and their applications in a variety of scientific fields. More detailed examples of applications are presented in following chapters in this section.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • P. Abbaszadeh, H. Moradkhani, H. Yan, Enhancing hydrologic data assimilation by evolutionary Particle Filter and Markov Chain Monte Carlo method. Adv. Water Resour. 111, 192–204 (2018). https://doi.org/10.1016/j.advwatres.2017.11.011

    Article  Google Scholar 

  • K.M. Andreadis, D.P. Lettenmaier, Assimilating remotely sensed snow observations into a macroscale hydrology model. Adv. Water Resour. 29, 872–886 (2006)

    Article  Google Scholar 

  • J.D. Annan, J.C. Hargreaves, N.R. Edwards, R. Marsh, Parameter estimation in an intermediate complexity Earth system model using an ensemble Kalman filter. Ocean Model. 8(1), 135–154 (2005)

    Article  Google Scholar 

  • M.S. Arulampalam, S. Maskell, N. Gordon, T. Clapp, A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)

    Article  Google Scholar 

  • D.M. Barker, W. Huang, Y.-R. Guo, A.J. Bourgeois, Q.N. Xiao, A three-dimensional variational data assimilation system for MM5: implementation and initial results. Mon. Weather Rev. 132(4), 897–914 (2004)

    Article  Google Scholar 

  • T. Bengtsson, P. Bickel, B. Li, Curse of dimensionality revisited: the collapse of importance sampling in very large scale systems, in IMS Collections: Probability and Statistics: Essays in Honor of David A. Freedman,, vol. 2, ed. by D. Nolan, T. Speed (Institute of Mathematical Statistics, Beachwood), pp. 316–334 (2008)

    Google Scholar 

  • N. Bulygina, H. Gupta, Estimating the uncertain mathematical structure of a water balance model via Bayesian data assimilation. Water Resour. Res. 45(12), W00B13 (2009). https://doi.org/10.1029/2007WR006749

    Article  Google Scholar 

  • N. Bulygina, H. Gupta, How Bayesian data assimilation can be used to estimate the mathematical structure of a model. Stoch. Environ. Res. Risk Assess. 24(6), 925 (2010). https://doi.org/10.1007/s00477-00010-00387-y

    Article  Google Scholar 

  • N. Bulygina, H. Gupta, Correcting the mathematical structure of a hydrological model via Bayesian data assimilation. Water Resour. Res. 47(5), W05514 (2011). https://doi.org/10.1029/2010WR009614

    Article  Google Scholar 

  • M.P. Clark, D.E. Rupp, R.A. Woods, X. Zheng, R.P. Ibbitt, A.G. Slater, J. Schmidt, M.J. Uddstrom, Hydrological data assimilation with the ensemble Kalman filter: use of streamflow observations to update states in a distributed hydrological model. Adv. Water Resour. 31, 1309 (2008)

    Article  Google Scholar 

  • W.T. Crow, E.F. Wood, The assimilation of remotely sensed soil brightness temperature imagery into a land surface model using ensemble Kalman filtering: a case study based on ESTAR measurements during SGP97. Adv. Water Resour. 26(2), 137–149 (2003)

    Article  Google Scholar 

  • G.J.M. De Lannoy, R.H. Reichle, P.R. Houser, V.R.N. Pauwels, N.E.C. Verhoest, Correcting for forecast bias in soil moisture assimilation with the ensemble Kalman filter. Water Resour. Res. 43, W09410 (2007). https://doi.org/10.1029/2006WR00544

    Article  Google Scholar 

  • G.J.M. De Lannoy, R.H. Reichle, K.R. Arsenault, P.R. Houser, S. Kumar, N.E.C. Verhoest, V. Pauwels, Multiscale assimilation of advanced microwave scanning radiometer–EOS snow water equivalent and moderate resolution imaging spectroradiometer snow cover fraction observations in northern Colorado. Water Resour. Res. 48, W01522 (2012). https://doi.org/10.1029/2011WR010588

  • P. De Rosnay, M. Drusch, D. Vasiljevic, G. Balsamo, C. Albergel, L. Isaksen, A simplified Extended Kalman Filter for the global operational soil moisture analysis at ECMWF. Q. J. R. Meteorol. Soc. 139(674), 1199–1213 (2013). https://doi.org/10.1002/qj.2023

    Article  Google Scholar 

  • C. DeChant, H. Moradkhani, Improving the characterization of initial condition for ensemble streamflow prediction using data assimilation. Hydrol. Earth Syst. Sci. 15, 3399–3410 (2011a). https://doi.org/10.5194/hess-15-3399.

    Article  Google Scholar 

  • C. DeChant, H. Moradkhani, Radiance data assimilation for operational snow and streamflow forecasting. Adv. Water Resour. 34(3), 351–364 (2011b)

    Article  Google Scholar 

  • C.M. DeChant, H. Moradkhani, Examining the effectiveness and robustness of sequential data assimilation methods for quantification of uncertainty in hydrologic forecasting. Water Resour. Res. 48(4), W04518 (2012)

    Article  Google Scholar 

  • C.M. DeChant, H. Moradkhani, Toward a reliable prediction of seasonal forecast uncertainty: addressing model and initial condition uncertainty with ensemble data assimilation and sequential Bayesian combination. J. Hydrol. 519, 2967–2977 (2014a). https://doi.org/10.1016/j.jhydrol.2014.05.045. Special issue on Ensemble Forecasting and data assimilation

    Article  Google Scholar 

  • C.M. DeChant, H. Moradkhani, Hydrologic prediction and uncertainty quantification, in Handbook of Engineering Hydrology, Modeling, Climate Change and Variability (CRC Press, Taylor & Francis Group, Boca Raton, 2014b), pp. 387–414

    Chapter  Google Scholar 

  • D.P. Dee et al., The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137(656), 553–597 (2011)

    Article  Google Scholar 

  • R. Douc, O. Cappe, Comparison of resampling schemes for particle filtering, paper presented at image and signal processing and analysis, 2005. ISPA 2005, in Proceedings of the 4th International Symposium on, 15–17 Sept 2005 (2005)

    Google Scholar 

  • M. Durand, S.A. Margulis, Effects of uncertainty magnitude and accuracy on assimilation of multiscale measurements for snowpack characterization. J. Geophys. Res. 113(D2), D02105 (2008)

    Google Scholar 

  • R.M. Errico, What is an adjoint model? Bull. Am. Meteorol. Soc. 78(11), 2577–2591 (1997)

    Article  Google Scholar 

  • G. Evensen, The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn. 53(4), 343–367 (2003)

    Article  Google Scholar 

  • Z. Ghahramani, S.T. Roweis, Learning nonlinear dynamical systems using an EM algorithm. Adv. Neural Inf. Process. Syst. 11, 431–437 (1999)

    Google Scholar 

  • M.E. Gharamti, J. Tjiputra, I. Bethke, A. Samuelsen, I. Skjelvan, M. Bentsen, L. Bertino, Ensemble data assimilation for ocean biogeochemical state and parameter estimation at different sites. Ocean Model. 112, 65–89 (2017)

    Article  Google Scholar 

  • R. Giering, Tangent Linear and Adjoint Model Compiler, Users Manual (Center for Global Change Sciences, Department of Earth, Atmospheric, and Planetary Science. MIT, Cambridge, 1997)

    Google Scholar 

  • N. Gordon, D. Salmond, A. Smith, Novel approach to nonlinear/non-Gaussian Bayesian state estimation. Proc. Inst. Elect. Eng. F. 140(2), 107–113 (1993)

    Article  Google Scholar 

  • P. Guingla, D. Antonio, R. De Keyser, G. De Lannoy, L. Giustarini, P. Matgen, V. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012)

    Article  Google Scholar 

  • C.M. Hoppe, H. Elbern, J. Schwinger, A variational data assimilation system for soil–atmosphere flux estimates for the Community Land Model (CLM3. 5). Geosci. Model Dev. 7(3), 1025–1036 (2014)

    Article  Google Scholar 

  • T. Hou, F. Kong, X. Chen, H. Lei, Impact of 3DVAR data assimilation on the prediction of heavy rainfall over Southern China. Adv. Meteorol. 2013, 1 (2013)

    Article  Google Scholar 

  • P.L. Houtekamer, H.L. Mitchell, Data assimilation using an ensemble Kalman filter technique. Mon. Weather Rev. 126(3), 796–811 (1998)

    Article  Google Scholar 

  • R.E. Kalman, A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(Series D), 35–45 (1960). https://doi.org/10.1115/1111.3662552

    Article  Google Scholar 

  • S. Kumar, C. Peters-Lidard, D. Mocko, R. Reichle, Y. Liu, K. Arsenault, Y. Xia, M. Ek, G. Riggs, B. Livneh, M Cosh, Assimilation of remotely sensed soil moisture and snow depth retrievals for drought estimation. J. Hydrometeorol. 15, 2446–2469 (2014). https://doi.org/10.1175/JHM-D-13-0132.1

    Article  Google Scholar 

  • H. Lee, D.J. Seo, Y. Liu, V. Koren, P. McKee, R. Corby, Variational assimilation of streamflow into operational distributed hydrologic models: effect of spatiotemporal scale of adjustment. Hydrol. Earth Syst. Sci. 16(7), 2233–2251 (2012)

    Article  Google Scholar 

  • M. Leisenring, H. Moradkhani, Snow water equivalent prediction using Bayesian data assimilation methods. Stoch. Environ. Res. Risk Assess. 25(2), 253–270 (2011)

    Article  Google Scholar 

  • M. Leisenring, H. Moradkhani, Analyzing the uncertainty of suspended sediment load prediction using sequential Monte Carlo methods. J. Hydrol. 468–469, 268–282 (2012). https://doi.org/10.1016/j.jhydrol.2012.08.049

    Article  Google Scholar 

  • Y. Liu, A.H. Weerts, M. Clark, H.J. Hendricks Franssen, S. Kumar, H. Moradkhani, D.J. Seo, D. Schwanenberg, P. Smith, A.I.J.M. van Dijk, N. van Velzen, M. He, H. Lee, S.J. Noh, O. Rakovec, P. Restrepo, Toward advancing data assimilation in operational hydrologic forecasting and water resources management: current status, challenges, and emerging opportunities. Hydrol. Earth Syst. Sci. 16, 3863–3887 (2012)

    Article  Google Scholar 

  • A.C. Lorenc, The potential of the ensemble Kalman filter for NWP – a comparison with 4D-Var. Q. J. R. Meteorol. Soc. 129(595), 3183–3203 (2003)

    Article  Google Scholar 

  • P. Matgen, R. Hostache, G. Schumann, L. Pfister, L. Hoffmann, H.H.G. Savenije, Towards an automated SAR-based flood monitoring system, Lessons learned from two case studies. Phys. Chem. Earth. 36(7–8), 241–252 (2011). https://doi.org/10.1016/j.pce.2010.12.009

    Article  Google Scholar 

  • C.L. Meng, Z.L. Li, X. Zhan, J.C. Shi, C. Y. Liu, Land surface temperature data assimilation and its impact on evapotranspiration estimates from the Common Land Model. Water Resour. Res. 45, W02421 (2009). https://doi.org/10.1029/2008WR006971

  • C. Montzka, H. Moradkhani, L. Weihermüller, H.J. Hendricks Franssen, M. Canty, H. Vereecken, Hydraulic parameter estimation by remotely-sensed top soil moisture observations with the particle filter. J. Hydrol. 399(3–4), 410–421 (2011). https://doi.org/10.1016/j.jhydrol.2011.01.020

    Article  Google Scholar 

  • C. Montzka, J. Grant, H. Moradkhani, H.J. Hendricks Franssen, L. Weihermüller, M. Drusch, H. Vereecken, Estimation of radiative transfer parameters from L-Band passive microwave brightness temperatures using data assimilation. Vadose Zone Hydrol. Special Issue of Remote Sensing. (2013). https://doi.org/10.2136/vzj2012.0040

  • H. Moradkhani, S. Sorooshian, H.V. Gupta, P.R. Houser, Dual state–parameter estimation of hydrological models using ensemble Kalman filter. Adv. Water Resour. 28(2), 135–147 (2005a)

    Article  Google Scholar 

  • H. Moradkhani, K.L. Hsu, H. Gupta, S. Sorooshian, Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter. Water Resour. Res. 41, W05012 (2005b)

    Article  Google Scholar 

  • H. Moradkhani, C.M. DeChant, S. Sorooshian, Evolution of ensemble data assimilation for uncertainty quantification using the Particle Filter-Markov Chain Monte Carlo method. Water Resour. Res. 48, W12520 (2012). https://doi.org/10.1029/2012WR012144

    Article  Google Scholar 

  • G.S. Nearing, H.V. Gupta, The quantity and quality of information in hydrologic models. Water Resour. Res. 51(1), 524–538 (2015)

    Article  Google Scholar 

  • G.S. Nearing, H.V. Gupta, W.T. Crow, Information loss in approximately bayesian estimation techniques: a comparison of generative and discriminative approaches to estimating agricultural productivity. J. Hydrol. 507, 163–173 (2013)

    Article  Google Scholar 

  • S.J. Noh, Y. Tachikawa, M. Shiiba, S. Kim, Applying sequential Monte Carlo methods into a distributed hydrologic model: lagged particle filtering approach with regularization. Hydrol. Earth Syst. Sci. 15(10), 3237 (2011)

    Article  Google Scholar 

  • S. Park, J.P. Hwang, E. Kim, H. Kang, A new evolutionary particle filter for the prevention of sample impoverishment. IEEE Trans. Signal Process. 13(4), 801–809 (2009)

    Google Scholar 

  • M. Parrish, H. Moradkhani, C.M. DeChant, Towards reduction of model uncertainty: integration of Bayesian model averaging and data assimilation. Water Resour. Res. 48, W03519 (2012). https://doi.org/10.1029/2011WR011116

    Article  Google Scholar 

  • S. Pathiraja, L. Marshall, A. Sharma, H. Moradkhani, Detecting non-stationary hydrologic model parameters in a paired catchment system using data assimilation. Adv. Water Resour. 94, 103–119 (2016a). https://doi.org/10.1016/j.advwatres.2016.04.021

    Article  Google Scholar 

  • S. Pathiraja, L. Marshall, A. Sharma, H. Moradkhani, Hydrologic modeling in dynamic catchments: a data assimilation approach. Water Resour. Res. (2016b). https://doi.org/10.1002/2015WR017192

  • S. Pathiraja, D. Anghileri, P. Burlando, A. Sharma, L. Marshall, H. Moradkhani, Time varying parameter models for catchments with land use change: the importance of model structure. Hydrol. Earth Syst. Sci. Discuss. (2017). https://doi.org/10.5194/hess-2017-382

  • S. Pathiraja, H. Moradkhani, L. Marshall, A. Sharma, G. Geenens, Data driven model uncertainty estimation in data assimilation. Water Resour. Res. (2018a). https://doi.org/10.1002/2018WR022627

  • S. Pathiraja, D. Anghileri, P. Burlando, A. Sharma, L. Marshall, H. Moradkhani, Insights on the impact of systematic model errors on data assimilation performance in changing catchments. Adv. Water Resour. (2018b). https://doi.org/10.1016/j.advwatres.2017.12.006

  • D.A. Plaza, R. De Keyser, G.J.M. De Lannoy, L. Giustarini, P. Matgen, V.R.N. Pauwels, The importance of parameter resampling for soil moisture data assimilation into hydrologic models using the particle filter. Hydrol. Earth Syst. Sci. 16(2), 375–390 (2012)

    Article  Google Scholar 

  • R.H. Reichle, D. Entekhabi, D.B. McLaughlin, Downscaling of radio brightness measurements for soil moisture estimation: a four-dimensional variational data assimilation approach. Water Resour. Res. 37(9), 2353–2364 (2001)

    Article  Google Scholar 

  • R.H. Reichle, D.B. McLaughlin, D. Entekhabi, Hydrologic data assimilation with the ensemble Kalman filter. Mon. Weather Rev. 130(1), 103–114 (2002)

    Article  Google Scholar 

  • J. Ruiz, M. Pulido, Parameter estimation using ensemble-based data assimilation in the presence of model error. Mon. Weather Rev. 143(5), 1568–1582 (2015)

    Article  Google Scholar 

  • P. Salamon, L. Feyen, Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter. J. Hydrol. 376(3), 428–442 (2009)

    Article  Google Scholar 

  • J. Samuel, P. Coulibaly, G. Dumedah, H. Moradkhani, Assessing model state variation in hydrologic data assimilation. J. Hydrol. 513, 127–141 (2014). https://doi.org/10.1016/j.jhydrol.2014.03.048

    Article  Google Scholar 

  • D.-J. Seo, V. Koren, N. Cajina, Real-time variational assimilation of hydrologic and hydrometeorological data into operational hydrologic forecasting. J. Hydrometeorol. 4(3), 627–641 (2003)

    Article  Google Scholar 

  • D.J. Seo, Y. Liu, H. Moradkhani, A. Weerts, Ensemble prediction and data assimilation for operational hydrology. J. Hydrol. 519, 2661–2662 (2014). https://doi.org/10.1016/j.jhydrol.2014.11.035

    Article  Google Scholar 

  • A.G. Slater, M.P. Clark, Snow data assimilation via an ensemble Kalman filter. J. Hydrometeorol. 7, 478 (2005)

    Article  Google Scholar 

  • P.J. Smith, G.D. Thornhill, S.L. Dance, A.S. Lawless, D.C. Mason, N.K. Nichols, Data assimilation for state and parameter estimation: application to morphodynamic modelling. Q. J. R. Meteorol. Soc. 139(671), 314–327 (2013)

    Article  Google Scholar 

  • C. Snyder, T. Bengtsson, P. Bickel, J. Anderson, Obstacles to high-dimensional particle filtering. Mon. Weather Rev. 136(12), 4629 (2008)

    Article  Google Scholar 

  • J.A. Vrugt, C.G.H. Diks, H.V. Gupta, W. Bouten, J.M. Verstraten, Improved treatment of uncertainty in hydrologic modeling: combining the strengths of global optimization and data assimilation. Water Resour. Res. 41(1), W01017 (2005). https://doi.org/10.1029/2004WR003059

    Article  Google Scholar 

  • A.H. Weerts, G.Y.H. El Serafy, Particle filtering and ensemble Kalman filtering for state updating with hydrological conceptual rainfall-runoff models. Water Resour. Res. 42, W09403 (2006). https://doi.org/10.1029/2005WR004093

  • J.S. Whitaker, T.M. Hamill, Ensemble data assimilation without perturbed observations. Monthly Weather Rev. 130(7), 1913–1924 (2002). https://doi.org/10.1175/1520-0493(2002)130<1913:EDAWPO>2.0.CO;2

    Article  Google Scholar 

  • R.D. Wilkinson, M. Vrettas, D. Cornford, J.E. Oakley, Quantifying simulator discrepancy in discrete-time dynamical simulators. J. Agric. Biol. Environ. Stat. 16(4), 554–570 (2011)

    Article  Google Scholar 

  • H. Yan, H. Moradkhani, Combined assimilation of streamflow and satellite soil moisture with the particle filter and geostatistical modeling. Adv. Water Resour. 94, 364–378 (2016). https://doi.org/10.1016/j.advwatres.2016.06.002

    Article  Google Scholar 

  • H. Yan, C.M. DeChant, H. Moradkhani, Improving soil moisture profile prediction with the Particle Filter-Markov Chain Monte Carlo method. IEEE Trans. Geosci. Remote Sens. (2015). https://doi.org/10.1109/TGRS.2015.2432067

  • H. Yan, H. Moradkhani, M. Zarekarizi, A probabilistic drought forecasting framework: a combined dynamical and statistical approach. J. Hydrol. 548, 291–304 (2017). https://doi.org/10.1016/j.jhydrol.2017.03.004

    Article  Google Scholar 

  • S. Yin, X. Zhu, Intelligent particle filter and its application to fault detection of nonlinear systems. IEEE Trans. Ind. Electron. 62(6), 3852–3861 (2015)

    Google Scholar 

  • D.a. Županski, F. Mesinger, Four-dimensional variational assimilation of precipitation data. Mon. Weather Rev. 123(4), 1112–1127 (1995)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Civil, Construction and Environmental Engineering, The University of Alabama, Tuscaloosa, AL, USA

    Hamid Moradkhani & Peyman Abbaszadeh

  2. Department of Geological Sciences, The University of Alabama, Tuscaloosa, AL, USA

    Grey Nearing

  3. Institute for Mathematics, University of Potsdam, Potsdam, Germany

    Sahani Pathiraja

Authors
  1. Hamid Moradkhani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grey Nearing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peyman Abbaszadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sahani Pathiraja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamid Moradkhani .

Editor information

Editors and Affiliations

  1. Beijing Normal University, Beijing, China

    Qingyun Duan

  2. European Centre for Medium-Range Weather Forecasts, Reading, United Kingdom

    Florian Pappenberger

  3. European Commission DG Joint Research Ce, Ispra, Italy

    Jutta Thielen

  4. National Center for Atmospheric Research, Boulder, Colorado, USA

    Andy Wood

  5. University of Reading, Reading, Berkshire, United Kingdom

    Hannah L. Cloke

  6. Consulting Hydrologist, Annapolis, USA

    John C. Schaake

Section Editor information

  1. Department of Civil & Environmental Engineering, Portland State University, Portland, Oregon, USA

    Hamid Moradkhani

  2. Inland Water Systems, Deltares, The Netherlands

    Albrecht Weerts Expert Hydrology and Forecasting

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer-Verlag GmbH Germany, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Moradkhani, H., Nearing, G., Abbaszadeh, P., Pathiraja, S. (2018). Fundamentals of Data Assimilation and Theoretical Advances. In: Duan, Q., Pappenberger, F., Thielen, J., Wood, A., Cloke, H., Schaake, J. (eds) Handbook of Hydrometeorological Ensemble Forecasting. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40457-3_30-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-40457-3_30-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-40457-3

  • Online ISBN: 978-3-642-40457-3

  • eBook Packages: Springer Reference Earth and Environm. ScienceReference Module Physical and Materials ScienceReference Module Earth and Environmental Sciences

Publish with us

Policies and ethics

Search

Navigation