Skip to main content
SpringerLink
Log in

Parameter estimation of subsurface flow models using iterative regularized ensemble Kalman filter

  • Original Paper
  • Published:
Stochastic Environmental Research and Risk Assessment Aims and scope Submit manuscript

Abstract

A new parameter estimation algorithm based on ensemble Kalman filter (EnKF) is developed. The developed algorithm combined with the proposed problem parametrization offers an efficient parameter estimation method that converges using very small ensembles. The inverse problem is formulated as a sequential data integration problem. Gaussian process regression is used to integrate the prior knowledge (static data). The search space is further parameterized using Karhunen–Loève expansion to build a set of basis functions that spans the search space. Optimal weights of the reduced basis functions are estimated by an iterative regularized EnKF algorithm. The filter is converted to an optimization algorithm by using a pseudo time-stepping technique such that the model output matches the time dependent data. The EnKF Kalman gain matrix is regularized using truncated SVD to filter out noisy correlations. Numerical results show that the proposed algorithm is a promising approach for parameter estimation of subsurface flow models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  • Allen M, Frame D, Kettleborough J, Stainforth D (2006) Model error in weather and climate forecasting. In: Palmer T, Hagedorn R (eds) Predictability of weather and climate. Cambridge University Press, Cambridge

  • Anderson JL (2001) An ensemble adjustment Kalman filter for data assimilation. Mon Weather Rev 129(12):2884–2903

    Article  Google Scholar 

  • Anderson JL (2003) A local least squares framework for ensemble filtering. Mon Weather Rev 131(4):634–642

    Article  Google Scholar 

  • Anderson JL, Anderson SL (1999) A monte carlo implementation of the nonlinear filtering problem to produce ensemble assimilation and forecasts. Mon Weather Rev 127:2741–2758

    Article  Google Scholar 

  • Anderson BDO, Moore JB (1979) Optimal filtering. Information and system sciences series. Prentice-Hall, Inc., Englewood Cliffs, NJ

    Google Scholar 

  • Bengtsson T, Snyder C, Nychka D (2003) Toward a nonlinear ensemble filter for high-dimensional systems. J Geophys Res 108(D24):8775–8785

    Google Scholar 

  • Blum J, Le Dimet FX, Navon IM (2008) Data assimilation for geophysical fluids. In: Ciarlet PG, Temam R, Tribbia J (eds) Computational methods for the atmosphere and the oceans. Handbook of numerical analysis, vol 14. Elsevier, Amsterdam, pp 385–442

  • Bulygina N, Gupta H (2010) How bayesian data assimilation can be used to estimate the mathematical structure of a model. Stoch Environ Res Risk Assess 24(6):925–937

    Article  Google Scholar 

  • Carrera J, Alcolea A, Medina A, Hidalgo J, Slooten LJ (2005) Inverse problem in hydrogeology. Hydrogeol J 13(1):206–222

    Article  Google Scholar 

  • Chen Z (2007) Reservoir simulation: mathematical techniques in oil recovery. Society for Industrial and Applied Mathematics, Philadelphia, PA

    Book  Google Scholar 

  • Chen Y, Oliver D (2012) Ensemble randomized maximum likelihood method as an iterative ensemble smoother. Math Geosci 44(1):1–26

    Article  Google Scholar 

  • Chilès JP, Delfiner P (1999) Geostatistics: modeling spatial uncertainty. Wiley, New York

    Book  Google Scholar 

  • Christie M, Blunt M (1995) Tenth SPE comparative solution project: a comparison of upscaling techniques. SPE Reserv Eval Eng 4:308–317

    Google Scholar 

  • Cohn SE (1997) An introduction to estimation theory. J Meteorol Soc Jpn Ser II 75(1B):257–288

    Google Scholar 

  • Dostert P, Efendiev Y, Mohanty B (2009) Efficient uncertainty quantification techniques in inverse problems for Richards’ equation using coarse-scale simulation models. Adv Water Resour 32(3):329–339

    Article  Google Scholar 

  • Dovera L, Della Rossa E (2011) Improved initial ensemble generation coupled with ensemble square root filters and inflation to estimate uncertainty. Comput Geosci 1–17

  • Efendiev Y, Datta-Gupta A, Ginting V, Ma X, Mallick B (2005) An efficient two-stage markov chain monte carlo method for dynamic data integration. Water Resour Res 41(12)

  • ELsheikh AH, Jackson MD, Laforce TC (2012) Bayesian reservoir history matching considering model and parameter uncertainties. Math Geosci 44(5):515–543

    Article  Google Scholar 

  • Evensen G (1994) Sequential data assimilation with a nonlinear quasi-geostrophic model using monte carlo methods to forecast error statistics. J Geophys Res 99(C5):10,143–10,162

    Article  Google Scholar 

  • Evensen G, van Leeuwen PJ (2000) An ensemble Kalman smoother for nonlinear dynamics. Mon Weather Rev 128(6):1852–1867

    Article  Google Scholar 

  • Fang H, Gong G, Qian M (1997) Annealing of iterative stochastic schemes. SIAM J Control Optim 35(6):1886–1907

    Article  Google Scholar 

  • Fang F, Pain CC, Navon IM, Piggott MD, Gorman GJ, Allison PA, Goddard AJH (2009) Reduced-order modelling of an adaptive mesh ocean model. Int J Numer Methods Fluids 59(8):827–851

    Article  Google Scholar 

  • Fu J, Gomez-Hernandez J (2009) A blocking markov chain monte carlo method for inverse stochastic hydrogeological modeling. Math Geosci 41(2):105–128

    Article  Google Scholar 

  • Gaspari G, Cohn SE (1999) Construction of correlation functions in two and three dimensions. Q J R Meteorol Soc 125(554):723–757

    Article  Google Scholar 

  • Gelfand S, Mitter S (1991) Simulated annealing type algorithms for multivariate optimization. Algorithmica 6(1):419–436

    Article  Google Scholar 

  • Ghanem RG, Spanos PD (1991) Stochastic finite elements: a spectral approach. Springer-Verlag New York, Inc., New York

    Book  Google Scholar 

  • Golub G, Van Loan C (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, Baltimore

  • Gu Y, Oliver DS (2007) An iterative ensemble Kalman filter for multiphase fluid flow data assimilation. SPE J 12(4):438–446

    Google Scholar 

  • Hansen C (1998) Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. SIAM, Philadelphia

  • Hillery A, Chin R (1991) Iterative wiener filters for image restoration. IEEE Trans Signal Process 39(8):1892–1899

    Article  Google Scholar 

  • Houtekamer PL, Mitchell HL (2001) A sequential ensemble Kalman filter for atmospheric data assimilation. Mon Weather Rev 129(1):123–137

    Article  Google Scholar 

  • Houtekamer PL, Mitchell HL (2005) Ensemble Kalman filtering. Q J R Meteorol Soc 131(613):3269–3289

    Article  Google Scholar 

  • Jazwinski AH (1970) Stochastic processes and filtering theory. Academic Press, New York

    Google Scholar 

  • Kac M, Siegert AJF (1947) An explicit representation of a stationary Gaussian process. Ann Math Stat 18:438–442

    Article  Google Scholar 

  • Karhunen K (1947) Über lineare Methoden in der Wahrscheinlichkeitsrechnung. Ann Acad Sci Fenn Ser A I Math Phys 1947(37):79

    Google Scholar 

  • Krymskaya MV, Hanea RG, Verlaan M (2009) An iterative ensemble Kalman filter for reservoir engineering applications. Comput Geosci 13(2):235–244

    Article  Google Scholar 

  • Kushner HJ (1987) Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: global minimization via monte carlo. SIAM J Appl Math 47(1):169–185

    Article  Google Scholar 

  • Li G, Reynolds AC (2009) Iterative ensemble Kalman filters for data assimilation. SPE J 14(3):496–505

    Google Scholar 

  • Loève M (1948) Fonctions aléatoires de second order. In: Levy P (ed) Processus Stochastiques et Movement Brownien. Hermann, Paris

  • Lorentzen RJ, Naevdal G (2011) An iterative ensemble Kalman filter. IEEE Trans Autom Control 56(8):1990–1995

    Article  Google Scholar 

  • Ma X, Al-Harbi M, Datta-Gupta A, Efendiev Y (2008) An efficient two-stage sampling method for uncertainty quantification in history matching geological models. SPE J 13(1):77–87

    Google Scholar 

  • MacKay DJC (1999) Comparison of approximate methods for handling hyperparameters. Neural Comput 11(5):1035–1068

    Article  Google Scholar 

  • McLaughlin D, Townley LR (1996) A reassessment of the groundwater inverse problem. Water Resour Res 32(5):1131–1161

    Article  Google Scholar 

  • Moradkhani H, Sorooshian S, Gupta HV, Houser PR (2005) Dual state–parameter estimation of hydrological models using ensemble Kalman filter. Adv Water Resour 28(2):135–147

    Article  Google Scholar 

  • Naevdal G, Johnsen L, Aanonsen S, Vefring E (2005) Reservoir monitoring and continuous model updating using ensemble Kalman filter. SPE J 10(1):66–74

    Google Scholar 

  • Navon IM (1998) Practical and theoretical aspects of adjoint parameter estimation and identifiability in meteorology and oceanography. Dyn Atmos Oceans 27(1–4):55–79

    Article  Google Scholar 

  • Nocedal J, Wright SJ (2006) Numerical Optimization, 2nd edn. Springer Verlag

  • Oliver D, Cunha L, Reynolds A (1997) Markov chain monte carlo methods for conditioning a permeability field to pressure data. Math Geol 29(1):61–91

    Article  Google Scholar 

  • Ott E, Hunt BR, Szunyogh I, Zimin AV, Kostelich EJ, Corazza M, Kalnay E, Patil DJ, Yorke JA (2004) A local ensemble Kalman filter for atmospheric data assimilation. Tellus A 56(5):415–428

    Article  Google Scholar 

  • Pham DT, Verron J, Roubaud MC (1998) A singular evolutive extended Kalman filter for data assimilation in oceanography. J Mar Syst 16(3–4):323–340

    Article  Google Scholar 

  • Rasmussen CE, Williams CKI (2005) Gaussian processes for machine learning (adaptive computation and machine learning). MIT Press, Cambridge

  • Sadegh P, Spall J (1998) Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans Autom Control 43(10):1480–1484

    Article  Google Scholar 

  • Sætrom J, Omre H (2011) Ensemble Kalman filtering with shrinkage regression techniques. Comput Geosci 15(2):271–292

    Article  Google Scholar 

  • Sakov P, Oliver DS, Bertino L (2012) An iterative enkf for strongly nonlinear systems. Mon Weather Rev 140(6):1988–2004

    Article  Google Scholar 

  • Simon E, Bertino L (2009) Application of the gaussian anamorphosis to assimilation in a 3-D coupled physical-ecosystem model of the north atlantic with the ENKF: a twin experiment. Ocean Sci 5(4):495–510

    Article  Google Scholar 

  • Smith KW (2007) Cluster ensemble Kalman filter. Tellus A 59(5):749–757

    Article  Google Scholar 

  • Sørensen JVT, Madsen H (2004) Data assimilation in hydrodynamic modelling: on the treatment of non-linearity and bias. Stoch Environ Res Risk Assess 18(4):228–244

    Article  Google Scholar 

  • Spall J (2003) Introduction to stochastic search and optimization: estimation, simulation, and control. Wiley-Interscience series in discrete mathematics and optimization. Wiley-Interscience, Hoboken

  • Sun AY, Morris A, Mohanty S (2009) Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data. Adv Water Resour 32(2):280–292

    Article  Google Scholar 

  • Thacker WC (1989) The role of the hessian matrix in fitting models to measurements. J Geophys Res 94(C5):6177–6196

    Article  Google Scholar 

  • Tippett MK, Anderson JL, Bishop CH, Hamill TM, Whitaker JS (2003) Ensemble square root filters. Mon Weather Rev 131(7):1485–1490

    Article  Google Scholar 

  • Tong J, Hu B, Yang J (2010) Using data assimilation method to calibrate a heterogeneous conductivity field conditioning on transient flow test data. Stoch Environ Res Risk Assess 24(8):1211–1223

    Article  Google Scholar 

  • Tong J, Hu B, Yang J (2012) Assimilating transient groundwater flow data via a localized ensemble Kalman filter to calibrate a heterogeneous conductivity field. Stoch Environ Res Risk Assess 26(3):467–478

    Article  Google Scholar 

  • Wan E, Van Der Merwe R (2000) The unscented Kalman filter for nonlinear estimation. In: Adaptive systems for signal processing, communications, and control symposium 2000. AS-SPCC. The IEEE 2000, Lake Louise, pp 153 –158

  • Zhang D, Lu Z, Chen Y (2007) Dynamic reservoir data assimilation with an efficient, dimension-reduced Kalman filter. SPE J 12(1):108–117

    Google Scholar 

  • Zhou E, Fu M, Marcus S (2008) A particle filtering framework for randomized optimization algorithms. In: Simulation conference, 2008. WSC 2008, Winter, pp 647–654

  • Zhou H, Gomez-Hernandez JJ, Franssen HJH, Li L (2011) An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering. Adv Water Resour 34(7):844–864

    Article  Google Scholar 

  • Zupanski M (2005) Maximum likelihood ensemble filter: theoretical aspects. Mon Weather Rev 133(6):1710–1726

    Article  Google Scholar 

  • Zupanski M, Navon IM, Zupanski D (2008) The maximum likelihood ensemble filter as a non-differentiable minimization algorithm. Q J R Meteorol Soc 134(633):1039–1050

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their insightful and constructive comments which helped enhance this manuscript. Dr. A. H. Elsheikh and Dr. Jefferson L. M. A. Gomes carried this work as part of activities of the Qatar Carbonates and Carbon Storage Research Centre (QCCSRC). They gratefully acknowledge the funding of QCCSRC provided jointly by Qatar Petroleum, Shell, and the Qatar Science and Technology Park. Professor I. M. Navon acknowledges the support of NSF/CMG grant ATM-0931198.

Author information

Authors and Affiliations

  1. Department of Earth Science and Engineering, Imperial College London, Prince Consort Road, London, SW7 2BP, UK

    A. H. ELSheikh, C. C. Pain, F. Fang & J. L. M. A. Gomes

  2. Department of Scientific Computing, Florida State University, Tallahassee, FL, 32306-4120, USA

    I. M. Navon

Authors
  1. A. H. ELSheikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. C. Pain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. F. Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. L. M. A. Gomes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. I. M. Navon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. H. ELSheikh.

Rights and permissions

Reprints and permissions

About this article

Cite this article

ELSheikh, A.H., Pain, C.C., Fang, F. et al. Parameter estimation of subsurface flow models using iterative regularized ensemble Kalman filter. Stoch Environ Res Risk Assess 27, 877–897 (2013). https://doi.org/10.1007/s00477-012-0613-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00477-012-0613-x

Keywords

Search

Navigation