Skip to main content
SpringerLink
Log in

Calibration of imperfect models to biased observations

  • Original Paper
  • Published:
Computational Geosciences Aims and scope Submit manuscript

Abstract

The problem of assimilating biased and inaccurate observations into inadequate models of the physical systems from which the observations were taken is common in the petroleum and groundwater fields. When large amounts of data are assimilated without accounting for model error and observation bias, predictions tend to be both overconfident and incorrect. In this paper, we propose a workflow for calibration of imperfect models to biased observations that involves model construction, model calibration, model criticism and model improvement. Model criticism is based on computation of model diagnostics which provide an indication of the validity of assumptions. During the model improvement step, we advocate identification of additional physically motivated parameters based on examination of data mismatch after calibration and addition of bias correction terms. If model diagnostics indicates the presence of residual model error after parameters have been added, then we advocate estimation of a “total” observation error covariance matrix, whose purpose is to reduce weighting of observations that cannot be matched because of deficiency of the model. Although the target applications of this methodology are in the subsurface, we illustrate the approach with two simplified examples involving prediction of the future velocity of fall of a sphere from models calibrated to a short-time series of biased measurements with independent additive random noise. The models into which the data are assimilated contain model errors due to neglect of physical processes and neglect of uncertainty in parameters. In every case, the estimated total error covariance is larger than the true observation covariance implying that the observations need not be matched to the accuracy of the measuring instrument. Predictions are much improved when all model improvement steps were taken.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aanonsen, S.I., Aavatsmark, I., Barkve, T., Cominelli, A., Gonard, R., Gosselin, O., Kolasinski, M., Reme, H.: Effect of scale dependent data correlations in an integrated history matching loop combining production data and 4D seismic data (SPE 79665). In: Proc SPE RSS, The Woodlands (2003)

  2. Aanonsen, S.I., Eydinov, D.: A multiscale method for distributed parameter estimation with application to reservoir history matching. Comput. Geosci. 10(1), 97–117 (2006)

    Article  Google Scholar 

  3. Achenbach, E.: Influence of surface roughness on cross-flow around a circular cylinder. J. Fluid Mech. 46(2), 321–335 (1971)

    Article  Google Scholar 

  4. Adam, L., Batzle, M., Brevik, I.: Gassmann’s fluid substitution and shear modulus variability in carbonates at laboratory seismic and ultrasonic frequencies. Geophysics 71(6), F173–F183 (2006)

    Article  Google Scholar 

  5. Auligné, T., McNally, A.P., Dee, D.P.: Adaptive bias correction for satellite data in a numerical weather prediction system. Q. J. Roy. Meteor. Soc. 133(624), 631–642 (2007)

    Article  Google Scholar 

  6. Aziz, K., Settari, A.: Petroleum reservoir simulation. Elsevier Applied Science Publishers, London (1979)

    Google Scholar 

  7. Bardsley, J., Solonen, A., Haario, H., Laine, M.: Randomize-then-optimize: a method for sampling from posterior distributions in nonlinear inverse problems. SIAM J. Sci. Comput. 36(4), A1895–A1910 (2014)

    Article  Google Scholar 

  8. Bayarri, M.J., Berger, J.O.: P values for composite null models. J. Am. Stat. Assoc. 95(452), 1127–1142 (2000)

    Google Scholar 

  9. Bayarri, M.J., Berger, J.O., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C.-H., Tu, J.: A framework for validation of computer models. Technometrics 49(2), 138–154 (2007)

    Article  Google Scholar 

  10. Bennett, N.D., Croke, B.F., Guariso, G., Guillaume, J.H., Hamilton, S.H., Jakeman, A.J., Marsili-Libelli, S., Newham, L.T., Norton, J.P., Perrin, C., Pierce, S.A., Robson, B., Seppelt, R., Voinov, A.A., Fath, B.D., Andreassian, V.: Characterising performance of environmental models. Environ. Model Softw. 40, 1–20 (2013)

    Article  Google Scholar 

  11. Box, G.E.P., Tiao, G.C.: Bayesian inference in statistical analysis. Addison-Wesley Publishing Company (1973)

  12. Brajard, J., Sirven, J., Talagrand, O.: Ensemble variational data assimilation with a shallow-water model: preliminary results. In: EGU General Assembly 2016, Held 17–22 April in Vienna, Austria (2016)

  13. Capen, E.C.: The difficulty of assessing uncertainty. J. Pet. Technol. 28(8), 843–850 (1976)

    Article  Google Scholar 

  14. Chapnik, B., Desroziers, G., Rabier, F., Talagrand, O.: Properties and first application of an error-statistics tuning method in variational assimilation. Q. J. Roy. Meteor. Soc. 130(601), 2253–2275 (2004)

    Article  Google Scholar 

  15. Chapnik, B., Desroziers, G., Rabier, F., Talagrand, O.: Diagnosis and tuning of observational error in a quasi-operational data assimilation setting. Q. J. R. Meteorol. Soc. 132(615), 543–565 (2006)

    Article  Google Scholar 

  16. Chen, Y., Oliver, D.S.: Parameterization techniques to improve mass conservation and data assimilation for ensemble Kalman filter (SPE 133560). In: SPE Western Regional Meeting, 27–29 May 2010, Anaheim, California, USA (2010)

  17. Chen, Y., Oliver, D.S.: Levenberg-Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification. Comput. Geosci. 17(4), 689–703 (2013)

    Article  Google Scholar 

  18. Chen, Y., Oliver, D.S.: History matching of the Norne full-field model with an iterative ensemble smoother. SPE Reserv. Eval. Eng. 17(2), 244–256 (2014)

    Article  Google Scholar 

  19. Chilès, J.-P., Delfiner, P.: Geostatistics: modeling spatial uncertainty, 2nd edn. Wiley, New York (2012)

    Book  Google Scholar 

  20. Christensen, R., Johnson, W., Branscum, A., Hanson, T.E.: Bayesian ideas and data analysis: an introduction for scientists and statisticians. CRC Press (2011)

  21. Cordua, K.S., Nielsen, L., Looms, M.C., Hansen, T.M., Binley, A.: Quantifying the influence of static-like errors in least-squares-based inversion and sequential simulation of cross-borehole ground penetrating radar data. J. Appl. Geophys. 68(1), 71–84 (2009)

    Article  Google Scholar 

  22. Dake, L.P.: The practice of reservoir engineering, volume 36 of Developments in Petroleum Science. Elsevier (1994)

  23. Dee, D.: On-line estimation of error covariance parameters for atmospheric data assimilation. Mon. Weather Rev. 123(4), 1128–1145 (1995)

    Article  Google Scholar 

  24. Dee, D.P.: Bias and data assimilation. Q. J. R. Meteorol. Soc. 131(613), 3323–3343 (2005)

    Article  Google Scholar 

  25. Dee, D.P., da Silva, A.M.: Maximum-likelihood estimation of forecast and observation error covariance parameters. Part I: methodology. Mon. Weather. Rev. 127(8), 1822–1834 (1999)

    Article  Google Scholar 

  26. Desroziers, G., Berre, L., Chabot, V., Chapnik, B.: A posteriori diagnostics in an ensemble of perturbed analyses. Mon. Weather Rev. 137(10), 3420–3436 (2009)

    Article  Google Scholar 

  27. Desroziers, G., Berre, L., Chapnik, B., Poli, P.: Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc. 131(613, C), 3385–3396 (2005)

    Article  Google Scholar 

  28. Desroziers, G., Ivanov, S.: Diagnosis and adaptive tuning of observation-error parameters in a variational assimilation. Q. J. R. Meteorol. Soc. 127(574, B), 1433–1452 (2001)

    Article  Google Scholar 

  29. Doherty, J.: Model-based environmental decision-making. www.pesthomepage.org/getfiles.php?file=models_and_decisions.doc (2010)

  30. Doherty, J., Christensen, S.: Use of paired simple and complex models to reduce predictive bias and quantify uncertainty. Water Resour. Res. 47(12), W12534 (2011)

    Article  Google Scholar 

  31. Doherty, J., Welter, D.: A short exploration of structural noise. Water Resour. Res. 46(5), W05525 (2010)

    Article  Google Scholar 

  32. Draper, D.: Assessment and propagation of model uncertainty. J. R. Stat. Soc. Ser. B Methodol. 57(1), 45–97 (1995)

    Google Scholar 

  33. Evin, G., Thyer, M., Kavetski, D., McInerney, D., Kuczera, G.: Comparison of joint versus postprocessor approaches for hydrological uncertainty estimation accounting for error autocorrelation and heteroscedasticity. Water Resour. Res. 50(3), 2350–2375 (2014)

    Article  Google Scholar 

  34. Fertig, E., Baek, S.-J., Hunt, B., Ott, E., Szunyogh, I., Aravéquia, J., Kalnay, E., Li, H., Liu, J.: Observation bias correction with an ensemble Kalman filter. Tellus A 61(2) (2009)

  35. Garthwaite, P.H., Kadane, J.B., O’Hagan, A.: Statistical methods for eliciting probability distributions. J. Am. Stat. Assoc. 100(470), 680–701 (2005)

    Article  Google Scholar 

  36. Gelman, A., Meng, X.-L.: Model checking and model improvement. In: Gilks, W.R., Richardson, S., Spiegelhalter, D.J. (eds.) Markov Chain Monte Carlo in Practice, pp. 189–201. Chapman & Hall, New York (1996)

    Google Scholar 

  37. Gelman, A., Meng, X.-L., Stern, H.: Posterior predictive assessment of model fitness via realized discrepancies. Stat. Sin. 6(4), 733–760 (1996)

    Google Scholar 

  38. Gelman, A., Shalizi, C.R.: Philosophy and the practice of Bayesian statistics. Br. J. Math. Stat. Psychol. 66(1), 8–38 (2013)

    Article  Google Scholar 

  39. Hansen, T.M., Cordua, K.S., Jacobsen, B.H., Mosegaard, K.: Accounting for imperfect forward modeling in geophysical inverse problems — exemplified for crosshole tomography. Geophysics 79(3), H1–H21 (2014)

    Article  Google Scholar 

  40. Higdon, D., Gattiker, J., Williams, B., Rightley, M.: Computer model calibration using high-dimensional output. J. Am. Stat. Assoc. 103(482), 570–583 (2008)

    Article  Google Scholar 

  41. Hunt, R.J., Doherty, J., Tonkin, M.J.: Are models too simple? Arguments for increased parameterization. Ground Water 45(3), 254–262 (2007)

    Article  Google Scholar 

  42. Jafarpour, B., Khodabakhshi, M.: A probability conditioning method (PCM) for nonlinear flow data integration into multipoint statistical facies simulation. Math. Geosci. 43, 133–164 (2011)

    Article  Google Scholar 

  43. Kennedy, M.C., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. Ser. B (Stat Methodol.) 63(3), 425–464 (2001)

    Article  Google Scholar 

  44. Kersting, A.B., Efurd, D.W., Finnegan, D.L., Rokop, D.J., Smith, D.K., Thompson, J.L.: Migration of plutonium in ground water at the Nevada Test Site. Nature 397(6714), 56–59 (1999)

    Article  Google Scholar 

  45. Kitanidis, P.K.: Quasi-linear geostatistical theory for inversing. Water Resour. Res. 31(10), 2411–2419 (1995)

    Article  Google Scholar 

  46. Kynn, M.: The heuristics and biases bias in expert elicitation. J. R. Stat. Soc. A. Stat. Soc. 171(1), 239–264 (2008)

    Google Scholar 

  47. Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. J. Multivar. Anal. 88(2), 365–411 (2004)

    Article  Google Scholar 

  48. Lødøen, O.P., Omre, H.: Scale-corrected ensemble Kalman filtering applied to production history conditioning in reservoir evaluation. SPE J. 13(2), 177–194 (2008)

    Article  Google Scholar 

  49. Lyotard, N., Shew, W.L., Bocquet, L., Pinton, J.-F.: Polymer and surface roughness effects on the drag crisis for falling spheres. Eur. Phys. J. B 60(4), 469–476 (2007)

    Article  Google Scholar 

  50. Mathai, A.M., Provost, S.B.: Quadratic forms in random variables: theory and applications. Marcel Dekker, Inc., New York (1992)

    Google Scholar 

  51. Mavko, G., Mukerji, T.: Bounds on low-frequency seismic velocities in partially saturated rocks. Geophysics 63(3), 918–924 (1998)

    Article  Google Scholar 

  52. Michel, Y.: Diagnostics on the cost-function in variational assimilations for meteorological models. Nonlinear Processes Geophys. 21(1), 187–199 (2014)

    Article  Google Scholar 

  53. Miyoshi, T., Kalnay, E., Li, H.: Estimating and including observation-error correlations in data assimilation. Inverse Prob. Sci. Eng. 21(3), 387–398 (2013)

    Article  Google Scholar 

  54. Moore, C., Doherty, J.: Role of the calibration process in reducing model predictive error. Water Resour. Res. 41(5), W05020 (2005)

    Article  Google Scholar 

  55. Mordant, N., Metz, P., Pinton, J.-F., Michel, O.: Acoustical technique for Lagrangian velocity measurement. Rev. Sci. Instrum. 76(2), 1–7 (2005)

    Article  Google Scholar 

  56. National Research Council: Assessing the reliability of complex models: mathematical and statistical foundations of verification, validation, and uncertainty quantification. National Academies Press. Committee on Mathematical Foundations of VV&UQ (2012)

  57. O’Hagan, A.: Eliciting expert beliefs in substantial practical applications. Journal of the Royal Statistical Society. Series D (The Statistician) 47(1), 21–35 (1998)

    Article  Google Scholar 

  58. Oliver, D.S.: Minimization for conditional simulation: Relationship to optimal transport. J. Comput. Phys. 265(0), 1–15 (2014)

    Article  Google Scholar 

  59. Oliver, D.S.: Metropolized randomized maximum likelihood for improved sampling from multimodal distributions. SIAM/ASA Journal on Uncertainty Quantification 5(1), 259–277 (2017)

    Article  Google Scholar 

  60. Oliver, D.S., Chen, Y.: Recent progress on reservoir history matching: a review. Comput. Geosci. 15(1), 185–221 (2011)

    Article  Google Scholar 

  61. Oliver, D.S., He, N., Reynolds, A.C.: Conditioning permeability fields to pressure data. In: Proceedings of the European Conference on the Mathematics of Oil Recovery, V, pp. 1–11 (1996)

  62. Oliver, D.S., Reynolds, A.C., Liu, N.: Inverse theory for petroleum reservoir characterization and history matching. Cambridge University Press, Cambridge (2008)

    Book  Google Scholar 

  63. Omre, H., Lødøen, O.P.: Improved production forecasts and history matching using approximate fluid-flow simulators. SPE J. 9(3), 339–351 (2004)

    Article  Google Scholar 

  64. Peters, L., Arts, R.J., Brouwer, G.K., Geel, C.R., Cullick, S., Lorentzen, R.J., Chen, Y., Dunlop, K.N.B., Vossepoel, F.C., Xu, R., Sarma, P., Alhutali, A.H., Reynolds, A.C.: Results of the Brugge benchmark study for flooding optimization and history matching. SPE Reserv. Evalu. Eng. 13(3), 391–405 (2010)

    Article  Google Scholar 

  65. Pringle, M.J., Lark, R.M.: Spatial analysis of model error, illustrated by soil Carbon dioxide emissions. Vadose Zone J. 5(1), 168–183 (2006)

    Article  Google Scholar 

  66. Rood, R.B.: The role of the model in the data assimilation system. In: Lahoz, W., Khattatov, B., Menard, R. (eds.) Data Assimilation: Making Sense of Observations, pp. 351–379. Springer (2010)

  67. Rubin, D.B.: Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Stat. 12(4), 1151–1172 (1984)

    Article  Google Scholar 

  68. Silverton, A., Warner, M., Umpleby, A., Morgan, J., Irabor, K.: Non- physical water density as a proxy to improve data fit during acoustic FWI. In: 76Th EAGE Conference and Exhibition 2014 (2014)

  69. Skjervheim, J.-A., Evensen, G., Aanonsen, S.I., Ruud, B.O., Johansen, T.A.: Incorporating 4D seismic data in reservoir simulation models using ensemble Kalman filter. SPE J. 12(3), 282–292 (2007)

    Article  Google Scholar 

  70. Smith, L.A.: Disentangling uncertainty and error: on the predictability of nonlinear systems. In: Mees, A.I. (ed.) Nonlinear Dynamics and Statistics, pp. 31–64. Birkhäuser (2000)

  71. Sønstabø, J.K., Hellevik, L.R.: Numerical methods for engineers: a digital compendium. lrhgit.github.io/tkt4140/allfiles/digital_compendium/main.html (2015)

  72. Stewart, L.M., Dance, S.L., Nichols, N.K.: Correlated observation errors in data assimilation. Int. J. Numer. Methods Fluids 56(8), 1521–1527 (2008)

    Article  Google Scholar 

  73. Talagrand, O.: A posteriori verification of analysis and assimilation algorithms. In: Proceedings of Workshop on Diagnosis of Data Assimilations Systems, 2–4 Nov 1998, ECMFW, Reading, UK, pp. 17–28 (1999)

  74. Talagrand, O.: Evaluation of assimilation algorithms. In: Lahoz, W., Khattatov, B., Menard, R. (eds.) Data Assimilation: Making Sense of Observations, pp. 217–240. Springer, Berlin (2010)

    Chapter  Google Scholar 

  75. Tarantola, A.: Inverse Problem Theory: Methods for Data Fitting and Model Parameter Estimation. Elsevier, Amsterdam (1987)

    Google Scholar 

  76. Ueno, G., Nakamura, N.: Bayesian estimation of the observation-error covariance matrix in ensemble-based filters. Q. J. Roy. Meteorol. Soc. 142(698), 2055–2080 (2016)

    Article  Google Scholar 

  77. Van Trees, H.L.: Detection, estimation, and modulation theory. Wiley (2004)

  78. Waller, J.A., Dance, S.L., Nichols, N.K.: Theoretical insight into diagnosing observation error correlations using observation-minus-background and observation-minus-analysis statistics. Q. J. Roy. Meteorol. Soc. 142(694), 418–431 (2016)

    Article  Google Scholar 

  79. Watson, T.A., Doherty, J.E., Christensen, S.: Parameter and predictive outcomes of model simplification. Water Resour. Res. 49(7), 3952–3977 (2013)

    Article  Google Scholar 

  80. Williams, G.J.J., Mansfield, M., MacDonald, D.G., Bush, M.D., et al.: Top-down reservoir modelling. In: SPE Annual Technical Conference and Exhibition Held in Houston, Texas, 26–29 September. Society of Petroleum Engineers (2004)

  81. Zhao, Y., Li, G., Reynolds, A.C.: Characterization of the measurement error in time-lapse seismic data and production data with an EM algorithm. Oil Gas Sci. Technol. 62(2, Sp. Iss. SI), 181–193 (2007)

    Article  Google Scholar 

Download references

Acknowledgements

Primary support for the authors has been provided by the CIPR/IRIS cooperative research project “4D Seismic History Matching” which is funded by industry partners Eni, Petrobras, and Total, as well as the Research Council of Norway through the PETROMAKS2 program.

Author information

Authors and Affiliations

  1. Uni Research CIPR, Bergen, Norway

    Dean S. Oliver & Miguel Alfonzo

  2. University of Bergen, Bergen, Norway

    Miguel Alfonzo

Authors
  1. Dean S. Oliver
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miguel Alfonzo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dean S. Oliver.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oliver, D.S., Alfonzo, M. Calibration of imperfect models to biased observations. Comput Geosci 22, 145–161 (2018). https://doi.org/10.1007/s10596-017-9678-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10596-017-9678-4

Keywords

Search

Navigation