Abstract
Existing model validation studies in geoscience often disregard or partly account for uncertainties in observations, model choices, and input parameters. In this work, we develop a statistical framework that incorporates a probabilistic modeling technique using a fully Bayesian approach to perform a quantitative uncertainty-aware validation. A Bayesian perspective on a validation task yields an optimal bias-variance trade-off against the reference data. It provides an integrative metric for model validation that incorporates parameter and conceptual uncertainty. Additionally, a surrogate modeling technique, namely Bayesian Sparse Polynomial Chaos Expansion, is employed to accelerate the computationally demanding Bayesian calibration and validation. We apply this validation framework to perform a comparative evaluation of models for coupling a free flow with a porous-medium flow. The correct choice of interface conditions and proper model parameters for such coupled flow systems is crucial for physically consistent modeling and accurate numerical simulations of applications. We develop a benchmark scenario that uses the Stokes equations to describe the free flow and considers different models for the porous-medium compartment and the coupling at the fluid–porous interface. These models include a porous-medium model using Darcy’s law at the representative elementary volume scale with classical or generalized interface conditions and a pore-network model with its related coupling approach. We study the coupled flow problems’ behaviors considering a benchmark case, where a pore-scale resolved model provides the reference solution. With the suggested framework, we perform sensitivity analysis, quantify the parametric uncertainties, demonstrate each model’s predictive capabilities, and make a probabilistic model comparison.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Data Availability
The Bayesian framework and the models’ source codes, as well as the reference data used in this study, are available at https://git.iws.uni-stuttgart.de/dumux-pub/mohammadi2022a.
References
Oberkampf, W.L., Roy, C.J.: Verification and Validation in Scientific Computing. Cambridge University Press, Cambridge (2010)
Oberkampf, W.L., Trucano, T.G.: Verification and validation benchmarks. Nucl. Eng. Des. 238, 716–743 (2008)
Oberkampf, W.L., Barone, M.F.: Measures of agreement between computation and experiment: validation metrics. J. Comput. Phys. 217(1), 5–36 (2006)
Ferson, S., Oberkampf, W.L., Ginzburg, L.: Model validation and predictive capability for the thermal challenge problem. Comput. Methods Appl. Mech. Eng. 197(29–32), 2408–2430 (2008)
Sankararaman, S., Ling, Y., Shantz, C., Mahadevan, S.: Inference of equivalent initial flaw size under multiple sources of uncertainty. Int. J. Fatigue 33(2), 75–89 (2011)
Hills, R.G., Leslie, I.H.: Statistical validation of engineering and scientific models: validation experiments to application. Technical report, Sandia National Lab, Albuquerque, NM (US) (2003)
Urbina, A., Paez, T.L., Urbina, A., Hasselman, T., Wathugala, W., Yap, K.: Assessment of model accuracy relative to stochastic system behavior. In: 44 Th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (2003)
Gelfand, A.E., Dey, D.K.: Bayesian model choice: asymptotics and exact calculations. J. R. Stat. Soc. B 56(3), 501–514 (1994)
Zhang, R., Mahadevan, S.: Bayesian methodology for reliability model acceptance. Reliab. Eng. Syst. Saf. 80(1), 95–103 (2003)
Mahadevan, S., Rebba, R.: Validation of reliability computational models using bayes networks. Reliab. Eng. Syst. Saf. 87(2), 223–232 (2005)
Geweke, J.: Bayesian model comparison and validation. Am. Econ. Rev. 97(2), 60–64 (2007)
Sankararaman, S., Mahadevan, S.: Model validation under epistemic uncertainty. Reliab. Eng. Syst. Saf. 96(9), 1232–1241 (2011)
Rebba, R., Mahadevan, S.: Computational methods for model reliability assessment. Reliab. Eng. Syst. Saf. 93(8), 1197–1207 (2008)
Sankararaman, S., Mahadevan, S.: Assessing the reliability of computational models under uncertainty. In: 54th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, pp. 1873 (2013)
Thacker, B.H., Paez, T.L.: A simple probabilistic validation metric for the comparison of uncertain model and test results. In: 16th AIAA Non-Deterministic Approaches Conference, pp. 0121 (2014)
Liu, Y., Chen, W., Arendt, P., Huang, H.-Z.: Toward a better understanding of model validation metrics. J. Mech. Des. 133(7) (2011)
Ling, Y., Mahadevan, S.: Quantitative model validation techniques: New insights. Reliab. Eng. Syst. Saf. 111, 217–231 (2013)
Hills, R.G.: Model validation: model parameter and measurement uncertainty. J. Heat Transfer 128(4), 339–351 (2006)
Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neural Comput. 4(1), 1–58 (1992)
Schöniger, A., Wöhling, T., Samaniego, L., Nowak, W.: Model selection on solid ground: rigorous comparison of nine ways to evaluate Bayesian model evidence. Water Resour. Res. 50(12), 9484–9513 (2014)
Schöniger, A., Wöhling, T., Nowak, W.: A statistical concept to assess the uncertainty in Bayesian model weights and its impact on model ranking. Water Resour. Res. 51(9), 7524–7546 (2015)
Draper, D.: Assessment and propagation of model uncertainty. J. R. Stat. Soc. Ser. B. Stat. Methodol. 57(1), 45–70 (1995)
Hoeting, J.A., Madigan, D., Raftery, A.E., Volinsky, C.T.: Bayesian model averaging: a tutorial. Statist. Sci. 14(4), 382–417 (1999)
Trotta, R.: Bayes in the sky: Bayesian inference and model selection in cosmology. Contemp. Phys. 49(2), 71–104 (2008)
Faust, J., Gilchrist, S., Wright, J.H., Zakrajšsek, E.: Credit spreads as predictors of real-time economic activity: a Bayesian model-averaging approach. Rev. Econ. Stat. 95(5), 1501–1519 (2013)
Hooten, M.B., Hobbs, N.T.: A guide to Bayesian model selection for ecologists. Ecol. Monogr. 85(1), 3–28 (2015)
Hoege, M., Guthke, A., Nowak, W.: The hydrologist’s guide to bayesian model selection, averaging and combination. J. Hydrol. 572, 96–107 (2019)
Schöniger, A., Illman, W.A., Wöhling, T., Nowak, W.: Finding the right balance between groundwater model complexity and experimental effort via Bayesian model selection. J. Hydrol. 531, 96–110 (2015)
Brunetti, C., Linde, N., Vrugt, J.A.: Bayesian model selection in hydrogeophysics: Application to conceptual subsurface models of the south oyster bacterial transport site, virginia, usa. Adv. Water Resour. 102, 127–141 (2017)
Wöhling, T., Schöniger, A., Gayler, S., Nowak, W.: Bayesian model averaging to explore the worth of data for soilplant model selection and prediction. Water Resour. Res. 51(4), 2825–2846 (2015)
Schäfer Rodrigues Silva, A., Guthke, A., Höge, M., Cirpka, O.A., Nowak, W.: Strategies for simplifying reactive transport models: A bayesian model comparison. Water Resour. Res. 56(11), 2020–028100 (2020)
Yoon, H.N., Marshall, L., Sharma, A., Kim, S.: Bayesian model calibration using surrogate streamflow in ungauged catchments. Water Resour. Res. 58(1), 2021–031287 (2022)
Mohammadi, F., Kopmann, R., Guthke, A., Oladyshkin, S., Nowak, W.: Bayesian selection of hydro-morphodynamic models under computational time constraints. Adv. Water Resour. 117, 53–64 (2018)
Beckers, F., Heredia, A., Noack, M., Nowak, W., Wieprecht, S., Oladyshkin, S.: Bayesian calibration and validation of a large-scale and time-demanding sediment transport model. Water Resour. Res. 56(7), 2019–026966 (2020)
Bazargan, H., Christie, M., Elsheikh, A.H., Ahmadi, M.: Surrogate accelerated sampling of reservoir models with complex structures using sparse polynomial chaos expansion. Adv. Water Res. 86, 385–399 (2015)
Bazargan, H., Christie, M.: Bayesian model selection for complex geological structures using polynomial chaos proxy. Comput. Geosci. 21(3), 533–551 (2017)
Scheurer, S., Schäfer Rodrigues Silva, A., Mohammadi, F., Hommel, J., Oladyshkin, S., Flemisch, B., Nowak, W.: Surrogatebased Bayesian comparison of computationally expensive models: application to microbially induced calcite precipitation. Comput. Geosci. 25(6), 1899–1917 (2021)
Elsheikh, A.H., Hoteit, I., Wheeler, M.F.: Efficient Bayesian inference of subsurface flow models using nested sampling and sparse polynomial chaos surrogates. Comput. Methods Appl. Mech. Engrg. 269, 515–537 (2014)
Wiener, N.: The homogeneous chaos. Amer. J. Math. 60(4), 897–936 (1938)
Oladyshkin, S., Nowak, W.: Data-driven uncertainty quantification using the arbitrary polynomial chaos expansion. Reliab. Eng. Syst. Saf. 106, 179–190 (2012)
Tipping, M.E.: Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1(3), 211–244 (2001)
Arjoune, Y., Kaabouch, N., El Ghazi, H., Tamtaoui, A.: Compressive sensing: performance comparison of sparse recovery algorithms. In: 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), pp. 1–7 (2017). IEEE
Lüthen, N., Marelli, S., Sudret, B.: Sparse polynomial chaos expansions: literature survey and benchmark. SIAM/ASA J. Uncertainty Quantif. 9(2), 593–649 (2021)
Tipping, M.E., Faul, A.: Fast marginal likelihood maximisation for sparse Bayesian models. In: Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics, pp. 3–6 (2003)
Eggenweiler, E., Rybak, I.: Effective coupling conditions for arbitrary flows in Stokes-Darcy systems. Multiscale Model. Simul. 19(2), 731–757 (2021)
Goyeau, B., Lhuillier, D., Gobin, D., Velarde, M.: Momentum transport at a fluid-porous interface. Int. J. Heat Mass Tran. 46, 4071–4081 (2003)
Angot, P., Goyeau, B., Ochoa-Tapia, J.A.: Asymptotic modeling of transport phenomena at the interface between a fluid and a porous layer: jump conditions. Phys. Rev. E 95, 063302 (2017)
Ochoa-Tapia, A.J., Whitaker, S.: Momentum transfer at the boundary between a porous medium and a homogeneous fluid. I: theoretical development. Int. J. Heat Mass Tran. 38, 2635–2646 (1995)
Le Bars, M., Worster, M.: Interfacial conditions between a pure fluid and a porous medium: implications for binary alloy solidification. J. Fluid Mech. 550, 149–173 (2006)
Beavers, G.S., Joseph, D.D.: Boundary conditions at a naturally permeable wall. J. Fluid Mech. 30, 197–207 (1967)
Rybak, I., Schwarzmeier, C., Eggenweiler, E., Rüde, U.: Validation and calibration of coupled porous-medium and free-flow problems using pore-scale resolved models. Comput. Geosci. 25(2), 621–635 (2021)
Eggenweiler, E., Rybak, I.: Unsuitability of the Beavers-Joseph interface condition for filtration problems. J. Fluid Mech. 892, 10 (2020)
Discacciati, M., Miglio, E., Quarteroni, A.: Mathematical and numerical models for coupling surface and groundwater flows. Appl. Num. Math. 43, 57–74 (2002)
Zampogna, G.A., Bottaro, A.: Fluid flow over and through a regular bundle of rigid fibres. J. Fluid Mech. 792, 5–35 (2016)
Discacciati, M., Quarteroni, A.: Navier-Stokes/Darcy coupling: modeling, analysis, and numerical approximation. Rev. Mat. Complut. 22, 315–426 (2009)
Saffman, P.G.: On the boundary condition at the surface of a porous medium. Stud. Appl. Math. 50, 93–101 (1971)
Hornung, U.: Homogenization and Porous Media. Springer, New York (1996)
Jäger, W., Mikelić, A.: Modeling effective interface laws for transport phenomena between an unconfined fluid and a porous medium using homogenization. Transp. Porous Med. 78, 489–508 (2009)
Jäger, W., Mikelić, A.: On the interface boundary conditions by Beavers, Joseph and Saffman. SIAM J. Appl. Math. 60, 1111–1127 (2000)
Yang, G., Coltman, E., Weishaupt, K., Terzis, A., Helmig, R., Weigand, B.: On the Beavers–Joseph interface condition for non-parallel coupled channel flow over a porous structure at high Reynolds numbers. Transp. Porous Med. 128, 431–457 (2019)
Carraro, T., Goll, C., Marciniak-Czochra, A., Mikelić, A.: Effective interface conditions for the forced infiltration of a viscous fluid into a porous medium using homogenization. Comput. Methods Appl. Mech. Engrg. 292, 195–220 (2015)
Lācis, U., Sudhakar, Y., Pasche, S., Bagheri, S.: Transfer of mass and momentum at rough and porous surfaces. J. Fluid Mech. 884, 21 (2020)
Lācis, U., Bagheri, S.: A framework for computing effective boundary conditions at the interface between free fluid and a porous medium. J. Fluid Mech. 812, 866–889 (2017)
Hanspal, N., Waghode, A., Nassehi, V., Wakeman, R.: Development of a predictive mathematical model for coupled Stokes/Darcy flows in cross-flow membrane filtration. Chem. Eng. J. 149, 132–142 (2009)
Discacciati, M., Gerardo-Giorda, L.: Optimized Schwarz methods for the Stokes-Darcy coupling. IMA J. Numer. Anal. 38, 1959–1983 (2018)
Angot, P., Goyeau, B., Ochoa-Tapia, J.A.: A nonlinear asymptotic model for the inertial flow at a fluid-porous interface. Adv. Water Res. 149, 103798 (2021)
Mierzwiczak, M., Fraska, A., Grabski, J.K.: Determination of the slip constant in the Beavers–Joseph experiment for laminar fluid flow through porous media using a meshless method. Math. Probl. Eng. 2019, 1494215 (2019)
Weishaupt, K., Joekar-Niasar, V., Helmig, R.: An efficient coupling of free flow and porous media flow using the pore-network modeling approach. J. Comput. Phys. X 1, 100011 (2019)
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian Data Analysis. CRC Press, London (2013)
Roy, C.J.: Errors and uncertainties: their sources and treatment. In: Computer Simulation Validation, pp. 119–141. Springer, Cham (2019)
Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer, New York (1999)
Liu, J.S.: Monte Carlo Strategies in Scientific Computing. Springer, New York (2008)
Weishaupt, K., Terzis, A., Zarikos, I., Yang, G., Flemisch, B., de Winter, D.A.M., Helmig, R.: A hybrid-dimensional coupled pore-network/free-flow model including pore-scale slip and its application to a micromodel experiment. Transp. Porous Med. 135(1), 243–270 (2020)
Koch, T., Weishaupt, K., Müller, J., Weigand, B., Helmig, R.: A (dual) network model for heat transfer in porous media. Transp. Porous Med. 140(1), 107–141 (2021)
Mehmani, Y., Tchelepi, H.A.: Minimum requirements for predictive pore-network modeling of solute transport in micromodels. Adv. Water Res. 108, 83–98 (2017)
Terzis, A., Zarikos, I., Weishaupt, K., Yang, G., Chu, X., Helmig, R., Weigand, B.: Microscopic velocity field measurements inside a regular porous medium adjacent to a low Reynolds number channel flow. Phys. Fluids 31, 042001 (2019)
Patzek, T.W., Silin, D.B.: Shape factor and hydraulic conductance in noncircular capillaries: I. One-phase creeping flow. J. Colloid Interface Sci. 236, 295–304 (2001)
Oostrom, M., Mehmani, Y., Romero-Gomez, P., Tang, Y., Liu, H., Yoon, H., Kang, Q., Joekar-Niasar, V., Balhoff, M., Dewers, T., Zhang, C.: Pore-scale and continuum simulations of solute transport micromodel benchmark experiments. Comput. Geosci. 20(4), 857–879 (2016)
Scheibe, T.D., Murphy, E.M., Chen, X., Rice, A.K., Carroll, K.C., Palmer, B.J., Tartakovsky, A.M., Battiato, I., Wood, B.D.: An analysis platform for multiscale hydrogeologic modeling with emphasis on hybrid multiscale methods. Groundwater 53(1), 38–56 (2015)
Balhoff, M.T., Thompson, K.E., Hjortso, M.: Coupling pore-scale networks to continuum-scale models of porous media. Comput. Geosci. 33(3), 393–410 (2007)
Balhoff, M.T., Thomas, S.G., Wheeler, M.F.: Mortar coupling and upscaling of pore-scale models. Comput. Geosci. 12(1), 15–27 (2007)
Mehmani, Y., Balhoff, M.T.: Bridging from pore to continuum: a hybrid mortar domain decomposition framework for subsurface flow and transport. Multiscale Model. Simul. 12(2), 667–693 (2014)
Beyhaghi, S., Xu, Z., Pillai, K.M.: Achieving the inside–outside coupling during network simulation of isothermal drying of a porous medium in a turbulent flow. Transp. Porous Med. 114(3), 823–842 (2016)
Blunt, M.J.: Multiphase Flow in Permeable Media: a Pore-scale Perspective. Cambridge University Press, Cambridge (2017)
Oladyshkin, S., Nowak, W.: The connection between Bayesian inference and information theory for model selection, information gain and experimental design. Entropy 21(11), 1081 (2019)
Angluin, D., Smith, C.H.: Inductive inference: theory and methods. ACM Comput. Surv. 15(3), 237–269 (1983)
Kass, R.E., Raftery, A.E.: Bayes factors. J. Amer. Statist. Assoc. 90(430), 773–795 (1995)
Jeffreys, H.: The Theory of Probability. Oxford University Press, Oxford (1961)
Marelli, S., Lüthen, N., Sudret, B.: UQLab user manual – Polynomial chaos expansions. Technical Report UQLab-V1.4-104, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland (2021)
Wagner, P.-R., Nagel, J., Marelli, S., Sudret, B.: UQLab user manual – Bayesian inversion for model calibration and validation. Technical Report UQLab-V1.4-113, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland (2021)
Berger, J.O.: Statistical Decision Theory and Bayesian Analysis. Springer, New York (2013)
Harlow, F.H., Welch, J.E.: Numerical calculation of time-dependent viscous incompressible flow of fluid with free surface. Phys. Fluids 8(12), 2182–2189 (1965)
Hackbusch, W.: On first and second order box schemes. Computing 41(4), 277–296 (1989)
Smith, A.F., Gelfand, A.E.: Bayesian statistics without tears: a sampling–resampling perspective. Amer. Statist. 46(2), 84–88 (1992)
Oladyshkin, S., Nowak, W.: Incomplete statistical information limits the utility of highorder polynomial chaos expansions. Reliab. Eng. Syst. Saf. 169, 137–148 (2018)
Mohammadi, F.: Development and Realization of Validation Benchmarks. arXiv:2011.13216 (2020)
Sokal, A.D.: Monte Carlo Methods in Statistical Mechanics: Foundations and New Algorithms Note to the Reader. In: Functional Integration, pp. 131–192 (1996)
Goodman, J., Weare, J.: Ensemble samplers with affine invariance. Commun. Appl. Math. Comput. Sci. 5(1), 65–80 (2010)
Foreman-Mackey, D., Farr, W.M., Sinha, M., Archibald, A.M., Hogg, D.W., Sanders, J.S., Zuntz, J., g. Williams, P.K., j. Nelson, A.R., de Val-Borro, M., Erhardt, T., Pashchenko, I., Pla, O.A.: emcee v3: a Python ensemble sampling toolkit for affineinvariant MCMC. J. Open Res. Softw 4(43), 1864 (2019)
Xu, T., Valocchi, A.J.: A Bayesian approach to improved calibration and prediction of groundwater models with structural error. Water Resour. Res. 51(11), 9290–9311 (2015)
Iooss, B., Lemaître, P.: In: Dellino, G., Meloni, C. (eds.) A Review on Global Sensitivity Analysis Methods, pp. 101–122. Springer, Boston (2015)
Sobol’, I.: Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp 1(4), 407–414 (1993)
Oladyshkin, S., De Barros, F., Nowak, W.: Global sensitivity analysis: a flexible and efficient framework with an example from stochastic hydrogeology. Adv. Water Res. 37, 10–22 (2012)
Sudret, B.: Global sensitivity analysis using polynomial chaos expansions. Reliab. Eng. Syst. Saf. 93(7), 964–979 (2008)
Oladyshkin, S., Mohammadi, F., Kroeker, I., Nowak, W.: Bayesian3 active learning for the Gaussian process emulator using information theory. Entropy 22(8), 890 (2020)
Kennedy, M.C., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. Ser. B Stat. Methodol. 63(3), 425–464 (2001)
Bayarri, M.J., Berger, J.O., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C.-H., Tu, J.: A framework for validation of computer models. Technometrics 49(2), 138–154 (2007)
Brynjarsdóttir, J., O’Hagan, A.: Learning about physical parameters: The importance of model discrepancy. Inverse Probl. 30(11), 114007 (2014)
Ling, Y., Mullins, J., Mahadevan, S.: Selection of model discrepancy priors in Bayesian calibration. J. Comput. Phys. 276, 665–680 (2014)
Gardner, P., Rogers, T., Lord, C., Barthorpe, R.: Learning model discrepancy: A Gaussian process and sampling-based approach. Mech. Syst. Sig. Process. 152, 107381 (2021)
Schneider, M., Gläser, D., Weishaupt, K., Coltman, E., Flemisch, B., Helmig, R.: Coupling staggered-grid and vertex-centered finite-volume methods for coupled porousmedium free-flow problems. arXiv:2112.11089 (2021)
Koch, T., Gläser, D., Weishaupt, K., Ackermann, S., Beck, M., Becker, B., Burbulla, S., Class, H., Coltman, E., Emmert, S., Fetzer, T., Grüninger, C., Heck, K., Hommel, J., Kurz, T., Lipp, M., Mohammadi, F., Scherrer, S., Schneider, M., Seitz, G., Stadler, L., Utz, M., Weinhardt, F., Flemisch, B.: Dumux 3-an open-source simulator for solving flow and transport problems in porous media with a focus on model coupling. Comput. Math. Appl. 81, 423–443 (2021)
Acknowledgements
The work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project Number 327154368 – SFB 1313.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Mohammadi, F., Eggenweiler, E., Flemisch, B. et al. A surrogate-assisted uncertainty-aware Bayesian validation framework and its application to coupling free flow and porous-medium flow. Comput Geosci 27, 663–686 (2023). https://doi.org/10.1007/s10596-023-10228-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10596-023-10228-z