Advertisement

Validation of Physical Models in the Presence of Uncertainty

  • Robert D. MoserEmail author
  • Todd A. Oliver
Reference work entry

Abstract

As the field of computational modeling continues to mature and simulation results are used to inform more critical decisions, validation of the physical models that form the basis of these simulations takes on increasing importance. While model validation is not a new concept, traditional techniques such as visual comparison of model outputs and experimental observations without accounting for uncertainties are insufficient for assessing model validity, particularly for the case where the intended purpose of the model is to make extrapolative predictions. This work provides an overview of validation of physical models in the presence of uncertainty. In particular, two issues are discussed: comparison of model outputs and observational data when both the model and observations are uncertain, and the process of building confidence in extrapolative predictions. For comparing uncertain model outputs and data, a Bayesian probabilistic perspective is adopted in which the problem of assessing the consistency of the model and the observations becomes one of Bayesian model checking. A broadly applicable approach to Bayesian model checking for physical models is described. For validating extrapolative predictions, a recently developed process termed predictive validation is discussed. This process relies on the ideas of Bayesian model checking but goes beyond comparison of model and data to assess the conditions necessary for reliable extrapolation using physics-based models.

Keywords

Extrapolative predictions Posterior predictive assessment Validation under uncertainty 

References

  1. 1.
    Adams, B.M., Ebeida, M.S., Eldred, M.S., Others: Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.2 User’s Manual. Sandia National Laboratories, Albuquerque (2014). https://dakota.sandia.gov/documentation.html
  2. 2.
    AIAA Computational Fluid Dynamics Committee on Standards: AIAA Guide for Verification and Validation of Computational Fluid Dynamics Simulations. AIAA Paper number G-077-1999 (1998)Google Scholar
  3. 3.
    Arlot, S., Celisse, A.: A survey of cross-validation procedures for model selection. Stat. Surv. 4, 40–79 (2010). doi:10.1214/09-SS054, http://dx.doi.org/10.1214/09-SS054 MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    ASME Committee V&V 10: Standard for Verification and Validation in Computational Solid Mechanics. ASME (2006)Google Scholar
  5. 5.
    Babuška, I., Nobile, F., Tempone, R.: Reliability of computational science. Numer. Methods Partial Differ. Equ. 23(4), 753–784 (2007). doi:10.1002/num.20263MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Box, G.E.P.: Sampling and Bayes’ inference in scientific modeling and robustness. R. Stat. Soc. Ser. A 143, 383–430 (1980)CrossRefzbMATHGoogle Scholar
  7. 7.
    Box, G., Tiao, G.C.: Bayesian Inference in Statistical Analysis. Wiley Classics, New York (1973)zbMATHGoogle Scholar
  8. 8.
    Cox, R.T.: The Algebra of Probable Inference. Johns Hopkins University Press, Baltimore (1961)zbMATHGoogle Scholar
  9. 9.
    Cui, T., Martin, J., Marzouk, Y.M., Solonen, A., Spantini, A.: Likelihood-informed dimension reduction for nonlinear inverse problems. Inverse Probl. 30(11), 114015 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Dubois, D., Prade, H.: Possibility Theory: An Approach to Computerized Processing of Uncertainty. Plenum Press, New York (1988)CrossRefzbMATHGoogle Scholar
  11. 11.
    Ferson, S., Ginzburg, L.R.: Different methods are needed to propagate ignorance and variability. Reliab. Eng. Syst. Saf. 54, 133–144 (1996)CrossRefGoogle Scholar
  12. 12.
    Fine, T.L.: Theories of Probability. Academic, New York (1973)zbMATHGoogle Scholar
  13. 13.
    Firm uses doe’s fastest supercomputer to streamline long-haul trucks. Office of Science, U.S. Department of Energy, Stories of Discovery and Innovation (2011). http://science.energy.gov/discovery-and-innovation/stories/2011/127008/
  14. 14.
    Gelman, A.: Comment: ‘Bayesian checking of the second levels of hierarchical models’. Stat. Sci. 22, 349–352 (2007). doi:doi:10.1214/07-STS235AGoogle Scholar
  15. 15.
    Gelman, A., Rubin, D.B.: Avoiding model selection in Bayesian social research. Sociol. Methodol. 25, 165–173 (1995)CrossRefGoogle Scholar
  16. 16.
    Gelman, A., Shalizi, C.R.: Philosophy and the practice of Bayesian statistics. Br. J. Math. Stat. Psychol. 66(1), 8–38 (2013)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Gelman, A., Meng, X.L., Stern, H.: Posterior predictive assessment of medel fitness via realized discrepancies. Statistica Sinica 6, 733–807 (1996)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian Data Analysis, 3rd edn. CRC Press, Boca Raton (2014)zbMATHGoogle Scholar
  19. 19.
    Hsu, J.: Multiple Comparisons: Theory and Methods. Chapman and Hall/CRC, London (1996)CrossRefzbMATHGoogle Scholar
  20. 20.
    Hyndman, R.J.: Computing and graphing highest density regions. Am. Stat. 50(2), 120–126 (1996)Google Scholar
  21. 21.
    Jaynes, E.T.: Probability Theory: The Logic of Science. Cambridge University Press, Cambridge/New York (2003)CrossRefzbMATHGoogle Scholar
  22. 22.
    Kanji, G.K.: 100 Statistical Tests, 3rd edn. Sage Publications, London/Thousand Oaks (2006)CrossRefGoogle Scholar
  23. 23.
    Le Maıtre, O., Knio, O., Najm, H., Ghanem, R.: Uncertainty propagation using wiener–haar expansions. J. Comput. Phys. 197(1), 28–57 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Li, J., Marzouk, Y.M.: Adaptive construction of surrogates for the Bayesian solution of inverse problems. SIAM J. Sci. Comput. 36(3), A1163–A1186 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Miller, R.G.J.: Simultaneous Statistical Inference, 2nd edn. Springer, New York (1981)CrossRefzbMATHGoogle Scholar
  26. 26.
    Miller, L.K.: Simulation-based engineering for industrial competitive advantage. Comput. Sci. Eng. 12(3), 14–21 (2010). doi:10.1109/MCSE.2010.71CrossRefGoogle Scholar
  27. 27.
    Najm, H.N.: Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics. Annu. Rev. Fluid Mech. 41, 35–52 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Oberkampf, W.L., Helton, J.C., Sentz, K.: Mathematical representation of uncertainty. AIAA 2001-1645Google Scholar
  29. 29.
    Oden, J.T., Belytschko, T., Fish, J., Hughes, T.J.R., Johnson, C., Keyes, D., Laub, A., Petzold, L., Srolovitz, D., Yip, S.: Revolutionizing engineering science through simulation: a report of the National Science Foundation blue ribbon panel on simulation-based engineering science (2006). http://www.nsf.gov/pubs/reports/sbes_final_report.pdf Google Scholar
  30. 30.
    Oliver, T.A., Terejanu, G., Simmons, C.S., Moser, R.D.: Validating predictions of unobserved quantities. Comput. Methods Appl. Mech. Eng. 283, 1310–1335 (2015). doi:http://dx.doi.org/10.1016/j.cma.2014.08.023, http://www.sciencedirect.com/science/article/pii/S004578251400293X
  31. 31.
    Petra, N., Martin, J., Stadler, G., Ghattas, O.: A computational framework for infinite-dimensional Bayesian inverse problems, Part II: stochastic Newton mcmc with application to ice sheet flow inverse problems. SIAM J. Sci. Comput. 36(4), A1525–A1555 (2014)zbMATHGoogle Scholar
  32. 32.
    Rubin, D.B.: Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Stat. 12, 1151–1172 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Sentz, K., Ferson, S.: Combination of evidence in Dempster–Shafer theory. Technical report SAND 2002-0835, Sandia National Laboratory (2002)Google Scholar
  34. 34.
    Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)zbMATHGoogle Scholar
  35. 35.
    Van Horn, K.S.: Constructing a logic of plausible inference: a guide to Cox’s theorem. Int. J. Approx. Reason. 34(1), 3–24 (2003)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.Department of Mechanical Engineering, Institute for Computational and Engineering SciencesThe University of Texas at AustinAustinUSA
  2. 2.Predictive Engineering and Computational Science,Institute for Computational and Engineering SciencesThe University of Texas at AustinAustinUSA
  3. 3.Institute for Computational and Engineering SciencesThe University of Texas at AustinAustinUSA

Personalised recommendations