Validation Using Bayesian Methods
Quantitative model validation is playing an increasingly important role in performance and reliability assessment of a complicated system whenever simulation is involved. This chapter discusses model validation from a Bayesian perspective, considering in particular data uncertainty. First, Bayes’ theorem is defined, then the Bayesian risk rule method is introduced. Explicit expressions for the Bayesian interval hypothesis testing approach are presented in both univariate and multivariate cases. The problem of non-normal validation data is addressed by the Box–Cox transformation. A generalized procedure is presented to implement Bayesian validation methods. Classic hypothesis testing method is utilized to conduct a comparison study. The impact of the data normality assumption and of the variation of the threshold on model assessment accuracy is investigated by using both classical and Bayesian approaches. The Bayesian methodology is illustrated with a reliability model of rotor blades, a univariate stochastic damage accumulation model, and a multivariate heat conduction problem.
KeywordsBayesian statistics Hypothesis testing Box–Cox transformation Bayes network Model validation Non-normality
- AIAA. (1998). Guide for the Verification and Validation of Computational Fluid Dynamics Simulations, American Institute of Aeronautics and Astronautics, AIAA-G-077-1998, Reston, Virginia, USA.Google Scholar
- Annis, C. (2002). Modeling high cycle fatigue with Markov chain Monte Carlo-a new look at an old idea. In: Proceedings of 43rd AIAA/ASME/ASCE/AHS/ASC Structure, Structural Dynamics and Materials Conference (AIAA-2002-1380) (pp. 1351–1361). Denver CO 2002.Google Scholar
- ASME. (2002). Verification & Validation (V&V) methodology and quantitative reliability at confidence: Basis for an investment strategy, PTC 60, UCRL-ID-150874, Livermore, CA: Lawrence Livermore National Laboratory.Google Scholar
- ASME. (2006). Guide for verification and validation in computational solid mechanics. ASME V&V 10-2006. New York, NY: American Society of Mechanical Engineers.Google Scholar
- Chen, W., Xiong, Y., Tsui, K. -L., & Wang, S. (2008). A design-driven validation approach using Bayesian prediction models. Journal of Mechanical Design, 130(2), 021101-1-12.Google Scholar
- DOD. (1996). Verification, validation, and accreditation (VV&A) recommended practices guide. Alexandria, VA: Department of Defense.Google Scholar
- DOE. (2000). Accelerated strategic computing initiative (ASCI) program plan. DOE/DP-99-000010592, Washington, DC: Department of Energy.Google Scholar
- Genz, A. (1992). Numerical computation of multivariate normal probabilities. Journal of Computational and Graphical Statistics, 1(2), 141–149.Google Scholar
- Hills, R.G., & Trucano, T. G. (2002). Statistical validation of engineering and scientific models: A maximum likelihood based metric. Technical Report Sand. No 2001-1783, Albuquerque, NM: Sandia National Laboratories.Google Scholar
- Migon H. S., & Gamerman, D. (1999). Statistical inference-an integrated approach. London: Arnold, a Member of the Holder Headline Group.Google Scholar
- NIST/SEMATECH (2005). e-Handbook of statistical methods, National Institute of Standards and Technology. Retrieved from http://www.itl.nist.gov/div898/handbook.
- Nowak, R., & Scott, C. (2004). The Bayes risk criterion in hypothesis testing. Connexions, Retrieved from http://cnx.rice.edu/content/m11533/1.6/.
- Roache, P. J. (1998). Verification and validation in computational science and engineering in: Science and engineering. Albuquerque, NM: Hermosa Publishers.Google Scholar