What is Validation of Computer Simulations? Toward a Clarification of the Concept of Validation and of Related Notions
This chapter clarifies the concept of validation of computer simulations by comparing various definitions that have been proposed for the notion. While the definitions agree in taking validation to be an evaluation, they differ on the following questions: (1) What exactly is evaluated—results from a computer simulation, a model, a computer code? (2) What are the standards of evaluation––truth, accuracy, and credibility or also something else? (3) What type of verdict does validation lead to––that the simulation is such and such good, or that it passes a test defined by a certain threshold? (4) How strong needs the case to be for the verdict? (5) Does validation necessarily proceed by comparing simulation outputs with measured data? Along with these questions, the chapter explains notions that figure prominently in them, e.g., the concepts of accuracy and credibility. It further discusses natural answers to the questions as well as arguments that speak in favor and against these answers. The aim is to obtain a better understanding of the options we have for defining validation and how they are related to each other.
KeywordsEvaluation Model Code Truth Accuracy Credibility Adequate representation Adequacy for purpose Explanation Data-driven validation Test
I am extremely grateful for extensive comments by William Oberkampf, Patrick Roache, and Nicole J. Saam. At a very different level, I wish to express my thanks to my mother, Bärbel Beisbart, who sadly passed away during the time when I was working on this chapter. I dedicate this chapter to her memory.
- AIAA. (1998). Guide for the verification and validation of computational fluid dynamics simulations, AIAA G-077–1998. American Institute of Aeronautics and Astronautics, Reston, VA.Google Scholar
- ASME. (2006). Guide for verification and validation in computational solid mechanics. American Society of Mechanical Engineers, ASME V&V 10-2006.Google Scholar
- Blackburn, S. (1999). Think. Oxford: Oxford University Press.Google Scholar
- Baumberger, C., Beisbart, C., & Brun, G. (2017a). What is understanding? An overview of recent debates in epistemology and philosophy of science. In S. Grimm, C. Baumberger, & S. Ammon (Eds.), Explaining understanding: New perspectives from epistemology and philosophy of science (pp. 1–34). New York: Routledge.Google Scholar
- Caldwell, S., & Morrison, R. J. (2000). Validation of longitudinal dynamic microsimulation models. Experience with CORSIM and DYNACAN. In: Mitton, L., Sutherland, H., & Weeks, M. J. (Eds.). Microsimulation modelling for policy analysis. challenges and innovations (pp. 200–225). Cambridge: Cambridge University Press.Google Scholar
- Carnap, R. (1950/1962) Logical foundations of probability. Chicago: University of Chicago Press.Google Scholar
- Ghetiu, T., Polack, F. A., & Bown, J. (2010). Argument-driven validation of computer simulations–A necessity rather than an option. In: VALID 2010. The Second International Conference on Advances in System Testing and Validation Lifecycle (pp. 1–4) August 22–27, 2010. Nice, France, IEEE Press.Google Scholar
- Hartmann, S. (1996). The world as a process: simulations in the natural and social sciences. In R. Hegselmann, et al. (Eds.), Modelling and simulation in the social sciences from the philosophy of science point of view, theory and decision library (pp. 77–100). Dordrecht: Kluwer.Google Scholar
- Hempel, C. G. (1945). Studies in the logic of confirmation (I.) (Vol. 54, No. 213, pp. 1–260) Mind, New Series.Google Scholar
- IEEE. (2012). IEEE standard for system and software verification and validation. In IEEE Std 1012-2012 (Revision of IEEE Std 1012-2004) (pp. 1–223), 25 May 2012. https://doi.org/10.1109/ieeestd.2012.6204026.
- Lacey, H. (1999). Is science value-free? Values and scientific understanding. London: Routledge.Google Scholar
- Lipton, P. (2000). Inference to the best explanation (2nd ed.). London: Routledge.Google Scholar
- Margolis, E., & Laurence, S. (2014). Concepts. In: E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy. https://plato.stanford.edu/archives/spr2014/entries/concepts.
- Oberkampf, W. & Roy, C. (2010). Verification and validation in scientific computing. Cambridge University Press.Google Scholar
- Pearl, J. (2000). Causality. Modeling, reasoning, and inference. Cambridge: Cambridge University Press.Google Scholar
- Roache, P. J. (1998). Verification and validation in computational science and engineering. Albuquerque, New Mexico: Hermosa Publishers.Google Scholar
- Roache, P. J. (2013). A defense of computational physics. Socorro, NM: Hermosa (revised printing).Google Scholar