Advertisement

What is Validation of Computer Simulations? Toward a Clarification of the Concept of Validation and of Related Notions

  • Claus BeisbartEmail author
Chapter
Part of the Simulation Foundations, Methods and Applications book series (SFMA)

Abstract

This chapter clarifies the concept of validation of computer simulations by comparing various definitions that have been proposed for the notion. While the definitions agree in taking validation to be an evaluation, they differ on the following questions: (1) What exactly is evaluated—results from a computer simulation, a model, a computer code? (2) What are the standards of evaluation––truth, accuracy, and credibility or also something else? (3) What type of verdict does validation lead to––that the simulation is such and such good, or that it passes a test defined by a certain threshold? (4) How strong needs the case to be for the verdict? (5) Does validation necessarily proceed by comparing simulation outputs with measured data? Along with these questions, the chapter explains notions that figure prominently in them, e.g., the concepts of accuracy and credibility. It further discusses natural answers to the questions as well as arguments that speak in favor and against these answers. The aim is to obtain a better understanding of the options we have for defining validation and how they are related to each other.

Keywords

Evaluation Model Code Truth Accuracy Credibility Adequate representation Adequacy for purpose Explanation Data-driven validation Test 

Notes

Acknowledgements

I am extremely grateful for extensive comments by William Oberkampf, Patrick Roache, and Nicole J. Saam. At a very different level, I wish to express my thanks to my mother, Bärbel Beisbart, who sadly passed away during the time when I was working on this chapter. I dedicate this chapter to her memory.

References

  1. AIAA. (1998). Guide for the verification and validation of computational fluid dynamics simulations, AIAA G-077–1998. American Institute of Aeronautics and Astronautics, Reston, VA.Google Scholar
  2. ASME. (2006). Guide for verification and validation in computational solid mechanics. American Society of Mechanical Engineers, ASME V&V 10-2006.Google Scholar
  3. Bogen, J., & Woodward, J. (1988). Saving the phenomena. Philosophical Review, 97(3), 303–352.CrossRefGoogle Scholar
  4. Blackburn, S. (1999). Think. Oxford: Oxford University Press.Google Scholar
  5. Bailer-Jones, D. M. (2003). When scientic models represent. International Studies in the Philosophy of Science, 17, 59–75.CrossRefGoogle Scholar
  6. Beisbart, C. (2012). How can computer simulations produce new knowledge? European Journal for Philosophy of Science, 2(3), 395–434.CrossRefGoogle Scholar
  7. Beisbart, C. (2014). Are we sims? How computer simulations represent and what this means for the simulation argument, The Monist 97/3 (2014, special issue edited by P. Humphreys), S. 399–417.CrossRefGoogle Scholar
  8. Brun, G. (2016). Explication as a method of conceptual re-engineering. Erkenntnis, 81, 1211–1241.MathSciNetCrossRefGoogle Scholar
  9. Beisbart, C. (2017). Advancing knowledge through computer simulations? A socratic exercise. In: M. Resch, A. Kaminski, P. Gehring (Eds.), The science and art of simulation I. Exploring–understanding–knowing (pp. 153–174). Cham:Springer.CrossRefGoogle Scholar
  10. Baumberger, C., Beisbart, C., & Brun, G. (2017a). What is understanding? An overview of recent debates in epistemology and philosophy of science. In S. Grimm, C. Baumberger, & S. Ammon (Eds.), Explaining understanding: New perspectives from epistemology and philosophy of science (pp. 1–34). New York: Routledge.Google Scholar
  11. Baumberger, C., Knutti, R., & Hirsch Hadorn, G. (2017b). Building confidence in climate model projections: An analysis of inferences from fit. WIREs Climate Change, 8, e454.  https://doi.org/10.1002/wcc.454.CrossRefGoogle Scholar
  12. Caldwell, S., & Morrison, R. J. (2000). Validation of longitudinal dynamic microsimulation models. Experience with CORSIM and DYNACAN. In: Mitton, L., Sutherland, H., & Weeks, M. J. (Eds.). Microsimulation modelling for policy analysis. challenges and innovations (pp. 200–225). Cambridge: Cambridge University Press.Google Scholar
  13. Cappelen, H. (2018). Fixing language: An essay on conceptual engineering. Oxford: Oxford University Press.CrossRefGoogle Scholar
  14. Carnap, R. (1950/1962) Logical foundations of probability. Chicago: University of Chicago Press.Google Scholar
  15. Grüne-Yanoff, T. (2009). The explanatory potential of artificial societies. Synthese, 169, 539–555.CrossRefGoogle Scholar
  16. Ghetiu, T., Polack, F. A., & Bown, J. (2010). Argument-driven validation of computer simulations–A necessity rather than an option. In: VALID 2010. The Second International Conference on Advances in System Testing and Validation Lifecycle (pp. 1–4) August 22–27, 2010. Nice, France, IEEE Press.Google Scholar
  17. Harman, G. H. (1965). The inference to the best explanation. Philosophical Review, 74(1), 88–95.CrossRefGoogle Scholar
  18. Hartmann, S. (1996). The world as a process: simulations in the natural and social sciences. In R. Hegselmann, et al. (Eds.), Modelling and simulation in the social sciences from the philosophy of science point of view, theory and decision library (pp. 77–100). Dordrecht: Kluwer.Google Scholar
  19. Hempel, C. G. (1945). Studies in the logic of confirmation (I.) (Vol. 54, No. 213, pp. 1–260) Mind, New Series.Google Scholar
  20. Hume, D. (1748). Essays concerning human understanding [now known as: An Enquiry Concerning Human Understanding], London: A. Millar, many new editions.CrossRefGoogle Scholar
  21. Humphreys, P. (2004). Extending ourselves: Computational science, empiricism, and scientific method. New York: Oxford University Press.CrossRefGoogle Scholar
  22. IEEE. (2012). IEEE standard for system and software verification and validation. In IEEE Std 1012-2012 (Revision of IEEE Std 1012-2004) (pp. 1–223), 25 May 2012.  https://doi.org/10.1109/ieeestd.2012.6204026.
  23. Kuhn, T. S. (1977). Objectivity, value judgment, and theory choice. In T. S. Kuhn (Ed.), The essential tension (pp. 320–339). Chicago: University of Chicago Press.CrossRefGoogle Scholar
  24. Künne, W. (2003). Conceptions of truth. Oxford: Clarendon Press.CrossRefGoogle Scholar
  25. Lacey, H. (1999). Is science value-free? Values and scientific understanding. London: Routledge.Google Scholar
  26. Lipton, P. (2000). Inference to the best explanation (2nd ed.). London: Routledge.Google Scholar
  27. Margolis, E., & Laurence, S. (2014). Concepts. In: E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy. https://plato.stanford.edu/archives/spr2014/entries/concepts.
  28. Naylor, T. H., & Finger, J. M. (1967). Verification of computer simulation models. Management Science, 14, B92–B101.CrossRefGoogle Scholar
  29. Norton, J. D. (2003). A material theory of induction. Philosophy of Science, 70, 647–670.CrossRefGoogle Scholar
  30. Oberkampf, W. & Roy, C. (2010). Verification and validation in scientific computing. Cambridge University Press.Google Scholar
  31. Oberkampf, W. L., & Tucano, T. G. (2008). Verification and validation benchmarks. Nuclear Engineering and Design, 238(3), 716–743.CrossRefGoogle Scholar
  32. Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263, 641–646.CrossRefGoogle Scholar
  33. Parker, W. S. (2008). Franklin, Holmes, and the epistemology of computer simulation. International Studies in the Philosophy of Science, 22(2), 165–183.MathSciNetCrossRefGoogle Scholar
  34. Parker, W. S. (2009). Confirmation and adequacy-for-purpose in climate modeling. Aristotelian Society Supplementary, 83, 233–249.CrossRefGoogle Scholar
  35. Pearl, J. (2000). Causality. Modeling, reasoning, and inference. Cambridge: Cambridge University Press.Google Scholar
  36. Resnik, D. B. (2003). Is the precautionary principle unscientific? Studies in History and Philosophy of Biological and Biomedical Sciences, 34, 329–344.  https://doi.org/10.1016/S1369-8486(02)00074-2.CrossRefGoogle Scholar
  37. Roache, P. J. (1998). Verification and validation in computational science and engineering. Albuquerque, New Mexico: Hermosa Publishers.Google Scholar
  38. Roache, P. J. (2009). Perspective: Validation—What does it mean?. Journals of Fluids Engineering; 131(3), 034503–034503-4.  https://doi.org/10.1115/1.3077134 (here quoted after the reprint in Roache 2013).CrossRefGoogle Scholar
  39. Roache, P. J. (2013). A defense of computational physics. Socorro, NM: Hermosa (revised printing).Google Scholar
  40. Schlesinger, S., et al. (1979). Terminology for model credibility. Simulation, 32, 103–104.CrossRefGoogle Scholar
  41. Suárez, M. (2004). An inferential conception of scientific representation. Philosophy of Science, 71, 767–779.CrossRefGoogle Scholar
  42. van Fraassen, B. (1980). The scientific image. Oxford: Clarendon Press.CrossRefGoogle Scholar
  43. Winsberg, E. (1999). Sanctioning models. The epistemology of simulation. Science in Context, 12, 275–292.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of BernBernSwitzerland

Personalised recommendations