Advertisement

Minds and Machines

, Volume 29, Issue 1, pp 169–186 | Cite as

Computer Modeling and Simulation: Increasing Reliability by Disentangling Verification and Validation

  • Vitaly PronskikhEmail author
Article
  • 64 Downloads

Abstract

Verification and validation (V&V) of computer codes and models used in simulations are two aspects of the scientific practice of high importance that recently have been discussed widely by philosophers of science. While verification is predominantly associated with the correctness of the way a model is represented by a computer code or algorithm, validation more often refers to the model’s relation to the real world and its intended use. Because complex simulations are generally opaque to a practitioner, the Duhem problem can arise with verification and validation due to their entanglement; such an entanglement makes it impossible to distinguish whether a coding error or the model’s general inadequacy to its target should be blamed in the case of a failure. I argue that a clear distinction between computer modeling and simulation has to be made to disentangle verification and validation. Drawing on that distinction, I suggest to associate modeling with verification and simulation, which shares common epistemic strategies with experimentation, with validation. To explain the reasons for their entanglement in practice, I propose a Weberian ideal–typical model of modeling and simulation as roles in practice. I examine an approach to mitigate the Duhem problem for verification and validation that is generally applicable in practice and is based on differences in epistemic strategies and scopes. Based on this analysis, I suggest two strategies to increase the reliability of simulation results, namely, avoiding alterations of verified models at the validation stage as well as performing simulations of the same target system using two or more different models. In response to Winsberg’s claim that verification and validation are entangled I argue that deploying the methodology proposed in this work it is possible to mitigate inseparability of V&V in many if not all domains where modeling and simulation are used.

Keywords

Modeling Computer simulations Verification Validation Experimentation 

Notes

Acknowledgements

The author is indebted to two anonymous reviewers for their careful reading of his manuscript and their many insightful comments and suggestions that helped improve the paper. I would like to thank the audience at Models and Simulations 6 conference for useful comments and discussions. I am grateful to Dr. Eric Winsberg and Dr. Arkadiy Lipkin for valuable feedback on the earlier versions of the manuscript. Fermi National Accelerator Laboratory is operated by the Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the U.S. Department of Energy, Office of Science, Office of High Energy Physics.

References

  1. Barberousse, A., Franceschelli, S., & Imbert, C. (2009). Computer simulations as experiments. Synthese, 169, 557–574.MathSciNetCrossRefGoogle Scholar
  2. Bruning, O., Cappi, R., Garoby, R., Grobner, O., Herr, W., Linnecar, T., Ostojic, R., Potter, K., Rossi, L., Ruggiero, F. (Eds.), Schindl, K., Stevenson, G., Tavian, L., Taylor, T., Tsesmelis, E., Weisse, E. & Zimmermann, F. (2002). LHC luminosity and energy upgrade: A feasibility study. CERN-LHC-Project-Report- 626, CERN.Google Scholar
  3. Chandrasekharan, S., Nersessian, N. J., & Subramanian, V. (2012). Computational modeling: Is this the end of thought experimenting in science? In J. Brown, M. Frappier, & L. Meynell (Eds.), Thought experiments in philosophy, science and the arts (pp. 239–260). London: Routledge.Google Scholar
  4. De Roeck, A., Gianotti, F., Morsch, A., & Pokorski, W. (2004). Simulation physics requirements from the LHC experiments, CERN-LCGAPP-2004-02.Google Scholar
  5. Deniau, L., Grote, H., Roy, G., & Schmidt, F. (2018). The MAD-X Program (Methodical Accelerator Design) Version 5.04.02 User’s reference manual. European Laboratory for Particle Physics. http://madx.web.cern.ch/madx/.
  6. Durán, J. M. (2018). Computer simulations as a technological singularity in the empirical sciences. In S. Armstrong, J. Miller, R. Yampolskiy, & V. Callaghan (Eds.), The technological singularity—A pragmatic perspective. Berlin: Springer.Google Scholar
  7. Franklin, A. (1994). How to avoid the experimenters’ regress. Studies in History and Philosophy of Science, 25(3), 463–491.CrossRefGoogle Scholar
  8. Franklin, A. (2012). Experiment in physics. In E. N. Zalta (Ed.) The Stanford encyclopedia of philosophy (Winter 2012 Edition). http://plato.stanford.edu/archives/win2012/entries/physics-experiment/.
  9. Frigg, R., & Reiss, J. (2009). The philosophy of simulation: Hot new issues or same old stew? Synthese, 169, 593–613.MathSciNetCrossRefGoogle Scholar
  10. Galison, P. (2003) The collective author. In P. Galison, M. Biagioli (Eds.) Scientific authorship: Credit and intellectual property in science (pp. 325–353). New York and Oxford: Routledge.Google Scholar
  11. Gorman, M. E. (2010). Trading zones and interactional expertise. In M. E. Gorman (Ed.), Creating new kinds of collaboration. Cambridge: MIT Press.Google Scholar
  12. Hartmann, S. (1996). The world as a process. Simulations in the natural and social sciences. In R. Hegselmann, U. Mueller, & K. Troitzch (Eds.), Modelling and simulation in natural sciences from the philosophy of science point of view (pp. 77–100). Dordrecht: Kluwer Academic Publishing.CrossRefGoogle Scholar
  13. Hasse, H., & Lenhard, J. (2017). Boon and bane: On the role of adjustable parameters in simulation models. In J. Lenhard & M. Carrier (Eds.), Mathematics as a tool. Boston studies in the philosophy and history of science (Vol. 327). Cham: Springer.Google Scholar
  14. Humphreys, P. (1991). Computer simulations. In A. Fine, M. Forbes, & L. Wessels (Eds.), PSA (Vol. 2, pp. 497–506). East Lansing: Philosophy of Science Association.Google Scholar
  15. Humphreys, P. (2004). Extending ourselves. Computational science, empiricism, and scientific method. Oxford: Oxford University Press.CrossRefGoogle Scholar
  16. Jebeile, J. (2012). Verification
 and 
validation
of
computer
simulations: A
philosophical
analysis. In Proceedings of the international conference models and simulations MS5 (p. 54), Helsinki.Google Scholar
  17. Keller, E. F. (2003). Models, simulation and computer experiments. In H. Radder (Ed.), The philosophy of scientific experimentation (p. 200). Pittsburgh: The University of Pittsburgh Press.Google Scholar
  18. Lenhard, J. (2007). Computer simulation: The cooperation between experimenting and modeling. Philosophy of Science, 74(2), 176–194.MathSciNetCrossRefGoogle Scholar
  19. Morrison, M. (2009). Models, measurement and computer simulation: the changing face of experimentation. Philosophical Studies, 143, 33–57.CrossRefGoogle Scholar
  20. Morrison, M. (2015). Reconstructing reality: Models, mathematics, and simulations. New York: Oxford University Press.CrossRefzbMATHGoogle Scholar
  21. Oberkampf, W. L. (2004). Verification and validation in computational simulation. http://www.psfc.mit.edu/ttf/2004/talks/oberkampf.pdf.
  22. Parke, E. C. (2014). Experiments, simulations, and epistemic privilege. Philosophy of Science, 81(4), 516–536.CrossRefGoogle Scholar
  23. Parker, W. S. (2009). Does matter really matter? Computer simulations, experiments, and materiality. Synthese, 169, 483–496.CrossRefGoogle Scholar
  24. Peck, S. L. (2004). Simulation as experiment: A philosophical reassessment for biological modeling. Trends in Ecology and Evolution, 19(10), 530–534.CrossRefGoogle Scholar
  25. Pronskikh, V. (2018). Linguistic privilege and justice: What can we learn from STEM? Fermilab publication FERMILAB-PUB-18-013.Google Scholar
  26. Spentzouris, P., & Amundson, J. (2004). Synergia: A modern tool for accelerator physics simulation. Fermilab preprint FERMILAB-CONF-04-488-CD. In Proceedings of 14th International Conference on Computing in High-Energy and Nuclear Physics (CHEP 2004), 27 Sep–1 Oct 2004 (pp. 223–236). Interlaken, Switzerland.Google Scholar
  27. Suppes, P. (1960). A comparison of the meaning and use of models in the mathematical and the empirical sciences. Synthese, 12, 287–300.MathSciNetCrossRefzbMATHGoogle Scholar
  28. The American Institute of Aeronautics and Astronautics (AIAA). (1998). Committee on standards in computational fluid dynamics. http://www.aiaa.org/.
  29. Trujillo-Gomez, S., Klypin, A., Primack, J., & Romanowsky, A. J. (2011). Galaxies in ΛCDM with halo abundance matching: Luminosity–velocity relation, baryonic mass–velocity relation, velocity function and clustering. arXiv:1005.1289v3 [astro-ph.CO], September 23, 2011.
  30. Weber, M. (1949). ‘Objectivity’ in social science and social policy. In The methodology of the social sciences (Eds. by E. A. Shils & H. A. Finch, Trans.) (pp. 49–112). New York: Free Press.Google Scholar
  31. Weisberg, M. (2013). Simulation and similarity. Using models to understand the world. Oxford: Oxford University Press.CrossRefGoogle Scholar
  32. Winsberg, E. (2009). Computer simulation and the philosophy of science. Philosophy Compass, 4, 835–845.  https://doi.org/10.1111/j.1747-9991.2009.00236.x.CrossRefGoogle Scholar
  33. Winsberg, E. (2010). Science in the age of computer simulations. Chicago, London: The University of Chicago Press.CrossRefGoogle Scholar
  34. Winsberg, E. (2014). Computer simulations in science. In E. N. Zalta (Ed.) The Stanford Encyclopedia of Philosophy (Fall 2014 Edition). http://plato.stanford.edu/archives/fall2014/entries/simulations-science/.
  35. Winsberg, E. (2018). Climate science and philosophy. Cambridge: Cambridge University Press.CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Fermi National Accelerator LaboratoryBataviaUSA

Personalised recommendations