Epistemic Entitlements and the Practice of Computer Simulation

Abstract

What does it mean to trust the results of a computer simulation? This paper argues that trust in simulations should be grounded in empirical evidence, good engineering practice, and established theoretical principles. Without these constraints, computer simulation risks becoming little more than speculation. We argue against two prominent positions in the epistemology of computer simulation and defend a conservative view that emphasizes the difference between the norms governing scientific investigation and those governing ordinary epistemic practices.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    See Kuhl et al. (2000) for an introduction to the general features of computer simulation in practice. For a historically important early introductory text see Naylor et al. (1966). These texts encompass the topics that we mean to capture using the term ‘computer simulation’.

  2. 2.

    The scientific role of mathematical models is a philosophically rich topic that falls beyond the scope of the present paper. For the most comprehensive discussion of the role of mathematical models in scientific reasoning see Pincock (2011). Pincock emphasizes that our confidence in computer simulations depends on our confidence in the prior mathematical model. He writes: “It is not always trivial to ensure that this has been done correctly, especially when computational or programming limitations force adjustments […] But if we have taken reasonable steps to ensure that [hardware failures or software bugs] did not occur, we can transfer the conclusion from the simulation back to the physical system” (2011, p. 80).

  3. 3.

    Some modeling (in geophysics for example) aims to explain static phenomena (modeling magnetic or gravitational fields). In other cases the use of closed-form equations may not need iterative solutions. While many of the points we discuss here also apply to non-dynamical cases, we focus on the dynamical cases here since they are most commonly discussed in the philosophical literature. We are grateful to an anonymous referee for reminding us to emphasize the static cases in which computer simulations are also used.

  4. 4.

    Philosophical treatments of the epistemology of computer simulations are in an early stage of development and work on this topic so far has been covered well by Eric Winsberg in his Stanford Encyclopedia of Philosophy article (2015).

  5. 5.

    In this paper we give particular attention to the first two questions. Thorough discussions of the two last questions can be found elsewhere (Symons and Horner 2014; Symons and Alvarado 2016; Alvarado and Humphreys 2017).

  6. 6.

    For example, policies regarding large-scale institutional interventions, life-critical systems assessments, existential risks, and the outcome of otherwise untested medical procedures would fall within this category.

  7. 7.

    See for example Barberrouse and Vorms (2014) and Beisbart (2017).

  8. 8.

    Saam (2017), for example, thinks that given the problem of epistemic opacity can be a serious obstacle to the reliability of simulation outcomes, only those computer simulations that are generated and treated as thought experiments (as opposed to laboratory experiments) should be considered in the social sciences. Simulations treated as experiments have at their core, significant issues concerning access or lack thereof to all the relevant steps of their processes (Humphreys 2009). Further, conventional error assessment strategies may be inadequate in contexts where software use is prominent (Symons and Horner 2014; Horner and Symons, forthcoming). In contrast, simulations that are treated merely as thought experiments, according to Saam, provide “well-founded answers to what-if-things-had-been-different questions” (Saam 2017, p. 81).

  9. 9.

    Notice that explicit theoretical or justificatory assumptions from the part of the first people to trust the oracle are not a necessary component of their reliance on the oracle. They could have begun to trust the oracles by default, accident or even superstition (See Skinner’s 1947 study “‘Superstition’ in the Pigeon”. In it pigeons would continue to perform behaviors they equivocally associated with food rewards solely because the behavior and the distribution coincided successfully in the past).

  10. 10.

    A position of this kind leads Quine to concede that he would have to count successful predictions from clairvoyants and telepaths as science (1973, see especially his 1990, pp 20–21). While genuine clairvoyance would be extremely interesting and useful, on the view that we defend here it would be a phenomenon that is largely orthogonal to science.

  11. 11.

    As helpfully noted by a reviewer of this paper, in the context of computer simulation, other secondary criteria are also at play for pragmatists. These include simplicity of the algorithm and greater unification of algorithmic calculations. However, these criteria will generally be subordinate to predictive power as a mark of success for the pragmatist.

  12. 12.

    We thank one of our anonymous referees for drawing our attention to this issue.

  13. 13.

    Thus, for example in cases where an applied mathematician uses techniques that require tacit knowledge or judgment developed through practice to reach some conclusion, we could recognize and rely on this fact in the same way we might recognize that someone who successfully rode a bicycle to work arrived to work without their being able to articulate explicitly, the details of how one rides a bike. The rider’s ability to ride a bicycle will not be entirely explainable by the rider, but the fact that he rode to work successfully can be regarded as undeniable and we can believe his claim to have made the journey by bicycle in virtue of knowing that he has the ability to ride a bicycle.

  14. 14.

    As we will see, even human expert credentials are not given out the same way we give out credentials to qualify as minimally reasonable interlocutors. I can trust my son, as Burge suggests, when he says that there is no more milk in the fridge based on a minimum intelligibility requirement and an assumption of healthy perceptual capacities. We would apply very different criteria in our choice of scientific experts.

  15. 15.

    The pragmatic approach seems to presume that science using computational methods is so fluid that the methods and instruments themselves can only be assessed in an ad hoc manner. The focus on reading simulations as sets of isolated measurement tasks misses, for example, the role of carefully curated data and principled theoretical background knowledge in the judgment of the expert interpreter.

  16. 16.

    An example of the pitfalls of atheoretical Big Data studies, is discussed by Lazer et al. (2014). Ultimately, many advocates of Big Data were misguided in spite of being able to point to well-established practices, successful predictive patterns and expertise within the data science community.

  17. 17.

    Burge’s account of the Acceptance Principle (as he acknowledges) is very similar in spirit to the Principle of Charity, as it figures in Quine (1960) and Davidson (1973). The principal difference between these principles is the role that Burge’s notion of preservation of content plays in his account.

  18. 18.

    In response to a referee comment, it should be mentioned that this emphasis on the apriori and the purely formal aspects of the target system contrasts sharply with the Materiality Thesis (Morgan 2005; Parker 2009), Morgan notes that a computer model is similar to its target system only in virtue of its form, while an experimental study that involves the target system itself is more likely to be generalizable in virtue of the material similarities between the object of the study and other instances of the kind (See also Roush 2015). Note that accepting the materiality thesis means that the devices by which information is manipulated bear significant epistemic import and that there are considerations other than the purely formal at stake. Thus, holders of the materiality thesis should be cautious with respect to the role of transparent conveyers in justification. (See also Barberousse et al. 2009).

  19. 19.

    Though they acknowledge that there is a substantial difference between computer assisted mathematical proofs, such as the ones Burge focused on and complex computer simulations in terms of content preservation, they justify the aprioricity of a scientist’s entitlement to trust a simulation in virtue of a second strategy, which we will inspect in detail in section three: trusting computer simulations, they argue, is like trusting expert testimony. Though expert testimony may be fallible, for this view, casting a general doubt on the practice absent specific reasonable doubt can be seen as irrational.

  20. 20.

    Similarly, when we rely on our senses we grant that when they are working the way they are supposed to they transmit information without altering it. That is, as explained above, they are transparent conveyers. Thus, though one can acknowledge their fallibility, in the absence of a plausible reason to doubt their well-functioning, it is rational to rely on the senses (Burge 1993).

  21. 21.

    As we will see below, whether one has reason not to doubt, no reason to doubt, or reason to trust represent significantly distinct challenges for this prerequisite.

  22. 22.

    Philosophy is another social practice that sets abnormally high epistemic standards. In our case, we aim high with respect to what should count as a rationally persuasive argument.

  23. 23.

    One can think of discretization as trying to approximate a circle by drawing one regular polygon after another with more sides each time starting form a square. Of course a square is a terrible circle, but a polygon with millions of sides may be visually indistinguishable for practical purposes. Nevertheless at each point, one is not drawing a continuous curve but rather a series of straight lines at an angle from each other.

  24. 24.

    McEvoy in his response to Tymoczko and Kitcher on the aprioricity of computer assisted mathematical proofs concedes as much by saying “What determines whether a proof is a priori is the type of inferential processes used to establish the conclusion of that proof. If the method of inference for any of the steps in the proof is a posteriori, it is a posteriori” (2008, p. 380).

  25. 25.

    This is especially the case now that memory has become so inexpensive in modern computing. The ‘true’ command in Unix, for example, which originally consisted of an empty file with nothing to execute grew to nearly 23,000 bytes from 1979 to 2012.

  26. 26.

    See Horner and Symons (forthcoming) for a review of the empirical literature on software error. They show that there has been a relatively consistent level of error reported in empirical studies from 1978 to 2018—for every 100 lines of code between 1 and 2 lines of code contain errors. (See also Symons and Horner 2017).

  27. 27.

    In saying this we are not discounting the serious time constraints faced by individual scientists in their careers e.g. the need to publish novel findings, the pressure of funding agencies, or the pursuit of tenure requirements. Rather, we are referring to scientific inquiry as a series of methods aimed at furnishing the best available understanding of our world. As such, scientist (from astrophysics to geology to biology- and some social sciences) can and ought to ensure a rigor in their methods that the nature of industry and war seldom afford.

  28. 28.

    It is unclear, for example, the extent to which Barberousse and Vorms assume a negative (entitlement to not doubt) or positive (entitlement to trust) role for entitlement in their arguments. However, this particular paragraph focuses on the completely distinct categories between kinds of entitlements (a kind of warrant that gives someone a reason to believe (x)) beyond their epistemic variety.

  29. 29.

    The five-sigma standard corresponds to a p value, or probability, of 3 × 10−7, or 1 in 3.5 million. In this case it is the probability that if the particle does not exist, the CERN team would find what they observed. It is extremely unlikely that they could have generated the data by accident. How unlikely? 1 in 3.5 million unlikely!

References

  1. Adler, J. (2015). Epistemological problems of testimony. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy (summer 2015 edition). https://plato.stanford.edu/archives/sum2015/entries/testimony-episprob/. Accessed 20 Dec 2018.

  2. Alvarado, R., & Humphreys, P. (2017). Big data, thick mediation, and representational opacity. New Literary History, 48(4), 729–749.

    Article  Google Scholar 

  3. Arkoudas, K., & Bringsjord, S. (2007). Computers, justification, and mathematical knowledge. Minds and Machines, 17(2), 185–202.

    Article  Google Scholar 

  4. Audi, R. (1997). The place of testimony in the fabric of knowledge and justification. American Philosophical Quarterly, 34(4), 405–422.

    Google Scholar 

  5. Barberousse, A., Franceschelli, S., & Imbert, C. (2009). Computer simulations as experiments. Synthese, 169(3), 557–574.

    MathSciNet  Article  Google Scholar 

  6. Barberousse, A., & Vorms, M. (2014). About the warrants of computer-based empirical knowledge. Synthese, 191(15), 3595–3620.

    Article  Google Scholar 

  7. Beebee, H. (2001). Transfer of warrant, begging the question and semantic externalism. The Philosophical Quarterly, 51(204), 356–374.

    Article  Google Scholar 

  8. Beisbart, C. (2017). Advancing knowledge through computer simulations? A socratic exercise. In M. Resch, A. Kaminski, & P. Gehring (Eds.), The science and art of simulation I (pp. 153–174). Berlin: Springer.

    Google Scholar 

  9. Borge, S. (2003). The word of others. Journal of Applied Logic, 1(1–2), 107–118.

    MathSciNet  Article  MATH  Google Scholar 

  10. Boschetti, F., Fulton, E., Bradbury, R., & Symons, J. (2012). What is a model, why people don’t trust them and why they should. In M. R. Raupach (Ed.), Negotiating our future: Living scenarios for Australia to 2050 (pp. 107–118). Australian Academy of Science.

  11. Burge, T. (1993). Content preservation. The Philosophical Review, 102(4), 457–488.

    Article  Google Scholar 

  12. Burge, T. (1998). Computer proof, apriori knowledge, and other minds: The sixth philosophical perspectives lecture. Noûs, 32(S12), 1–37.

    MathSciNet  Article  MATH  Google Scholar 

  13. Davidson, D. (1973). Radical interpretation. Dialectica, 27(3–4), 313–328.

    Article  Google Scholar 

  14. Davies, M. (2004) II—Martin Davies: Epistemic entitlement, warrant transmission and easy knowledge. In Aristotelian Society supplementary volume (Vol. 78(1)). Oxford: The Oxford University Press.

  15. Dretske, F. (2000). Entitlement: Epistemic rights without epistemic duties? Philosophy and Phenomenological Research, 60(3), 591–606.

    Article  Google Scholar 

  16. Fresco, N., & Primiero, G. (2013). Miscomputation. Philosophy & Technology, 26(3), 253–272.

    Article  Google Scholar 

  17. Frigg, R., & Reiss, J. (2009). The philosophy of simulation: Hot new issues or same old stew? Synthese, 169(3), 593–613.

    MathSciNet  Article  Google Scholar 

  18. Gramelsberger, G. (2011). Generation of evidence in simulation runs: Interlinking with models for predicting weather and climate change. Simulation & Gaming, 42(2), 212–224.

    Article  Google Scholar 

  19. Holzmann, G. J. (2015). Code inflation. IEEE Software, 2, 10–13.

    Article  Google Scholar 

  20. Horner, J., & Symons, J. (2014). Reply to Angius and Primiero on software intensive science. Philosophy & Technology, 27(3), 491–494.

    Article  Google Scholar 

  21. Horner, J, & Symons, J. (forthcoming). Understanding error rates in software engineering: Conceptual, empirical, and experimental approaches.

  22. Hubig, C, & Kaminski, A. (2017). Outlines of a pragmatic theory of truth and error in computer simulation. In M. Resch, A. Kaminski, & P. Gehring (Eds.), The science and art of simulation I (pp. 121–136). Cham: Springer.

    Google Scholar 

  23. Humphreys, P. (2004). Extending ourselves: Computational science, empiricism, and scientific method. Oxford: Oxford University Press.

    Google Scholar 

  24. Humphreys, P. (2009). The philosophical novelty of computer simulation methods. Synthese, 169(3), 615–626.

    MathSciNet  Article  Google Scholar 

  25. Jenkins, C. S. (2007). Entitlement and rationality. Synthese, 157(1), 25–45.

    Article  Google Scholar 

  26. Kuhl, F., Dahmann, J., & Weatherly, R. (2000). Creating computer simulation systems: An introduction to the high level architecture. Upper Saddle River: Prentice Hall.

    Google Scholar 

  27. Lackey, J. (1999). Testimonial knowledge and transmission. The Philosophical Quarterly, 49(197), 471–490.

    Article  Google Scholar 

  28. Lazer, D., Kennedy, R., King, G., et al. (2014). The parable of Google Flu: Traps in big data analysis. Science, 434, 343.

    Google Scholar 

  29. McEvoy, M. (2008). The epistemological status of computer-assisted proofs. Philosophia Mathematica, 16(3), 374–387.

    MathSciNet  Article  MATH  Google Scholar 

  30. McEvoy, M. (2013). Experimental mathematics, computers and the a priori. Synthese, 190(3), 397–412.

    MathSciNet  Article  MATH  Google Scholar 

  31. McGlynn, A. (2014). On Epistemic Alchemy. In D. Dodd, & E. Zardini (Eds.), Scepticism and Perceptual Justification. (pp. 173–189), OUP Oxford.

  32. Moretti, L., & Piazza, T. (2013). When warrant transmits and when it doesn’t: Towards a general framework. Synthese, 190(13), 2481–2503.

    Article  Google Scholar 

  33. Morgan, M. S. (2005). Experiments versus models: New phenomena, inference and surprise. Journal of Economic Methodology, 12(2), 317–329.

    Article  Google Scholar 

  34. Morrison, M. (2015). Reconstructing reality. Oxford: Oxford University Press.

    Google Scholar 

  35. Naylor, T. H., Balintfy, J. L., Burdick, D. S., & Chu, K. (1966). Computer simulation techniques. New York: Wiley.

    Google Scholar 

  36. Newman, J. (2015). Epistemic opacity, confirmation holism and technical debt: Computer simulation in the light of empirical software engineering. In International conference on history and philosophy of computing (pp. 256–272). Springer.

  37. Nola, R., & Sankey, H. (2014). Theories of scientific method: An introduction. Abingdon: Routledge.

    Google Scholar 

  38. Norton, S., & Suppe, F. (2001). Why atmospheric modeling is good science (pp. 67–105). Changing the atmosphere: Expert knowledge and environmental governance.

    Google Scholar 

  39. O’Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. New York: Broadway Books.

    Google Scholar 

  40. Oreskes, N. (2004). The scientific consensus on climate change. Science, 306(5702), 1686–1686.

    Article  Google Scholar 

  41. Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263(5147), 641–646.

    Article  Google Scholar 

  42. Parker, W. S. (2009). Does matter really matter? Computer simulations, experiments, and materiality. Synthese, 169(3), 483–496.

    Article  Google Scholar 

  43. Pincock, C. (2011). Mathematics and scientific representation. Oxford: Oxford University Press.

    Google Scholar 

  44. Pryor, J. (2012). When warrant transmits. In W. Crispin (Ed.), Mind, meaning, and knowledge: Themes from the philosophy of Crispin Wright (pp. 269–303). Oxford: University Press.

    Google Scholar 

  45. Quine, W. V. (1973). The roots of reference. La Salle, Ill: Open Court.

    Google Scholar 

  46. Quine, W. (1960). Word and object. MIT press.

  47. Resch, M. M., Kaminski, A., & Gehring, P. (Eds.). (2017). The science and art of simulation I: Exploring-understanding-knowing. Berlin: Springer.

    Google Scholar 

  48. Resnik, M. (1997). Mathematics as a science of patterns. New York: Oxford University Press.

    Google Scholar 

  49. Roush, S. (2015). The epistemic superiority of experiment to simulation. Synthese, 169, 1–24.

    Google Scholar 

  50. Ruphy, S. (2011). Limits to modeling: Balancing ambition and outcome in astrophysics and cosmology. Simulation & Gaming, 42(2), 177–194.

    Article  Google Scholar 

  51. Ruphy, S. (2015). Computer simulations: A new mode of scientific inquiry? In S. O. Hansen (Ed.), The role of technology in science: Philosophical perspectives (pp. 131–148). Dordrecht: Springer.

    Google Scholar 

  52. Saam, N. J. (2017). Understanding social science simulations: Distinguishing two categories of simulations. In M. Resch, A. Kaminski, & P. Gehring (Eds.), The science and art of simulation I (pp. 67–84). Cham: Springer.

    Google Scholar 

  53. Steadman, I. (2013). Big data and the death of the theorist. Wired Online, 25, 2013.

    Google Scholar 

  54. Symons, J. (2008). Computational models of emergent properties. Minds and Machines, 18(4), 475–491.

    Article  Google Scholar 

  55. Symons, J., & Alvarado, R. (2016). Can we trust big data? Applying philosophy of science to software. Big Data & Society, 3(2), 2053951716664747.

    Article  Google Scholar 

  56. Symons, J., & Boschetti, F. (2013). How computational models predict the behavior of complex systems. Foundations of Science, 18(4), 809–821.

    Article  Google Scholar 

  57. Symons, J., & Horner, J. (2014). Software intensive science. Philosophy & Technology, 27(3), 461–477.

    Article  Google Scholar 

  58. Symons, J., & Horner, J. (2017). On some limits to model-based proof of software correctness. In T. Powers (Ed.), Philosophy and computing: Essays in epistemology, philosophy of mind, logic, and ethics. Berlin: Springer.

    Google Scholar 

  59. Tymoczko, T. (1979). The four-color problem and its philosophical significance. Journal of Philosophy, 76, 57–82.

    Article  Google Scholar 

  60. Vallor, S. (2017). AI and the automation of wisdom. In T. Powers (Ed.), Philosophy and computing: Essays in epistemology, philosophy of mind, logic, and ethics. Philosophical Studies Series (Vol. 128, pp. 161–178). Berlin: Springer.

    Google Scholar 

  61. Williams, M. (2000). Dretske on epistemic entitlement. Philosophy and Phenomenological Research, 60(3), 607–612.

    Article  Google Scholar 

  62. Wright, C., & Davies M. (2004) On epistemic entitlement. In Proceedings of the aristotelian society, supplementary volumes (Vol. 78, pp. 167–245). www.jstor.org/stable/4106950. Accessed 20 Dec 2018.

  63. Winsberg, E. (2010). Science in the age of computer simulation. Chicago: University of Chicago Press.

    Google Scholar 

  64. Winsberg, E. (2015). Computer simulations in science. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy (summer 2015 edition). http://plato.stanford.edu/archives/sum2015/entries/simulations-science/. Accessed 20 Dec 2018.

Download references

Acknowledgements

This paper has benefited greatly from the work of two referees for this journal. We sincerely thank both of them for their detailed criticisms and thoughtful questions. We are grateful also to Samuel Arbesman, Jack Horner, Paul Humphreys, and Andreas Kaminski for discussions that contributed to the development of this paper. This work is supported by The National Security Agency through the Science of Security initiative contract #H98230-18-D-0009.

Author information

Affiliations

Authors

Corresponding author

Correspondence to John Symons.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by The National Security Agency through the Science of Security initiative contract #H98230-18-D-0009.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Symons, J., Alvarado, R. Epistemic Entitlements and the Practice of Computer Simulation. Minds & Machines 29, 37–60 (2019). https://doi.org/10.1007/s11023-018-9487-0

Download citation

Keywords

  • Computer simulation
  • Trust
  • Epistemology
  • Entitlements
  • Models