Philosophical Studies

, Volume 172, Issue 9, pp 2299–2313 | Cite as

How much evidence should one collect?

  • Remco Heesen


A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their justifications are to be successful, they need to rely on the finite sequence being either indefinitely increasing or of a large size. These assumptions are often not met in practice. This paper analyzes a simple model of collecting evidence and finds that the practice of collecting only very small sets of evidence before taking a question to be settled is rationally justified. This shows that the appeal to long run results can be used neither to explain the success of actual scientific practice nor to give a rational reconstruction of that practice.


Philosophy of science Evidence Rational choice Formal epistemology Bayesian epistemology Sequential decision problems 



Thanks to Kevin Zollman, Kevin Kelly, Liam Bright, Adam Brodie, and an anonymous referee for valuable comments and discussion.


  1. Boring, E. G. (1950). A history of experimental psychology (2nd ed.). New Jersey: Prentice-Hall.Google Scholar
  2. Casella, G., & Berger, R. L. (2001). Statistical inference (2nd ed.). Belmont: Duxbury.Google Scholar
  3. DeGroot, M. H. (2004). Optimal statistical decisions. New Jersey: Wiley.CrossRefGoogle Scholar
  4. Earman, J. (1992). Bayes or bust? A critical examination of Bayesian confirmation theory. Cambridge: MIT Press.Google Scholar
  5. Edwards, W., Lindman, H., & Savage, L. J. (1963). Bayesian statistical inference for psychological research. Psychological Review, 70(3), 193–242.CrossRefGoogle Scholar
  6. Friedman, M. (1979). Truth and confirmation. The Journal of Philosophy, 76(7), 361–382.CrossRefGoogle Scholar
  7. Hempel, C. G. (1945a). Studies in the logic of confirmation (I.). Mind, 54(213), 1–26.CrossRefGoogle Scholar
  8. Hempel, C. G. (1945b). Studies in the logic of confirmation (II.). Mind, 54(214), 97–121.CrossRefGoogle Scholar
  9. Howson, C., & Urbach, P. (1989). Scientific reasoning: the Bayesian approach. La Salle: Open Court.Google Scholar
  10. Kelly, K. T. (1991). Reichenbach, induction, and discovery. Erkenntnis, 35(1/3), 123–149.Google Scholar
  11. Kelly, K. T. (1996). The logic of reliable inquiry. Oxford: Oxford University Press.Google Scholar
  12. Lai, T. L. (1981). Asymptotic optimality of invariant sequential probability ratio tests. The Annals of Statistics, 9(2), 318–333.CrossRefGoogle Scholar
  13. Liu, Y., & Blostein, S. D. (1992). Optimality of the sequential probability ratio test for nonstationary observations. IEEE Transactions on Information Theory, 38(1), 177–182.CrossRefGoogle Scholar
  14. Olesko, K. M., & Holmes, F. L. (1993). Experiment, quantification, and discovery: Helmholtz’s early physiological researches, 1843–50. In D. Cahan (Ed.), Hermann von Helmholtz and the foundations of Nineteenth-century science, Chap. 2 (pp. 50–108). Berkeley: University of California Press.Google Scholar
  15. Peirce, C. S. (1931 [1878]). How to make our ideas clear. In C. Hartstone & P. Weiss (Eds.), Collected Papers of Charles Sanders Peirce (Vol. V, pp. 5.388–5.410). Cambridge: Harvard University Press.Google Scholar
  16. Popper, K. (1959). The logic of scientific discovery. London: Hutchinson.Google Scholar
  17. Reichenbach, H. (1938). Experience and prediction: An analysis of the foundations and the structure of knowledge. Chicago: The University of Chicago Press.CrossRefGoogle Scholar
  18. Schurz, G. (2008). The meta-inductivist’s winning strategy in the prediction game: A new approach to Hume’s problem. Philosophy of Science, 75(3), 278–305.CrossRefGoogle Scholar
  19. Stein, C. (1946). A note on cumulative sums. The Annals of Mathematical Statistics, 17(4), 498–499.CrossRefGoogle Scholar
  20. von Helmholtz, H. L. F. (1850). Messungen über den zeitlichen Verlauf der Zuckung animalischer Muskeln und die Fortpflanzungsgeschwindigkeit der Reizung in den Nerven Archiv für Anatomie, Physiologie und wissenschaftliche Medicin (pp. 276–364).Google Scholar
  21. Wald, A. (1947). Sequential analysis. New York: Wiley.Google Scholar
  22. Wald, A., & Wolfowitz, J. (1948). Optimum character of the sequential probability ratio test. The Annals of Mathematical Statistics, 19(3), 326–339.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  1. 1.Department of PhilosophyCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations