Abstract
Theories of statistical testing may be seen as attempts to provide systematic means for evaluating scientific conjectures on the basis of incomplete or inaccurate observational data. The Neyman-Pearson Theory of Testing (NPT) has purported to provide an objective means for testing statistical hypotheses corresponding to scientific claims. Despite their widespread use in science, methods of NPT have themselves been accused of failing to be objective; and the purported objectivity of scientific claims based upon NPT has been called into question. The purpose of this paper is first to clarify this question by examining the conceptions of (I) the function served by NPT in science, and (II) the requirements of an objective theory of statistics upon which attacks on NPT's objectivity are based. Our grounds for rejecting these conceptions suggest altered conceptions of (I) and (II) that might avoid such attacks. Second, we propose a reformulation of NPT, denoted by NPT*, based on these altered conceptions, and argue that it provides an objective theory of statistics. The crux of our argument is that by being able to objectively control error frequencies NPT* is able to objectively evaluate what has or has not been learned from the result of a statistical test.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Birnbaum, A.: 1977, ‘The Neyman-Pearson Theory as Decision Theory, and as Inference Theory; With a Criticism of the Lindley-Savage Argument for Bayesian Theory’,Synthese 36, 19–50.
Carnap, R.: 1950,Logical Foundations of Probability, University of Chicago Press, Chicago.
Dempster, A. P.: 1971, ‘Model Searching and Estimation in the Logic of Inference’, in V. P. Godambe and D. A. Sprott (eds.),Foundations of Statistical Inference, Holt, Rinehart and Winston of Canada, Toronto, 56–77.
Edwards, A. W. F.: 1971, ‘Science, Statistics and Society’,Nature 233, 17–19.
Fetzer, J. H.: 1981,Scientific Knowledge, Reidel, Dordrecht.
Fisher, R. A.: 1955, ‘Statistical Methods and Scientific Induction’,Journal of the Royal Statistical Society (B)17, 69–78.
Giere, R. N.: 1976, ‘Empirical Probability, Objective Statistical Methods and Scientific Inquiry’, in W. L. Harper and C. A. Hooker (eds.),Foundations of Probability Theory, Statistical Inference and Statistical Theories of Science, Vol. II, Reidel, Dordrecht, 63–101.
Giere, R. N.: 1977, ‘Testing vs. Information Models of Statistical Inference’, in R. G. Colodny (ed.),Logic Laws and Life, University of Pittsburgh Press, Pittsburgh, 19–70.
Good, I. J.: 1976, ‘The Bayesian Influence, or How to Sweep Subjectivism Under the Carpet’, in W. L. Harper and C. A. Hooker (eds.),Foundations of Probability Theory, Statistical Inference and Statistical Theories of Science, Vol. II, Reidel, Dordrecht, 125–174.
Good, I. J.: 1981, ‘Some Logic and History of Hypothesis Testing’, in J. C. Pitt (ed.),Philosophy in Economics, Reidel, Dordrecht, 149–174.
Hacking, I.: 1965,Logic of Statistical Inference, Cambridge University Press, Cambridge.
Hacking, I.: 1980, ‘The Theory of Probable Inference: Neyman, Peirce and Braithwaite’, in D. H. Mellor (ed.),Science, Belief and Behavior: Essays in Honor of R. B. Braithwaite, Cambridge University Press, Cambridge, 141–160.
Kalbfleisch, J. G.: 1979,Probability and Statistical Inference, Vol. II, Springer-Verlag, New York.
Kempthorne, O. and Folks, L.: 1971, ‘Probability, Statistics, and Data Analysis’, Iowa State University Press, Ames.
Kyburg, H. E. Jr.: 1971, ‘Probability and Informative Inference’, in V. P. Godambe and D. A. Sprott (eds.),Foundations of Statistical Inference, Holt, Rinehart and Winston of Canada, Toronto, 82–103.
Kyburg, H. E. Jr.: 1974,The Logical Foundations of Statistical Inference, Reidel, Dordrecht.
Lehmann, E. L.: 1959,Testing Statistical Hypotheses, John Wiley, New York.
Levi, I.: 1980,The Enterprise of Knowledge, MIT Press, Cambridge.
Lindley, D. V.: 1976, ‘Bayesian Statistics’, in W. L. Harper and C. A. Hooker (eds.),Foundations of Probability Theory, Statistical Inference and Statistical Theories of Science, Vol. II, Reidel, Dordrecht, 353–363.
Mayo, D.: 1981a, ‘In Defense of the Neyman-Pearson Theory of Confidence Intervals’,Philosophy of Science 48, 269–280.
Mayo, D.: 1981b, ‘Testing Statistical Testing’, in J. C. Pitt (ed.),Philosophy in Economics, Reidel, Dordrecht, 175–203.
Mayo, D.: 1982, ‘On After-Trial Criticisms of Neyman-Pearson Theory of Statistics’, in P. Asquith (ed.),PSA 1982, Vol. 1, East Lansing Philosophy of Science Association, 145–158.
Neyman, J.: 1950,First Course in Probability and Statistics, Henry Holt, New York.
Neyman, J.: 1971, Comments on R. M. Royall, ‘Linear Regression Models in Finite Population Sampling Theory’, in V. P. Godambe and D. A. Sprott (eds.),Foundations of Statistical Inference, Holt, Rinehart and Winston of Canada, Toronto, 276–278.
Neyman, J. and Pearson, E. S.: 1933, ‘On the Problem of the Most Efficient Tests of Statistical Hypotheses’, inPhilosophical Transactions of the Royal Society A, 231, 289–337. (As reprinted inJoint Statistical Papers, University of California Press, Berkeley, 1967, 276–283.)
Neyman, J. and Pearson, E. S.: 1936, ‘Contributions to the Theory of Testing Statistical Hypotheses’,Statistical Research Memoirs 1, 1–37. (As reprinted inJoint Statistical Papers, University of California Press, Berkeley, 1967, 203–239.)
Pearson, E. S.: 1955, ‘Statistical Concepts in Their Relation to Reality’,Journal of the Royal Statistical Society B,17, 204–207.
Popper, K. R.: 1972,Objective Knowledge, Oxford University Press, Oxford.
Rosenkrantz, R. D.: 1977,Inference, Method and Decision, Reidel, Dordrecht.
Rubin, H.: 1971, ‘Occam's Razor Needs New Blades’, in V. P. Godambe and D. A. Sprott (eds.),Foundations of Statistical Inference, 372–374.
Savage, L.: 1954,The Foundations of Statistics, Wiley & Sons, New York.
Scheffler, I.: 1967,Science and Subjectivity, Bobbs-Merrill, New York.
Seidenfeld, T.: 1979,Philosophical Problems of Statistical Inference, Reidel, Dordrecht.
Sokal, R. R. and Hunter, P. E.: 1955, ‘A Morphometric Analysis of DDT-Resistant and Non-Resistant Housefly Strains’,Annals of the Entomology Society of America 48, 499–507.
Sokal, R. R. and Rohlf, F. J.: 1969,Biometry, W. H. Freeman, San Francisco.
Spielman, S.: 1972, ‘A Reflection on the Neyman-Pearson Theory of Testing’,British Journal for the Philosophy of Science 24, 201–222.
Author information
Authors and Affiliations
Additional information
I would like to thank Ronald Giere for very useful comments and numerous valuable conversations concerning this paper and his own work. This research was carried out during tenure of a National Endowment for the Humanities Fellowship; I gratefully acknowledge this support.
Rights and permissions
About this article
Cite this article
Mayo, D.G. An objective theory of statistical testing. Synthese 57, 297–340 (1983). https://doi.org/10.1007/BF01064701
Issue Date:
DOI: https://doi.org/10.1007/BF01064701