Abstract
The notion of a severe test has played an important methodological role in the history of science. But it has not until recently been analyzed in any detail. We develop a generally Bayesian analysis of the notion, compare it with Deborah Mayo’s error-statistical approach by way of sample diagnostic tests in the medical sciences, and consider various objections to both. At the core of our analysis is a distinction between evidence and confirmation or belief. These notions must be kept separate if mistakes are to be avoided; combined in the right way, they provide an adequate understanding of severity.
Those who think that the weight of the evidence always enables you to choose between hypotheses “ignore one of the factors (the prior probability) altogether, and treat the other (the likelihood) as though it ...meant something other than it actually does. This is the same mistake as is made by someone who has scruples about measuring the arms of a balance (having only a tape measure at his disposal ...), but is willing to assert that the heavier load will always tilt the balance (thereby implicitly assuming, although without admitting it, that the arms are of equal length!). (Bruno de Finetti, Theory of Probability)2
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
P. Achinstein (1983) The Nature of Explanation Clarendon Press Oxford
P. Achinstein (Eds) (1984) The Concept of Evidence Clarendon Press Oxford
P. Achinstein (2001) The Book of Evidence Clarendon Press Oxford
Bandyopadhyay, P. and Boik, R.: 1999, ‘The Curve-Fitting Problem: A Bayesian Rejoinder’, Philosophy of Science (Proceedings), S390–S402.
J. Berger (1985) Statistical Decision Theory and Bayesian Analysis EditionNumber2 Springer New York
J. Berger et al. (1987) ArticleTitle‘Testing a Precise Hypothesis with Discussion’ Statistical Science 3 317–352
J. Berger et al. (1996) ArticleTitle‘The Intrinsic Bayes Factor for Model Selection and Prediction’ Journal of American Statistical Association 91 109–122
J. Berger et al. (1997) ArticleTitle‘Unified Frequentist and Bayesian Testing of a Precise Hypothesis (with Discussion)’ Statistical Science 12 133–160
G. Bernard (1987) ‘Comments’ Bernard Groot ParticleDe Lindley Smith (Eds) Bayesian Statistics NumberInSeriesVol II. North-Holland Publishing Company Amsterdam 57–60
D. Berry (1996) Statistics: A Bayesian Perspective Duxbury Press Belmont
D. Berry et al. (1996) Bayesian Biostatistics Marcel Dekker Inc. New York
A. Birnbaum (1969) ‘Concepts of Statistical Evidence’ Morgenbesser Suppes White (Eds) Philosophy, Science, and Method: Essays in Honor of Ernest Nagel St. Martin’s Press New York 112–143
B. Brody (1970) Readings in the Philosophy of Science Prentice-Hall Englewood Cliffs, N.J.
Carnap, R.: 1949, ‘The Two Concepts of Probability’, in Feigl and Sellars (eds), Readings in Philosophical Analysis, Appleton-Century-Crofts
Carnap, R.: 1955, ‘Statistical and Inductive Probability’, pamphlet published by the Galois Institute of Mathematics, Brooklyn, N.Y., reprinted in Brody (1970), pp. 440–450.
De Finetti, B.: 1951, ‘Recent Suggestions for the Reconciliation of Theories of Probability’, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley & Los Angeles, pp. 217–225.
Demissie et al. (1998) ArticleTitle‘Bayesian Estimation of the Asthma Prevalence, and Comparison of Exercise and Questionnaire Diagnostics in the Absence of a Gold Standard’ Annals of Epidemiology April 8 IssueID3 201–208
P. Duhem (1962) The Aim and Structure of Physical Theory Atheneum New York
J. Earman (1992) Bayes or Bust? MIT Cambridge, MA
Edwards Lindman Savage (1963) ArticleTitle‘Bayesian Statistical Inference for Psychological Research’ Psychological Review 70 IssueID3 193–242
E. Eells (1993) ‘Probability, Inference, and Decision’ J. Fetzer (Eds) Foundations of Philosophy of Science Paragon Press New York 192–208
Epstein et al. (1996) ArticleTitle‘Bayesian Imputation of Predictive Values when Covariate Information is Available and Gold Standard analysis is Unavailable’ Statistics in Medicine, March 15 IssueID5 463–476
M. Forster (2000) ‘The New Science of Simplicity’ H.A. Kreuzenkamp (Eds) et al. Simplicity, Inference and Economic Modeling Cambridge University Press Cambridge
R. Galen R. Gambino (1975) Beyond Normality: The Predictive Value and Efficiency of Medical Diagnoses John Wiley & Sons New York
A. Gelman et al. (2000) Bayesian Data Analysis Chapman & Hall London
R. Giere (1997) ArticleTitle‘Scientific Inference: Two Points of View’ Philosophy of Science (supplement) 64 S180–S184
C. Glymour (1980) Theory and Evidence Princeton University Press Princeton
Good, J.: 1983, ‘Some Logic and History of Hypothesis Testing’, in Good Thinking, University of Minnesota Press, Minneapolis, pp. 129–148
J. Good (1992) ArticleTitle‘The Bayesian/Non-Bayesian Compromise’ Journal of the American Statistical Association 87 597–606
Grandy, R.: XXXX, ‘Some Comments on Confirmation and Selective Confirmation’, Philosophical Studies 18, 19–24.
T. Greenhalgh (2001) How to Read a Paper: The Basics of Evidence Based Medicine BMJ Books London
J. Groopman (1999) ArticleTitle‘Contagion’ The New Yorker, September 13 34–49
A. Guth (1997) The Inflationary Universe Addison-Wesley Boston
I. Hacking (1965) Logic of Statistical Inference Cambridge University Press Cambridge
G. Hellman (1997) ArticleTitle‘Bayes and Beyond’ Philosophy of Science 64 191–221
C. Hempel (1965) Aspects of Scientific Explanation Free Press New York
P. Horwich (1982) Probability and Evidence Cambridge University Press Cambridge
C. Howson (1997a) ArticleTitle‘Error Probabilities in Error’ Philosophy of Science (Supplement) 64 S185–S194
C. Howson (1997) ArticleTitle‘The Logic of Induction’ Philosophy of Science 64 268–290 Occurrence Handle10.1086/392551
C. Howson (2000) Hume’s Problem: Induction and the Justification of Belief Clarendon Press Oxford
R. Jeffrey (1956) ArticleTitle‘Valuation and Acceptance of Scientific Hypotheses’ Philosophy of Science 33 236–246
Jeffrey, R. et al.: 1988, ‘Probabilizing Pathology’, Proceedings of the Aristotelian Society, vol. 89, part 3
R. Jeffrey (1992) ‘Radical Probabilism’, Probability and the Art of Judgment Cambridge University Press Cambridge
D. Johnston (1990a) ArticleTitle‘Interpreting Statistical Insignificance: A Bayesian Perspective’ Psychological Reports 6 115–121
D. Johnston (1990b) ArticleTitle‘Sample Size and Strength of Evidence’ Abacus 26 17–35
Kyburg H. 1970, ‘Conjunctivitis’, in Epistemology and Inference, University of Minnesota Press, Minneapolis
I. Levi (1960) ArticleTitle‘Does Science Make Value Judgments?’ Journal of Philosophy 57 345–357
D. Mayo (1996) Error and the Growth of Experimental Knowledge The University of Chicago Press Chicago
Mayo, D.: 1997a, ‘Duhem’s Problem, The Bayesian Way, and Error Statistics, or “What’s Belief Got to Do With It?” and “Response to Howson and Lauden” ’, Philosophy of Science (June), 222–244, 323–333.
D. Mayo (1997b) ArticleTitle‘Error Statistics and Learning from Error: Making a Virtue of Necessity’ Philosophy of Science (Supplement) 64 S195–S212
D. Mayo M. Kruse (2001) ‘Principles of Inferences and Their Consequences’ Cornfield Williamson (Eds) Foundations of Bayesianism Kluwer Academic Publishers Dordrecht 381–404
Pagano et al. (2000) Principles of Biostatistics EditionNumber2 Duxbury Thomson Learning Pacific Grove, CA
K. Popper (1959) The Logic of Scientific Discovery, English Edition Hutchinson London
A. Raftery (1995) ‘Bayesian Model Selection in Social Research’ Marsden (Eds) Sociological Methodology Blackwell Cambridge, MA
A. Raftery et al. (1997) ArticleTitle‘Computing Bayes Factor by Combining Simulation and Asymptotic Approximations’ Journal of American Statistical Association 92 IssueID439 903–915
J. Rawls (1971) A Theory of Justice Harvard University Press Cambridge, MA
K. Rothman et al. (1998) Modern Epidemiology EditionNumber2 Lippincott-Raven Philadelphia
R. Royall (1997) Statistical Evidence: A Likelihood Paradigm Chapman & Hall London
Royall, R.: 2000, ‘On the Probability of Observing Misleading Statistical Evidence (with Discussion)’, Journal of American Statistical Association 95(482).
R. Royall (2001) ‘The Likelihood Paradigm for Statistical Evidence’ M. Taper (Eds) et al. The Nature ofScientific Evidence University of Chicago Press Chicago 760–780
L. Savage (Eds) (1962) The Foundations of Statistical Inference: A Discussion Methuen London
M. Schulzer (1994) ArticleTitle‘Diagnostic Tests: A Statistical Review’ Muscle and Nerve, July 17 IssueID7 815–819
T. Seidenfeld (1985) ‘Comment on Good’s Philosophy’ Bernard Groot ParticleDe Lindley Smith (Eds) Bayesian Statistics NumberInSeriesVol II North-Holland Publishing Company Amsterdam 264–266
G. Shafer V. Vovk (2001) Probability and Finance John Wiley New York
E. Sober (1993) ArticleTitle‘Epistemology for Empiricists’ Midwest Studies in Philosophy XVIII 39–61
A. Spanos (1999) Probability Theory and Statistical Inference Cambridge University Press Cambridge
K. Staley (2001) ArticleTitle‘What Experiment Did We Just Do? Counterfactual Error Statistics and Uncertainties About the Reference Class’ Philosophy of Science 69 279–299
D. Steel (2003) ArticleTitle‘A Bayesian Way to Make Stopping Rules Matter’ Erkenntnis 58 213–222 Occurrence Handle10.1023/A:1022034427230
B. Fraassen Particlevan (1980) The Scientific Image Oxford University Press London
G. Wheeler (2000) ArticleTitle‘Error Statistics and Duhem’s Problem’ Philosophy of Science 67 410–420 Occurrence Handle10.1086/392788
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bandyopadhyay, P.S., Brittan, G.G. Acceptibility, Evidence, and Severity. Synthese 148, 259–293 (2006). https://doi.org/10.1007/s11229-004-6222-6
Issue Date:
DOI: https://doi.org/10.1007/s11229-004-6222-6