Skip to main content

A frequentist framework of inductive reasoning

Abstract

A betting game establishes a sense in which confidence measures, confidence distributions in the form of probability measures, are the only reliable inferential probability distributions. In addition, because confidence measures are Kolmogorov probability distributions, they are as coherent as Bayesian posterior distributions in their avoidance of sure loss under the usual Dutch-book betting game.

Although a confidence measure can be computed without any prior, previous knowledge can be incorporated into confidence-based reasoning by combining the confidence measure from the observed data with one or more independent confidence measures representing previous agent opinion. The representation of subjective knowledge in terms of confidence measures rather than more general priors preserves approximate frequentist validity and thus reliability in the first game.

This is a preview of subscription content, access via your institution.

References

  • Armendt, B. (1992). Dutch strategies for diachronic rules: when believers see the sure loss coming. In PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association 1992, pp. 217–229.

  • Barnard, G.A. (1987). R. A. Fisher: a true Bayesian? International Statistical Review, 55, 183–189.

    MathSciNet  MATH  Article  Google Scholar 

  • Barndorff-nielsen, O.E. and Cox, D.R. (1994). Inference and Asymptotics. CRC Press, London.

    MATH  Google Scholar 

  • Berger, J.O., Bernardo, J.M. and Sun, D. (2009). The formal definition of reference priors. Ann. Statist., 37, 905–938.

    MathSciNet  MATH  Article  Google Scholar 

  • Berger, J.O. (2004). The case for objective Bayesian analysis. Bayesian Anal., 1, 1–17.

    Google Scholar 

  • Bernardo, J.M. (1979). Reference posterior distributions for Bayesian inference. J. R. Stat. Soc. Ser. B, 41, 113–147.

    MathSciNet  MATH  Google Scholar 

  • Bernardo, J.M. (1997). Noninformative priors do not exist: a discussion. J. Statist. Plann. Inference, 65, 159–189.

    MathSciNet  Article  Google Scholar 

  • Bickel, D.R. (2006). Incorporating expert knowledge into frequentist results by combining subjective prior and objective posterior distributions: a generalization of confidence distribution combination. Technical Report, Pioneer Hi-Bred International, arXiv:math.ST/0602377v2.

  • Bickel, D.R. (2011). Estimating the null distribution to adjust observed confidence levels for genome-scale screening. Biometrics, 67, 363–370.

    MathSciNet  MATH  Article  Google Scholar 

  • Bickel, D.R. (2012a). Blending Bayesian and frequentist methods according to the precision of prior information with applications to hypothesis testing. Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/23124.

  • Bickel, D.R. (2012b). Coherent frequentism: a decision theory based on confidence sets. Comm. Statist. Theory Methods, 41, 1478–1496.

    MathSciNet  MATH  Article  Google Scholar 

  • Bickel, D.R. (2012c). Controlling the degree of caution in statistical inference with the Bayesian and frequentist approaches as opposite extremes. Electron. J. Stat., 6, 686–709.

    Article  Google Scholar 

  • Bickel, D.R. (2012d). Empirical Bayes interval estimates that are conditionally equal to unadjusted confidence intervals or to default prior credibility intervals. Stat. Appl. Genet. Mol. Biol., 11, art. 3.

    Google Scholar 

  • Bickel, D.R. (2012e). A prior-free framework of coherent inference and its derivation of simple shrinkage estimators. Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/23093.

  • Bickel, D.R. (2012f). The strength of statistical evidence for composite hypotheses: Inference to the best explanation. Statistica Sinica, 22, 1147–1198.

    MathSciNet  MATH  Google Scholar 

  • Brazzale, A.R., Davison, A.C. and Reid, N. (2007). Applied Asymptotics: Case Studies in Small-sample Statistics. Cambridge University Press, Cambridge.

    MATH  Book  Google Scholar 

  • Buehler, R.J. (1977). Conditional confidence statements and confidence estimators: Comment. J. Amer. Statist. Assoc., 72, 813–814.

    MathSciNet  Google Scholar 

  • Carnap, R. (1971). A basic system of inductive logic, part 1. Studies in Inductive Logic and Probability, Vol. 1. University of California Press, Berkeley, pp. 3–165.

    Google Scholar 

  • Chaloner, K. (1996). The elicitation of prior distributions. Bayesian Biostatistics. Marcel Dekker, New York.

    Google Scholar 

  • Clarke, B. (2007). Information optimality and Bayesian modelling. J. Econometrics, 138, 405–429.

    MathSciNet  Article  Google Scholar 

  • Cornfield, J. (1969). The Bayesian outlook and its application. Biometrics, 25, 617–657.

    MathSciNet  Article  Google Scholar 

  • Cox, D.R. (1958). Some problems connected with statistical inference. Annals of Mathematical Statistics, 29, 357–372.

    MathSciNet  MATH  Article  Google Scholar 

  • Craig, P.S., Goldstein, M., Seheult, A.H. and Smith, J.A. (1998). Constructing partial prior specifications for models of complex physical systems. The Statistician, 47, 37–53.

    Google Scholar 

  • Datta, G.S. and Mukerjee, R. (2004). Probability Matching Priors: Higher Order Asymptotics. Springer, New York.

    MATH  Book  Google Scholar 

  • De Finetti, B. (1970). Theory of Probability: a Critical Introductory Treatment, 1st Edition. John Wiley and Sons Ltd, New York.

    Google Scholar 

  • Dempster, A.P. (2008). The Dempster-Shafer calculus for statisticians. Internat. J. Approx. Reason., 48, 365–377.

    MathSciNet  MATH  Article  Google Scholar 

  • Edwards, A.W.F. (1992). Likelihood. Johns Hopkins Press, Baltimore.

    MATH  Google Scholar 

  • Efron, B. (1993). Bayes and likelihood calculations from confidence intervals. Biometrika, 80, 3–26.

    MathSciNet  MATH  Article  Google Scholar 

  • Efron, B. (1998). R. A. Fisher in the 21st century, invited paper presented at the 1996 R. A. Fisher lecture. Statist. Sci., 13, 95–114.

    MathSciNet  MATH  Article  Google Scholar 

  • Efron, B. and Tibshirani, R. (1998). The problem of regions. Ann. Statist. 26, 1687–1718.

    MathSciNet  MATH  Article  Google Scholar 

  • Fisher, R.A. (1960). Scientific thought and the refinement of human reasoning. J. Oper. Res. Soc. Japan, 3, 1–10.

    Google Scholar 

  • Fisher, R.A. (1973). Statistical Methods and Scientific Inference. Hafner Press, New York.

    MATH  Google Scholar 

  • Fraser, D. (1978). Inference and Linear Models. McGraw-Hill, New York.

    Google Scholar 

  • Fraser, D.A.S. (1968). The Structure of Inference. John Wiley, New York.

    MATH  Google Scholar 

  • Fraser, D.A.S. (1977). Confidence, posterior probability, and the buehler example. Ann. Statist., 5, 892–898.

    MathSciNet  MATH  Article  Google Scholar 

  • Fraser, D.A.S. (1991). Statistical inference: likelihood to significance. J. Amer. Statist. Assoc., 86, 258–265.

    MathSciNet  MATH  Article  Google Scholar 

  • Fraser, D.A.S. (2004). Ancillaries and conditional inference. Statist. Sci., 19, 333–351.

    MathSciNet  MATH  Article  Google Scholar 

  • Fraser, D.A.S. (2006). Did Lindley get the argument the wrong way around? Technical Report, Department of Statistics, University of Toronto.

  • Fraser, D.A.S. and Reid, N. (2002). Strong matching of frequentist and Bayesian parametric inference. J. Statist. Plann. Inference, 103, 263–285.

    MathSciNet  MATH  Article  Google Scholar 

  • Freedman, D.A. and Purves, R.A. (1969). Bayes’ method for bookies. Annals of Mathematical Statistics, 40, 1177–1186.

    MathSciNet  MATH  Article  Google Scholar 

  • Garthwaite, P.H., Kadane, J.B. and O’hagan, A. (2005). Statistical methods for eliciting probability distributions. J. Amer. Statist. Assoc., 100, 680–700.

    MathSciNet  MATH  Article  Google Scholar 

  • Gleser, L.J. (2002). [setting confidence intervals for bounded parameters]: Comment. Statist. Sci., 17, 161–163.

    Google Scholar 

  • Goldstein, M. (1997). Prior inferences for posterior judgements. In Structures and Norms in Science: Volume Two of the Tenth International Congress of Logic, Methodology and Philosophy of Science, Florence, August 1995 (M. L. D. Chiara, K. Doets, D. Mundici, & J. van Benthem Eds.). New York, Springer, pp. 55–71.

  • Goldstein, M. (2001). Avoiding foregone conclusions: Geometric and foundational analysis of paradoxes of finite additivity. J. Statist. Plann. Inference, 94, 73–87.

    MathSciNet  MATH  Article  Google Scholar 

  • Goldstein, M. (2006). Subjective Bayesian analysis: principles and practice. Bayesian Anal., 1, 403–420.

    MathSciNet  Article  Google Scholar 

  • Grundy, P.M. (1956). Fiducial distributions and prior distributions: an example in which the former cannot be associated with the latter. J. R. Stat. Soc. Ser. B, 18, 217–221.

    MathSciNet  MATH  Google Scholar 

  • Hacking, I. (1965). Logic of Statistical Inference. Cambridge University Press, Cambridge.

    MATH  Google Scholar 

  • Hacking, I. (2001). An introduction to probability and inductive logic. Cambridge University Press, Cambridge.

    MATH  Book  Google Scholar 

  • Hannig, J. (2009). On generalized fiducial inference. Statist. Sinica, 19, 491–544.

    MathSciNet  MATH  Google Scholar 

  • Hannig, J. and Xie, M. (2009). A note on Dempster-Shafer recombination of confidence distributions. Electron. J. Stat., 6, 1943–1966.

    Article  Google Scholar 

  • Heath, D. and Sudderth, W. (1978). On finitely additive priors, coherence, and extended admissibility. Ann. Statist., 6, 333–345.

    MathSciNet  MATH  Article  Google Scholar 

  • Heath, D. and Sudderth, W. (1989). Coherent inference from improper priors and from finitely additive priors. Ann. Statist., 17, 907–919.

    MathSciNet  MATH  Article  Google Scholar 

  • Helland, I.S. (2004). Statistical inference under symmetry. International Statistical Review, 72, 409–422.

    Article  Google Scholar 

  • Hurwicz, L. (1951). The generalized Bayes-minimax principle: a criterion for decision-making under uncertainty. Cowles Commission Discussion Paper 355.

  • Hwang, J.T., Casella, G., Robert, C., Wells, M.T. and Farrell, R.H. (1992). Estimation of accuracy in testing. Ann. Statist., 20, 490–509.

    MathSciNet  MATH  Article  Google Scholar 

  • Jaffray, J.-Y. (1989). Linear utility theory for belief functions. Oper. Res. Lett., 8, 107–112.

    MathSciNet  MATH  Article  Google Scholar 

  • Jeffrey, R. (1986). Probabilism and induction. Topoi, 5, 51–58.

    MathSciNet  Article  Google Scholar 

  • Kempthorne, O. (1976). Comment on E. T. Jaynes, ‘Confidence intervals vs Bayesian intervals’. Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science. D. Reidel, Dordrecht-Holland, Ch. Confidence intervals vs Bayesian intervals, pp. 220–228.

  • Kiefer, J. (1977). Conditional confidence statements and confidence estimators: rejoinder. J. Amer. Statist. Assoc., 72, 822–827.

    MathSciNet  Google Scholar 

  • Kohlas, J. and Monney, P.-A. (2008). An algebraic theory for statistical information based on the theory of hints. Internat. J. Approx. Reason., 48, 378–398.

    MathSciNet  MATH  Article  Google Scholar 

  • Kyburg, H.E. (2007). Probability and Inference. Texts in Philosophy 2. College Publications, London, Ch. Bayesian inference with evidential probability, pp. 281–296.

  • Kyburg, H.E. and Teng, C.M. (2001). Uncertain Inference. Cambridge University Press, Cambridge.

    MATH  Book  Google Scholar 

  • Lele, S.R. (2004). Elicit data, not prior: On using expert opinion in ecological studies. The Nature of Scientific Evidence: Statistical, Philosophical, and Empirical Considerations. University of Chicago Press, Chicago, pp. 410–436.

    Google Scholar 

  • Lindley, D.V. (1958). Fiducial distributions and Bayes’ theorem. J. R. Stat. Soc. Ser. B, 20, 102–107.

    MathSciNet  MATH  Google Scholar 

  • Liu, R. and Singh, K. (1997). Notions of limiting P values based on data depth and bootstrap. J. Amer. Statist. Assoc., 92, 266–277.

    MathSciNet  MATH  Google Scholar 

  • Maher, P. (1992). Diachronic rationality. Philos. Sci., 59, 120–141.

    MathSciNet  Article  Google Scholar 

  • Mccullagh, P. (2002). What is a statistical model? Ann. Statist., 30, 1225–1267.

    MathSciNet  MATH  Article  Google Scholar 

  • Paris, J.B. (1994). The Uncertain Reasoner’s Companion: A Mathematical Perspective. Cambridge University Press, New York.

    MATH  Google Scholar 

  • Polansky, A.M. (2007). Observed Confidence Levels: Theory and Application. Chapman and Hall, New York.

    Book  Google Scholar 

  • Robins, J. and Wasserman, L. (2000). Conditioning, likelihood, and coherence: A review of some foundational concepts. J. Amer. Statist. Assoc., 95, 1340–1346.

    MathSciNet  MATH  Article  Google Scholar 

  • Royall, R. (1997). Statistical Evidence: A Likelihood Paradigm. CRC Press, New York.

    MATH  Google Scholar 

  • Royall, R. (2000). On the probability of observing misleading statistical evidence. J. Amer. Statist. Assoc., 95, 760–768.

    MathSciNet  MATH  Article  Google Scholar 

  • Savage, L.J. (1954). The Foundations of Statistics. John Wiley and Sons, New York.

    MATH  Google Scholar 

  • Scheffe, H. (1977). A note on a reformulation of the s-method of multiple comparison. J. Amer. Statist. Assoc., 72, 143–146.

    MathSciNet  MATH  Google Scholar 

  • Schervish, M.J. (1995). Theory of Statistics. Springer-Verlag, New York.

    MATH  Book  Google Scholar 

  • Schweder, T., Hjort, N.L. (2002). Confidence and likelihood. Scand. J. Stat., 29, 309–332.

    MathSciNet  MATH  Article  Google Scholar 

  • Seidenfeld, T. (2007). Probability and Inference. Texts in Philosophy 2. College Publications, London, Ch. Forbidden fruit: When epistemological probability may not take a bite of the Bayesian apple, pp. 267–279.

  • Shafer, G. (2011). A betting interpretation for probabilities and Dempster-Shafer degrees of belief. Internat. J. Approx. Reason., 52, 127–136.

    MathSciNet  MATH  Article  Google Scholar 

  • Sharma, S.S. (1980). On hacking’s fiducial theory of inference. Canad. J. Statist., 8, 227–233.

    MathSciNet  MATH  Article  Google Scholar 

  • Singh, K., Xie, M. and Strawderman, W.E. (2005). Combining information from independent sources through confidence distributions. Ann. Statist., 33, 159–183.

    MathSciNet  MATH  Article  Google Scholar 

  • Sprott, D.A. (2000). Statistical Inference in Science. Springer, New York.

    MATH  Google Scholar 

  • Troffaes, M.C.M. (2007). Decision making under uncertainty using imprecise probabilities. Internat. J. Approx. Reason., 45, 17–29.

    MathSciNet  MATH  Article  Google Scholar 

  • Ville, J. (1939). Gauthier-Villars, Paris.

  • Vos, P. (2008). Boyles, R.A. (2008), “The role of likelihood in interval estimation,” The American Statistician, 62, 22–26: Comment by Vos and reply. Amer. Statist., 62, 274–275.

    Article  Google Scholar 

  • Wilkinson, G.N. (1977). On resolving the controversy in statistical inference (with discussion). J. R. Stat. Soc. Ser. B, 39, 119–171.

    MathSciNet  MATH  Google Scholar 

  • Williamson, J. (2009). Objective Bayesianism, Bayesian conditionalisation and voluntarism. Synthese, 178, 1–19.

    Google Scholar 

  • Zabell, S.L. (1992). R. A. Fisher and the fiducial argument. Statist. Sci., 7, 369–387.

    MathSciNet  MATH  Article  Google Scholar 

Download references

Acknowledgement.

I am grateful to the anonymous referee for comments that lead to several improvements in clarity and completeness, most notably the inclusion of Sections 3.2.2 and 5.2. Matthias Kohl kindly provided R code (R Development Core Team, 2004) used to compute the convolution of the double exponential distribution (“R-help” list message posted on 12/23/05). I thank Mark Cooper for helpful feedback and Jean Peccoud, Mark Whitsitt, Chris Martin, and Bob Merrill for their support of the seed of this paper (Bickel, 2006) at Pioneer Hi-Bred, International. Subsequent developments were partially supported by the Canada Foundation for Innovation and by the Ministry of Research and Innovation of Ontario.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to David R. Bickel.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bickel, D.R. A frequentist framework of inductive reasoning. Sankhya A 74, 141–169 (2012). https://doi.org/10.1007/s13171-012-0020-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13171-012-0020-x

Keywords and phrases.

  • Artificial intelligence
  • betting
  • coherence
  • confidence distribution
  • confidence posterior
  • expert system
  • foundations of statistics
  • inductive reasoning
  • interpretation of probability
  • machine learning
  • personal probability
  • prior elicitation
  • subjective probability

AMS (2000) subject classification.

  • Primary 62A01
  • Secondary 62C99