A betting game establishes a sense in which confidence measures, confidence distributions in the form of probability measures, are the only reliable inferential probability distributions. In addition, because confidence measures are Kolmogorov probability distributions, they are as coherent as Bayesian posterior distributions in their avoidance of sure loss under the usual Dutch-book betting game.
Although a confidence measure can be computed without any prior, previous knowledge can be incorporated into confidence-based reasoning by combining the confidence measure from the observed data with one or more independent confidence measures representing previous agent opinion. The representation of subjective knowledge in terms of confidence measures rather than more general priors preserves approximate frequentist validity and thus reliability in the first game.
Keywords and phrases.Artificial intelligence betting coherence confidence distribution confidence posterior expert system foundations of statistics inductive reasoning interpretation of probability machine learning personal probability prior elicitation subjective probability
AMS (2000) subject classification.Primary 62A01 Secondary 62C99
I am grateful to the anonymous referee for comments that lead to several improvements in clarity and completeness, most notably the inclusion of Sections 3.2.2 and 5.2. Matthias Kohl kindly provided R code (R Development Core Team, 2004) used to compute the convolution of the double exponential distribution (“R-help” list message posted on 12/23/05). I thank Mark Cooper for helpful feedback and Jean Peccoud, Mark Whitsitt, Chris Martin, and Bob Merrill for their support of the seed of this paper (Bickel, 2006) at Pioneer Hi-Bred, International. Subsequent developments were partially supported by the Canada Foundation for Innovation and by the Ministry of Research and Innovation of Ontario.
- Armendt, B. (1992). Dutch strategies for diachronic rules: when believers see the sure loss coming. In PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association 1992, pp. 217–229.Google Scholar
- Berger, J.O. (2004). The case for objective Bayesian analysis. Bayesian Anal., 1, 1–17.Google Scholar
- Bickel, D.R. (2006). Incorporating expert knowledge into frequentist results by combining subjective prior and objective posterior distributions: a generalization of confidence distribution combination. Technical Report, Pioneer Hi-Bred International, arXiv:math.ST/0602377v2.Google Scholar
- Bickel, D.R. (2012a). Blending Bayesian and frequentist methods according to the precision of prior information with applications to hypothesis testing. Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/23124.
- Bickel, D.R. (2012d). Empirical Bayes interval estimates that are conditionally equal to unadjusted confidence intervals or to default prior credibility intervals. Stat. Appl. Genet. Mol. Biol., 11, art. 3.Google Scholar
- Bickel, D.R. (2012e). A prior-free framework of coherent inference and its derivation of simple shrinkage estimators. Working Paper, University of Ottawa, deposited in uO Research at http://hdl.handle.net/10393/23093.
- Carnap, R. (1971). A basic system of inductive logic, part 1. Studies in Inductive Logic and Probability, Vol. 1. University of California Press, Berkeley, pp. 3–165.Google Scholar
- Chaloner, K. (1996). The elicitation of prior distributions. Bayesian Biostatistics. Marcel Dekker, New York.Google Scholar
- Craig, P.S., Goldstein, M., Seheult, A.H. and Smith, J.A. (1998). Constructing partial prior specifications for models of complex physical systems. The Statistician, 47, 37–53.Google Scholar
- De Finetti, B. (1970). Theory of Probability: a Critical Introductory Treatment, 1st Edition. John Wiley and Sons Ltd, New York.Google Scholar
- Fisher, R.A. (1960). Scientific thought and the refinement of human reasoning. J. Oper. Res. Soc. Japan, 3, 1–10.Google Scholar
- Fraser, D. (1978). Inference and Linear Models. McGraw-Hill, New York.Google Scholar
- Fraser, D.A.S. (2006). Did Lindley get the argument the wrong way around? Technical Report, Department of Statistics, University of Toronto.Google Scholar
- Gleser, L.J. (2002). [setting confidence intervals for bounded parameters]: Comment. Statist. Sci., 17, 161–163.Google Scholar
- Goldstein, M. (1997). Prior inferences for posterior judgements. In Structures and Norms in Science: Volume Two of the Tenth International Congress of Logic, Methodology and Philosophy of Science, Florence, August 1995 (M. L. D. Chiara, K. Doets, D. Mundici, & J. van Benthem Eds.). New York, Springer, pp. 55–71.Google Scholar
- Hurwicz, L. (1951). The generalized Bayes-minimax principle: a criterion for decision-making under uncertainty. Cowles Commission Discussion Paper 355.Google Scholar
- Kempthorne, O. (1976). Comment on E. T. Jaynes, ‘Confidence intervals vs Bayesian intervals’. Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science. D. Reidel, Dordrecht-Holland, Ch. Confidence intervals vs Bayesian intervals, pp. 220–228.Google Scholar
- Kyburg, H.E. (2007). Probability and Inference. Texts in Philosophy 2. College Publications, London, Ch. Bayesian inference with evidential probability, pp. 281–296.Google Scholar
- Lele, S.R. (2004). Elicit data, not prior: On using expert opinion in ecological studies. The Nature of Scientific Evidence: Statistical, Philosophical, and Empirical Considerations. University of Chicago Press, Chicago, pp. 410–436.Google Scholar
- Seidenfeld, T. (2007). Probability and Inference. Texts in Philosophy 2. College Publications, London, Ch. Forbidden fruit: When epistemological probability may not take a bite of the Bayesian apple, pp. 267–279.Google Scholar
- Ville, J. (1939). Gauthier-Villars, Paris.Google Scholar
- Williamson, J. (2009). Objective Bayesianism, Bayesian conditionalisation and voluntarism. Synthese, 178, 1–19.Google Scholar