Abstract
Priors elicited according to maximal entropy rules have been used for years in objective and subjective Bayesian analysis. However, when the prior knowledge remains fuzzy or dubious, they often suffer from impropriety which can make them uncomfortable to use. In this article we suggest the formal elicitation of an encompassing family for the standard maximal entropy (ME) priors and the maximal data information (MDI) priors, which can lead to obtain proper families. An interpretation is given in the objective framework of channel coding. In a subjective framework, the performance of the method is shown in a reliability context when flat but proper priors are elicited for the Weibull lifetime distributions. Such priors appear as practical tools for sensitivity studies.
Similar content being viewed by others
References
Andrieu C, Doucet A, Fitzgerald WJ, Pérez JM (2001) Bayesian computational approaches to model selection. In: Smith RL, Young PC, Walkden A(eds) Nonlinear and non-Gaussian signal processing. Cambridge University Press, London
Bacha M, Celeux G, Idée E, Lannoy A, Vasseur D (1998) Estimation de modèles de durées de vie fortement censurées. Eyrolles
Beirlant J, Dudewicz E, Gyorfi L, van der Meulen E (1997) Nonparametric entropy estimation: an overview. Int J Math Stat Sci 6: 17–39
Berger JO (1985) Statistical decision theory and Bayesian analysis. Springer, New York
Berger J (2006) The Case for objective bayesian analysis. Bayesian Anal 1: 385–402
Berger JO, Bernardo JM (1992) On the development of reference priors (with discussion). In: Bernardo JM, Berger JO, Dawid AP, Smith AFM(eds) Bayesian statistics, vol 4. Oxford University Press, USA, pp 35–60
Berger JO, Sun D (1993) Bayesian analysis for the poly-Weibull distribution. J Am Stat Assoc 88: 1412–1418
Berger JO, Sun D (1994) Bayesian sequential reliability for Weibull and related distributions. Ann Inst Stat Math 46: 221–249
Bernardo JM (1997) Noninformative priors do not exist: a discussion. J Stat Plan Inference 65: 159–189
Billy F, Bousquet N, Celeux G (2006) Modelling and eliciting expert knowledge with fictitious data. In: Proceedings of the workshop on the use of expert judgement for decision-making, CEA Cadarache
Bousquet N (2006a) A Bayesian analysis of industrial lifetime data with Weibull distributions, HAL-INRIA research report RR-6025
Bousquet N (2006b) Analyse bayésienne de la durée de vie de composants industriels (Elements of Bayesian analysis for the prediction of the lifetime of industrial components). Ph.D. thesis, Parix XI University
Bousquet N, Celeux G (2006) Bayesian agreement between prior and data. In: Proceedings of the ISBA congress, Benidorm, Spain
Celeux G, Marin J-M, Robert CP (2006) Iterated importance sampling in missing data problems. Comput Stat Data An 12: 3386–3404
Clarke B, Barron AR (1994) Jeffreys’ prior is asymptotically least favorable under entropy risk. J Stat Plan Inference 41: 37–60
Clarke B (2007) Information optimality and Bayesian modelling. J Econom 138: 405–449
Cooke R (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New York
Cover TM, Thomas JA (1991) Elements of information theory. Wiley, New York
Craig PS, Goldstein M, Seheult AH, Smith JA (1998) Constructing partial prior specifications for models of complex physical systems. The Statistician 47: 37–53
Forrester Y (2005) The quality of expert judgement: an interdisciplinary investigation. Ph.D. thesis, University of Maryland
Friedlander MP, Gupta MR (2006) On minimizing distortion and relative entropy. IEEE Trans Inform Theory 52: 238–245
Dai Y-S, Xie M, Long Q, Ng S-H (2007) Uncertainty analysis in software reliability modeling by bayesian analysis with maximum-entropy principle. Trans Soft Eng 33: 781–795
Garthwaite PH, Kadane JB, O’Hagan A (2005) Statistical methods for eliciting probability distributions. J Am Stat Assoc 100: 680–701
Gelfand AE, Mallick BK, Dey DK (1995) Modeling expert opinion arising as a partial probabilistic specification. J Am Stat Assoc 90: 598–604
Goldstein M (2006) Subjective Bayesian analysis: principles and practice. Bayesian Anal 1: 403–420
Goossens LHJ, Cooke RM (2006) Expert judgement—calibration and combination. In: Proceedings of the workshop on the use of expert judgement for decision-making, CEA Cadarache
Guida M, Calabria R, Pulcini G (1989) Bayes inference for a non-homogeneous Poisson process with power intensity law reliability. IEEE Trans Inform Theory 5: 603–609
Hill SD, Spall JC (1994) Sensitivity of a Bayesian analysis to the prior distribution. IEEE Trans Inform Theory 24: 216–221
Jaynes ET (1957) Information theory and statistical mechanics. Phys Rev 106, 620–630, and 108, 171–190
Jaynes ET (1982) On the rationale of maximum entropy methods. Proc IEEE 70: 939–952
Jaynes ET (1988) The Relation of bayesian and maximum entropy methods (510Kb). In: Erickson GJ, Smith CR(eds) Maximum-entropy and Bayesian methods in science and engineering, vol 1. Kluwer, Dordrecht, pp 25–29
Jaynes ET (2003) Probability theory: the logic of science. Cambridge University Press, London
Kadane JB, Wolfson J (1998) A experiences in elicitation. The Statistician 47: 3–19
Kaminskiy MP, Krivtsov VV (2005) A simple procedure for Bayesian estimation of the Weibull distribution. IEEE Trans Inform Theory 54: 612–616
Kass RE, Wasserman L (1996) The selection of prior distributions by formal rules. J Am Stat Assoc 91: 1343–1370
Lannoy A, Procaccia H (2003) L’utilisation du jugement d’expert en sûreté de fonctionnement. Tec & Doc
Lawless JF (2003) Statistical models and methods for lifetime data, 2nd edn. Wiley, London
Le Besnerais G, Bercher J-F, Demoment G (1999) A new look at entropy for solving linear inverse problems. IEEE Trans Inform Theory 45: 1565–1578
Linnemann JT (2000) Upper limits and priors. FNAL CL Worshop (with notes added in January 2003)
Luttrell SP (1985) The use of transinformation in the design of data sampling schemes for inverse problems. Inverse Probl 1: 199–218
Miller DJ, Yan L (2000) Approximate maximum entropy joint feature inference consistent with arbitrary lower order probability constraints: application to statistical classification. Neural Comput 12: 2175–2208
Natarajan R, McCulloch CE (1998) Gibbs sampling with diffuse proper priors: a valid approach to data-driven inference?. J Comput Graph Stat 7: 267–277
Oakley JE, O’Hagan A (2007) Uncertainty in prior elicitations: a nonparametric approach. Biometrika 94: 427–441
O’Hagan A (1998) Eliciting expert beliefs in substantial practical applications. The Statistician 47(1): 21–35
O’Hagan A (2006) Research in elicitation. In: Upadhyay SK, Singh U, Dey DK(eds) Bayesian statistics and its applications. Anamaya, New Delhi, pp 375–382
O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T (2006) Uncertain judgements: eliciting expert probabilities. Wiley, Chichester
Press SJ (2003) Subjective and objective Bayesian statistics, 2nd edn. Wiley, New York
Robert CP (2001) The Bayesian choice. a decision-theoretic motivation, 2nd edn. Springer, Heidelberg
Shannon CE (1948) A mathematical theory of communication. Bell Syst Technol J 27:379–423, 623–656
Shulman N, Feder M (2004) The uniform distribution as a universal prior. IEEE Trans Inform Theory 50: 1356–1362
Singpurwalla ND (2006) Reliability and risk: a Bayesian perspective. Wiley, London
Skilling J (1989) Maximum entropy and Bayesian methods. Kluwer, Dordrecht
Smith, CR, Erickson, G, Neudorfer, PO (eds) (1992) Maximum entropy and Bayesian methods (fundamental theories of physics). Kluwer Academic Publisher, Dordrecht
Soofi ES (1992) Information theory and Bayesian statistics. In: Berry DA, Chaloner KM, Geweke JK(eds) Bayesian analysis in statistics and econometrics in honor of Arnold Zellner. Wiley, New York, pp 179–189
Soofi ES (1994) Capturing the intangible concept of information. J Am Stat Assoc 89: 1243–1254
Soofi ES (2000) Principal information theoretic approaches. J Am Stat Assoc 95: 1349–1353
Sun D (1997) A note on noninformative priors for Weibull distributions. J Stat Plan Inference 61: 319–338
van Noortwijk JM, Dekker R, Cooke RM, Mazzuchi TA (1992) Expert judgment in maintenance optimization. IEEE Trans Reliab 41: 427–431
Venegaz-Martinez F (2004) On information measures and prior distributions: a synthesis. Morfismos 8: 27–50
Zellner A (1977) Maximal data information prior distributions. In: Aykae A, Brumat C (eds) New developments in the applications of Bayesian methods, Amsterdam
Zellner A, Sinha SK (1990) A note on the prior distributions of Weibull parameters. SCIMA 19: 5–13
Zellner A (1991) Bayesian methods and entropy in economics and econometrics. In: Grandy WT Jr, Schick LH(eds) Maximum entropy and Bayesian methods. Kluwer, Boston, pp 17–31
Zellner A (1996) Models, prior information and Bayesian analysis. J Econom 75: 51–68
Zellner A (1997) Bayesian analysis in econometrics and statistics: the Zellner view and papers. Edward Elgar, Cheltenham, UK
Zellner A (2002) Information processing and Bayesian analysis. J Econom 107: 41–50
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bousquet, N. Eliciting vague but proper maximal entropy priors in Bayesian experiments. Stat Papers 51, 613–628 (2010). https://doi.org/10.1007/s00362-008-0149-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00362-008-0149-9
Keywords
- Bayesian inference
- Expert opinion
- Kullback–Leibler distance
- Shannon’s entropy
- Noninformative priors
- Channel coding
- Sensitivity study
- Weibull