Skip to main content
SpringerLink
Log in

Eliciting vague but proper maximal entropy priors in Bayesian experiments

  • Regular Article
  • Published:
Statistical Papers Aims and scope Submit manuscript

Abstract

Priors elicited according to maximal entropy rules have been used for years in objective and subjective Bayesian analysis. However, when the prior knowledge remains fuzzy or dubious, they often suffer from impropriety which can make them uncomfortable to use. In this article we suggest the formal elicitation of an encompassing family for the standard maximal entropy (ME) priors and the maximal data information (MDI) priors, which can lead to obtain proper families. An interpretation is given in the objective framework of channel coding. In a subjective framework, the performance of the method is shown in a reliability context when flat but proper priors are elicited for the Weibull lifetime distributions. Such priors appear as practical tools for sensitivity studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andrieu C, Doucet A, Fitzgerald WJ, Pérez JM (2001) Bayesian computational approaches to model selection. In: Smith RL, Young PC, Walkden A(eds) Nonlinear and non-Gaussian signal processing. Cambridge University Press, London

    Google Scholar 

  • Bacha M, Celeux G, Idée E, Lannoy A, Vasseur D (1998) Estimation de modèles de durées de vie fortement censurées. Eyrolles

  • Beirlant J, Dudewicz E, Gyorfi L, van der Meulen E (1997) Nonparametric entropy estimation: an overview. Int J Math Stat Sci 6: 17–39

    MATH  MathSciNet  Google Scholar 

  • Berger JO (1985) Statistical decision theory and Bayesian analysis. Springer, New York

    MATH  Google Scholar 

  • Berger J (2006) The Case for objective bayesian analysis. Bayesian Anal 1: 385–402

    Article  MathSciNet  Google Scholar 

  • Berger JO, Bernardo JM (1992) On the development of reference priors (with discussion). In: Bernardo JM, Berger JO, Dawid AP, Smith AFM(eds) Bayesian statistics, vol 4. Oxford University Press, USA, pp 35–60

    Google Scholar 

  • Berger JO, Sun D (1993) Bayesian analysis for the poly-Weibull distribution. J Am Stat Assoc 88: 1412–1418

    Article  MATH  MathSciNet  Google Scholar 

  • Berger JO, Sun D (1994) Bayesian sequential reliability for Weibull and related distributions. Ann Inst Stat Math 46: 221–249

    Article  MATH  MathSciNet  Google Scholar 

  • Bernardo JM (1997) Noninformative priors do not exist: a discussion. J Stat Plan Inference 65: 159–189

    Article  MathSciNet  Google Scholar 

  • Billy F, Bousquet N, Celeux G (2006) Modelling and eliciting expert knowledge with fictitious data. In: Proceedings of the workshop on the use of expert judgement for decision-making, CEA Cadarache

  • Bousquet N (2006a) A Bayesian analysis of industrial lifetime data with Weibull distributions, HAL-INRIA research report RR-6025

  • Bousquet N (2006b) Analyse bayésienne de la durée de vie de composants industriels (Elements of Bayesian analysis for the prediction of the lifetime of industrial components). Ph.D. thesis, Parix XI University

  • Bousquet N, Celeux G (2006) Bayesian agreement between prior and data. In: Proceedings of the ISBA congress, Benidorm, Spain

  • Celeux G, Marin J-M, Robert CP (2006) Iterated importance sampling in missing data problems. Comput Stat Data An 12: 3386–3404

    Article  MathSciNet  Google Scholar 

  • Clarke B, Barron AR (1994) Jeffreys’ prior is asymptotically least favorable under entropy risk. J Stat Plan Inference 41: 37–60

    Article  MATH  MathSciNet  Google Scholar 

  • Clarke B (2007) Information optimality and Bayesian modelling. J Econom 138: 405–449

    Article  Google Scholar 

  • Cooke R (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New York

    Google Scholar 

  • Cover TM, Thomas JA (1991) Elements of information theory. Wiley, New York

    Book  MATH  Google Scholar 

  • Craig PS, Goldstein M, Seheult AH, Smith JA (1998) Constructing partial prior specifications for models of complex physical systems. The Statistician 47: 37–53

    Google Scholar 

  • Forrester Y (2005) The quality of expert judgement: an interdisciplinary investigation. Ph.D. thesis, University of Maryland

  • Friedlander MP, Gupta MR (2006) On minimizing distortion and relative entropy. IEEE Trans Inform Theory 52: 238–245

    Article  MathSciNet  Google Scholar 

  • Dai Y-S, Xie M, Long Q, Ng S-H (2007) Uncertainty analysis in software reliability modeling by bayesian analysis with maximum-entropy principle. Trans Soft Eng 33: 781–795

    Article  Google Scholar 

  • Garthwaite PH, Kadane JB, O’Hagan A (2005) Statistical methods for eliciting probability distributions. J Am Stat Assoc 100: 680–701

    Article  MATH  MathSciNet  Google Scholar 

  • Gelfand AE, Mallick BK, Dey DK (1995) Modeling expert opinion arising as a partial probabilistic specification. J Am Stat Assoc 90: 598–604

    Article  MATH  MathSciNet  Google Scholar 

  • Goldstein M (2006) Subjective Bayesian analysis: principles and practice. Bayesian Anal 1: 403–420

    Article  MathSciNet  Google Scholar 

  • Goossens LHJ, Cooke RM (2006) Expert judgement—calibration and combination. In: Proceedings of the workshop on the use of expert judgement for decision-making, CEA Cadarache

  • Guida M, Calabria R, Pulcini G (1989) Bayes inference for a non-homogeneous Poisson process with power intensity law reliability. IEEE Trans Inform Theory 5: 603–609

    Google Scholar 

  • Hill SD, Spall JC (1994) Sensitivity of a Bayesian analysis to the prior distribution. IEEE Trans Inform Theory 24: 216–221

    MathSciNet  Google Scholar 

  • Jaynes ET (1957) Information theory and statistical mechanics. Phys Rev 106, 620–630, and 108, 171–190

    Google Scholar 

  • Jaynes ET (1982) On the rationale of maximum entropy methods. Proc IEEE 70: 939–952

    Article  Google Scholar 

  • Jaynes ET (1988) The Relation of bayesian and maximum entropy methods (510Kb). In: Erickson GJ, Smith CR(eds) Maximum-entropy and Bayesian methods in science and engineering, vol 1. Kluwer, Dordrecht, pp 25–29

    Google Scholar 

  • Jaynes ET (2003) Probability theory: the logic of science. Cambridge University Press, London

    MATH  Google Scholar 

  • Kadane JB, Wolfson J (1998) A experiences in elicitation. The Statistician 47: 3–19

    Google Scholar 

  • Kaminskiy MP, Krivtsov VV (2005) A simple procedure for Bayesian estimation of the Weibull distribution. IEEE Trans Inform Theory 54: 612–616

    Google Scholar 

  • Kass RE, Wasserman L (1996) The selection of prior distributions by formal rules. J Am Stat Assoc 91: 1343–1370

    Article  MATH  Google Scholar 

  • Lannoy A, Procaccia H (2003) L’utilisation du jugement d’expert en sûreté de fonctionnement. Tec & Doc

  • Lawless JF (2003) Statistical models and methods for lifetime data, 2nd edn. Wiley, London

    MATH  Google Scholar 

  • Le Besnerais G, Bercher J-F, Demoment G (1999) A new look at entropy for solving linear inverse problems. IEEE Trans Inform Theory 45: 1565–1578

    Article  MATH  MathSciNet  Google Scholar 

  • Linnemann JT (2000) Upper limits and priors. FNAL CL Worshop (with notes added in January 2003)

  • Luttrell SP (1985) The use of transinformation in the design of data sampling schemes for inverse problems. Inverse Probl 1: 199–218

    Article  MATH  MathSciNet  Google Scholar 

  • Miller DJ, Yan L (2000) Approximate maximum entropy joint feature inference consistent with arbitrary lower order probability constraints: application to statistical classification. Neural Comput 12: 2175–2208

    Article  Google Scholar 

  • Natarajan R, McCulloch CE (1998) Gibbs sampling with diffuse proper priors: a valid approach to data-driven inference?. J Comput Graph Stat 7: 267–277

    Article  Google Scholar 

  • Oakley JE, O’Hagan A (2007) Uncertainty in prior elicitations: a nonparametric approach. Biometrika 94: 427–441

    Article  MATH  MathSciNet  Google Scholar 

  • O’Hagan A (1998) Eliciting expert beliefs in substantial practical applications. The Statistician 47(1): 21–35

    MathSciNet  Google Scholar 

  • O’Hagan A (2006) Research in elicitation. In: Upadhyay SK, Singh U, Dey DK(eds) Bayesian statistics and its applications. Anamaya, New Delhi, pp 375–382

    Google Scholar 

  • O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T (2006) Uncertain judgements: eliciting expert probabilities. Wiley, Chichester

    Book  MATH  Google Scholar 

  • Press SJ (2003) Subjective and objective Bayesian statistics, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  • Robert CP (2001) The Bayesian choice. a decision-theoretic motivation, 2nd edn. Springer, Heidelberg

    Google Scholar 

  • Shannon CE (1948) A mathematical theory of communication. Bell Syst Technol J 27:379–423, 623–656

    Google Scholar 

  • Shulman N, Feder M (2004) The uniform distribution as a universal prior. IEEE Trans Inform Theory 50: 1356–1362

    Article  MathSciNet  Google Scholar 

  • Singpurwalla ND (2006) Reliability and risk: a Bayesian perspective. Wiley, London

    MATH  Google Scholar 

  • Skilling J (1989) Maximum entropy and Bayesian methods. Kluwer, Dordrecht

    MATH  Google Scholar 

  • Smith, CR, Erickson, G, Neudorfer, PO (eds) (1992) Maximum entropy and Bayesian methods (fundamental theories of physics). Kluwer Academic Publisher, Dordrecht

    Google Scholar 

  • Soofi ES (1992) Information theory and Bayesian statistics. In: Berry DA, Chaloner KM, Geweke JK(eds) Bayesian analysis in statistics and econometrics in honor of Arnold Zellner. Wiley, New York, pp 179–189

    Google Scholar 

  • Soofi ES (1994) Capturing the intangible concept of information. J Am Stat Assoc 89: 1243–1254

    Article  MATH  MathSciNet  Google Scholar 

  • Soofi ES (2000) Principal information theoretic approaches. J Am Stat Assoc 95: 1349–1353

    Article  MATH  MathSciNet  Google Scholar 

  • Sun D (1997) A note on noninformative priors for Weibull distributions. J Stat Plan Inference 61: 319–338

    Article  MATH  Google Scholar 

  • van Noortwijk JM, Dekker R, Cooke RM, Mazzuchi TA (1992) Expert judgment in maintenance optimization. IEEE Trans Reliab 41: 427–431

    Article  Google Scholar 

  • Venegaz-Martinez F (2004) On information measures and prior distributions: a synthesis. Morfismos 8: 27–50

    Google Scholar 

  • Zellner A (1977) Maximal data information prior distributions. In: Aykae A, Brumat C (eds) New developments in the applications of Bayesian methods, Amsterdam

  • Zellner A, Sinha SK (1990) A note on the prior distributions of Weibull parameters. SCIMA 19: 5–13

    MATH  Google Scholar 

  • Zellner A (1991) Bayesian methods and entropy in economics and econometrics. In: Grandy WT Jr, Schick LH(eds) Maximum entropy and Bayesian methods. Kluwer, Boston, pp 17–31

    Google Scholar 

  • Zellner A (1996) Models, prior information and Bayesian analysis. J Econom 75: 51–68

    Article  MATH  MathSciNet  Google Scholar 

  • Zellner A (1997) Bayesian analysis in econometrics and statistics: the Zellner view and papers. Edward Elgar, Cheltenham, UK

  • Zellner A (2002) Information processing and Bayesian analysis. J Econom 107: 41–50

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, Laval University, 1045 av. de la Médecine, Quebec, QC, G1V 0A6, Canada

    Nicolas Bousquet

Authors
  1. Nicolas Bousquet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolas Bousquet.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bousquet, N. Eliciting vague but proper maximal entropy priors in Bayesian experiments. Stat Papers 51, 613–628 (2010). https://doi.org/10.1007/s00362-008-0149-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00362-008-0149-9

Keywords

Mathematics Subject Classification (2000)

Search

Navigation