Skip to main content

Intrinsic losses

Theory and Decision volume 40, pages 191–214 (1996)Cite this article

Abstract

Since the choice of a particular loss function strongly influences the resulting inference, it seems necessary to rely on “intrinsic” losses when no information is available about the utility function of the decision-maker, rather than to call for classical losses like the squared error loss. Since this setting is quite similar to the derivation of noninformative priors in Bayesian analysis, we first recall the conditions of this derivation and deduce from these conditions some requirements on the intrinsic losses. It then appears that these loss functions should only depend on the sampling distribution and that they should be independent of the parameterization of the distribution. The resulting estimators are therefore transformation equivariant. We study the properties of two natural intrinsic losses, namely entropy and Hellinger losses, and show that they can be expressed in closed form for exponential families. Moreover, the entropy loss also provides analytic expressions of Bayes estimators under conjugate priors; the derivation of Bayes estimators associated with the Hellinger loss is more cumbersome, as shown in Poisson and Gamma cases, while leading to similar estimators.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Abramowitz, M. and Stegun, I.: 1964, Handbook of Mathematical Functions, Dover.

  • Berger, J.O.: 1985, Statistical Decision Theory and Bayesian Analysis (2nd edition). Springer-Verlag, New York.

    Google Scholar 

  • Berger, J.O. and Bernardo, J.M.: 1989, ‘Estimating a product of means Bayesian analysis with reference priors’, J. Amer. Statist. Assoc. 84, 200–207.

    Google Scholar 

  • Berger, J.O. and Wolpert, R.: 1988, The Likelihood Principle (2nd edition). IMS Lecture Notes, Monograph Series 6, Hay ward, California.

  • Bernardo, J.M. 1979, ‘Reference posterior distributions for Bayesian inference (with discussion)’, J. Royal Statist. Soc. (Ser. B) 41, 113–147.

    Google Scholar 

  • Bernardo, J.M. and Smith, A.F.M.: 1994, Bayesian Theory, J. Wiley, New York.

    Google Scholar 

  • Birgé, L.: 1980, Approximation dans les espaces métriques et théorie de l'estimation, Thèse d'Etat, Univ. de Paris VII

  • Birgé, L.: 1983, ‘Robust testing for independent non identically distributed variables and Markov chains’, Lecture Notes in Statistics 16, Springer-Verlag, New York.

    Google Scholar 

  • Brown, L.D.: 1986, Foundations of Exponential Families, IMS Lecture Notes, Monograph Series 9, Hay ward, California.

  • Brown, L.D.: 1980, ‘Examples of Berger's phenomenon in the estimation of independent normal means’, Ann. Statist. 9, 1289–1300.

    Google Scholar 

  • DeGroot, M.: 1970, Optimal Statistical Decisions, McGraw-Hill, New York.

    Google Scholar 

  • Fishburn, P.: 1988, Non-linear Preferences and Utility Theory, Harvester Wheatsheaf, Brighton, Sussex.

    Google Scholar 

  • Gauβ, C.F.: 1810, Méthode des moindres carrés. Mémoire sur la combination des observations, Trad. J. Bertrand. Mallet-Bachelier, Paris (1985).

    Google Scholar 

  • Gibbs, J.W.: 1876, Collected Works and Commentary, vol. II, A. Haas (Ed.) Yale University Press, New Haven (1936).

    Google Scholar 

  • Gouriéroux, C. and Monfort, A.: 1994, ‘Testing non-nested hypotheses’, Handbook of Econometrics, Vol. IV, R.F. Engle and D.L. McFadden (Eds.), Elsevier, Amsterdam.

    Google Scholar 

  • Gutiérrez-Peña, E.: 1992, ‘Expected logarithmic divergence for exponential families’, in Bayesian Statistics 4, J.O. Bernardo, A.P. Dawid and A.F.M. Smith (eds.). Oxford University Press, London, pp. 669–674.

    Google Scholar 

  • Huber, P.: 1964, ‘Robust estimation of a location parameter’, Ann. Math. Statist. 35, 73–101.

    Google Scholar 

  • Jaynes, E.t.: 1989, Papers on Probability, Statistics and Statistical Physics, R.D. Rosenkrantz (Ed.), Kluwer Academic Publishers, Dordrecht.

    Google Scholar 

  • Jeffreys, H.: 1961, Theory of Probability (3rd edition), Oxford University Press, London.

    Google Scholar 

  • Keating, J.P., Mason, R.L. and Sen, P.K.: 1993, Pitman Measure of Closeness: Comparison of Statistical Estimators, SIAM, Philadelphia.

    Google Scholar 

  • Le Cam, L.: 1982, ‘On the risk of Bayes estimates’, in Statistical Decision Theory and Related Topics III, Vol. 2, J.L. Berger and S.S. Gupta (Eds.) Academic Press, New York.

    Google Scholar 

  • Le Cam, L.: 1986, Asymptotic Methods in Statistical Decision Theory, Springer-Verlag, New York.

    Google Scholar 

  • Le Cam, L. and Yang, G.L.: 1990, Asymptotics in Statistics, Springer-Verlag, New York.

    Google Scholar 

  • Lehmann, E. Lehmann, E.L.: 1993, Theory of Point Estimation J. Wiley, New York.

    Google Scholar 

  • Lehmann, E.L. and Casella, G.: 1995, Theory of Point Estimation (revised edition). Wadsworth, Pacific Grove, California.

    Google Scholar 

  • Lindley, D.: 1985, Making Decisions, Wiley, New York.

    Google Scholar 

  • McCulloch, R. and Rossi, P.E.: 1992, ‘Bayes factors for nonlinear hypotheses and likelihood distributions’, Biometrika 79, 663–676.

    Google Scholar 

  • Olver, F.W.J.: 1974, Asymptotics and Special Functions, Academic-Press, New York.

    Google Scholar 

  • Pitman, E.: 1937, ‘The closest estimates of statistical parameters’, Proc. Cambridge Phil. Soc. 33, 212–222.

    Google Scholar 

  • Rao, C.R.: 1981, ‘Some comments on the minimum mean square error as a criterion of estimation’, in Statistics and Related Topics, M. Csörgo, D. Dawson, J.N.K. Rao, and A. Saleh (Eds.), pp. 123–143.

  • Robert, C.P.: 1994, The Bayesian Choice, Springer-Verlag, New York.

    Google Scholar 

  • Robert, C.P., Hwang, J.T.G. and Strawderman, W.E.: 1993, ‘Is Pitman nearness a reasonable criterion? (with discussion)’, J. American Statistical Assoc. 88(1), 57–76.

    Google Scholar 

  • Rubin, H.: 1987, ‘A weak system of axioms for “rational” behavior and the nonseparability of utility from prior’, Statist. Decisions 5, 47–58.

    Google Scholar 

  • Rukhin, A.: 1978, ‘Universal Bayes estimators’, Ann. Statist. 6, 345–351.

    Google Scholar 

  • Shannon, C.: 1948, ‘A mathematical theory of communication’, Bell System Tech. J. 27, 379–423 and 623–656.

    Google Scholar 

  • Stigler, S.: 1986, The History of Statistics Belknap, Harvard.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INSEE, Centre de Recherche en Economie et Statistique, Timbre J320-3 Av. P. Larousse, 92245, Malakoff cedex, France

    Christian P. Robert

Authors
  1. Christian P. Robert
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Robert, C.P. Intrinsic losses. Theor Decis 40, 191–214 (1996). https://doi.org/10.1007/BF00133173

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00133173

AMS Subject Classification (1990)

  • 62J99
  • 62F15
  • 62C10
  • 62C15

Key words

  • Utility theory
  • non-informative prior
  • distributional distance
  • entropy
  • Hellinger distance
  • conjugate prior
  • Fisher information
  • exponential families
  • bayes estimator