, Volume 41, Issue 1, pp 109–119 | Cite as

An information theoretic argument for the validity of the exponential model

  • Konstantinos Zografos
  • Kosmas Ferentinos


Based on the Cramér-Rao inequality (in the multiparameter case) the lower bound of Fisher information matrix is achieved if and only if the underlying distribution is ther-parameter exponential family. This family and the lower bound of Fisher information matrix are characterized when some constraints in the form of expected values of some statistics are available. If we combine the previous results we can find the class of parametric functions and the corresponding UMVU estimators via Cramér-Rao inequality.

Key Words and Phrases

Fisher information Cramér-Rao inequality UMVU estimators exponential family 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bartlett MS (1982) The “ideal” estimation equation. In: Essays in Statistical Science (papers in honour of Moran PA). J Appl Probab Special 19A:187–200Google Scholar
  2. Bartlett MS (1983) Examples of minimum variance estimation. Aust J Stat 25:174–181Google Scholar
  3. Bellman R (1970) Introduction to matrix analysis. McGraw-Hill, New YorkGoogle Scholar
  4. Brown LD (1986) Fundamentals of statistical exponential families with applications in statistical decision theory. Institute of mathematical statistics. Lecture notes-Monograph series 9Google Scholar
  5. Campbell LL (1970) Equivalence of Gauss's principle and minimum discrimination information estimation of probabilities. Ann Math Stat 41:1011–1015Google Scholar
  6. Drygas H (1987) On the multivariate Cramér-Rao inequality. Stat Hefte 28:69–71Google Scholar
  7. Fabian V, Hannan J (1977) On the Cramér-Rao inequality. Ann Stat 5:197–205Google Scholar
  8. Ferentinos K, Papaioannou T (1981) New parametric measures of information. Information and control 51:193–208Google Scholar
  9. Ferentinos K (1984) Note on the use of the Cramér-Rao inequality for finding uniformly minimum variance unbiased estimators. Metron 42:127–131Google Scholar
  10. Frieden BR (1988) Applications of optics and wave mechanics of the criterion of maximum Cramér-Rao bound. J Mod Opt 35:1297–1316Google Scholar
  11. Holevo AS (1982) Probabilistic and statistical aspects of quantum theory. North-HollandGoogle Scholar
  12. Huber PJ (1981) Robust statistics. John Wiley, New YorkGoogle Scholar
  13. Jarrett RG (1984) Bounds and expansions for Fisher information when the moments are known. Biometrika 71:101–113Google Scholar
  14. Jaynes ET (1957) Information theory and statistical mechanics. Physical Review 106:620–630Google Scholar
  15. Jonson RA, Ladala J, Liu ST (1979) Differential relations in the original parameters which determine first two moments of the multiparameter exponential family. Ann Stat 7:232–235Google Scholar
  16. Joshi VM (1976) On the attainment of the Cramér-Rao lower bound. Ann Stat 4:998–1002Google Scholar
  17. Kagan AM (1985) Information property of exponential families. Theory Probab. Appl 30:831–835Google Scholar
  18. Kagan AM (1989) A multivariate analog of the Cramér theorem on components of the Gaussian distribution. In: Stability problems for stochastic models, Lecture notes in mathematics 1412:68–77Google Scholar
  19. Kagan AM, Linnik Yu V, Rao CR (1973) Characterization problems of mathematical statistics. John Wiley, New YorkGoogle Scholar
  20. Kapur JN (1989) Maximum entropy models in science and engineering. Wiley eastern limitedGoogle Scholar
  21. Koshevnik Yu A, Levit B Ya (1976) On a non parametric analogue of the information matrix. Theory Probab Appl 21: 738–753Google Scholar
  22. Kullback S (1959) Information theory and statistics. John Wiley, New YorkGoogle Scholar
  23. Kullback S (1985) Fisher information. In: Encyclopedia of statistical sciences, Kotz S, Johnson NL (Eds) 3:115–118Google Scholar
  24. Lehmann E (1983) Theory of point estimation. John Wiley, New YorkGoogle Scholar
  25. Levit B Ya (1975) On the efficiency of a class of non parametric estimates. Theory Probab Appl 20:723–740Google Scholar
  26. Li G, Cheng P (1983) Some results on Cramér-Rao type bounds. Kexue Tongbao (English Ed.) 28:5–9Google Scholar
  27. Mardia KV, Kent JT, Bibby JM (1979) Multivariate analysis. Academic PressGoogle Scholar
  28. Mead LR, Papanicolaou N (1984) Maximum entropy in the problem of moments. J Math Phys 25:2404–2417Google Scholar
  29. Papathanasiou V (1990) Characterizations of the multidimensional exponential families by Cacoullos type inequalities. J Multivariate Anal 35:102–107Google Scholar
  30. Shore JE, Johnson RW (1980) Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross entropy. IEEE Trans. Inf Theory IT-26:26–37Google Scholar
  31. Wijsman RA (1973) On the attainment of the Cramér-Rao lower bound. Ann Stat 1:538–542Google Scholar
  32. Zacks S (1971) The theory of statistical inference. John Wiley, New YorkGoogle Scholar

Copyright information

© Physica-Verlag 1994

Authors and Affiliations

  • Konstantinos Zografos
    • 1
  • Kosmas Ferentinos
    • 1
  1. 1.Section of Probability-Statistics and Operational ResearchUniversity of Ioannina, Department of MathematicsIoanninaGreece

Personalised recommendations