Behaviormetrika

, Volume 44, Issue 1, pp 57–95 | Cite as

A family of the adjusted estimators maximizing the asymptotic predictive expected log-likelihood

Original Paper

Abstract

A family of the estimators adjusting the maximum likelihood estimator by a higher-order term maximizing the asymptotic predictive expected log-likelihood is introduced under possible model misspecification. The negative predictive expected log-likelihood is seen as the Kullback–Leibler distance plus a constant between the adjusted estimator and the population counterpart. The vector of coefficients in the correction term for the adjusted estimator is given explicitly by maximizing a quadratic form. Examples using typical distributions in statistics are shown.

Keywords

Expected log-likelihood Kullback–Leibler distance Mean square error Asymptotic expectation Shrinkage Asymptotic bias 

Notes

Acknowledgements

This work was partially supported by a Grant-in-Aid for Scientific Research from the Japanese Ministry of Education, Culture, Sports, Science and Technology (JSPS KAKENHI, Grant No. 26330031).

References

  1. Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Petrov BN, Csáki F (eds) Proceedings of the 2nd international symposium on information theory. Académiai Kiado, Budapest, pp 267–281Google Scholar
  2. Bjornstad JF (1990) Predictive likelihood: a review. Stat Sci 5:242–265MathSciNetCrossRefMATHGoogle Scholar
  3. DeGroot MH, Schervish MJ (2002) Probability and statistics, 3rd edn. Addison-Wesley, BostonGoogle Scholar
  4. Fisher RA (1956) Statistical methods and scientific inference. Oliver and Boyd, EdinburghMATHGoogle Scholar
  5. Giles DEA, Rayner AC (1979) The mean squared errors of the maximum likelihood and natural-conjugate Bayes regression estimators. J Econometr 11:319–334MathSciNetCrossRefMATHGoogle Scholar
  6. Gruber MHJ (1998) Improving efficiency by shrinkage: The James-Stein and ridge regression estimators. Marcel Dekker, New YorkMATHGoogle Scholar
  7. Hinkley D (1979) Predictive likelihood. Ann Stat 7:718–728MathSciNetCrossRefMATHGoogle Scholar
  8. Konishi S, Kitagawa G (1996) Generalized information criteria in model selection. Biometrika 83:875–890MathSciNetCrossRefMATHGoogle Scholar
  9. Konishi S, Kitagawa G (2003) Asymptotic theory for information criteria in model selection—functional approach. J Stat Plan Inference 114:45–61MathSciNetCrossRefMATHGoogle Scholar
  10. Konishi S, Kitagawa G (2008) Information criteria and statistical modeling. Springer, New-YorkCrossRefMATHGoogle Scholar
  11. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22:79–86MathSciNetCrossRefMATHGoogle Scholar
  12. Lawless JF, Fredette M (2005) Frequentist prediction intervals and predictive distributions. Biometrika 92:529–542MathSciNetCrossRefMATHGoogle Scholar
  13. Lejeune M, Faulkenberry GD (1982) A simple predictive density function. J Am Stat Assoc 77:654–657MathSciNetCrossRefMATHGoogle Scholar
  14. Leonard T (1982) Comment (on Lejeune & Faulkenberry, 1982). J Am Stat Assoc 77:657–658Google Scholar
  15. Ogasawara H (2010) Asymptotic expansions for the pivots using log-likelihood derivatives with an application in item response theory. J Multivar Anal 101:2149–2167MathSciNetCrossRefMATHGoogle Scholar
  16. Ogasawara H (2013) Asymptotic cumulants of the estimator of the canonical parameter in the exponential family. J Stat Plan Inference 143:2142–2150MathSciNetCrossRefMATHGoogle Scholar
  17. Ogasawara H (2014a) Supplement to the paper “Asymptotic cumulants of the estimator of the canonical parameter in the exponential family”. Econ Rev (Otaru University of Commerce), 65 (2 & 3), 3–16. Permalink: http://hdl.handle.net/10252/5399. Accessed 20 Nov 2016
  18. Ogasawara H (2014b) Optimization of the Gaussian and Jeffreys power priors with emphasis on the canonical parameters in the exponential family. Behaviormetrika 41:195–223CrossRefGoogle Scholar
  19. Ogasawara H (2015) Bias adjustment minimizing the asymptotic mean square error. Commun Stat Theory Methods 44:3503–3522MathSciNetCrossRefMATHGoogle Scholar
  20. Ogasawara H (2016) Optimal information criteria minimizing their asymptotic mean square errors. Sankhyā B 78:152–182MathSciNetCrossRefMATHGoogle Scholar
  21. Ogasawara H (2017) Asymptotic cumulants of some information criteria. J Jpn Soc Comput Stat (to appear)Google Scholar
  22. Sakamoto Y, Ishiguro M, Kitagawa G (1986) Akaike information criterion statistics. Reidel, DordrechtMATHGoogle Scholar
  23. Stone M (1977) An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion. J R Stat Soc B 39:44–47MathSciNetMATHGoogle Scholar
  24. Takeuchi K (1976) Distributions of information statistics and criteria of the goodness of models. Math Sci 153:12–18 (in Japanese) Google Scholar
  25. Takezawa K (2012) A revision of AIC for normal error models. Open J Stat 2:309–312CrossRefGoogle Scholar
  26. Takezawa K (2014) Estimation of the exponential distribution from a viewpoint of prediction. In: Proceedings of 2014 Japanese Joint Statistical Meeting, 305. University of Tokyo, Tokyo, Japan (in Japanese)Google Scholar
  27. Takezawa K (2015) Estimation of the exponential distribution in the light of future data. Br J Math Comput Sci 5:128–132CrossRefGoogle Scholar

Copyright information

© The Behaviormetric Society 2016

Authors and Affiliations

  1. 1.Department of Information and Management ScienceOtaru University of CommerceOtaruJapan

Personalised recommendations