Abstract
The decomposition of the Kullback-Leibler risk of the maximum likelihood estimator (MLE) is discussed in relation to the Stein estimator and the conditional MLE. A notable correspondence between the decomposition in terms of the Stein estimator and that in terms of the conditional MLE is observed. This decomposition reflects that of the expected log-likelihood ratio. Accordingly, it is concluded that these modified estimators reduce the risk by reducing the expected log-likelihood ratio. The empirical Bayes method is discussed from this point of view.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amari, S-I. (1985).Differential-Geometrical Methods in Statistics, Springer, Berlin.
Andersen, E. B. (1970). Asymptotic properties of conditional maximum-likelihood estimators,J. Roy. Statist. Soc. Ser. B,32, 283–301.
Barndorff-Nielsen, O. (1978).Information and Exponential Families in Statistical Theory, Wiley, New York.
Berger, J. (1980). Improving on inadmissible estimators in continuous exponential families with applications to simultaneous estimation of gamma scale parameters,Ann. Statist.,8, 545–571.
Blaesild, P. and Jensen, J. L. (1985). Saddlepoint formulas for reproductive exponential models,Scand. J. Statist.,12, 193–202.
Brandwein, A. C. and Strawderman, W. (1990). Stein estimation: The spherically symmetric case,Statist. Sci.,5, 356–369.
Brown, L. D. (1979). A heuristic method for determining admissibility of estimators—with applications,Ann. Statist.,7, 960–994.
Cassella, G. (1985). An introduction to empirical Bayes data analysis,Amer. Statist.,39, 83–87.
Cox, D. R. and Reid, N. (1987). Parameter orthogonality and approximate conditional inference (with discussion),J. Roy. Statist. Soc. Ser. B,49, 1–39.
Dey, D. K., Ghosh, M. and Srinivasan, M. (1987). Simultaneous estimation of parameters under entropy loss,J. Statist. Plann. Inference,15, 347–363.
Efron, B. and Morris, C. (1973). Stein's estimation rules and its competitors—An empirical Bayes approach,J. Amer. Statist. Assoc.,68, 117–130.
Fisher, R. A. (1935). The logic of inductive inference,J. Roy. Statist. Soc.,98, 39–54.
Godambe, V. P. (1984). On ancillarity and Fisher information in the presence of a nuisance parameter,Biometrika,71, 626–629.
James, W. and Stein, C. (1961). Estimation with quadratic loss,Proc. Fourth Berkley Symp. on Math. Statist. Prob., Vol. 1, 361–380, Univ. of California Press, Berkeley.
Jorgensen, B. (1987). Exponential dispersion model (with discussion),J. Roy. Statist. Soc. Ser. B,49, 127–162.
Kullback, S. (1959).Information Theory and Statistics, Wiley, New York.
Kullback, S. and Leibler, R. A. (1951). On information and sufficiency,Ann. Math. Statist.,22, 79–86.
Lindsay, B. (1982). Conditional score functions: Some optimality results,Biometrika,69, 503–512.
McCullagh, P. and Nelder, J. A. (1989).Generalized Linear Models, 2nd ed., Chapman and Hall, London.
Morris, C. N. (1982). Natural exponential families with quadratic variance functions,Ann. Statist.,10, 65–80.
Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications (with discussion),J. Amer. Statist. Assoc.,78, 47–65.
Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear model (with discussion),J. Roy. Statist. Soc. Ser. A,34, 370–384.
Neyman, J. and Scott, E. L. (1948). Consistent estimates based on partially consistent observations,Econometrica,16, 1–32.
Saville, D. J. and Wood, G. R. (1991).Statistical Methods: The Geometric Approach, Springer, New York.
Simon, G. (1973). Additivity of information in exponential family probability laws,J. Amer. Statist. Assoc.,68, 478–482.
Stein, C. M. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution,Proc. Third Berkeley Symp. on Math. Statist. Prob., Vol. 1, 197–206, Univ. of California Press, Berkeley.
Stein, C. M. (1962). Confidence sets for the mean of a multivariate normal distribution (with discussion),J. Roy. Statist. Soc. Ser. B,24, 265–296.
Yanagimoto, T. (1987). A notion of an obstructive residual likelihood,Ann. Inst. Statist. Math.,39, 247–261.
Yanagimoto, T. (1991). Estimating a model through the conditional MLE,Ann. Inst. Statist. Math.,43, 735–746.
Yanagimoto, T. and Anraku, K. (1989). Possible superiority of the conditional MLE over the unconditional MLE,Ann. Inst. Statist. Math.,41, 269–278.
Yanagimoto, T. and Yanagimoto, M. (1987). The use of the marginal likelihood for a diagnostic test for the goodness of fit of the simple regression model,Technometrics,29, 95–101.
Author information
Authors and Affiliations
About this article
Cite this article
Yanagimoto, T. The Kullback-Leibler risk of the Stein estimator and the conditional MLE. Ann Inst Stat Math 46, 29–41 (1994). https://doi.org/10.1007/BF00773590
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00773590