Abstract
For estimating an unknown parameter θ, the likelihood principle yields the maximum likelihood estimator. It is often favoured especially by the applied statistician, for its good properties in the large sample case. In this paper, a large deviation expansion for the distribution of the maximum likelihood estimator is obtained. The asymptotic expansion provides a useful tool to approximate the tail probability of the maximum likelihood estimator and to make statistical inference. Theoretical and numerical examples are given. Numerical results show that the large deviation approximation performs much better than the classical normal approximation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bahadur, R. R. (1971).Some limit theorems in statistics, Regional Conference Series in Applied Mathematics, Society for Industrial and Applied Mathematics, Philadelphia.
Bahadur, R. R. and Rao, R. R. (1960). On deviations of the sample means,Ann. Math. Statist.,31, 1015–1027.
Bai, Z. C. and Fu, J. C. (1986). Likelihood principle and maximum likelihood estimator of location parameter for Cauchy distribution,Canad. J. Statist.,15, 137–146.
Barndorff-Nielsen, O. and Cox, D. R. (1979). Edgeworth and saddlepoint approximations with statistical applications,J. Roy. Statist. Soc. Ser. B,41, 279–312.
Cheng, P. and Fu, J. C. (1986). On a fundamental optimality of the maximum likelihood estimator,Statist. Probab. Lett.,4, 173–178.
Chernoff, H. (1952). A measure of asymptotic efficiency for tests of an hypothesis based on the sum of observations,Ann. Math. Statist.,23, 493–502.
Chung, K. L. (1974).A Course of Probability Theory, Academic Press, New York.
Cramer, H. (1937). Random variables and probability distributions,Cambridge Tracts in Math.,36.
Daniels, H. E. (1954). Saddlepoint approximations in statistics,Math. Math. Statist.,25, 631–650.
Daniels, H. E. (1987). Tail probability approximations in statistics,Internat. Statist. Rev.,55, 37–48.
Efron, B. (1975). Defining the curvature of a statistical problem,Ann. Statist.,3, 1189–1242.
Field, C. A. (1982). Small sample asymptotic expansion for multivariateM-estimators,Ann. Statist.,10, 672–689.
Fu, J. C. (1973). On a theorem of Bahadur on the rate of convergence of point estimators,Ann. Statist.,1, 741–749.
Fu, J. C. (1975). The rate of convergence of consistent point estimators,Ann. Statist.,3, 234–240.
Fu, J. C. (1982). Large sample point estimator: a large deviation theory approach,Ann. Statist.,10, 762–771.
Fu, J. C. (1985). On exponential rates of likelihood ratio estimators for location parameters,Statist. Probab. Lett.,3, 101–106.
Gradshteyn, I. S. and Ryzhik, I. M. (1965).Tables of Integrals, Series and Products, Academic Press, New York.
Hougaard, P. (1985). Saddlepoint approximation for curved exponential families,Statist. Probab. Lett.,3, 161–166.
Kester, A. and Kallenberg, W. C. M. (1986). Large deviations of estimators,Ann. Statist.,14, 648–664.
Kraft, C. H. and LeCam, L. M. (1956). A remark on the roots of maximum likelihood equation,Ann. Math. Statist.,27, 1174–1177.
Lugannani, R. and Rice, S. O. (1980). Saddlepoint approximation for the distribution of the sum of independent random variables,Adv. in Appl. Probab.,12, 475–490.
Olver, F. W. J. (1974).Asymptotic and Special Functions, Academic Press, New York.
Petrov, V. V. (1975).Sums of Independent Random Variables, Springer, Berlin.
Reeds, J. A. (1985). Asymptotic number of roots of Cauchy location likelihood equations,Ann. Statist.,13, 775–784.
Rubin, H. and Rukhin, A. L. (1983). Convergence rates of large deviation probabilities for point estimators,Statist. Probab. Lett.,1, 197–202.
Sievers, G. (1978). Estimates of location: a large deviation comparison,Ann. Statist.,6, 610–618.
Weiss, L. and Wolfowitz, J. (1966). Generalized maximum likelihood estimators,Teoriya Vyerogatnostey,11, 68–93.
Weiss, L. and Wolfowitz, J. (1967). Maximum probability estimators,Ann. Inst. Statist. Math.,19, 193–200.
Weiss, L. and Wolfowitz, J. (1970). Maximum probability estimators and asymptotic sufficiency,Ann. Inst. Statist. Math.,22, 225–244.
Author information
Authors and Affiliations
Additional information
This work is supported in part by the Natural Science and Engineering Research Council of Canada under grant NSERC A-9216.
This author is also partially supported by the National Science Foundation of China.
About this article
Cite this article
Fu, J.C., Li, G. & Zhao, D.L.C. On large deviation expansion of distribution of maximum likelihood estimator and its application in large sample estimation. Ann Inst Stat Math 45, 477–498 (1993). https://doi.org/10.1007/BF00773350
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00773350