Abstract
Let \({\mathcal{P}}\) be a nonparametric probability model consisting of smooth probability densities and let \({\hat{p}_{n}}\) be the corresponding maximum likelihood estimator based on n independent observations each distributed according to the law \({\mathbb{P}}\) . With \(\hat{\mathbb{P}}_{n}\) denoting the measure induced by the density \({\hat{p}_{n}}\) , define the stochastic process \({\hat{\nu}}_{n}: f\longmapsto \sqrt{n} \int fd({\hat{\mathbb{P}}}_{n} -\mathbb{P})\) where f ranges over some function class \({\mathcal{F}}\) . We give a general condition for Donsker classes \({\mathcal{F}}\) implying that the stochastic process \(\hat{\nu}_{n}\) is asymptotically equivalent to the empirical process in the space \({\ell ^{\infty }(\mathcal{F})}\) of bounded functions on \({ \mathcal{F}}\) . This implies in particular that \(\hat{\nu}_{n}\) converges in law in \({\ell ^{\infty }(\mathcal{F})}\) to a mean zero Gaussian process. We verify the general condition for a large family of Donsker classes \({\mathcal{ F}}\) . We give a number of applications: convergence of the probability measure \({\hat{\mathbb{P}}_{n}}\) to \({\mathbb{P}}\) at rate \({\sqrt{n}}\) in certain metrics metrizing the topology of weak(-star) convergence; a unified treatment of convergence rates of the MLE in a continuous scale of Sobolev-norms; \({\sqrt{n}}\) -efficient estimation of nonlinear functionals defined on \({\mathcal{P}}\) ; limit theorems at rate \({\sqrt{n}}\) for the maximum likelihood estimator of the convolution product \({\mathbb{P\ast P}}\) .
References
Adams R.A., Fournier J.F. (2003) Sobolev spaces, 2nd edn. Academic, New York
Bickel J.P., Ritov Y. (1988) Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhya Ser. A 50, 381–393
Bickel J.P., Ritov Y. (2003) Nonparametric estimators which can be ‘plugged-in’. Ann. Stat. 31, 1033–1053
Birgé L., Massart P. (1993) Rates of convergence of minimum contrast estimators. Probab. Theory Relat. Fields 97, 113–150
Birgé L., Massart P. (1995) Estimation of integral functionals of a density. Ann. Stat. 23, 11–29
Dieudonné J. (1960) Foundations of Modern Analysis. Academic, New York
Donoho, D.L., Liu R.C.: Geometrizing rates of convergence II, III. Ann. Stat. 19, 633–667, 668–701 (1991)
Dudley R.M. (1999) Uniform Central Limit Theorems. Cambridge University Press, Cambridge
Dudley R.M. (2002) Real Analysis and Probability. Cambridge University Press, Cambridge
Dunford N., Schwartz J.T. (1966) Linear Operators. Part I: General Theory. Interscience, New York
Frees E.W. (1994) Estimating densities of functions of observations. J. Am. Stat. Assoc. 89, 517–525
Giné E. (1975) Invariant tests for uniformity on compact Riemannian manifolds based on Sobolev-norms. Ann. Stat. 3, 1243–1266
Giné, E., Mason, D.M.: On local U-statistic processes and the estimation of densities of functions of several variables. Ann. Stat. (in press) (2006)
Giné E., Zinn J. (1986) Empirical processes indexed by Lipschitz functions. Ann. Probab. 14, 1329–1338
Hall P., Marron J.S. (1987) Estimation of integrated squared density derivatives. Stat. Probab. Lett. 6, 109–115
Kiefer J., Wolfowitz J. (1976) Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrscheinlichkeitstheorie verw. Gebiete 34, 73–85
Laurent B. (1996) Efficient estimation of integral functionals of a density. Ann. Stat. 24, 659–681
Laurent B. (1997) Estimation of integral functionals of a density and its derivatives. Bernoulli 3, 181–211
Leeb H., Pötscher B.M. (2006) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results. Econom. Theory 22, 69–97
Lions J.L., Magenes E. (1972) Non-Homogeneous Boundary Value Problems and Applications I. Springer, Berlin Heidelberg New York
Nickl, R.: Empirical and Gaussian processes on Besov classes. In: Giné, E., Kolchinskii, V., Li, W., Zinn, J. (eds.) High Dimensional Probability IV, IMS Lecture Notes (in press) (2006a)
Nickl, R.: On convergence and convolutions of random signed measures (preprint) (2006b)
Nickl, R.: Uniform central limit theorems for density estimators (preprint) (2006c)
Nickl, R., Pötscher, B.M.: Bracketing metric entropy rates and empirical central limit theorems for function classes of Besov-and Sobolev-type. J. Theor. Probab. (in press) (2005)
Pötscher B.M., Prucha I.R. (1997) Dynamic Nonlinear Econometric. Models Asymptotic Theory. Springer, Berlin Heidelberg New York
Radulovic, D., Wegkamp, M.: Weak convergence of smoothed empirical processes. Beyond Donsker classes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High Dimensional Probability II, Progr. Probab. 47, pp. 89–105 Birkhäuser, Boston (2000)
Radulovic D., Wegkamp M. (2003) Necessary and sufficient conditions for weak convergence of smoothed empirical processes. Stat. Probab. Lett.61, 321–336
Rost, D.: Limit theorems for smoothed empirical processes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High dimensional probability II, Progr. Probab. 47, pp. 107–113 Birkhäuser, Boston (2000)
Rufibach, K., Dümbgen, L.: Maximum likelihood estimation of a log-concave density. Basic properties and consistency (preprint) (2004)
Schick A., Wefelmeyer W. (2004) Root n consistent density estimators for sums of independent random variables. J. Nonparametr. Stat. 16, 925–935
Schmeisser H.-J., Triebel H. (1987) Topics in Fourier Analysis and Function Spaces. Wiley, New York
Stone C.J. (1980) Optimal rates of convergence for nonparametric estimators. Ann. Stat. 8, 1348–1360
Strassen V., Dudley R.M. (1969) The central limit theorem and \(\varepsilon \)-entropy. Probability and information theory. Lect. Notes Math. 1247, 224–231
Triebel H. (1983) Theory of Function Spaces. Birkhäuser, Basel
van de Geer S. (1993) Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Stat. 21, 14–44
van de Geer S. (2000) Empirical Processes in M-estimation. Cambridge University Press, Cambridge
van der Vaart A.W. (1994) Weak convergence of smoothed empirical processes. Scand. J. Stat. 21, 501–504
van der Vaart A.W., Wellner J.A. (1996) Weak Convergence and Empirical Processes. Springer, Berlin Heidelberg New York
von Mises R. (1947) On the asymptotic distribution of differentiable statistical functions. Ann. Math. Stat. 20, 309–348
Wong W.H., Severini T.A. (1991) On maximum likelihood estimation in infinite dimensional parameter spaces. Ann. Stat. 19, 603–632
Wong W.H., Shen X. (1995) Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Stat. 23, 339–362
Yukich J.E. (1992) Weak convergence of smoothed empirical processes. Scand. J. Stat. 19, 271–279
Author information
Authors and Affiliations
Corresponding author
Additional information
An erratum to this article can be found at http://dx.doi.org/10.1007/s00440-007-0136-4
Rights and permissions
About this article
Cite this article
Nickl, R. Donsker-type theorems for nonparametric maximum likelihood estimators. Probab. Theory Relat. Fields 138, 411–449 (2007). https://doi.org/10.1007/s00440-006-0031-4
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00440-006-0031-4
Keywords
- Nonparametric maximum likelihood estimator
- Uniform central limit theorem
- Plug-in property
- Differentiable functionals
- Convolution products
Mathematical Subject Classification (2000)
- Primary 60F05
- Primary 62G07
- Secondary 62F12
- Secondary 46F05