Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
  1. Home
  2. Probability Theory and Related Fields
  3. Article
Donsker-type theorems for nonparametric maximum likelihood estimators
Download PDF
Download PDF
  • Published: 24 October 2006

Donsker-type theorems for nonparametric maximum likelihood estimators

  • Richard Nickl1,2 

Probability Theory and Related Fields volume 138, pages 411–449 (2007)Cite this article

  • 291 Accesses

  • 23 Citations

  • Metrics details

An Erratum to this article was published on 10 January 2008

Abstract

Let \({\mathcal{P}}\) be a nonparametric probability model consisting of smooth probability densities and let \({\hat{p}_{n}}\) be the corresponding maximum likelihood estimator based on n independent observations each distributed according to the law \({\mathbb{P}}\) . With \(\hat{\mathbb{P}}_{n}\) denoting the measure induced by the density \({\hat{p}_{n}}\) , define the stochastic process \({\hat{\nu}}_{n}: f\longmapsto \sqrt{n} \int fd({\hat{\mathbb{P}}}_{n} -\mathbb{P})\) where f ranges over some function class \({\mathcal{F}}\) . We give a general condition for Donsker classes \({\mathcal{F}}\) implying that the stochastic process \(\hat{\nu}_{n}\) is asymptotically equivalent to the empirical process in the space \({\ell ^{\infty }(\mathcal{F})}\) of bounded functions on \({ \mathcal{F}}\) . This implies in particular that \(\hat{\nu}_{n}\) converges in law in \({\ell ^{\infty }(\mathcal{F})}\) to a mean zero Gaussian process. We verify the general condition for a large family of Donsker classes \({\mathcal{ F}}\) . We give a number of applications: convergence of the probability measure \({\hat{\mathbb{P}}_{n}}\) to \({\mathbb{P}}\) at rate \({\sqrt{n}}\) in certain metrics metrizing the topology of weak(-star) convergence; a unified treatment of convergence rates of the MLE in a continuous scale of Sobolev-norms; \({\sqrt{n}}\) -efficient estimation of nonlinear functionals defined on \({\mathcal{P}}\) ; limit theorems at rate \({\sqrt{n}}\) for the maximum likelihood estimator of the convolution product \({\mathbb{P\ast P}}\) .

Download to read the full article text

Working on a manuscript?

Avoid the common mistakes

References

  1. Adams R.A., Fournier J.F. (2003) Sobolev spaces, 2nd edn. Academic, New York

    MATH  Google Scholar 

  2. Bickel J.P., Ritov Y. (1988) Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhya Ser. A 50, 381–393

    MATH  MathSciNet  Google Scholar 

  3. Bickel J.P., Ritov Y. (2003) Nonparametric estimators which can be ‘plugged-in’. Ann. Stat. 31, 1033–1053

    Article  MATH  MathSciNet  Google Scholar 

  4. Birgé L., Massart P. (1993) Rates of convergence of minimum contrast estimators. Probab. Theory Relat. Fields 97, 113–150

    Article  MATH  Google Scholar 

  5. Birgé L., Massart P. (1995) Estimation of integral functionals of a density. Ann. Stat. 23, 11–29

    Article  MATH  Google Scholar 

  6. Dieudonné J. (1960) Foundations of Modern Analysis. Academic, New York

    MATH  Google Scholar 

  7. Donoho, D.L., Liu R.C.: Geometrizing rates of convergence II, III. Ann. Stat. 19, 633–667, 668–701 (1991)

    Google Scholar 

  8. Dudley R.M. (1999) Uniform Central Limit Theorems. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  9. Dudley R.M. (2002) Real Analysis and Probability. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  10. Dunford N., Schwartz J.T. (1966) Linear Operators. Part I: General Theory. Interscience, New York

    Google Scholar 

  11. Frees E.W. (1994) Estimating densities of functions of observations. J. Am. Stat. Assoc. 89, 517–525

    Article  MATH  MathSciNet  Google Scholar 

  12. Giné E. (1975) Invariant tests for uniformity on compact Riemannian manifolds based on Sobolev-norms. Ann. Stat. 3, 1243–1266

    Article  MATH  Google Scholar 

  13. Giné, E., Mason, D.M.: On local U-statistic processes and the estimation of densities of functions of several variables. Ann. Stat. (in press) (2006)

  14. Giné E., Zinn J. (1986) Empirical processes indexed by Lipschitz functions. Ann. Probab. 14, 1329–1338

    Article  MATH  MathSciNet  Google Scholar 

  15. Hall P., Marron J.S. (1987) Estimation of integrated squared density derivatives. Stat. Probab. Lett. 6, 109–115

    Article  MATH  MathSciNet  Google Scholar 

  16. Kiefer J., Wolfowitz J. (1976) Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrscheinlichkeitstheorie verw. Gebiete 34, 73–85

    Article  MATH  MathSciNet  Google Scholar 

  17. Laurent B. (1996) Efficient estimation of integral functionals of a density. Ann. Stat. 24, 659–681

    Article  MATH  MathSciNet  Google Scholar 

  18. Laurent B. (1997) Estimation of integral functionals of a density and its derivatives. Bernoulli 3, 181–211

    Article  MATH  MathSciNet  Google Scholar 

  19. Leeb H., Pötscher B.M. (2006) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results. Econom. Theory 22, 69–97

    Article  MATH  Google Scholar 

  20. Lions J.L., Magenes E. (1972) Non-Homogeneous Boundary Value Problems and Applications I. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  21. Nickl, R.: Empirical and Gaussian processes on Besov classes. In: Giné, E., Kolchinskii, V., Li, W., Zinn, J. (eds.) High Dimensional Probability IV, IMS Lecture Notes (in press) (2006a)

  22. Nickl, R.: On convergence and convolutions of random signed measures (preprint) (2006b)

  23. Nickl, R.: Uniform central limit theorems for density estimators (preprint) (2006c)

  24. Nickl, R., Pötscher, B.M.: Bracketing metric entropy rates and empirical central limit theorems for function classes of Besov-and Sobolev-type. J. Theor. Probab. (in press) (2005)

  25. Pötscher B.M., Prucha I.R. (1997) Dynamic Nonlinear Econometric. Models Asymptotic Theory. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  26. Radulovic, D., Wegkamp, M.: Weak convergence of smoothed empirical processes. Beyond Donsker classes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High Dimensional Probability II, Progr. Probab. 47, pp. 89–105 Birkhäuser, Boston (2000)

  27. Radulovic D., Wegkamp M. (2003) Necessary and sufficient conditions for weak convergence of smoothed empirical processes. Stat. Probab. Lett.61, 321–336

    Article  MATH  MathSciNet  Google Scholar 

  28. Rost, D.: Limit theorems for smoothed empirical processes. In: Giné, E., Mason, D.M., Wellner, J.A. (eds.) High dimensional probability II, Progr. Probab. 47, pp. 107–113 Birkhäuser, Boston (2000)

  29. Rufibach, K., Dümbgen, L.: Maximum likelihood estimation of a log-concave density. Basic properties and consistency (preprint) (2004)

  30. Schick A., Wefelmeyer W. (2004) Root n consistent density estimators for sums of independent random variables. J. Nonparametr. Stat. 16, 925–935

    Article  MATH  MathSciNet  Google Scholar 

  31. Schmeisser H.-J., Triebel H. (1987) Topics in Fourier Analysis and Function Spaces. Wiley, New York

    Google Scholar 

  32. Stone C.J. (1980) Optimal rates of convergence for nonparametric estimators. Ann. Stat. 8, 1348–1360

    Article  MATH  Google Scholar 

  33. Strassen V., Dudley R.M. (1969) The central limit theorem and \(\varepsilon \)-entropy. Probability and information theory. Lect. Notes Math. 1247, 224–231

    MathSciNet  Google Scholar 

  34. Triebel H. (1983) Theory of Function Spaces. Birkhäuser, Basel

    Google Scholar 

  35. van de Geer S. (1993) Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Stat. 21, 14–44

    Article  MathSciNet  Google Scholar 

  36. van de Geer S. (2000) Empirical Processes in M-estimation. Cambridge University Press, Cambridge

    Google Scholar 

  37. van der Vaart A.W. (1994) Weak convergence of smoothed empirical processes. Scand. J. Stat. 21, 501–504

    MathSciNet  Google Scholar 

  38. van der Vaart A.W., Wellner J.A. (1996) Weak Convergence and Empirical Processes. Springer, Berlin Heidelberg New York

    Google Scholar 

  39. von Mises R. (1947) On the asymptotic distribution of differentiable statistical functions. Ann. Math. Stat. 20, 309–348

    Article  Google Scholar 

  40. Wong W.H., Severini T.A. (1991) On maximum likelihood estimation in infinite dimensional parameter spaces. Ann. Stat. 19, 603–632

    Article  MATH  MathSciNet  Google Scholar 

  41. Wong W.H., Shen X. (1995) Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Stat. 23, 339–362

    Article  MATH  MathSciNet  Google Scholar 

  42. Yukich J.E. (1992) Weak convergence of smoothed empirical processes. Scand. J. Stat. 19, 271–279

    MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Vienna, Vienna, Austria

    Richard Nickl

  2. Department of Mathematics, University of Connecticut, 196, Auditorium Road, Storrs, CT, 06269-3009, USA

    Richard Nickl

Authors
  1. Richard Nickl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard Nickl.

Additional information

An erratum to this article can be found at http://dx.doi.org/10.1007/s00440-007-0136-4

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nickl, R. Donsker-type theorems for nonparametric maximum likelihood estimators. Probab. Theory Relat. Fields 138, 411–449 (2007). https://doi.org/10.1007/s00440-006-0031-4

Download citation

  • Received: 24 September 2005

  • Revised: 14 September 2006

  • Published: 24 October 2006

  • Issue Date: July 2007

  • DOI: https://doi.org/10.1007/s00440-006-0031-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Nonparametric maximum likelihood estimator
  • Uniform central limit theorem
  • Plug-in property
  • Differentiable functionals
  • Convolution products

Mathematical Subject Classification (2000)

  • Primary 60F05
  • Primary 62G07
  • Secondary 62F12
  • Secondary 46F05
Download PDF

Working on a manuscript?

Avoid the common mistakes

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature