Estimation of Smooth Functionals of Location Parameter in Gaussian and Poincaré Random Shift Models

Estimation of Smooth functionals

Abstract

Let E be a separable Banach space and let \(f:E\mapsto {\mathbb {R}}\) be a smooth functional. We discuss a problem of estimation of f(𝜃) based on an observation X = 𝜃 + ξ, where 𝜃E is an unknown parameter and ξ is a mean zero random noise, or based on n i.i.d. observations from the same random shift model. We develop estimators of f(𝜃) with sharp mean squared error rates depending on the degree of smoothness of f for random shift models with distribution of the noise ξ satisfying Poincaré type inequalities (in particular, for some log-concave distributions). We show that for sufficiently smooth functionals f these estimators are asymptotically normal with a parametric convergence rate. This is done both in the case of known distribution of the noise and in the case when the distribution of the noise is Gaussian with covariance being an unknown nuisance parameter.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    The definition and relevant properties of j-th order Fréchet derivatives could be found, e.g., in Cartan (1971), Sections 2.1, 5.1, 5.3.

  2. 2.

    In fact, a more general result was proved in Koltchinskii and Zhilova (2020) for spaces Cs,γ(E) of functionals of smoothness s whose derivatives are allowed to grow as ∥𝜃γ. For simplicity, we consider here only the case of γ = 0 (both f and its derivatives are uniformly bounded).

  3. 3.

    Note that these asymptotic relationships hold in the case when E = En depends on n, for instance, when \(E={\mathbb {R}}^{d}\) with d = dn.

  4. 4.

    A standard way to write Poincaré inequality for r.v. in \({\mathbb {R}}^{d}\) is:

    $$ \begin{array}{@{}rcl@{}} \text{Var}(g(\xi)) \leq C {\mathbb{E}} \|\nabla g(\xi)\|^{2}, \end{array} $$

    where ∇g is the gradient of function g that, by Rademacher’s theorem, exists a.s. for all locally Lipschitz functions and ∥⋅∥ is the standard Euclidean norm.

  5. 5.

    Here \(\|\cdot \|_{\psi _{1}}\) is the Orlicz ψ-norm corresponding to the sub-exponential tails; see Vershynin (2018), Chapter 2 for the definitions.

References

  1. Barthe, F. and Klartag, B. (2019). Spectral gaps, symmetries and log-concave perturbations. arXiv:1907.01823.

  2. Bickel, P. and Ritov, Y. (1988). Estimating integrated square density derivatives: sharp best order of convergence estimates. Sankhya 50, 381–393.

    MathSciNet  MATH  Google Scholar 

  3. Birgé, L. and Massart, P. (1995). Estimation of integral functionals of a density. Ann. Stat. 23, 11–29.

    MathSciNet  Article  Google Scholar 

  4. Cai, T. T. and Low, M. (2005). Non-quadratic estimators of a quadratic functional. Ann. Stat. 33, 2930–2956.

    Article  Google Scholar 

  5. Cartan, H. (1971). Differential calculus, Houghton Mifflin Co.

  6. Collier, O., Comminges, L. and Tsybakov, A. (2017). Minimax estimation of linear and quadratic functionals on sparsity classes. Ann. Stat. 45, 923–958.

    MathSciNet  Article  Google Scholar 

  7. Donoho, D. and Nussbaum, M. (1990). Minimax quadratic estimation of a quadratic functional. J. Complexity 6, 290–323.

    MathSciNet  Article  Google Scholar 

  8. Fisher, R. A. (1922). On the mathematical foundation of theoretical statistics. Philos. Trans. R. Soc. Lond. Ser. 222, 309–368.

    Article  Google Scholar 

  9. Fisher, R. A. (1925). Theory of statistical estimation. Proc. Camb. Philos. Soc. 22, 700–725.

    Article  Google Scholar 

  10. Hàjek, J. (1972). Local asymptotic minimax and admissibility in estimation. Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability 1, 175–194.

    MathSciNet  MATH  Google Scholar 

  11. Han, Y., Jiao, J. and Mukherjee, R. (2020). On estimation of lr-norms in Gaussian white noise model. Probability Theory and Related Fields. to appear arXiv:1710.03863.

  12. Ibragimov, I. A. and Khasminskii, R. Z. (1981). Statistical estimation: asymptotic theory. Springer, New York.

    Google Scholar 

  13. Ibragimov, I. A., Nemirovski, A. S. and Khasminskii, R. Z. (1987). Some problems of nonparametric estimation in Gaussian white noise. Theory of Probab. and Appl. 31, 391–406.

    Article  Google Scholar 

  14. Koltchinskii, V. (2020). Asymptotically efficient estimation of smooth functionals of covariance operators. J. European Math. Society. to appear. arxiv:1710.09072.

  15. Koltchinskii, V. (2018). Asymptotic efficiency in High-Dimensional covariance estimation. Proc. ICM 2018 Rio de Janeiro 3, 2891–2912.

    MathSciNet  MATH  Google Scholar 

  16. Koltchinskii, V., Löffler, M. and Nickl, R. (2020). Efficient Estimation of Linear Functionals of Principal Components. Ann. Stat. 48, 464–490.

    MathSciNet  Article  Google Scholar 

  17. Koltchinskii, V. and Zhilova, M. (2020). Efficient Estimation of Smooth Functionals in Gaussian Shift Models. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques. to appear. arXiv:1810.02767.

  18. Koltchinskii, V. and Zhilova, M. (2019). Estimation of smooth functionals in normal models: bias reduction and asymptotic efficiency. arXiv:1912.08877.

  19. Le Cam, L. (1953). On some asymptotic properties of maximum likelihood estimates and related Bayes estimates. Univ. Calif. publ. Stat. 1, 277–330.

    MathSciNet  Google Scholar 

  20. Le Cam, L. (1972). Limits of experiments. Proc. Sixth Berkeley Symp. Math. Statist. Prob., I, L. Le Cam, J. Neyman and E.L. Scott (Eds.), University of California Press, 245-261.

  21. Ledoux, M. (2001). The concentration of measure phenomenon. American Mathematical Society.

  22. Lee, Y. -T. and Vempala, S. (2017). Eldan’s stochastic localization and the KLS hyperplane conjecture: an improved lower bound for expansion. In: Foundations of Computer Science (FOCS) 2017, IEEE 58th Annual Symposium, pp. 998–1007.

  23. Levit, B. (1975). On the efficiency of a class of non-parametric estimates. Theory Probab. Appl. 20, 723–740.

    Article  Google Scholar 

  24. Levit, B. (1978). Asymptotically efficient estimation of nonlinear functionals. Probl. Peredachi Inf. (Problems of Information Transmission) 14, 65–72.

    MathSciNet  Google Scholar 

  25. Lepski, O., Nemirovski, A. and Spokoiny, V. (1999). On estimation of the Lr norm of a regression function. Probab. Theory Related Fields 113, 221–253.

    MathSciNet  Article  Google Scholar 

  26. Mukherjee, R., Newey, W. and Robins, J. (2017). Semiparametric efficient empirical higher order influence function estimators. arXiv:1705.07577.

  27. Nemirovski, A. (1990). On necessary conditions for the efficient estimation of functionals of a nonparametric signal which is observed in white noise. Theory Probab. Appl. 35, 94–103.

    MathSciNet  Article  Google Scholar 

  28. Nemirovski, A. (2000). Topics in Non-parametric Statistics. Ecole d’Ete de Probabilites de Saint-Flouŕ. Lecture Notes in Mathematics, V. 1738, Springer, New York.

  29. Rao, C. R. (1963). Criteria of estimation in large samples. Sankhya, Ser A. 25, 189–206.

    MathSciNet  MATH  Google Scholar 

  30. Robins, J., Li, L., Tchetgen, E. and van der Vaart, A. (2008). Higher order influence functions and minimax estimation of nonlinear functionals. IMS Collections Probability and Statistics: Essays in Honor of David A. Freedman 2, 335–421.

    MathSciNet  MATH  Google Scholar 

  31. Robins, J., Li, L., Tchetgen, E. and van der Vaart, A. (2016). Asymptotic normality of quadratic estimators. Stoch. Process. Their Appl. 126, 3733–3759.

    MathSciNet  Article  Google Scholar 

  32. van der Vaart, A. (2014). Higher order tangent spaces and influence functions. Stat. Sci. 29, 679–686.

    MathSciNet  Article  Google Scholar 

  33. Vershynin, R. (2018). High-Dimensional Probability. An Introduction with Applications in Data Science. Cambridge University Press.

  34. Zhou, F. and Li, P. (2019). A Fourier Analytical Approach to Estimation of Smooth Functions in Gaussian Shift Model. arXiv:1911.02010.

Download references

Acknowledgements

V. Koltchinskii was supported in part by NSF Grant DMS-1810958. M. Zhilova was supported in part by NSF Grant DMS-1712990.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Vladimir Koltchinskii.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Koltchinskii, V., Zhilova, M. Estimation of Smooth Functionals of Location Parameter in Gaussian and Poincaré Random Shift Models. Sankhya A (2021). https://doi.org/10.1007/s13171-020-00232-1

Download citation

Keywords

  • Smooth functionals
  • Efficiency
  • Random shift model
  • Poincaré inequality
  • Normal approximation.

AMS (2000) subject classification.

  • Primary 62H12; Secondary 62G20
  • 62H25
  • 60B20