Propagation-Separation Approach for Local Likelihood Estimation

Probability Theory and Related Fields volume 135, pages335–362(2006)Cite this article

Abstract

The paper presents a unified approach to local likelihood estimation for a broad class of nonparametric models, including e.g. the regression, density, Poisson and binary response model. The method extends the adaptive weights smoothing (AWS) procedure introduced in Polzehl and Spokoiny (2000) in context of image denoising. The main idea of the method is to describe a greatest possible local neighborhood of every design point X i in which the local parametric assumption is justified by the data. The method is especially powerful for model functions having large homogeneous regions and sharp discontinuities. The performance of the proposed procedure is illustrated by numerical examples for density estimation and classification. We also establish some remarkable theoretical nonasymptotic results on properties of the new algorithm. This includes the ``propagation'' property which particularly yields the root-n consistency of the resulting estimate in the homogeneous case. We also state an ``oracle'' result which implies rate optimality of the estimate under usual smoothness conditions and a ``separation'' result which explains the sensitivity of the method to structural changes.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Bickel, P.J., Klaassen, C.A.J., Ritov, Y., Wellner, J.A.: Efficient and Adaptive Estimation for Semiparametric Models. Springer, 1998

  2. 2.

    Cai, Z., Fan, J., Li, R.: Efficient estimation and inference for varying coefficients models. J. Amer. Statist. Ass., 95, 888–902 (2000)

    Article  MATH  Google Scholar 

  3. 3.

    Cai, Z., Fan, J., Yao, Q.: Functional-coefficient regression models for nonlinear time series. J. Amer. Statist. Ass. 95, 941–956 (2000)

    Article  MATH  Google Scholar 

  4. 4.

    Fan, J., Farmen, M., Gijbels, I.: Local maximum likelihood estimation and Inference. J. Royal Statist. Soc. Ser. B, 60, 591–608 (1998)

    Article  MATH  Google Scholar 

  5. 5.

    Fan, J., Gijbels, I.: Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. Royal Statist. Soc. B 57, 371–394 (1995)

    MATH  Google Scholar 

  6. 6.

    Fan, J., Gijbels, I.: Local polynomial modelling and its applications. Chapman & Hall, London, 1996

  7. 7.

    Fan, J., Marron, J.S.: Fast implementations of nonparametric curve estimators. J. Comp. Graph. Statist. 3, 35–56 (1994)

    Article  Google Scholar 

  8. 8.

    Fan, J., Zhang, C., Zhang, J.: Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29, 153–193 (2001)

    MATH  MathSciNet  Google Scholar 

  9. 9.

    Fan, J., Zhang, J.: Sieve empirical likelihood ratio tests for nonparametric functions. Ann. Statist. 32, 1858–1907 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  10. 10.

    Friedman, J.H.: Greedy function approximation: A gradient boosting machine. Ann. Statist. 29 (5), 11891232 (2001)

    Google Scholar 

  11. 11.

    Grama, I., Polzehl, J. Spokoiny, V.: Adaptive estimation for varying coefficient generalized linear models. Manuscript in preparation 2003

  12. 12.

    Hastie, T.J., Tibshirani, R.J.: Varying-coefficient models (with discussion). J. Royal Statist. Soc. Ser. B 55, 757–796 (1993)

    MATH  MathSciNet  Google Scholar 

  13. 13.

    Hastie, T.J., Tibshirani, R.J., Friedman, J.: hThe Elements of Statistical Learning. Springer, 2001

  14. 14.

    Korostelev, A., Tsybakov, A.: Minimax Theory of Image Reconstruction. Springer Verlag, New York–Heidelberg–Berlin, 1993

  15. 15.

    Lepski, O., Mammen, E., Spokoiny, V.: Ideal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selection. Annals of Statistics 25 (3), 929–947 (1997)

    MathSciNet  Google Scholar 

  16. 16.

    Loader, C.R.: Local likelihood density estimation. Academic Press 1996

  17. 17.

    Müller, H.: Change-points in nonparametric regression analysis. Ann. Statist. 20, 737–761 (1992)

    MATH  MathSciNet  Google Scholar 

  18. 18.

    Müller, H.G., Song, K.S.: Maximum estimation of multidimensional boundaries. J. Multivariate Anal. 50 (2), 265–281 (1994)

    Article  Google Scholar 

  19. 19.

    Polzehl, J., Spokoiny, V.: Adaptive weights smoothing with applications to image segmentation. J. of Royal Stat. Soc. 62, Series B, 335–354 (2000)

  20. 20.

    Polzehl, J., Spokoiny, V.: Local likelihood modeling by adaptive weights smoothing. WIAS-Preprint No. 787, 2002

  21. 21.

    Polzehl, J., Spokoiny, V.: Image denoising: pointwise adaptive approach. Annals of Statistics 31, 30–57 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  22. 22.

    Polzehl, J., Spokoiny, V., Starica, C.: When did the 2001 recession really end? WIAS-Preprint No. 934, 2004

  23. 23.

    Polzehl, J., Spokoiny, V.: Varying coefficient GARCH versus local constant volatility modeling. Comparison of the predictive power. WIAS-Preprint No. 977, 2004

  24. 24.

    Polzehl, J., Spokoiny, V.: Spatially adaptive regression estimation: Propagation-separation approach. WIAS-Preprin No. 998, 2004c

  25. 25.

    Qiu, P.: Discontinuous regression surface fitting. Ann. Statist. 26 (6), 2218–2245 (1998)

    Google Scholar 

  26. 26.

    Spokoiny, V.: Estimation of a function with discontinuities via local polynomial fit with an adaptive window choice. Ann. Statist. 26 (4), 1356–1378 (1998)

    MathSciNet  Google Scholar 

  27. 27.

    Staniswalis, J.C.: The kernel estimate of a regression function in likelihood-based models. Journal of the American Statistical Association 84, 276–283 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  28. 28.

    Tibshirani, J.R., Hastie, T.J.: Local likelihood estimation. Journal of the American Statistical Association 82, 559–567 (1987)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Affiliations

  1. Weierstrass-Institute, Mohrenstr. 39, 10117, Berlin, Germany

    Jörg Polzehl & Vladimir Spokoiny

Authors
  1. Jörg Polzehl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimir Spokoiny
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Spokoiny.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Polzehl, J., Spokoiny, V. Propagation-Separation Approach for Local Likelihood Estimation. Probab. Theory Relat. Fields 135, 335–362 (2006). https://doi.org/10.1007/s00440-005-0464-1

Download citation

Mathematics Subject Classification (2000)

  • 62G05
  • Secondary: 62G07
  • 62G08
  • 62G32
  • 62H30

Key words or phrases

  • Adaptive weights
  • Local likelihood
  • Exponential family
  • Propagation
  • Separation
  • Density estimation
  • Classification