Probability Theory and Related Fields

, Volume 135, Issue 3, pp 335–362 | Cite as

Propagation-Separation Approach for Local Likelihood Estimation

  • Jörg Polzehl
  • Vladimir Spokoiny


The paper presents a unified approach to local likelihood estimation for a broad class of nonparametric models, including e.g. the regression, density, Poisson and binary response model. The method extends the adaptive weights smoothing (AWS) procedure introduced in Polzehl and Spokoiny (2000) in context of image denoising. The main idea of the method is to describe a greatest possible local neighborhood of every design point X i in which the local parametric assumption is justified by the data. The method is especially powerful for model functions having large homogeneous regions and sharp discontinuities. The performance of the proposed procedure is illustrated by numerical examples for density estimation and classification. We also establish some remarkable theoretical nonasymptotic results on properties of the new algorithm. This includes the ``propagation'' property which particularly yields the root-n consistency of the resulting estimate in the homogeneous case. We also state an ``oracle'' result which implies rate optimality of the estimate under usual smoothness conditions and a ``separation'' result which explains the sensitivity of the method to structural changes.

Mathematics Subject Classification (2000)

62G05 Secondary: 62G07 62G08 62G32 62H30 

Key words or phrases

Adaptive weights Local likelihood Exponential family Propagation Separation Density estimation Classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bickel, P.J., Klaassen, C.A.J., Ritov, Y., Wellner, J.A.: Efficient and Adaptive Estimation for Semiparametric Models. Springer, 1998Google Scholar
  2. 2.
    Cai, Z., Fan, J., Li, R.: Efficient estimation and inference for varying coefficients models. J. Amer. Statist. Ass., 95, 888–902 (2000)CrossRefMATHGoogle Scholar
  3. 3.
    Cai, Z., Fan, J., Yao, Q.: Functional-coefficient regression models for nonlinear time series. J. Amer. Statist. Ass. 95, 941–956 (2000)CrossRefMATHGoogle Scholar
  4. 4.
    Fan, J., Farmen, M., Gijbels, I.: Local maximum likelihood estimation and Inference. J. Royal Statist. Soc. Ser. B, 60, 591–608 (1998)CrossRefMATHGoogle Scholar
  5. 5.
    Fan, J., Gijbels, I.: Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. Royal Statist. Soc. B 57, 371–394 (1995)MATHGoogle Scholar
  6. 6.
    Fan, J., Gijbels, I.: Local polynomial modelling and its applications. Chapman & Hall, London, 1996Google Scholar
  7. 7.
    Fan, J., Marron, J.S.: Fast implementations of nonparametric curve estimators. J. Comp. Graph. Statist. 3, 35–56 (1994)CrossRefGoogle Scholar
  8. 8.
    Fan, J., Zhang, C., Zhang, J.: Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29, 153–193 (2001)MATHMathSciNetGoogle Scholar
  9. 9.
    Fan, J., Zhang, J.: Sieve empirical likelihood ratio tests for nonparametric functions. Ann. Statist. 32, 1858–1907 (2004)CrossRefMATHMathSciNetGoogle Scholar
  10. 10.
    Friedman, J.H.: Greedy function approximation: A gradient boosting machine. Ann. Statist. 29 (5), 11891232 (2001)Google Scholar
  11. 11.
    Grama, I., Polzehl, J. Spokoiny, V.: Adaptive estimation for varying coefficient generalized linear models. Manuscript in preparation 2003Google Scholar
  12. 12.
    Hastie, T.J., Tibshirani, R.J.: Varying-coefficient models (with discussion). J. Royal Statist. Soc. Ser. B 55, 757–796 (1993)MATHMathSciNetGoogle Scholar
  13. 13.
    Hastie, T.J., Tibshirani, R.J., Friedman, J.: hThe Elements of Statistical Learning. Springer, 2001Google Scholar
  14. 14.
    Korostelev, A., Tsybakov, A.: Minimax Theory of Image Reconstruction. Springer Verlag, New York–Heidelberg–Berlin, 1993Google Scholar
  15. 15.
    Lepski, O., Mammen, E., Spokoiny, V.: Ideal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selection. Annals of Statistics 25 (3), 929–947 (1997)MathSciNetGoogle Scholar
  16. 16.
    Loader, C.R.: Local likelihood density estimation. Academic Press 1996Google Scholar
  17. 17.
    Müller, H.: Change-points in nonparametric regression analysis. Ann. Statist. 20, 737–761 (1992)MATHMathSciNetGoogle Scholar
  18. 18.
    Müller, H.G., Song, K.S.: Maximum estimation of multidimensional boundaries. J. Multivariate Anal. 50 (2), 265–281 (1994)CrossRefGoogle Scholar
  19. 19.
    Polzehl, J., Spokoiny, V.: Adaptive weights smoothing with applications to image segmentation. J. of Royal Stat. Soc. 62, Series B, 335–354 (2000)Google Scholar
  20. 20.
    Polzehl, J., Spokoiny, V.: Local likelihood modeling by adaptive weights smoothing. WIAS-Preprint No. 787, 2002Google Scholar
  21. 21.
    Polzehl, J., Spokoiny, V.: Image denoising: pointwise adaptive approach. Annals of Statistics 31, 30–57 (2003)CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Polzehl, J., Spokoiny, V., Starica, C.: When did the 2001 recession really end? WIAS-Preprint No. 934, 2004Google Scholar
  23. 23.
    Polzehl, J., Spokoiny, V.: Varying coefficient GARCH versus local constant volatility modeling. Comparison of the predictive power. WIAS-Preprint No. 977, 2004Google Scholar
  24. 24.
    Polzehl, J., Spokoiny, V.: Spatially adaptive regression estimation: Propagation-separation approach. WIAS-Preprin No. 998, 2004cGoogle Scholar
  25. 25.
    Qiu, P.: Discontinuous regression surface fitting. Ann. Statist. 26 (6), 2218–2245 (1998)Google Scholar
  26. 26.
    Spokoiny, V.: Estimation of a function with discontinuities via local polynomial fit with an adaptive window choice. Ann. Statist. 26 (4), 1356–1378 (1998)MathSciNetGoogle Scholar
  27. 27.
    Staniswalis, J.C.: The kernel estimate of a regression function in likelihood-based models. Journal of the American Statistical Association 84, 276–283 (1989)CrossRefMATHMathSciNetGoogle Scholar
  28. 28.
    Tibshirani, J.R., Hastie, T.J.: Local likelihood estimation. Journal of the American Statistical Association 82, 559–567 (1987)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  1. 1.Weierstrass-InstituteBerlinGermany

Personalised recommendations