Abstract
The paper presents a unified approach to local likelihood estimation for a broad class of nonparametric models, including e.g. the regression, density, Poisson and binary response model. The method extends the adaptive weights smoothing (AWS) procedure introduced in Polzehl and Spokoiny (2000) in context of image denoising. The main idea of the method is to describe a greatest possible local neighborhood of every design point X i in which the local parametric assumption is justified by the data. The method is especially powerful for model functions having large homogeneous regions and sharp discontinuities. The performance of the proposed procedure is illustrated by numerical examples for density estimation and classification. We also establish some remarkable theoretical nonasymptotic results on properties of the new algorithm. This includes the ``propagation'' property which particularly yields the root-n consistency of the resulting estimate in the homogeneous case. We also state an ``oracle'' result which implies rate optimality of the estimate under usual smoothness conditions and a ``separation'' result which explains the sensitivity of the method to structural changes.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bickel, P.J., Klaassen, C.A.J., Ritov, Y., Wellner, J.A.: Efficient and Adaptive Estimation for Semiparametric Models. Springer, 1998
Cai, Z., Fan, J., Li, R.: Efficient estimation and inference for varying coefficients models. J. Amer. Statist. Ass., 95, 888–902 (2000)
Cai, Z., Fan, J., Yao, Q.: Functional-coefficient regression models for nonlinear time series. J. Amer. Statist. Ass. 95, 941–956 (2000)
Fan, J., Farmen, M., Gijbels, I.: Local maximum likelihood estimation and Inference. J. Royal Statist. Soc. Ser. B, 60, 591–608 (1998)
Fan, J., Gijbels, I.: Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. Royal Statist. Soc. B 57, 371–394 (1995)
Fan, J., Gijbels, I.: Local polynomial modelling and its applications. Chapman & Hall, London, 1996
Fan, J., Marron, J.S.: Fast implementations of nonparametric curve estimators. J. Comp. Graph. Statist. 3, 35–56 (1994)
Fan, J., Zhang, C., Zhang, J.: Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29, 153–193 (2001)
Fan, J., Zhang, J.: Sieve empirical likelihood ratio tests for nonparametric functions. Ann. Statist. 32, 1858–1907 (2004)
Friedman, J.H.: Greedy function approximation: A gradient boosting machine. Ann. Statist. 29 (5), 11891232 (2001)
Grama, I., Polzehl, J. Spokoiny, V.: Adaptive estimation for varying coefficient generalized linear models. Manuscript in preparation 2003
Hastie, T.J., Tibshirani, R.J.: Varying-coefficient models (with discussion). J. Royal Statist. Soc. Ser. B 55, 757–796 (1993)
Hastie, T.J., Tibshirani, R.J., Friedman, J.: hThe Elements of Statistical Learning. Springer, 2001
Korostelev, A., Tsybakov, A.: Minimax Theory of Image Reconstruction. Springer Verlag, New York–Heidelberg–Berlin, 1993
Lepski, O., Mammen, E., Spokoiny, V.: Ideal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selection. Annals of Statistics 25 (3), 929–947 (1997)
Loader, C.R.: Local likelihood density estimation. Academic Press 1996
Müller, H.: Change-points in nonparametric regression analysis. Ann. Statist. 20, 737–761 (1992)
Müller, H.G., Song, K.S.: Maximum estimation of multidimensional boundaries. J. Multivariate Anal. 50 (2), 265–281 (1994)
Polzehl, J., Spokoiny, V.: Adaptive weights smoothing with applications to image segmentation. J. of Royal Stat. Soc. 62, Series B, 335–354 (2000)
Polzehl, J., Spokoiny, V.: Local likelihood modeling by adaptive weights smoothing. WIAS-Preprint No. 787, 2002
Polzehl, J., Spokoiny, V.: Image denoising: pointwise adaptive approach. Annals of Statistics 31, 30–57 (2003)
Polzehl, J., Spokoiny, V., Starica, C.: When did the 2001 recession really end? WIAS-Preprint No. 934, 2004
Polzehl, J., Spokoiny, V.: Varying coefficient GARCH versus local constant volatility modeling. Comparison of the predictive power. WIAS-Preprint No. 977, 2004
Polzehl, J., Spokoiny, V.: Spatially adaptive regression estimation: Propagation-separation approach. WIAS-Preprin No. 998, 2004c
Qiu, P.: Discontinuous regression surface fitting. Ann. Statist. 26 (6), 2218–2245 (1998)
Spokoiny, V.: Estimation of a function with discontinuities via local polynomial fit with an adaptive window choice. Ann. Statist. 26 (4), 1356–1378 (1998)
Staniswalis, J.C.: The kernel estimate of a regression function in likelihood-based models. Journal of the American Statistical Association 84, 276–283 (1989)
Tibshirani, J.R., Hastie, T.J.: Local likelihood estimation. Journal of the American Statistical Association 82, 559–567 (1987)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Polzehl, J., Spokoiny, V. Propagation-Separation Approach for Local Likelihood Estimation. Probab. Theory Relat. Fields 135, 335–362 (2006). https://doi.org/10.1007/s00440-005-0464-1
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00440-005-0464-1