Abstract
Knowledge of the probability distribution of error in a regression problem plays an important role in verification of an assumed regression model, making inference about predictions, finding optimal regression estimates, suggesting confidence bands and goodness of fit tests as well as in many other issues of the regression analysis. This article is devoted to an optimal estimation of the error probability density in a general heteroscedastic regression model with possibly dependent predictors and regression errors. Neither the design density nor regression function nor scale function is assumed to be known, but they are suppose to be differentiable and an estimated error density is suppose to have a finite support and to be at least twice differentiable. Under this assumption the article proves, for the first time in the literature, that it is possible to estimate the regression error density with the accuracy of an oracle that knows “true” underlying regression errors. Real and simulated examples illustrate importance of the error density estimation as well as the suggested oracle methodology and the method of estimation.
Similar content being viewed by others
References
Akritas M.G., Van Keilegom I., (2001). Non-parametric estimation of the residual distribution. Scandinavian Journal of Statistics, 28, 549–567
Bickel P.J., Ritov Y., (2003). Nonparametric estimators that can be “plugged–in”. The Annals of Statistics, 31, 1033–1053
Brown L.D., Low M.G., Zhao L.H. (1997). Superefficiency in nonparametric function estimation. The Annals of Statistics, 25, 2607–2625
Butzer P.L., Nessel R.J. (1971). Fourier analysis and approximation. New York, Academic press
Cheng F. (2002). Error density and distribution function estimation in nonparametric regression models. Ph.D. Thesis. Department of Statistics and Probability, Michigan State Univeristy.
Cheng F. (2004). Weak and strong uniform consistency of a kernel error density estimator in nonparametric regression. Journal of Statistical Planning and Inference, 116, 95–108
Donoho D., Johnstone D. (1995). Adaptation to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90, 1200–1224
Efromovich S. (1985). Nonparametric estimation of a density with unknown smoothness. Theory of Probability and its Applications, 30, 557–568
Efromovich S. (1998). Simultaneous sharp estimation of functions and their derivatives. The Annals of Statistics, 26, 273–278
Efromovich S. (1999), Nonparametric curve estimation: methods, theory and applications. Berlin Heidelberg New York, Springer
Efromovich S. (2001). Density estimation under random censorship and order restrictions: from asymptotic to small sample sizes. Journal of the American Statistical Association, 94, 667–685
Efromovich S. (2004). Distribution estimation for biased data. Journal of Statistical Planning and Inference, 124, 1–43
Efromovich S. (2005). Estimation of the density of regression errors. The Annals of Statistics, 33, 2194–2227
Efromovich S. (2006). Adaptive estimation of the error density in nonparametric regression with small sample size. Journal of Statistical Planning and Inference (in press).
Eubank R.L. (1999). Nonparametric regression and spline smoothing. New York, Marcel Dekker
Fan J., Gijbels I. (1996). Local polynomial modeling and its applications. New York, Chapman & Hall
Hart J. (1997). Nonparametric smoothing and lack-of-fit tests. Berlin Heidelberg New York, Springer
Müller U.U., Schick A., Wefelmeyer W. (2004), Estimating linear functionals of the error distribution in nonparametric regression. Journal of Statistical Planning and Inference, 116: 75–94
Neter J., Kutner M., Nachtsheim C., Wasserman W. (1996). Applied linear models (4th ed). Boston, McGraw-Hill
Wasserman L. (2005). All of nonparametric statistics. Berlin Heidelberg New York, Springer
Zhang C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estimation. The Annals of Statistics, 33, 54–100
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Efromovich, S. Optimal nonparametric estimation of the density of regression errors with finite support. AISM 59, 617–654 (2007). https://doi.org/10.1007/s10463-006-0067-3
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-006-0067-3