Abstract
Estimation of the mean function in nonparametric regression is usefully separated into estimating the means at the observed factor levels—a one-way layout problem—and interpolation between the estimated means at adjacent factor levels. Candidate penalized least squares (PLS) estimators for the mean vector of a one-way layout are expressed as shrinkage estimators relative to an orthogonal regression basis determined by the penalty matrix. The shrinkage representation of PLS suggests a larger class of candidate monotone shrinkage (MS) estimators. Adaptive PLS and MS estimators choose the shrinkage vector and penalty matrix to minimize estimated risk. The actual risks of shrinkage-adaptive estimators depend strongly upon the economy of the penalty basis in representing the unknown mean vector. Local annihilators of polynomials, among them difference operators, generate penalty bases that are economical in a range of examples. Diagnostic techniques for adaptive PLS or MS estimators include basis-economy plots and estimates of loss or risk.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrews, D. F. and Herzberg, A. M. (1985). Data: A Collection of Problems from Many Fields for the Student and Research Worker, Springer, New York.
Beran, R. (1996). Confidence sets centered at C p estimators, Ann. Inst. Statist. Math., 48, 1–15.
Beran, R. (2000). REACT scatterplot smoothers: Superefficiency through basis economy, J. Amer. Statist. Assoc., 63, 155–171.
Beran, R. and Dümbgen, L. (1998). Modulation of estimators and confidence sets, Ann. Statist., 26, 1826–1856.
Buja, A., Hastie, T. and Tibshirani, R. (1989). Linear smoothers and additive models (with discussion), Ann. Statist., 17, 453–555.
Chu, C.-K. and Marron, J. S. (1991). Choosing a kernel regression estimator, Statist. Sci., 6, 404–436.
Heckman, N. E. and Ramsay, J. O. (2000). Penalized regression with model-based penalties, Canad. J. Statist., 28, 241–258.
Mallows, C. L. (1973). Some comments on C p, Technometrics, 15, 661–676.
Pinsker, M. S. (1980). Optimal filtration of square-integrable signals in Gaussian noise, Problems Inform. Transmission, 16, 120–133.
Rice, J. (1984). Bandwidth choice for nonparametric regression, Ann. Statist., 12, 1215–1230.
Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference, Wiley, New York.
Simonoff, J. S. (1996). Smoothing Methods in Statistics, Springer, New York.
Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution, Proc. Third Berkeley Symp. on Math. Statist. Prob., Vol. 1 (ed. J. Neyman), 197–206, University of California Press, Berkeley.
Stein, C. (1981). Estimation of the mean of a multivariate normal distribution, Ann. Statist., 9, 1135–1151.
Venables, W. N. and Ripley, B. D. (1999). Modern Applied Statistics with S-PLUS, 3rd ed., Springer, New York.
Author information
Authors and Affiliations
About this article
Cite this article
Beran, R. Improving Penalized Least Squares through Adaptive Selection of Penalty and Shrinkage. Annals of the Institute of Statistical Mathematics 54, 900–917 (2002). https://doi.org/10.1023/A:1022479822263
Issue Date:
DOI: https://doi.org/10.1023/A:1022479822263