Abstract
A shrinkage-type variable selection procedure for varying coefficient models is routinely established in the least-squares (LS) framework. Although the LS method has favorable properties for a large class of error distributions, it will break down if the error variance is infinite and is adversely affected by outliers and heavy-tail distributions. To overcome these issues, we propose a robust shrinkage method termed regularized Walsh-average (RWA) that can construct robust nonparametric variable selection and robust coefficient estimation simultaneously. Theoretical analysis reveals RWA works beautifully, including consistency in variable selection and oracle property in estimation, even when error variance is infinite. More important property is that when error variance is finite, compared with the LS based estimators, the asymptotic relative efficiency of the new estimator is at least 0.8896, a relatively high level. Furthermore, a robust BIC-type criterion, which can identify the true model consistently, is suggested for shrinkage parameter selection. Numerical studies also confirm our theories.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Breiman, L. (1995). Better subset selection using nonnegative garrote. Techonometrics, 37, 373–384.
Cantoni, E., & Ronchetti, E. (2001). Resistant selection of the smoothing parameter for smoothing splines. Statistics and Computing, 11, 141–146.
Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348–1360.
Fan, J., & Li, R. (2004). New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. Journal of the American Statistical Association, 99, 710–723.
Feng, L., Zou, C., & Wang, Z. (2012). Local Walsh-average regression. Journal of Multivariate Analysis, 106, 36–48.
Fu, W. (1998). Penalized regression: the bridge versus the LASSO. Journal of Computational and Graphical Statistics, 7, 397–416.
Hettmansperger, T., & McKean, J. (1998). Robust Nonparametric Statistical Methods. London and New York: Arnold/Wiley.
Hodges, J., & Lehmann, E. (1956). The efficiency of some nonparametric competitors of the t-Test. The Annals of Mathematical Statistics, 27, 324–335.
Hunter, D., & Li, R. (2005). Variable selection using MM algorithms. The Annals of Statistics, 33, 1617–1642.
Janson, S. (1987). Maximal spacing in several dimensions. The Annals of Probability, 15, 274–280.
Kai, B., Li, R., & Zou, H. (2010). Local composite quantile regression smoothing: an efficient and safe alternative to local polynomial regression. Journal of the Royal Statistical Society. B, 72, 49–69.
Kai, B., Li, R., & Zou, H. (2011). New efficient estimation and variable selection methods for semiparametric varying-coefficient partially linear models. The Annals of Statistics, 39, 305–332.
Knight, K., & Fu, W. (2000). Asymptotics for LASSO-type estimators. The Annals of Statistics, 28, 1356–1378.
Leng, C. (2010). Variable selection and coefficient estimation via regularization rank estimation. Statistica Sinica, 20, 167–181.
Li, R., & Liang, H. (2008). Variable selection in semiparametric regression model. The Annals of Statistics, 36, 261–286.
Mack, Y., & Silverman, B. (1982). Weak and strong uniform consistency of kernel regression estimates. Probability Theory and Related Fields, 61, 405–415.
Noh, H., Chung, K., & Keilegom, I. (2012). Variable selection of varying coefficient models in quantile regression. Electronic Journal of Statistics, 6, 1220–1238.
Shang, S., Zou, C., & Wang, Z. (2012). Local Walsh-average regression for semiparametric varying-coefficient models. Statistics & Probability Letters, 82, 1815–1822.
Tang, Y., Wang, H., & Zhu, Z. (2013). Variable selection in quantile varying coefficient models with longitudinal data. Computational Statistics and Data Analysis, 57, 435–449.
Tang, Y., Wang, H., Zhu, Z., & Song, X. (2012). A unified variable selection approach for varying coefficient models. Statistica Sinica, 22, 601–628.
Tibshirani, R. J. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society. Series B, 58, 267–288.
Wang, L., Kai, B., & Li, R. (2009). Local rank inference for varying coefficient models. Journal of the American Statistical Association, 488, 1631–1645.
Wang, L., & Li, R. (2009). Weighted Wilcoxon-type smoothly clipped absolute deviation method. Biometrics, 65, 564–571.
Wang, L., Li, H., & Huang, J. (2008). Variable selection in nonparametric varying-coefficient models for analysis of repeated measurements. Journal of the American Statistical Association, 103, 1556–1569.
Wang, H., Li, G., & Tsai, C. (2007). Regression coefficient and autoregressive order shrinkage and selection via LASSO. Journal of the Royal Statistical Society. B, 69, 63–78.
Wang, H., & Xia, Y. (2009). Shrinkage estimation of the varying coefficient model. Journal of the American Statistical Association, 104, 747–757.
Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society. B, 68, 49–67.
Yuan, M., & Lin, Y. (2007). On the nonnegative garrote estimator. Journal of the Royal Statistical Society. B, 69, 143–161.
Zhang, W., Lee, S., & Song, X. (2002). Local polynomial fitting in semivarying coefficient models. Journal of Multivariate Analysis, 82, 166–188.
Zhao, W., Zhang, R., Lv, Y., & Liu, J. (2012). Variable selection of the quantile varying coefficient regression models. Journal of the Korean Statistical Association, 103, 1556–1569.
Zou, H. (2006). The adaptive LASSO and its oracle properties. Journal of the American Statistical Association, 101, 1418–1429.
Zou, H., & Yuan, M. (2008). Composite quantile regression and the oracle model selection theory. The Annals of Statistics, 36, 1108–1126.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, K., Lin, L. Walsh-average based variable selection for varying coefficient models. J. Korean Stat. Soc. 44, 95–110 (2015). https://doi.org/10.1016/j.jkss.2014.05.006
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1016/j.jkss.2014.05.006