Abstract
In this paper we discuss variable selection in a class of single-index models in which we do not assume the error term as additive. Following the idea of sufficient dimension reduction, we first propose a unified method to recover the direction, then reformulate it under the least square framework. Differing from many other existing results associated with nonparametric smoothing methods for density function, the bandwidth selection in our proposed kernel function essentially has no impact on its root-n consistency or asymptotic normality. To select the important predictors, we suggest using the adaptive lasso method which is computationally efficient. Under some regularity conditions, the adaptive lasso method enjoys the oracle property in a general class of single-index models. In addition, the resulting estimation is shown to be asymptotically normal, which enables us to construct a confidence region for the estimated direction. The asymptotic results are augmented through comprehensive simulations, and illustrated by an analysis of air pollution data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altham P.M.E. (1984) Improving the precision of estimation by fitting a generalized linear model and quasi-likelihood. Journal of the Royal Statistical Society: Series B 46: 118–119
Carroll R.J., Fan J., Gijbels I., Wand M.P. (1997) Generalized partially linear single-index models. Journal of the American Statistical Association 92: 477–489
Chen C.H., Li K.C. (1998) Can SIR be as popular as multiple linear regression?. Statistica Sinica 8: 289–316
Cook R.D. (1998) Regression graphics: Ideas for studying regressions through graphics. Wiley & Sons, New York
Cook R.D. (2004) Testing predictor contributions in sufficient dimension reduction. Annals of Statistics 32: 1061–1092
Cook R.D., Ni L. (2005) Sufficient dimension reduction via inverse regression: A minimum discrepancy approach. Journal of the American Statistical Association 100: 410–428
Cook R.D., Weisberg S. (1991) Discussion to “Sliced inverse regression for dimension reduction”. Journal of the American Statistical Association 86: 316–342
Donoho D.L., Johnstone I.M. (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika 81: 425–455
Efron B., Hastie T., Johnstone I., Tibshirani R. (2004) Least angle regression. Annals of Statistics 32: 407–499
Fan J.Q., Li R. (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96: 1348–1360
Fan J.Q., Peng H. (2004) Nonconcave penalized likelihood with a diverging number of parameters. Annals of Statistics 32: 928–961
Geyer C. (1994) On the asymptotics of constraint M-estimation. Annals of Statistics 22: 1993–2010
Hall P., Li K.C. (1993) On almost linearity of low dimensional projection from high dimensional data. Annals of Statistics 21: 867–889
Härdle W., Hall P., Ichimura H. (1993) Optimal smoothing in single-index models. Annals of Statistics 21: 157–178
Härdle W., Mammen E. (1993) Comparing nonparametric versus parametric regression fits. Annals of Statistics 21: 1926–1947
Härdle W., Stoker T.M. (1989) Investigating smooth multiple regression by the method of average derivatives. Journal of the American Statistical Association 84: 986–995
Hristache M., Juditsky A., Spokoiny V. (2001) Direct estimation of the index coefficient in a single-index model. Annals of Statistics 29: 595–623
Huber P.J. (1981) Robust statistics. Wiley & Sons, New York
Knight K., Fu W.J. (2000) Asymptotics for lasso-type estimators. Annals of Statistics 28: 1356–1378
Kong E., Xia Y.C. (2007) Variable selction for the single-index model. Biometrika 94: 217–229
Li K.C. (1991) Sliced inverse regression for dimension reduction (with discussion). Journal of the American Statistical Association 86: 316–342
Li K.C., Duan N.H. (1989) Regression analysis under link violation. Annals of Statistics 17: 1009–1052
Li L.X. (2007) Sparse sufficient dimension reduction. Biometrika 92: 603–613
Li L.X., Nachtsheim C.J. (2006) Sparse sliced inverse regression. Technometrics 48: 503–510
NiL. Cook R.D., Tsai C.L. (2005) A note on shrinkage sliced inverse regression. Biometrika 92: 242–247
Powell J.L., Stock J.H., Stoker T.M. (1989) Semiparametric estimation of index coeffcients. Econometrika 57: 1403–1430
Serfling R.J. (1980) Approximation theorems of mathematical statistics. John Wiley & Sons Inc., New York
Stein C. (1981) Estimation the mean of a multivariate normal distribution. Annals of Statistics 9: 1135–1151
Wang H., Leng C.L. (2007) Unified lasso estimation via least square approximation. Journal of the American Statistical Association 102: 1039–1048
Wang H., Li R., Tsai C.L. (2007) Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94: 553–568
Xia Y.C., Tong H., Li W.K., Zhu L.X. (2002) An adaptive estimation of optimal regression subspace. Journal of the Royal Statistical Society: Series B 64: 363–410
Zhu L.X., Fang K.T. (1996) Asymptotics for the kernel estimates of sliced inverse regression. Annals of Statistics 24: 1053–1067
Zhu L.P., Zhu L.X. (2007) On kernel method for sliced average variance estimation. Journal of Multivariate Analysis 98: 970–991
Zhu L.P., Zhu L.X. (2009) Nonconcave penalized inverse regression in single-index models with high dimensional predictors. Journal of Multivariate Analysis 100: 862–875
Zou H. (2006) The adaptive Lasso and its oracle properties. Journal of the American Statistical Association 101: 1418–1429
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Zhu, LP., Qian, LY. & Lin, JG. Variable selection in a class of single-index models. Ann Inst Stat Math 63, 1277–1293 (2011). https://doi.org/10.1007/s10463-010-0287-4
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-010-0287-4