Stable direction recovery in single-index models with a diverging number of predictors

Abstract

Large dimensional predictors are often introduced in regressions to attenuate the possible modeling bias. We consider the stable direction recovery in single-index models in which we solely assume the response Y is independent of the diverging dimensional predictors X when β ^T₀ X is given, where β ₀ is a p _n × 1 vector, and p _n → ∞ as the sample size n → ∞. We first explore sufficient conditions under which the least squares estimation β _n0 recovers the direction β ₀ consistently even when p _n = o($\sqrt n$). To enhance the model interpretability by excluding irrelevant predictors in regressions, we suggest an ℓ ₁-regularization algorithm with a quadratic constraint on magnitude of least squares residuals to search for a sparse estimation of β ₀. Not only can the solution β _n of ℓ ₁-regularization recover β ₀ consistently, it also produces sufficiently sparse estimators which enable us to select “important” predictors to facilitate the model interpretation while maintaining the prediction accuracy. Further analysis by simulations and an application to the car price data suggest that our proposed estimation procedures have good finite-sample performance and are computationally efficient.