In this paper, the high-dimensional sparse linear regression model is considered, where the overall number of variables is larger than the number of observations. Many penalized regularization approaches including LASSO, group LASSO, and Elastic-Net, typically focus on selecting variables with strong effects. This may result in biased prediction, especially when weak signals outnumber strong signals. To solve this problem, we incorporate weak signals in variable selection and estimation. We propose a two-stage procedure, consisting of variable selection and post-selection estimation. The variable selection is done using the LASSO and Elastic-Net penalties to detect weak signals, whereas the post-selection estimation involves by shrinking a post-selection weighted ridge estimator in the direction of a selected candidate subset from the first stage. Monte-Carlo simulation experiment is conducted to evaluate the performance of each estimator in terms of the relative mean squared error. As a particular example, we apply the proposed method to analyze the GDP growth data.
Sparse estimation Data LASSO Elastic-Net Post-selection shrinkage estimation
This is a preview of subscription content, log in to check access.
Belloni, A., Chernozhukov, V.: Least squares after model selection in high-dimensional sparse models. Bernoulli 19(2), 521–547 (2009)MathSciNetCrossRefGoogle Scholar
Gao, X., Ahmed, S.E., Feng, Y.: Post selection shrinkage estimation for high-dimensional data analysis. Appl. Stoch. Models Bus. Ind. 33, 97–120 (2006)MathSciNetzbMATHGoogle Scholar
Leng, C., Lin, Y., Wahba, G.: Selection a note on the lasso and related procedures in model selection. Stat. Sin. 16(4), 1273–1284 (2006)MathSciNetzbMATHGoogle Scholar
Liu, H., Yu, B.: Asymptotic properties of Lasso+mLS and Lasso+Ridge in sparse high-dimensional linear regression. Electron. J. Stat. 7, 3124–3169 (2006)MathSciNetCrossRefGoogle Scholar