Shrinkage and Sparse Estimation for High-Dimensional Linear Models

  • M. Noori Asl
  • H. Bevrani
  • R. Arabi Belaghi
  • Syed Ejaz Ahmed
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1001)


In this paper, the high-dimensional sparse linear regression model  is considered, where the overall number of variables is larger than the number of observations. Many penalized regularization approaches including LASSO, group LASSO, and Elastic-Net, typically focus on selecting variables with strong effects. This may result in biased prediction, especially when weak signals outnumber strong signals. To solve this problem, we incorporate weak signals in variable selection and estimation. We propose a two-stage procedure, consisting of variable selection and post-selection estimation. The variable selection is done using the LASSO and Elastic-Net penalties to detect weak signals, whereas the post-selection estimation involves by shrinking a post-selection weighted ridge estimator in the direction of a selected candidate subset from the first stage. Monte-Carlo simulation experiment is conducted to evaluate the performance of each estimator in terms of the relative mean squared error. As a particular example, we apply the proposed method to analyze the GDP growth data.


Sparse estimation Data LASSO Elastic-Net Post-selection shrinkage estimation 


  1. 1.
    Belloni, A., Chernozhukov, V.: Least squares after model selection in high-dimensional sparse models. Bernoulli 19(2), 521–547 (2009)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Gao, X., Ahmed, S.E., Feng, Y.: Post selection shrinkage estimation for high-dimensional data analysis. Appl. Stoch. Models Bus. Ind. 33, 97–120 (2006)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Leng, C., Lin, Y., Wahba, G.: Selection a note on the lasso and related procedures in model selection. Stat. Sin. 16(4), 1273–1284 (2006)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Liu, H., Yu, B.: Asymptotic properties of Lasso+mLS and Lasso+Ridge in sparse high-dimensional linear regression. Electron. J. Stat. 7, 3124–3169 (2006)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B 58(1), 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Zou, H.: The adaptive Lasso and its oracle properties. J. Am. Stat. Assoc. 101(476), 1418–1429 (2006)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B 67(2), 301–320 (2005)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • M. Noori Asl
    • 1
  • H. Bevrani
    • 1
  • R. Arabi Belaghi
    • 1
  • Syed Ejaz Ahmed
    • 2
  1. 1.Department of Statistics, Faculty of Mathematical ScienceUniversity of TabrizTabrizIran
  2. 2.Department of Mathematics & StatisticsBrock UniversitySt. CatharinesCanada

Personalised recommendations