Weak Signals in High-Dimensional Logistic Regression Models
In this work, we addressed parameter estimation and prediction in the high-dimensional sparse logistic regression model through both Monte Carlo simulations and application to real data. We applied two well-known penalized maximum likelihood (ML) methods (LASSO and aLASSO) for variable screening. There may exist overfitting from LASSO or underfitting from aLASSO, making ML estimators based on these models inefficient. Hence, after performing variable selection, we proposed post-selection improved estimation based on linear shrinkage, pretest, and James-Stein shrinkage strategies, which efficiently combine overfitted and underfitted ML estimators. Regardless of the correctness in the variable selection stage, the proposed estimators were shown to be more efficient than the classical ML estimators, which were severely affected by inappropriate variable selection.
KeywordsHigh-dimensional sparse logistic Monte Carlo simulation Penalized maximum likelihood Linear shrinkage Pretest James-Stein shrinkage
The research of Professor S. Ejaz Ahmed was partially supported by the Natural Sciences and Engineering Research Council of Canada.
- 1.Agresti, A.: Foundations of Linear and Generalized Linear Models. Wiley, New York (2015)Google Scholar
- 3.Ahmed, S.E.: Penalty, Shrinkage and Pretest Strategies: Variable Selection and Estimation. Springer (2014)Google Scholar
- 4.Ahmed, S.E., Yüzbaşı, B.: Big data analytics: integrating penalty strategies. Int. J. Manag. Sci. Eng. Manag. 11(2), 105–115 (2016)Google Scholar
- 11.Li, Y., Hong, H.G., Ahmed, S.E., Li, Y.: Weak signals in high-dimensional regression: Detection, estimation and prediction. Appl. Stoch. Model. Bus. Ind. (2018)Google Scholar
- 13.Myers, R.H., Montgomery, D.C., Vining, G.G., Robinson, T.J.: Generalized Linear Models: With Applications in Engineering and the Sciences, vol. 791. Wiley, New York (2012)Google Scholar
- 14.Reangsephet, O., Lisawadi, S., Ahmed, S.E.: A comparison of pretest, stein-type and penalty estimators in logistic regression model. In: International Conference on Management Science and Engineering Management, pp. 19–34. Springer (2017)Google Scholar
- 15.Reangsephet, O., Lisawadi, S., Ahmed, S.E.: Improving estimation of regression parameters in negative binomial regression model. In: International Conference on Management Science and Engineering Management, pp. 265–275. Springer (2018)Google Scholar
- 16.Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 267–288 (1996)Google Scholar
- 17.Towell, G.G., Shavlik, J.W., Noordewier, M.O.: Refinement of approximate domain theories by knowledge-based neural networks. In: Proceedings of the Eighth National Conference on Artificial Intelligence, Boston, MA (1990)Google Scholar
- 18.Yuzbasi, B., Arashi, M., Ahmed, S.E.: Big data analysis using shrinkage strategies (2017). arXiv:170405074
- 19.Yüzbaşı, B., Arashi, M., Ahmed, S.E.: Shrinkage estimation strategies in generalized ridge regression models under low/high-dimension regime (2017). arXiv:170702331