A Comparison of Pretest, Stein-Type and Penalty Estimators in Logistic Regression Model

  • Orawan ReangsephetEmail author
  • Supranee Lisawadi
  • Syed Ejaz Ahmed
Conference paper
Part of the Lecture Notes on Multidisciplinary Industrial Engineering book series (LNMUINEN)


Various estimators are proposed based on the preliminary test and Stein-type strategies to estimate the parameters in a logistic regression model when it is priori suspected that some parameters may be restricted to a subspace. Two different penalty estimators as LASSO and ridge regression are also considered. A Monte Carlo simulation experiment was conducted for different combinations, and the performance of each estimator was evaluated in terms of simulated relative efficiency. The positive-part Stein-type shrinkage estimator is recommended for use since its performance is robust regardless of the reliability of the subspace information. The proposed estimators are applied to a real dataset to appraise their performance.


Monte Carlo simulation Logistic regression model Likelihood ratio test Preliminary test estimator Shrinkage estimator Penalty estimator 



We would like to thank all the referees, the Editor, and the Associate Editor for their valuable suggestions on the revision of the article.


  1. 1.
    Ahmed SE (1992) Shrinkage preliminary test estimation in multivariate normal distributionsm. J Stat Comput Simul 43(3–4):177–195Google Scholar
  2. 2.
    Ahmed SE, Amezziane M (2016) Shrinkage-based semiparametric density estimation. Statistical MethodologyGoogle Scholar
  3. 3.
    Ahmed SE, Raheem E, Hossain S (2011) Absolute Penalty Estimation. Springer International Encyclopedia of statistical science, New YorkGoogle Scholar
  4. 4.
    Ahmed SE, Yüzbaşı B (2016) Big data analytics: integrating penalty strategies. International Journal of Management Science & Engineering Management 11(2016):105–115Google Scholar
  5. 5.
    Al-Momani M, Hussein AA, Ahmed SE (2016) Penalty and related estimation strategies in the spatial error model: estimation strategies in sem modelGoogle Scholar
  6. 6.
    Gao X, Ahmed SE, Feng Y (2016) Post selection shrinkage estimation for high-dimensional data analysis. Appl Stoch Models Bus IndGoogle Scholar
  7. 7.
    Gourieroux CS, Monfort A (1981) Asymptotic properties of the maximum likelihood estimator in dichotomous logit models. J Econometrics 17(1):83–97Google Scholar
  8. 8.
    Hilbe JM (2009) Logistic Regression Models. Chapman and Hall, Boca RatonGoogle Scholar
  9. 9.
    Hoerl AE, Kennard RW (2000) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 42(1):80–86Google Scholar
  10. 10.
    Hosmer DW, Lemeshow S (2000) Applied Logistic Regression. WileyGoogle Scholar
  11. 11.
    Hossain S, Ahmed S (2014) Shrinkage estimation and selection for a logistic regression modelGoogle Scholar
  12. 12.
    Hossain S, Ahmed SE, Doksum KA (2014) Shrinkage, pretest, and penalty estimators in generalized linear models. Stat Methodol 24(2):52–68Google Scholar
  13. 13.
    Lisawadi S, Shah MKA, Ahmed SE (2016) Model selection and post estimation based on a pretest for logistic regression models. J Stat Comput Simul 86(17):1–17Google Scholar
  14. 14.
    Rossouw JE, Du PJ, et al (1983) Coronary risk factor screening in three rural communities. The CORISs baseline study. S Afr Med J 64(12):430–436Google Scholar
  15. 15.
    Tibshirani R (2011) Regression shrinkage and selection via the lasso. J Roy Stat Soc 58(3):267–288Google Scholar
  16. 16.
    Yüzbaşı B, Ahmed SE (2015) Shrinkage ridge regression estimators in high-dimensional linear models. Springer, HeidelbergGoogle Scholar
  17. 17.
    Yüzbaşı B, Ahmed SE (2016) Shrinkage and penalized estimation in semi-parametric models with multicollinear data. J Stat Comput SimulGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Orawan Reangsephet
    • 1
    Email author
  • Supranee Lisawadi
    • 1
  • Syed Ejaz Ahmed
    • 2
  1. 1.Department of Mathematics and StatisticsThammasat UniversityBangkokThailand
  2. 2.Faculty of Mathematics and ScienceBrock UniversitySt. CatharinesCanada

Personalised recommendations