Skip to main content
SpringerLink
Log in

Efficient Shrinkage for Generalized Linear Mixed Models Under Linear Restrictions

  • Published:
Sankhya A Aims and scope Submit manuscript

Abstract

In this paper, we consider the pretest, shrinkage, and penalty estimation procedures for generalized linear mixed models when it is conjectured that some of the regression parameters are restricted to a linear subspace. We develop the statistical properties of the pretest and shrinkage estimation methods, which include asymptotic distributional biases and risks. We show that the pretest and shrinkage estimators have a significantly higher relative efficiency than the classical estimator. Furthermore, we consider the penalty estimator LASSO (Least Absolute Shrinkage and Selection Operator), and numerically compare its relative performance with that of the other estimators. A series of Monte Carlo simulation experiments are conducted with different combinations of inactive predictors, and the performance of each estimator is evaluated in terms of the simulated mean squared error. The study shows that the shrinkage and pretest estimators are comparable to the LASSO estimator when the number of inactive predictors in the model is relatively large. The estimators under consideration are applied to a real data set to illustrate the usefulness of the procedures in practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahmed, S., Hussein, A. and Sen, P. (2006). Risk comparison of some shrinkage M-estimators in linear models. Journal of Nonparametric Statistics 18, 401–415.

    Article  MathSciNet  MATH  Google Scholar 

  • Ahmed, S. E., Hossain, S. and Doksum, K. A. (2012). LASSO And shrinkage estimation in Weibull censored regression models. Journal of Statistical Planning and Inference 142, 1273–1284.

    Article  MathSciNet  MATH  Google Scholar 

  • Arnold, T. B. and Tibshirani, R. J. (2016). Efficient implementations of the generalized Lasso dualpath algorithm. J. Comput. Graph. Stat. 25, 1–27.

    Article  Google Scholar 

  • Chen, F. and Nkurunziza, S. (2015). A class of Stein-rules in multivariate regression model with structural changes. Scand. J. Stat. 43, 83–102.

    Article  MathSciNet  MATH  Google Scholar 

  • Davis, K. A., Park, C. G. and Sinha, S. K. (2012). Testing for generalized linear mixed models with cluster correlated data under linear inequality constraints. Can. J. Stat. 40, 243–258.

    Article  MathSciNet  MATH  Google Scholar 

  • Diggle, P. J., Heagerty, P., Liang, K -Y. and Zeger, S. L. (2002). Analysis of Longitudinal Data, 2nd edn. Oxford University Press, Oxford.

    MATH  Google Scholar 

  • Fahrmeir, L. and Kneib, T. (2011). Bayesian smoothing and regression for longitudinal, spatial and event history data. Oxford University Press, Oxford.

    Book  MATH  Google Scholar 

  • Fallahpour, S., Ahmed, S. E. and Doksum, K. A. (2012). ℓ 1 penalty and shrinkage estimation in partially linear models with random coefficient autoregressive errors. Appl. Stoch. Model. Bus. Ind. 28, 236–250.

    Article  MathSciNet  MATH  Google Scholar 

  • Fan, J. and Lv, J. (2010). A selective overview of variable selection in high dimensional feature space. Stat. Sin. 20, 101–148.

    MathSciNet  MATH  Google Scholar 

  • Goeman, J. J. (2010). Penalized estimation in the Cox proportional hazards model. Biom. J. 52, 70–84.

    MathSciNet  MATH  Google Scholar 

  • Groll, A. and Tutz, G. (2014). Variable selection for generalized linear mixed models by l 1 penalized estimation. Stat. Comput. 24, 137–154.

    Article  MathSciNet  MATH  Google Scholar 

  • Hossain, S. and Ahmed, S. E. (2014). Shrinkage estimation and selection for a logistic regression model. CRM Proceedings-Contemporary Mathematics 622, 159–176.

    Article  MathSciNet  MATH  Google Scholar 

  • Hossain, S., Ahmed, S. E. and Doksum, K. A. (2015). Shrinkage, pretest, and penalty estimators in generalized linear models. Stat. Methodology 24, 52–68.

    Article  MathSciNet  Google Scholar 

  • Jamshidian, M. (2004). On algorithms for restricted maximum likelihood estimation. Comput. Stat. Data Anal. 45, 137–157.

    Article  MathSciNet  MATH  Google Scholar 

  • Judge, G. G. and Bock, M. E. (1978). The statistical implication of pretest and Stein-Rule estimators in econometrics. North-Holland, Amsterdam.

    MATH  Google Scholar 

  • Li, Q. and Shao, J. (2015). Regularizing LASSO: A consistent variable selection method. Stat. Sin. 25, 975–992.

    MathSciNet  MATH  Google Scholar 

  • Lian, H. (2012). Shrinkage estimation for identification of linear components in additive models. Statistics & Probability Letters 82, 225–231.

    Article  MathSciNet  MATH  Google Scholar 

  • Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. J. R. Stat. Soc. Ser. B Methodol. 44, 226–233.

    MathSciNet  MATH  Google Scholar 

  • Lu, X. and Su, L. (2016). Shrinkage estimation of dynamic panel data models with interactive fixed effects. J. Econ. 190, 148–175.

    Article  MathSciNet  MATH  Google Scholar 

  • Luenberger, D. G. and Ye, Y. (2008). Linear and nonlinear programming. Springer Science & Business Media, New York.

    MATH  Google Scholar 

  • McCulloch, C. E., Searle, S. R. and Neuhaus, J. M. (2008). Generalized, Linear, and Mixed Models. Wiley, Hoboken.

    MATH  Google Scholar 

  • Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. R. Stat. Soc. Ser. A 135, 370–384.

    Article  Google Scholar 

  • Schelldorfer, J., Meier, L. and Bühlmann, P. (2014). GLMMLasso: an algorithm for high-dimensional generalized linear mixed models using l 1-penalization. J. Comput. Graph. Stat. 23, 460–477.

    Article  Google Scholar 

  • Simon, N., Friedman, J., Hastie, T. and Tibshirani, R. (2014). A sparse-group LASSO. J. Comput. Graph. Stat. 22, 231–245.

    Article  MathSciNet  Google Scholar 

  • Sommer, A., Katz, J. and Tarwotjo, I. (1984). Increased risk of respiratory infection and diarrhea in children with pre-existing mild vitamin a deficiency. Am. J. Clin. Nutr. 40, 1090–1095.

    Article  Google Scholar 

  • Thomson, T., Hossain, S. and Ghahramani, M. (2014). Application of shrinkage estimation in linear regression models with autoregressive errors. J. Stat. Comput. Simul. 85, 3335–3351.

    Article  MathSciNet  Google Scholar 

  • Thomson, T., Hossain, S. and Ghahramani, M. (2015). Efficient estimation for time series following generalized linear models 58, 493–513.

    Google Scholar 

  • Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Ser. B 58, 267–288.

    MathSciNet  MATH  Google Scholar 

  • Wu, C. O. and Chiang, C. (2000). Kernel smoothing on varying coefficient models with longitudinal dependent variable. Stat. Sin. 10, 433–456.

    MathSciNet  MATH  Google Scholar 

  • Zhang, T. and Zou, H. (2014). Sparse precision matrix estimation via LASSO penalized D-trace loss. Biometrika 101, 103–120.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Winnipeg, Winnipeg, MB, Canada

    T. Thomson

  2. Department of Mathematics and Statistics, University of Winnipeg, 515 Portage Ave, Winnipeg, MB, Canada

    S. Hossain

Authors
  1. T. Thomson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Hossain.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thomson, T., Hossain, S. Efficient Shrinkage for Generalized Linear Mixed Models Under Linear Restrictions. Sankhya A 80, 385–410 (2018). https://doi.org/10.1007/s13171-017-0122-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13171-017-0122-6

Keywords

AMS (2000) subject classification

Search

Navigation