Abstract
This paper is concerned with the selection and estimation of fixed effects in linear mixed models while the random effects are treated as nuisance parameters. We propose the non-penalty James–Stein shrinkage and pretest estimation methods based on linear mixed models for longitudinal data when some of the fixed effect parameters are under a linear restriction. We establish the asymptotic distributional biases and risks of the proposed estimators, and investigate their relative performance with respect to the unrestricted maximum likelihood estimator (UE). Furthermore, we investigate the penalty (LASSO and adaptive LASSO) estimation methods and compare their relative performance with the non-penalty pretest and shrinkage estimators. A simulation study for various combinations of the inactive covariates shows that the shrinkage estimators perform better than the penalty estimators in certain parts of the parameter space. This particularly happens when there are many inactive covariates in the model. It also shows that the pretest, shrinkage, and penalty estimators all outperform the UE. We further illustrate the proposed procedures via a real data example.
Similar content being viewed by others
References
Ahmed SE, Rohatgi VJ (1996) Shrinkage estimation in a randomized response model. Metrika 43:17–30
Ahmed SE, Hussein A, Sen PK (2006) Risk comparison of some shrinkage M-estimators in linear models. J Nonparametr Stat 18(3):401–415
Ahmed SE, Hossain S, Doksum KA (2012) LASSO and shrinkage estimation in Weibull censored regression models. J Stat Plan Inference 142(6):1273–1284
Ahn M, Zhang HH, Lu W (2012) Moment-based method for random effects selection in linear mixed models. Stat Sin 22(4):1539–1562
Bondell HD, Krishna A, Ghosh SK (2010) Joint variable selection for fixed and random effects in linear mixed-effects models. Biometrics 66(4):1069–1077
Datta G, Ghosh M (2012) Small area shrinkage estimation. Stat Sci 27(1):95–114
Diggle PJ, Heagerty P, Liang KY, Zeger SL (2002) Analysis of longitudinal data. Oxford University Press, Oxford
Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96(456):1348–1360
Fan YY, Li RZ (2012) Variable selection in linear mixed effects models. Ann Stat 40(4):2043–2068
Fitzmaurice GM, Laird NM, Ware JH (2011) Longitudinal data analysis. Wiley, New York
Gao X, Ahmed SE, Fen Y (2017) Post selection shrinkage estimation for high-dimensional data analysis. Appl Stoch Models Bus Ind 33:97–120
Gumedze FN, Dunne TT (2011) Parameter estimation and inference in the linear mixed model. Linear Algebra Appl 435(8):1920–1944
Gupta A, Saleh AKME, Sen PK (1989) Improved estimation in a contingency table: independence structure. J Am Stat Assoc 84(406):525–532
Harville D (1977) Maximum likelihood approaches to variance component estimation and to related problems. J Am Stat Assoc 358:320–340
Hedeker D, Gibbons RD (2006) Longitudinal data analysis. Wiley, New York
Hossain S, Doksum KA, Ahmed SE (2009) Positive-part shrinkage and absolute penalty estimators in partially linear models. Linear Algebra Appl 430:2749–2761
Hossain S, Ahmed SE, Doksum KA (2015) Shrinkage, \(l_1\) penalty and pretest estimators for generalized linear models. Stat Methodol 24:52–68
Hossain S, Ahmed SE, Yi Y, Chen B (2016) Shrinkage and pretest estimators for longitudinal data analysis under partially linear models. J Nonparametr Stat 28(3):531–549
Ibrahim JG, Zhu HT, Garcia R, Guo RX (2011) Fixed and random effects selection in mixed effects models. Biometrics 67(2):495–503
Judge GG, Bock ME (1978) The statistical implication of pretest and Stein-rule estimators in econometrics. North-Holland, Amsterdam
Judge GG, Mittelhammaer RC (2004) A semiparametric basis for combining estimation problem under quadratic loss. J Am Stat Assoc 99(466):479–487
Kemper HCG (1995) The Amsterdam Growth Study: a longitudinal analysis of health, fitness and lifestyle. HK Sport Science Monograph Series, vol 6. Human Kinetics Publishers, Champaign
Laird NM, Ware JH (1982) Random-effects models for longitudinal data. Biometrics 38(4):963–974
Lindstrom MJ, Bates DM (1988) Newton–Raphson and EM algorithms for linear mixed-effects models for repeated measures data. J Am Stat Assoc 83(404):1014–1022
Longford NT (1993) Random coefficient models. Oxford University Press, Oxford
McCulloch CE, Searle SR, Neuhaus JR (2008) Generalized, linear, and mixed models, 2nd edn. Wiley, Hoboken, New Jersey
Müller S, Scealy JL, Welsh AH (2013) Model selection in linear mixed models. Stat Sci 28(2):135–167
Pan J, Shang J (2017) Adaptive LASSO for linear mixed model selection via profile log-likelihood. Commun Stat Theory Methods. https://doi.org/10.1080/03610926.2017.1332219
Peng H, Lu Y (2012) Model selection in linear mixed effect models. J Multivar Anal 109:109–129
Schelldorfer J, Bühlmann P, van de Geer S (2011) Estimation for high-dimensional linear mixed effects models using \(l_1\)-penalization. Scand J Stat 38(2):197–214
Searle SR, Casella G, McCulloch CE (2006) Variance components. Wiley, New York
Thomson T, Hossain S, Ghahramani M (2015) Application of shrinkage estimation in linear regression models with autoregressive errors. J Stat Comput Simul 85(16):1580–1592
Tibshirani R (1996) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58(1):267–288
Twisk JWR (2003) Applied longitudinal data analysis for epidemiology—a practical guide. Cambridge University Press, New York
Verbeke G, Molenberghs G (2009) Linear mixed models for longitudinal data. Springer Series in Statistics: corrected edition, New York
Zeng T, Hill RC (2016) Shrinkage estimation in the random parameters logit model. Open J Stat 6:667–674
Zou H (2006) The adaptive lasso and its oracle properties. J Am Stat Assoc 101(476):1418–1429
Acknowledgements
We would like to thank the referees, editor and associate editor for their valuable suggestions in the revision of this paper. The research of Shakhawat Hossain and Ejaz Ahmed was supported by the Natural Sciences and the Engineering Research Council of Canada.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hossain, S., Thomson, T. & Ahmed, E. Shrinkage estimation in linear mixed models for longitudinal data. Metrika 81, 569–586 (2018). https://doi.org/10.1007/s00184-018-0656-1
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00184-018-0656-1