Abstract
In this paper, we propose to jointly model the conditional mean and variance components associated with the response in multilevel data. We set a generalized linear mixed model (GLMM) for the mean and a generalized linear model (GLM) for the variance components. The variable selection method of our choice is the smoothly clipped absolute deviation (SCAD) penalty, a penalized likelihood variable selection procedure, which shrinks the coefficients of redundant variables to 0 while simultaneously estimating the coefficients of the remaining important covariates. To assess the performance of the proposed procedures, we carry out real data analysis as well as extensive simulation studies, and compare to a similar process which excludes variable selection. We conclude that our method outperforms a simple joint mean-variance modelling approach, in both identifying the important components in the joint models and also producing more efficient estimation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aitkin M., Longford N. (1986) Statistical modelling issues in school effectiveness studies. J. Royal Stat. Soc. Series A, 144: 1–43
Antoniadis A. (1997) Wavelets in statistics: a review (with discussion). J. Italian Stat. Assoc., 6: 97–144
Bondell H.D., Krishna A., Ghosh S.K. (2010) Joint variable selection for fixed and random effects in linear mixed-effects models. Biometrics, 66: 1069–1077
Breiman L. (1996) Heuristics of instability and stabilization in model selection. Ann. Stat., 24: 2350–2383
Breslow N.E., Clayton D.G. (1993) Approximate inference in generalized linear mixed models. J. Am. Stat. Assoc., 88: 9–25
Cai B., Dunson D.B. (2006) Bayesian covariance selection in generalized linear mixed models. Biometrics, 62: 446–457
Chen Z., Dunson D.B. (2003) Random effects selection in linear mixed models. Biometrics, 59: 762–769
Chiu T.Y.M., Leonard T., Tsui K.W. (1996) The matrix-logarithmic covariance model. J. Am. Stat. Assoc., 91: 198–210
Cox D.R., Hinkley D.V. (1974) Theoretical statistics, 1st ed, 1st edn. Chapman and Hall, London
Daniels M.J., Zhao Y.D. (2003) Modelling the random effects covariance matrix in longitudinal data. Stat. Med., 22: 1631–1647
Fan J., Li R. (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc., 96: 1348–1360
Frank I.E., Friedman J.H. (1993) A statistical view of some chemometrics regression tools. Technometrics, 35: 109–148
Gelman A., Hill J. (2006) Data analysis using regression and multilevel/hierarchical models. Cambridge University Press
Goldstein H. (1991) Nonlinear multilevel models, with an application to discrete response data. Biometrika, 78: 45–51
Goldstein H. (1995) Multilevel statistical models, 2nd ed. London: Edward Arnold; New York: Halstead Press
Hall P., Lee E.R., Park B.U. (2009) Bootstrap-based penalty choice for the LASSO, achieving oracle performance. Statistica Sinica, 19: 449–471
Harvey A.C. (1976) Estimating regression models with multiplicative heteroscedasticity. Econometrica, 44: 461–465
Hoerl A.E., Kennard R.W. (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12: 55–67
Ibrahim J.G., Zhu H., Garcia R.I., Guo R. (2010) Fixed and random effects selection in mixed effects models. Biometrics, 67: 495–503
Kou C., Pan J.X. (2009) Variable selection for joint mean and covariance models via penalized likelihood. http://www.manchester.ac.uk/mims/eprints. Accessed 18 March 2013
Lee Y., Nelder J.A. (2006) Double hierarchical generalized linear models (with discussion). Applied Statistics, 55: 139–185
Lee Y., Nelder J.A., Pawitan Y. (2006) Generalized linear models with random effects, unified analysis via H-likelihood. Chapman and Hall/CRC, Boca Raton
Lehmann E.L. (1983) Theory of point estimation. Wadsworth and Brooks/Cole, Pacific Grove
McCullagh P., Nelder J.A. (1989) Generalized linear models, 2nd ed. Chapman and Hall, London
Ni X., Zhang D., Zhang H.H. (2010) Variable selection for semiparametric mixed models in longitudinal studies. Biometrics, 66: 79–88
Pan J.X., MacKenzie G. (2003) Model selection for joint mean-covariance structures in longitudinal studies. Biometrika, 90: 239–244
Pan J.X., MacKenzie G. (2006) Regression models for covariance structures in longitudinal studies. Stat. Model., 6: 43–57
Pan J.X., MacKenzie G. (2007) Modelling conditional covariance in the linear mixed model. Stat. Model., 7: 49–71
Park R.E. (1966) Estimation with heteroscedastic error terms. Econometrica, 34: 888
Phadke M.S., Kackar R.N., Speeney D.V., Grieco M.J. (1983) Off-line quality control for integrated circuit fabrication using experimental design. Bell Syst. Tech. J., 62: 1273–1309
Pötscher B.M., Leeb H. (2009) On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD and thresholding. J. Multivariate Anal., 100: 2065–2082
Pötscher B.M., Schneider U. (2010) Confidence sets based on penalized maximum likelihood estimators in Gaussian regression. Electron. J. Stat., 4: 334–360
Pourahmadi M. (1999) Joint mean-covariance models with applications to longitudinal data: unconstrained parameterization. Biometrika, 86: 677–690
Pourahmadi M. (2000) Maximum likelihood estimation of generalised linear models for multivariate normal covariance matrix. Biometrika, 87: 425–435
Raudenbush S. W., Bryk A. S. (2013) Hierarchical linear models: applications and data analysis methods, 2nd ed. Sage Publications Inc, Newbury Park
Rigby R.A., Stasinopoulos D.M. (1996) A semi-parametric additive model for variance heterogeneity. Stat. Comput., 6: 57–65
Rigby R.A., Stasinopoulos D.M. (2005) Generalized additive models for location, scale and shape. Appl. Stat., 54: 507–554
Rutemiller H.C., Bowers D.A. (1968) Estimation in a heteroscedastic regression model. J. Am. Stat. Assoc., 63: 552–557
Scott M. A., Simonoff J. S., Marx B. D. (2013) The SAGE handbook of multilevel modeling. Sage Publications Ltd, London
Smyth G.K., Verbyla A.P. (2009) Leverage adjustments for dispersion modelling in generalized nonlinear models. Australian New Zealand J. Stat. 51: 433–448
Tibshirani R.J. (1996) Regression shrinkage and selection via the LASSO. J. Royal Stat. Soc. Series B, 58: 267–288
Vonesh E.F. (1996) A note on the use of Laplace’s approximation for nonlinear mixed-effects models. Biometrika, 83: 447–452
Wang H., Li R., Tsai C.-L. (2007) Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika, 94: 553–568
Wolfinger R.D., Tobias R.D. (1998) Joint estimation of location, dispersion, and random effects in robust design. Technometrics, 40: 62–71
Ye H.J., Pan J.X. (2006) Modelling of covariance structures in generalized estimating equations for longitudinal data. Biometrika, 93: 927–941
Zhang Y., Li R., Tsai C.-L. (2010) Regularization parameter selections via generalized information criterion. J. Am. Stat. Assoc., 105: 312–323
Zhang Z.Z., Wang D.R. (2011) Simultaneous variable selection for heteroscedastic regression models. Sci. China Math., 54: 515–530
Zou H. (2006) The adaptive lasso and its oracle properties. J. Am. Stat. Assoc., 101: 1418–1429
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Charalambous, C., Pan, J. & Tranmer, M. Variable Selection in Joint Mean and Dispersion Models via Double Penalized Likelihood. Sankhya B 76, 276–304 (2014). https://doi.org/10.1007/s13571-014-0079-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13571-014-0079-6
Keywords and phrases
- Generalized linear mixed models
- hierarchical data
- h-likelihood
- modelling of mean and covariance structures
- smoothly clipped absolute deviation penalty.