Statistics and Computing

, Volume 28, Issue 4, pp 753–774 | Cite as

Extended differential geometric LARS for high-dimensional GLMs with general dispersion parameter

  • Hassan Pazira
  • Luigi Augugliaro
  • Ernst Wit


A large class of modeling and prediction problems involves outcomes that belong to an exponential family distribution. Generalized linear models (GLMs) are a standard way of dealing with such situations. Even in high-dimensional feature spaces GLMs can be extended to deal with such situations. Penalized inference approaches, such as the \(\ell _1\) or SCAD, or extensions of least angle regression, such as dgLARS, have been proposed to deal with GLMs with high-dimensional feature spaces. Although the theory underlying these methods is in principle generic, the implementation has remained restricted to dispersion-free models, such as the Poisson and logistic regression models. The aim of this manuscript is to extend the differential geometric least angle regression method for high-dimensional GLMs to arbitrary exponential dispersion family distributions with arbitrary link functions. This entails, first, extending the predictor–corrector (PC) algorithm to arbitrary distributions and link functions, and second, proposing an efficient estimator of the dispersion parameter. Furthermore, improvements to the computational algorithm lead to an important speed-up of the PC algorithm. Simulations provide supportive evidence concerning the proposed efficient algorithms for estimating coefficients and dispersion parameter. The resulting method has been implemented in our R package (which will be merged with the original dglars package) and is shown to be an effective method for inference for arbitrary classes of GLMs.


High-dimensional inference Generalized linear models Least angle regression Predictor–corrector algorithm Dispersion paremeter 



We would like to thank the editor and anonymous reviewers for valuable comments which improved the presentation of the paper.

Supplementary material

11222_2017_9761_MOESM1_ESM.pdf (66 kb)
Supplementary material 1 (pdf 65 KB)


  1. Aho, K., Derryberry, D., Peterson, T.: Model selection for ecologists: the worldviews of AIC and BIC. Ecology 95(3), 631–636 (2014)CrossRefGoogle Scholar
  2. Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19, 716–723 (1974)MathSciNetCrossRefMATHGoogle Scholar
  3. Allgower, E., Georg, K.: Introduction to Numerical Continuation Methods. Society for Industrial and Applied Mathematics, New York (2003)CrossRefMATHGoogle Scholar
  4. Arlot, S., Celisse, A.: A survey of cross-validation procedures for model selection. Stat. Surv. 4, 40–79 (2010)MathSciNetCrossRefMATHGoogle Scholar
  5. Augugliaro, L., Mineo, A.M., Wit, E.C.: Differential geometric least angle regression: a differential geometric approach to sparse generalized linear models. J. R. Stat. Soc. B 75(3), 471–498 (2013)MathSciNetCrossRefGoogle Scholar
  6. Augugliaro, L., Mineo, A.M., Wit, E.C.: dglars: an R package to estimate sparse generalized linear models. J. Stat. Softw. 59(8), 1–40 (2014a)CrossRefGoogle Scholar
  7. Augugliaro, L.: dglars: Differential Geometric LARS (dgLARS) Method. R package version 1.0.5. (2014b)
  8. Augugliaro, L., Mineo, A.M., Wit, E.C.: A differential geometric approach to generalized linear models with grouped predictors. Biometrika 103, 563–593 (2016)MathSciNetCrossRefGoogle Scholar
  9. Augugliaro, L., Pazira, H.: dglars: Differential Geometric Least Angle Regression. R package version 2.0.0. (2017)
  10. Burnham, K.P., Anderson, D.R.: Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach, \(2^{{\rm nd}}\) edn. Springer, New York (2002)Google Scholar
  11. Candes, E.J., Tao, T.: The dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2313–2351 (2007)MathSciNetCrossRefMATHGoogle Scholar
  12. Chen, Y., Du, P., Wang, Y.: Variable selection in linear models. Wiley Interdiscip. Rev. Comput. Stat. 6, 1–9 (2014)CrossRefGoogle Scholar
  13. Cordeiro, G.M., McCullagh, P.: Bias correction in generalized linear models. J. R. Stat. Soc. B 53(3), 629–643 (1991)MathSciNetMATHGoogle Scholar
  14. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)MathSciNetCrossRefMATHGoogle Scholar
  15. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)MathSciNetCrossRefMATHGoogle Scholar
  16. Fan, J., Lv, J.: Sure independence screening for ultrahigh dimensional feature space. J. R. Stat. Soc. B 70(5), 849–911 (2008)MathSciNetCrossRefGoogle Scholar
  17. Fan, J., Guo, S., Hao, N.: Variance estimation using refitted cross-validation in ultrahigh dimensional regression. J. R. Stat. Soc. B 74(1), 37–65 (2012)MathSciNetCrossRefGoogle Scholar
  18. Farrington, C.P.: On assessing goodness of fit of generalized linear model to sparse data. J. R. Stat. Soc. B 58(2), 349–360 (1996)MathSciNetMATHGoogle Scholar
  19. Friedman, J., Hastie, T., RTibshirani: glmnet: Lasso and Elastic-Net Regularized Generalized Linear Models. R Package Version 1.1-5. (2010b)
  20. Hastie, T., Efron, B.: lars: Least Angle Regression, Lasso and Forward Stagewise. R Package Version 1.2. (2013)
  21. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York (2009)CrossRefMATHGoogle Scholar
  22. Hoerl, A.E., Kennard, R.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)CrossRefMATHGoogle Scholar
  23. Ishwaran, H., Kogalur, U.B., Rao, J.: spikeslab: prediction and variable selection using spike and slab regression. R J. 2(2), 68–73 (2010a)Google Scholar
  24. Ishwaran, H., Kogalur, U.B., Rao, J.: spikeslab: prediction and variable selection using spike and slab regression. R package version 1.1.2. (2010b)
  25. James, G., Radchenko, P.: A generalized dantzig selector with shrinkage tuning. Biometrika 96, 323–337 (2009)MathSciNetCrossRefMATHGoogle Scholar
  26. Jorgensen, B.: Exponential dispersion models. J. R. Stat. Soc. B 49, 127–162 (1987)MathSciNetMATHGoogle Scholar
  27. Jorgensen, B.: The Theory of Dispersion Models. Chapman & Hall, London (1997)MATHGoogle Scholar
  28. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)MathSciNetCrossRefMATHGoogle Scholar
  29. Li, K.C.: Asymptotic optimality for \(c_p\), \(c_l\), cross-validation and generalized cross-validation: discrete index set. Ann. Stat. 15, 958–975 (1987)Google Scholar
  30. Littell, R.C., Stroup, W.W., Feund, R.J.: SAS for Linear Models, 4th edn. Sas Institute Inc., Cary (2002)Google Scholar
  31. McCullagh, P., Nelder, J.A.: Generalized Liner Models. Chapman & Hall, London (1989)CrossRefMATHGoogle Scholar
  32. McQuarrie, A.D.R., Tsai, C.L.: Regression and Time Series Model Selection, 1st edn. World Scientific Publishing Co. Pte. Ltd, Singapore (1998)CrossRefMATHGoogle Scholar
  33. Meng, R.: Estimation of dispersion parameters in glms with and without random effects. Master’s thesis, Stockholm University (2004)Google Scholar
  34. Park, M.Y., Hastie, T.: glmpath: \(L_1\) Regularization Path for Generalized Linear Models and Cox Proportional Hazards Model. R Package Version 0.94. (2007b)
  35. Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Numerical Recipes in Fortran 77: The Art of Scientific Computing, 2nd edn. Cambridge University Press, England (1992)MATHGoogle Scholar
  36. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)MathSciNetCrossRefMATHGoogle Scholar
  37. Shao, J.: An asymptotic theory for linear model selection. Stat. Sin. 7, 221–264 (1997)MathSciNetMATHGoogle Scholar
  38. Shibata, R.: An optimal selection of regression variables. Biometrika 68, 45–54 (1981)MathSciNetCrossRefMATHGoogle Scholar
  39. Shibata, R.: Approximation efficiency of a selection procedure for the number of regression variables. Biometrika 71, 43–49 (1984)MathSciNetCrossRefMATHGoogle Scholar
  40. Stone, M.: Asymptotics for and against cross-validation. Biometrika 64, 29–35 (1977)MathSciNetCrossRefMATHGoogle Scholar
  41. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58(1), 267–288 (1996)MathSciNetMATHGoogle Scholar
  42. Ultricht, J., Tutz, G.: Combining quadratic penalization and variable selection via forward boosting. Tech. Rep., Department of Statistics, Munich University, Technical Reports No. 99 (2011)Google Scholar
  43. Vos, P.W.: A geometric approach to detecting influential cases. Ann. Stat. 19, 1570–1581 (1991)MathSciNetCrossRefMATHGoogle Scholar
  44. Whittaker, E.T., Robinson, G.: The Calculus of Observations: An Introduction to Numerical Analysis, 4th edn. Dover Publications, New York (1967)Google Scholar
  45. Wood, S.N.: Generalized Additive Models: An Introduction with R. Chapman & Hall/CRC, Boca Raton (2006)MATHGoogle Scholar
  46. Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)MathSciNetCrossRefMATHGoogle Scholar
  47. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 67(2), 301–320 (2005a)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.University of GroningenGroningenThe Netherlands
  2. 2.University of PalermoPalermoItaly

Personalised recommendations