Advertisement

Statistics and Computing

, Volume 29, Issue 3, pp 559–569 | Cite as

Selecting the tuning parameter in penalized Gaussian graphical models

  • Antonino AbbruzzoEmail author
  • Ivan Vujačić
  • Angelo M. Mineo
  • Ernst C. Wit
Article
  • 193 Downloads

Abstract

Penalized inference of Gaussian graphical models is a way to assess the conditional independence structure in multivariate problems. In this setting, the conditional independence structure, corresponding to a graph, is related to the choice of the tuning parameter, which determines the model complexity or degrees of freedom. There has been little research on the degrees of freedom for penalized Gaussian graphical models. In this paper, we propose an estimator of the degrees of freedom in \(\ell _1\)-penalized Gaussian graphical models. Specifically, we derive an estimator inspired by the generalized information criterion and propose to use this estimator as the bias term for two information criteria. We called these tuning parameter selectors GAIC and GBIC. These selectors can be used to choose the tuning parameter, i.e., the optimal tuning parameter is the minimizer of GAIC or GBIC. A simulation study shows that GAIC tends to improve the performance of both AIC-type and CV-type model selectors, in terms of estimation quality (entropy loss function) in high-dimensional setting. Moreover, GBIC model selector improves the performance of both BIC-type and CV-type model selectors, in terms of support recovery (F-score). A data analysis shows that GBIC selects a tuning parameter that produces a sparser graph with respect to BIC and a CV-type model selector (KLCV).

Keywords

Penalized likelihood Kullback–Leibler divergence Model complexity Model selection Generalized information criterion 

Notes

Acknowledgements

The project was partially supported by the “European Cooperation in Science & Technology” (COST) funding: action number CA15109.

Supplementary material

11222_2018_9823_MOESM1_ESM.pdf (83 kb)
Supplementary material 1 (pdf 82 KB)

References

  1. Bogdan, M., Ghosh, J.K., Doerge, R.W.: Modifying the Schwarz Bayesian information criterion to locate multiple interacting quantitative trait loci. Genetics 167(2), 989–999 (2004)CrossRefGoogle Scholar
  2. Broman, K.W., Speed, T.P.: A model selection approach for the identification of quantitative trait loci in experimental crosses. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 64(4), 641–656 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  3. Cai, T., Liu, W., Luo, X.: A constrained \(\ell_1\) minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 1063(494), 594–607 (2011)CrossRefzbMATHGoogle Scholar
  4. Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31(4), 377–403 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  5. Chen, J., Chen, Z.: Extended Bayesian information criteria for model selection with large model spaces. Biometrika 95(3), 759–771 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  6. d’Aspremont, A., Banerjee, O., El Ghaoui, L.: First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30(1), 56–66 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  7. Dong, X., Wahba, G.: A generalized approximate cross validation for smoothing splines with non-Gaussian data. Stat. Sin. 6(3), 675–692 (1996)MathSciNetzbMATHGoogle Scholar
  8. Edwards, D.: Introduction to Graphical Modelling. Springer, Berlin (2000)CrossRefzbMATHGoogle Scholar
  9. Fan, J., Feng, Y., Wu, Y.: Network exploration via the adaptive lasso and scad penalties. Ann. Appl. Stat. 3(2), 521–541 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  10. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  11. Foygel, R., Drton, M.: Extended bayesian information criteria for gaussian graphical models. Adv. Neural Inf. Process. Syst. 23, 604–612 (2010)Google Scholar
  12. Fried, R., Vogel, D.: On robust gaussian Graphical Modelling. Recent Developments in Applied Probability and Statistics. Springer, Berlin (2009)Google Scholar
  13. Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)CrossRefzbMATHGoogle Scholar
  14. Gao, X., Pu, D.Q., Wu, Y., Xu, H.: Tuning parameter selection for penalized likelihood estimation of Gaussian graphical model. Stat. Sin. 22(3), 1123 (2012)MathSciNetzbMATHGoogle Scholar
  15. Konishi, S., Kitagawa, G.: Generalised information criteria in model selection. Biometrika 83(4), 875–890 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  16. Konishi, S., Kitagawa, G.: Information Criteria and Statistical Modeling. Springer, Berlin (2008)CrossRefzbMATHGoogle Scholar
  17. Lam, C., Fan, J.: Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Stat. 37(6B), 4254 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  18. Lauritzen, S.L.: Graphical Models. Oxford University Press, Oxford (1996)zbMATHGoogle Scholar
  19. Lian, H.: Shrinkage tuning parameter selection in precision matrices estimation. J. Stat. Plan. Inference 141(8), 2839–2848 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  20. Liu, H., Roeder, K., Wasserman, L.: Stability approach to regularization selection (stars) for high dimensional graphical models. Adv. Neural Inf. Process. Syst. 23, 1432–1440 (2010)Google Scholar
  21. Magnus, J.R., Neudecker, H.: Matrix Differential Calculus with Application in Statistics and Econometrics. Wiley Series in Probability and Statistics. Wiley, Hoboken (2007)Google Scholar
  22. Nishii, R.: Asymptotic properties of criteria for selection of variables in multiple regression. Ann. Stat. 12(2), 758765 (1984)MathSciNetCrossRefGoogle Scholar
  23. Penny, W.: Kullback–Leibler divergences of normal, gamma. Dirichlet and Wishart densities. Tech. Rep., Welcome Department of Cognitive Neurology (2001)Google Scholar
  24. Rothman, A.J., Bickel, P.J., Levina, E., Zhu, J.: Sparse permutation invariant covariance estimation. Electr. J. Stat. 2, 494–515 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  25. Rue, H., Held, L.: Gaussian Markov Random Fields: Theory and Applications. CRC Press, Boca Raton (2005)CrossRefzbMATHGoogle Scholar
  26. Schmidt, M., Fung, G. and Rosales, R.: Fast optimization methods for l1 regularization: A comparative study and two new approaches. In: Proceedings of European Conference on Machine Learning, pp. 286–297. Springer (2007)Google Scholar
  27. Siegmund, D.: Model selection in irregular problems: applications to mapping quantitative trait loci. Biometrika 94(3), 553–568 (2004)MathSciNetzbMATHGoogle Scholar
  28. Zhang, Y., Li, R., Tsai, C.-L.: Regularization parameter selections via generalized information criterion. J. Am. Stat. Assoc. 105(489), 312323 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  29. Wang, H., Li, R., Tsai, C.-L.: Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 91(4), 785–800 (2007)MathSciNetzbMATHGoogle Scholar
  30. Zou, H.: The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 101(476), 1418–1429 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  31. Whittaker, J.: Graphical Models in Applied Multivariate Statistics. Wiley Publishing, Hoboken (2009)zbMATHGoogle Scholar
  32. Yuan, M., Lin, Y.: Model selection and estimation in the Gaussian graphical model. Biometrika 94(1), 19–35 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  33. Zhao, T., Liu, H., Roeder, K., Lafferty, J., Wasserman, L.: The huge package for high-dimensional undirected graph estimation in R. J. Mach. Learn. Res. 13(4), 1059–1062 (2012)MathSciNetzbMATHGoogle Scholar
  34. Vujac̆ić, I., Abbruzzo, A., Wit, E.: A computationally fast alternative to cross-validation in penalized Gaussian graphical models. J. Stat. Comput. Simul. 85(18), 3628–3640 (2015)MathSciNetCrossRefGoogle Scholar
  35. Wille, A., Zimmermann, P., Vranová, E., Fürholz, A., Laule, O., Bleuler, S., Hennig, L., Prelić, A., von Rohr, P., Thiele, L., Zitzler, E., Gruissem, W., Buhlmann, P.: Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biol. 5(11), R92 (2004)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Economics, Business and StatisticsUniversity of PalermoPalermoItaly
  2. 2.PRA Health Sciences, Early Development ServicesGroningenThe Netherlands
  3. 3.Johann Bernoulli InstituteUniversity of GroningenGroningenThe Netherlands
  4. 4.Institute of Computation ScienceUSILuganoSwitzerland

Personalised recommendations