Abstract
In this paper, we introduce a simple but efficient greedy algorithm, called SINCO, for the Sparse INverse COvariance selection problem, which is equivalent to learning a sparse Gaussian Markov Network, and empirically investigate the structure-recovery properties of the algorithm. Our approach is based on a coordinate ascent method which naturally preserves the sparsity of the network structure. We show that SINCO is often comparable to, and, in various cases, outperforms commonly used approaches such as glasso [7] and COVSEL [1], in terms of both structure-reconstruction error (particularly, false positive error) and computational time. Moreover, our method has the advantage of being easily parallelizable. Finally, we show that SINCO’s greedy nature allows reproduction of the regularization path behavior by applying the method to one (sufficiently small) instance of the regularization parameter λ only; thus, SINCO can obtain a desired number of network links directly, without having to tune the λ parameter. We evaluate our method empirically on various simulated networks and real-life data from biological and neuroimaging applications.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Banerjee, O., El Ghaoui, L., d’Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. Journal of Machine Learning Research 9, 485–516 (2008)
Barabasi, A.-L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)
Buhl, S.L.: On the existence of maximum likelihood estimators for graphical gaussian models. Scandinavian Journal of Statistics 20(3), 263–270 (1993)
Dempster, A.P.: Covariance selection. Biometrics 28(1), 157–175 (1972)
Duchi, J., Gould, S., Koller, D.: Projected subgradient methods for learning sparse gaussians. In: Proc. of UAI-08 (2008)
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Statist. 32(1), 407–499 (2004)
Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics (2007)
Lauritzen, S.: Graphical Models. Oxford University Press, Oxford (1996)
Lu, Z.: Smooth optimization approach for sparse covariance selection. SIAM Journal on Optimization 19(4), 1807–1827 (2009)
Marlin, B., Murphy, K.: Sparse gaussian graphical models with unknown block structure. In: Proc. of ICML-09 (2009)
Meinshausen, N., Buhlmann, P.: High dimensional graphs and variable selection with the Lasso. Annals of Statistics 34(3), 1436–1462 (2006)
Mitchell, T.M., Hutchinson, R., Niculescu, R.S., Pereira, F., Wang, X., Just, M., Newman, S.: Learning to decode cognitive states from brain images. Machine Learning 57, 145–175 (2004)
Ravikumar, P., Wainwright, M., Raskutti, G., Yu, B.: Model selection in Gaussian graphical models: High-dimensional consistency of l1-regularized MLE. In: NIPS-08 (2008)
Rothman, A., Bickel, P., Levina, E., Zhu, J.: Sparse permutation invariant covariance estimation. Electronic Journal of Statistics (2), 494–515 (2008)
Schmidt, M., van den Berg, E., Friedlander, M., Murphy, K.: Optimizing costly functions with simple constraints: A limited-memory projected quasi-newton algorithm. In: Proc. of AISTATS-09 (2009)
Stolovitzky, G., Prill, R.J., Califano, A.: Lessons from the dream2 challenges. Annals of the New York Academy of Sciences (1158), 159–195 (2009)
Tibshirani, R.: Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B 58(1), 267–288 (1996)
Wainwright, M., Ravikumar, P., Lafferty, J.: High-Dimensional Graphical Model Selection Using ℓ1-Regularized Logistic Regression. In: NIPS 19, pp. 1465–1472 (2007)
Wen, Z., Goldfarb, D., Ma, S., Scheinberg, K.: Row by row methods for semidefinite programming. Technical report, Department of IEOR, Columbia University (2009)
Wolkowicz, H., Saigal, R., Vanenberghe, L. (eds.): Handbook of Semidefinite Programming. Kluwer Academic Publishers, Dordrecht (2000)
Yuan, M., Lin, Y.: Model Selection and Estimation in the Gaussian Graphical Model. Biometrika 94(1), 19–35 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Scheinberg, K., Rish, I. (2010). Learning Sparse Gaussian Markov Networks Using a Greedy Coordinate Ascent Approach. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2010. Lecture Notes in Computer Science(), vol 6323. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15939-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-15939-8_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15938-1
Online ISBN: 978-3-642-15939-8
eBook Packages: Computer ScienceComputer Science (R0)