Group Sparse Inverse Covariance Selection with a Dual Augmented Lagrangian Method
Sparse Inverse Covariance Selection (SICS) is a popular tool identifying an intrinsic relationship between continuous random variables. In this paper, we treat the extension of SICS to the grouped feature model in which the state-of-the-art SICS algorithm is no longer applicable. Such an extended model is essential when we aim to find a group-wise relationships between sets of variables, e.g. unknown interactions between groups of genes. We tackle the problem with a technique called Dual Augmented Lagrangian (DAL) that provides an efficient method for grouped sparse problems. We further improve the DAL framework by combining the Alternating Direction Method of Multipliers (ADMM), which dramatically simplifies the entire procedure of DAL and reduce the computational cost. We also provide empirical comparisons of the proposed DAL–ADMM algorithm against existing methods.
KeywordsSparse Inverse Covariance Selection Dual Augmented Lagrangian Alternating Direction Method of Multipliers
Unable to display preview. Download preview PDF.
- 5.Scheinberg, K., Ma, S., Goldfarb, D.: Sparse inverse covariance selection via alternating linearization methods. In: Advances in Neural Information Processing Systems, vol. 23, pp. 2101–2109 (2010)Google Scholar
- 6.Duchi, J., Gould, S., Koller, D.: Projected subgradient methods for learning sparse gaussians. In: Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, pp. 145–152 (2008)Google Scholar
- 10.Yuan, X.: Alternating direction methods for sparse covariance selection (preprint 2009), http://www.optimization-online.org/DB_FILE/2009/09/2390.pdf
- 12.Hsieh, C., Sustik, M., Dhillon, I., Ravikumar, P.: Sparse inverse covariance matrix estimation using quadratic approximation. In: Advances in Neural Information Processing Systems, vol. 24, pp. 2330–2338 (2011)Google Scholar
- 15.Schmidt, M., Van Den Berg, E., Friedlander, M., Murphy, K.: Optimizing costly functions with simple constraints: A limited-memory projected quasi-newton algorithm. In: Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, pp. 456–463 (2009)Google Scholar