Abstract
The precision matrix is the inverse of the covariance matrix. Estimating large sparse precision matrices is an interesting and a challenging problem in many fields of sciences, engineering, humanities and machine learning problems in general. Recent applications often encounter high dimensionality with a limited number of data points leading to a number of covariance parameters that greatly exceeds the number of observations, and hence the singularity of the covariance matrix. Several methods have been proposed to deal with this challenging problem, but there is no guarantee that the obtained estimator is positive definite. Furthermore, in many cases, one needs to capture some additional information on the setting of the problem. In this paper, we introduce a criterion that ensures the positive definiteness of the precision matrix and we propose the inner-outer alternating direction method of multipliers as an efficient method for estimating it. We show that the convergence of the algorithm is ensured with a sufficiently relaxed stopping criterion in the inner iteration. We also show that the proposed method converges, is robust, accurate and scalable as it lends itself to an efficient implementation on parallel computers.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anbari, M.E., Alam, S., Bensmail, H.: COFADMM: a computational features selection with alternating direction method of multipliers. Procedia Comput. Sci. 29, 821–830 (2014)
Baggag, A., Sameh, A.: A nested iterative scheme for indefinite linear systems in particulate flows. Comput. Methods Appl. Mech. Eng. 193(21–22), 1923–1957 (2004)
Bien, J., Tibshirani, R.: Sparse estimation of a covariance matrix. Biometrika 98, 807–820 (2011)
Bredel, M., Bredel, C., Juric, D., Harsh, G.R., Vogel, H., Recht, L.D., Sikic, B.I.: High-resolution genome-wide mapping of genetic alterations in human glial brain tumors. Cancer Res. 65(10), 4088–4096 (2005)
Cai, T., Zhou, H.: A constrained \(l_{1}\) minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106, 594–607 (2011)
Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)
Duchi, J., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the l1-ball for learning in high dimensions. In: Proceedings of the 25th International Conference on Machine Learning, pp. 272–279. ACM (2008)
Honorio, J., Jaakkola, T.S: Inverse covariance estimation for high-dimensional data in linear time, space: spectral methods for riccati and sparse models. arXiv preprint arXiv:1309.6838 (2013)
Higham, N.J.: Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl. 103, 103–118 (1988)
Lai, W.R., Johnson, M.D., Kucherlapati, R., Park, P.J.: Comparative analysis of algorithms for identifying amplifications and deletions in array CGH data. Bioinform. 21(19), 3763–3770 (2005)
Maurya, A.: A well-conditioned and sparse estimation of covariance and inverse covariance matrices using a joint penalty. J. Mach. Learn. Res. 17, 1–28 (2016)
Mazumder, R., Hastie, T.: Exact covariance thresholding into connected components for large-scale graphical lasso. J. Mach. Learn. Res. 13(Mar), 781–794 (2012)
Ravikumar, P., Wainwright, M., Raskutti, G., Yu, B.: High-dimensional covariance estimation by minimizing \(l_{1}\)-penalized log-determinant divergence. Electron. J. Statist. 5, 935–980 (2011)
Rothman, A.J., Bickel, P.J., Levina, E., Zhu, J., et al.: Sparse permutation invariant covariance estimation. Electron. J. Stat. 2, 494–515 (2008)
Schwertman, N.C., Allen, D.M.: Smoothing an indefinite variance-covariance matrix. J. Stat. Comput. Simul. 9(3), 183–194 (1979)
Witten, D.M., Tibshirani, R.: Covariance-regularized regression and classification for high dimensional problems. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 71(3), 615–636 (2009)
Xue, L., Ma, S., Zou, H.: Positive-definite 1-penalized estimation of large covariance matrices. J. Am. Stat. Assoc. 107(500), 1480–1491 (2012)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Baggag, A., Bensmail, H., Srivastava, J. (2017). Advanced Computation of Sparse Precision Matrices for Big Data. In: Kim, J., Shim, K., Cao, L., Lee, JG., Lin, X., Moon, YS. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2017. Lecture Notes in Computer Science(), vol 10235. Springer, Cham. https://doi.org/10.1007/978-3-319-57529-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-57529-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57528-5
Online ISBN: 978-3-319-57529-2
eBook Packages: Computer ScienceComputer Science (R0)