Abstract
This paper considers the problem of positive semidefinite factorization (PSD factorization), a generalization of exact nonnegative matrix factorization. Given an m-by-n nonnegative matrix X and an integer k, the PSD factorization problem consists in finding, if possible, symmetric k-by-k positive semidefinite matrices \(\{A^1,\ldots ,A^m\}\) and \(\{B^1,\ldots ,B^n\}\) such that \(X_{i,j}=\text {trace}(A^iB^j)\) for \(i=1,\ldots ,m\), and \(j=1,\ldots ,n\). PSD factorization is NP-hard. In this work, we introduce several local optimization schemes to tackle this problem: a fast projected gradient method and two algorithms based on the coordinate descent framework. The main application of PSD factorization is the computation of semidefinite extensions, that is, the representations of polyhedrons as projections of spectrahedra, for which the matrix to be factorized is the slack matrix of the polyhedron. We compare the performance of our algorithms on this class of problems. In particular, we compute the PSD extensions of size \(k=1+ \lceil \log _2(n) \rceil \) for the regular n-gons when \(n=5\), 8 and 10. We also show how to generalize our algorithms to compute the square root rank (which is the size of the factors in a PSD factorization where all factor matrices \(A^i\) and \(B^j\) have rank one) and completely PSD factorizations (which is the special case where the input matrix is symmetric and equality \(A^i=B^i\) is required for all i).
Similar content being viewed by others
Notes
Example 5.2 provides an explicit PSD factorization of size 4 for \(S_8\). For \(S_7\), we were not able to obtain such an exact factorization of size 4, although we have tried many different initializations. It is possible that \({{\mathrm{{{\mathrm{rank}}}_{{{\mathrm{psd}}}}}}}(S_7) = 5\) since there is no result about the monotonicity of the PSD rank of regular n-gons (this is, as far as we know, an open question). In fact, [12] showed that monotonicity does not hold for the PSD rank over the complex numbers with \({{\mathrm{rank}}}_{{{\mathrm{psd}}}}^{\mathbb {C}}(S_6) = 3 < 4 \le {{\mathrm{rank}}}_{{{\mathrm{psd}}}}^{\mathbb {C}}(S_5)\).
References
Berman, A., Shaked-Monderer, N.: Completely Positive Matrices. World Scientific, Singapore (2003)
Burer, S., Monteiro, R.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95(2), 329–357 (2003)
Cardano, G.: Ars Magna or the Rules of Algebra. Dover Publications, Mineola (1968)
Cichocki, A., Phan, A.H.: Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans. Fundam. Electron. E92–A(3), 708–721 (2009)
Cichocki, A., Zdunek, R., Amari, S.i.: Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. In: International Conference on Independent Component Analysis and Signal Separation, pp. 169–176. Springer (2007)
Fawzi, H., Gouveia, J., Parrilo, P., Robinson, R., Thomas, R.: Positive semidefinite rank. Math. Program. 153(1), 133–177 (2015)
Fawzi, H., Gouveia, J., Robinson, R.: Rational and real positive semidefinite rank can be different. Oper. Res. Lett. 44(1), 59–60 (2016)
Fiorini, S., Massar, S., Pokutta, S., Tiwary, H., de Wolf, R.: Linear vs. semidefinite extended formulations: exponential separation and strong lower bounds. In: Proceedings of the 44th Annual ACM Symposium on Theory of Computing, pp. 95–106. ACM (2012)
Fiorini, S., Rothvoss, T., Tiwary, H.: Extended formulations for polygons. Discrete Comput. Geom. 48(3), 658–668 (2012)
Gillis, N.: The why and how of nonnegative matrix factorization. In: Suykens, J., Signoretto, M., Argyriou, A. (eds.) Regularization, Optimization, Kernels, and Support Vector Machines‘. Chapman & Hall/CRC, Boca Raton (2014). Machine Learning and Pattern Recognition Series
Gillis, N., Glineur, F.: Accelerated multiplicative updates and hierarchical ALS algorithms for nonnegative matrix factorization. Neural Comput. 24(4), 1085–1105 (2012)
Goucha, A., Gouveia, J., Silva, P.: On ranks of regular polygons. SIAM J. Discrete Math. 31(4), 2612–2625 (2017)
Gouveia, J., Parrilo, P., Thomas, R.: Lifts of convex sets and cone factorizations. Math. Oper. Res. 38(2), 248–264 (2013)
Gouveia, J., Robinson, R., Thomas, R.: Worst-case results for positive semidefinite rank. Math. Program. 153(1), 201–212 (2015)
Gribling, S., de Laat, D., Laurent, M.: Matrices with high completely positive semidefinite rank. Linear Algebra Appl. 513, 122–148 (2017)
Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear gauss-seidel method under convex constraints. Oper. Res. Lett. 26(3), 127–136 (2000)
Ho, N.D.: Nonnegative matrix factorization algorithms and applications. Ph.D. thesis, Univertsité catholique de Louvain (2008)
Hsieh, C.J., Dhillon, I.: Fast coordinate descent methods with variable selection for non-negative matrix factorization. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1064–1072. ACM (2011)
Kaibel, V.: Extended formulations in combinatorial optimization. Optima 85, 2–7 (2011)
Kuang, D., Yun, S., Park, H.: SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering. J. Glob. Optim. 62(3), 545–574 (2015)
Kubjas, K., Robeva, E., Robinson, R.: Positive semidefinite rank and nested spectrahedra. Linear Multilinear Algebra. 1–23 (2017)
Lee, T., Theis, D.O.: Support-based lower bounds for the positive semidefinite rank of a nonnegative matrix. (2012). arXiv preprint arXiv:1203.3961
Löfberg, J.: Yalmip: A toolbox for modeling and optimization in matlab. In: IEEE International Symposium on Computer Aided Control Systems Design, 2004, pp. 284–289. IEEE (2004)
Nesterov, Y.: A method of solving a convex programming problem with convergence rate 0(1/k2). Sov. Math. Dokl. 27, 372–376 (1983)
Prakash, A., Sikora, J., Varvitsiotis, A., Wei, Z.: Completely positive semidefinite rank. Math. Program. 1–35 (2016)
Shitov, Y.: The complexity of positive semidefinite matrix factorization. SIAM J. Optim. 27(3), 1898–1909 (2017)
Sturm, J.F.: Using sedumi 1.02, a matlab toolbox for optimization over symmetric cones. Optim. Methods Softw. 11(1–4), 625–653 (1999)
Vandaele, A., Gillis, N., Glineur, F.: On the linear extension complexity of regular n-gons. Linear Algebra Appl. 521, 217–239 (2017)
Vandaele, A., Gillis, N., Glineur, F., Tuyttens, D.: Heuristics for exact nonnegative matrix factorization. J. Glob. Optim. 65(2), 369–400 (2016)
Vandaele, A., Gillis, N., Lei, Q., Zhong, K., Dhillon, I.: Efficient and non-convex coordinate descent for symmetric nonnegative matrix factorization. IEEE Trans. Signal Process. 64(21), 5571–5584 (2016)
Wright, S.: Coordinate descent algorithms. Math. Program. 151(1), 3–34 (2015)
Yannakakis, M.: Expressing combinatorial optimization problems by linear programs. J. Comput. Syst. Sci. 43(3), 441–466 (1991)
Author information
Authors and Affiliations
Corresponding author
Additional information
Nicolas Gillis acknowledges the support by the F.R.S.-FNRS (incentive Grant for scientific research No. F.4501.16) and by the ERC (starting Grant No. 679515). This paper presents research results of the Concerted Research Action (ARC) programme supported by the Federation Wallonia-Brussels (contract ARC 14/19-060).
Rights and permissions
About this article
Cite this article
Vandaele, A., Glineur, F. & Gillis, N. Algorithms for positive semidefinite factorization. Comput Optim Appl 71, 193–219 (2018). https://doi.org/10.1007/s10589-018-9998-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-018-9998-x