QSDPNAL: a two-phase augmented Lagrangian method for convex quadratic semidefinite programming

Abstract

In this paper, we present a two-phase augmented Lagrangian method, called QSDPNAL, for solving convex quadratic semidefinite programming (QSDP) problems with constraints consisting of a large number of linear equality and inequality constraints, a simple convex polyhedral set constraint, and a positive semidefinite cone constraint. A first order algorithm which relies on the inexact Schur complement based decomposition technique is developed in QSDPNAL-Phase I with the aim of solving a QSDP problem to moderate accuracy or using it to generate a reasonably good initial point for the second phase. In QSDPNAL-Phase II, we design an augmented Lagrangian method (ALM) wherein the inner subproblem in each iteration is solved via inexact semismooth Newton based algorithms. Simple and implementable stopping criteria are designed for the ALM. Moreover, under mild conditions, we are able to establish the rate of convergence of the proposed algorithm and prove the R-(super)linear convergence of the KKT residual. In the implementation of QSDPNAL, we also develop efficient techniques for solving large scale linear systems of equations under certain subspace constraints. More specifically, simpler and yet better conditioned linear systems are carefully designed to replace the original linear systems and novel shadow sequences are constructed to alleviate the numerical difficulties brought about by the crucial subspace constraints. Extensive numerical results for various large scale QSDPs show that our two-phase algorithm is highly efficient and robust in obtaining accurate solutions. The software reviewed as part of this submission was given the DOI (Digital Object Identifier) https://doi.org/10.5281/zenodo.1206980.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    In this case, the optimal solution set to (P) is necessarily a singleton though (D) may have multiple solutions.

  2. 2.

    https://www.rmicri.org.

References

  1. 1.

    Alfakih, A.Y., Khandani, A., Wolkowicz, H.: Solving Euclidean distance matrix completion problems via semidefinite programming. Comput. Optim. Appl. 12, 13–30 (1999)

    MathSciNet  Article  Google Scholar 

  2. 2.

    Bauschke, H.H., Borwein, J.M., Li, W.: Strong conical hull intersection property, bounded linear regularity, Jameson’s property (G), and error bounds in convex optimization. Math. Program. 86, 135–160 (1999)

    MathSciNet  Article  Google Scholar 

  3. 3.

    Biswas, P., Liang, T.C., Toh, K.-C., Wang, T.C., Ye, Y.: Semidefinite programming approaches for sensor network localization with noisy distance measurements. IEEE Trans. Autom. Sci. Eng. 3, 360–371 (2006)

    Article  Google Scholar 

  4. 4.

    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)

    Google Scholar 

  5. 5.

    Burkard, R.E., Karisch, S.E., Rendl, F.: QAPLIB—a quadratic assignment problem library. J. Global Optim. 10, 391–403 (1997)

    MathSciNet  Article  Google Scholar 

  6. 6.

    Chen, L., Sun, D.F., Toh, K.-C.: An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming. Math. Program. 161, 237–270 (2017)

    MathSciNet  Article  Google Scholar 

  7. 7.

    Cui, Y., Sun, D.F., Toh, K.-C.: On the asymptotic superlinear convergence of the augmented Lagrangian method for semidefinite programming with multiple solutions, arXiv:1610.00875 (2016)

  8. 8.

    Clarke, F.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

    Google Scholar 

  9. 9.

    Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)

    Google Scholar 

  10. 10.

    Han, D., Sun, D., Zhang, L.: Linear rate convergence of the alternating direction method of multipliers for convex composite programming. Math. Oper. Res. (2017). https://doi.org/10.1287/moor.2017.0875

    MathSciNet  Article  Google Scholar 

  11. 11.

    Higham, N.J.: Computing the nearest correlation matrix—a problem from finance. IMA J. Numer. Anal. 22, 329–343 (2002)

    MathSciNet  Article  Google Scholar 

  12. 12.

    Hiriart-Urruty, J.-B., Strodiot, J.-J., Nguyen, V.H.: Generalized Hessian matrix and second-order optimality conditions for problems with \({C}^{1,1}\) data. Appl. Math. Optim. 11, 43–56 (1984)

    MathSciNet  Article  Google Scholar 

  13. 13.

    Jiang, K., Sun, D.F., Toh, K.-C.: An inexact accelerated proximal gradient method for large scale linearly constrained convex SDP. SIAM J. Optim. 22, 1042–1064 (2012)

    MathSciNet  Article  Google Scholar 

  14. 14.

    Jiang, K., Sun, D.F., Toh, K.-C.: A partial proximal point algorithm for nuclear norm regularized matrix least squares problems. Math. Program. Comput. 6, 281–325 (2014)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Krislock, N., Lang, J., Varah, J., Pai, D.K., Seidel, H.-P.: Local compliance estimation via positive semidefinite constrained least squares. IEEE Trans. Robot. 20, 1007–1011 (2004)

    Article  Google Scholar 

  16. 16.

    Li, L., Toh, K.-C.: An inexact interior point method for l1-regularized sparse covariance selection. Math. Program. Comput. 2, 291–315 (2010)

    MathSciNet  Article  Google Scholar 

  17. 17.

    Li, X.D.: A Two-Phase Augmented Lagrangian Method for Convex Composite Quadratic Programming, PhD thesis, Department of Mathematics, National University of Singapore (2015)

  18. 18.

    Li, X.D., Sun, D.F., Toh, K.-C.: A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions. Math. Program. 155, 333–373 (2016)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Nie, J.W., Yuan, Y.X.: A predictor-corrector algorithm for QSDP combining Dikin-type and Newton centering steps. Ann. Oper. Res. 103, 115–133 (2001)

    MathSciNet  Article  Google Scholar 

  20. 20.

    Pang, J.-S., Sun, D.F., Sun, J.: Semismooth homeomorphisms and strong stability of semidefinite and Lorentz complementarity problems. Math. Oper. Res. 28, 39–63 (2003)

    MathSciNet  Article  Google Scholar 

  21. 21.

    Povh, J., Rendl, F.: Copositive and semidefinite relaxations of the quadratic assignment problem. Discrete Optim. 6, 231–241 (2009)

    MathSciNet  Article  Google Scholar 

  22. 22.

    Qi, H.D.: Local duality of nonlinear semidefinite programming. Math. Oper. Res. 34, 124–141 (2009)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Qi, H.D., Sun, D.F.: A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl. 28, 360–385 (2006)

    MathSciNet  Article  Google Scholar 

  24. 24.

    Rockafellar, R.T.: Conjugate Duality and Optimization, CBMS-NSF Regional Conf. Ser. Appl. Math. vol. 16. SIAM, Philadelphia (1974)

  25. 25.

    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 97–116 (1976)

    MathSciNet  Article  Google Scholar 

  27. 27.

    Sun, D.F., Sun, J.: Semismooth matrix-valued functions. Math. Oper. Res. 27, 150–169 (2002)

    MathSciNet  Article  Google Scholar 

  28. 28.

    Sun, D.F., Toh, K.-C., Yang, L.: A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim. 25, 882–915 (2015)

    MathSciNet  Article  Google Scholar 

  29. 29.

    Sun, D.F., Toh, K.-C., Yang, L.: An efficient inexact ABCD method for least squares semidefinite programming. SIAM J. Optim. 26, 1072–1100 (2016)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Sun, J., Zhang, S.: A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur. J. Oper. Res. 207, 1210–1220 (2010)

    MathSciNet  Article  Google Scholar 

  31. 31.

    Toh, K.-C.: An inexact primal-dual path following algorithm for convex quadratic SDP. Math. Program. 112, 221–254 (2008)

    MathSciNet  Article  Google Scholar 

  32. 32.

    Toh, K.-C., Tütüncü, R., Todd, M.: Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems. Pac. J. Optim. 3, 135–164 (2007)

    MathSciNet  MATH  Google Scholar 

  33. 33.

    Yang, L., Sun, D.F., Toh, K.-C.: SDPNAL+: a majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints. Math. Program. Comput. 7, 331–366 (2015)

    MathSciNet  Article  Google Scholar 

  34. 34.

    Zhao, X.Y.: A Semismooth Newton-CG Augmented Lagrangian Method for Large Scale Linear and Convex Quadratic SDPs, PhD thesis, Department of Mathematics, National University of Singapore (2009)

  35. 35.

    Zhao, X.Y., Sun, D.F., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)

    MathSciNet  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Xudong Li.

Additional information

An earlier version of this paper was made available in arXiv as arXiv:1512.08872. The software that was reviewed as part of this submission has been issued the Digital Object Identifier https://doi.org/10.5281/zenodo.1206980.

D. Sun: On leave from Department of Mathematics, National University of Singapore. This author’s research is partially supported by a start-up research grant from the Hong Kong Polytechnic University.

K.-C. Toh: This author’s research is supported in part by the Ministry of Education, Singapore, Academic Research Fund under Grant R-146-000-256-114.

Appendix

Appendix

Along with the paper, we provide a Matlab implementation of our algorithm. The software package has been issued the Digital Object Identifier https://doi.org/10.5281/zenodo.1206980. Here, as a short users’ guide, we present some general description of the structure of our software.

Installation Qsdpnal is a Matlab software package developed under Matlab version 8.0 or above. It includes some C subroutines written to carry out certain operations for which Matlab is not efficient at. These subroutines are called within Matlab via the Mex interface. The user can simply follow the steps below to install Qsdpnal within Matlab:

(a):

unzip QSDPNAL+v0.1.zip;

(b):

run Matlab in the directory QSDPNAL+v0.1;

After that, to see whether you have installed Qsdpnal correctly, try the following steps:

figuref

In the above, startup.m sets up the paths for Qsdpnal in Matlab and qsdpdemo.m is a demo file illustrating how to solve a QSDP problem (31) in Qsdpnal.

Copyright Qsdpnal: A Matlab software for convex quadratic semidefinite programming with inequality, equality and bound constraints, is copyrighted in 2016 by Xudong Li, Defeng Sun and Kim-Chuan Toh. The software Qsdpnal is distributed under the GNU General Public License 2.0. You can redistribute it and/or modify it under the terms of the GNU General Public License 2.0 as published by the Free Software Foundation Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. For commercial applications that may be incompatible with this license, please contact the authors to discuss alternatives. This software is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

Main function and data structure Qsdpnal is an extension of the solvers Sdpnal [34] and Sdpnal+ [33]. The internal implementation of Qsdpnal thus follows in part the data structures and design framework of the above two solvers. A casual user does not need to understand the internal implementation of Qsdpnal.

In the Qsdpnal solver, the main routine is qsdpnal.m, whose calling syntax is as follows:

figureg

Input arguments

  • blk: a cell array describing the conic structure of the QSDP problem.

  • Q, AEt, bE, C, AIt, bI: data of QSDP problem (31). If the linear map \(\mathcal{A}_I\) is not present, one can set AIt = [], bI = [].

  • options: a structure array of parameters.

Output arguments The names chosen for the output arguments explain their contents. The argument X is a solution to problem (31) up to the desired accuracy tolerance. The argument info is a structure array which records various performance measures of the solver. For example

figureh

correspond to the measures \(\eta _Z\), \(\eta _{I_1}\), \(\eta _{I_2}\) and \(\eta _{I_3}\) defined in (35), respectively. The argument runhist is a structure array which records the history of various performance measures during the course of running qsdpnal. For example,

figurei

record the primal and dual objective values, primal and dual infeasibilities at each iteration, respectively.

Data structure The format of the input data in Qsdpnal is similar to those in Sdpnal [34] and Sdpnal+ [33]. For problem (31), we set

figurej

where \({\bar{n}} = n(n+1)/2\) and the self-adjoint linear operator \(\mathcal{Q}\) is stored as a function handle which takes any matrix \(X\in \mathcal{S}^n\) as input and output the matrix \(\mathcal{Q}X\in \mathcal{S}^n\). For the sake of computational efficiency, following the same approach used in Sdpnal [34] and Sdpnal+ [33], we store the linear map \(\mathcal{A}_E\) in vectorized form as a single \({\bar{n}}\times m_E\) matrix AEt. The same procedure also applies to the linear operator \(\mathcal{A}_I\), i.e., AIt is stored in the same format as AEt. The data of the simple nonempty closed convex polyhedral set \(\mathcal{K}\) is encoded in the argument options. For example, if \(\mathcal{K}=\{ X\in \mathcal{S}^n\mid L\le X \le U\}\) with \(L,U\in S^n\) being given matrices, we set

figurek

The default is options.L = [], options.U = []. If \(\mathcal{K}= \{X\in \mathcal{S}^n\mid X\ge 0\}\), then one can simply set options.nonnegative = 1.

Apart from the information of \(\mathcal{K}\), various other parameters used in our solver qsdpnal.m are set in the structure array options. We list here the important parameters which the user is likely to reset.

  • options.stoptol: accuracy tolerance to terminate the algorithm, default is \(10^{-6}\).

  • options.maxiter: maximum number of iterations allowed, default is 5000.

  • options.sGSstoptol: accuracy tolerance to use for Qsdpnal-Phase I (SCB_qsdp.m) when generating a starting point for the algorithm in the second phase of qsdpnal.m (default = \(10^{-4}\)).

  • options.sGSmaxiter: maximum number of Qsdpnal-Phase I iterations allowed for generating a starting point (default = 1500).

Examples Next, we present two examples of using Qsdpnal for solving two QSDP problems in detail. The segment below illustrates how one can solve the instance be250.2 of QSDP-BIQ problems.

figurel

The next example is using qsdpnal.m to solve the instance chr15c of QSDP-QAP problems.

figurem

In the above codes, randQXfun generates the self-adjoint positive semidefinite linear operator \(\mathcal{Q}\) in (37) with randomly generated matrices \(A,B\in \mathcal{S}^n_+\).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Li, X., Sun, D. & Toh, K. QSDPNAL: a two-phase augmented Lagrangian method for convex quadratic semidefinite programming. Math. Prog. Comp. 10, 703–743 (2018). https://doi.org/10.1007/s12532-018-0137-6

Download citation

Keywords

  • Quadratic semidefinite programming
  • Schur complement
  • Augmented Lagrangian
  • Inexact semismooth Newton method

Mathematics Subject Classification

  • 90C06
  • 90C20
  • 90C22
  • 90C25
  • 65F10