IPRSDP: a primal-dual interior-point relaxation algorithm for semidefinite programming

Abstract

We propose an efficient primal-dual interior-point relaxation algorithm based on a smoothing barrier augmented Lagrangian, called IPRSDP, for solving semidefinite programming problems in this paper. The IPRSDP algorithm has three advantages over classical interior-point methods. Firstly, IPRSDP does not require the iterative points to be positive definite. Consequently, it can easily be combined with the warm-start technique used for solving many combinatorial optimization problems, which require the solutions of a series of semidefinite programming problems. Secondly, the search direction of IPRSDP is symmetric in itself, and hence the symmetrization procedure is not required any more. Thirdly, with the introduction of the smoothing barrier augmented Lagrangian function, IPRSDP can provide the explicit form of the Schur complement matrix. This enables the complexity of forming this matrix in IPRSDP to be comparable to or lower than that of many existing search directions. The global convergence of IPRSDP is established under suitable assumptions. Numerical experiments are made on the SDPLIB set, which demonstrate the efficiency of IPRSDP.

Appendix: Some proofs

Proof of Lemma 3.5

“$\Longleftarrow$” First assume that $X\succ 0,\, S\succ 0,\, XS = \mu I$, which implies that $XS+SX=2\mu I$. Therefore,

$$\begin{aligned} \begin{array}{rl} &{}(S-\rho X)^2+ 4\rho \mu I\\ &{}\quad = S^2-\rho (SX+XS)+\rho ^2X^2+4\rho \mu I \\ &{}\quad = S^2+\rho (SX+XS)+\rho ^2 X^2\\ &{}\quad = (S+\rho X)^2, \end{array} \end{aligned}$$

and

$$\begin{aligned} Z(X,S;\mu ,\rho ) -X = \dfrac{((S-\rho X)^2+4\rho \mu I)^{\frac{1}{2}}-(S+\rho X)}{2\rho } \\ = \dfrac{S+\rho X-(S+\rho X)}{2\rho } = 0. \end{aligned}$$

“$\Longrightarrow$" Assume that $Z(X,S;\mu ,\rho ) -X = 0$, which results in $((S-\rho X)^2+4\rho \mu I)^{\frac{1}{2}}=S+\rho X$. After squaring both sides of the equation, we get

$$\begin{aligned} (S-\rho X)^2+4\rho \mu I = (S+\rho X)^2,\,\, S+\rho X \succ 0. \end{aligned}$$

This is equivalent to

$$\begin{aligned} XS+SX=2\mu I,\,\, S+\rho X \succ 0. \end{aligned}$$

(61)

Suppose there is an eigenvalue decomposition $X=Q^{\top }\Omega Q$, where $Q\in {\mathbb {R}}^{n\times n}$ is an orthogonal matrix, $\Omega = \text {Diag}(\omega _1,\dots ,\omega _n)$. Then (61) can be reformulated as

$$\begin{aligned} \Omega QSQ^{\top }+QSQ^{\top }\Omega = 2\mu I,\,\, QSQ^{\top }+\rho \Omega \succ 0. \end{aligned}$$

(62)

Define $\Xi = QSQ^{\top }=(\xi _{ij})_{1\le i\le j \le n}$. Then (62) can be equivalently written as:

$$\begin{aligned} \Omega \Xi +\Xi \Omega = 2\mu I,\,\, \Xi +\rho \Omega \succ 0, \end{aligned}$$

(63)

or written in a component form:

$$\begin{aligned} (\omega _{i}+\omega _{j})\xi _{ij}=\left\{ \begin{array}{ll} 2\mu , &{}\text {if }i = j;\\ 0, &{} \text {if }i \ne j, \end{array} \right. \forall i,j= 1,\dots ,n, \,\Xi +\rho \Omega \succ 0, \end{aligned}$$

(64)

If $i=j$, $2\omega _i \xi _{ii} = 2\mu$ and $\rho \omega _i+ \xi _{ii} > 0$, this implies $\omega _i > 0,\forall i = 1,\dots ,n$. As a result, X is a symmetric positive definite matrix. We can demonstrate that S is a symmetric positive definite matrix in a similar manner.

Next we prove that $XS = \mu I$. Notice that (64) implies $\xi _{ij}=0$ for all $i\ne j$. As a result, the matrix $\Xi$ is diagonal. Due to $\Omega \Xi +\Xi \Omega = 2\mu I$, we can get $\Omega \Xi = \mu I$. Then, we obtain $Q^{\top } \Omega \Xi Q = Q^{\top } \Omega QSQ^{\top }Q = XS = \mu I$ by multiplying this equation left by $Q^{\top }$ and right by Q. $\square$

Proof of Theorem 4.2

The directional derivative of the merit function $\phi _{(\mu ^{(k)},\rho ^{(k)})}(w)$ at the point $(\mu ^{(k)},w^{(k)})$ along the direction $(\Delta \mu ^{(k)},\Delta w^{(k)})$ is

$$\begin{aligned} \phi ^{'}_{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)};\Delta \mu ^{(k)},\Delta w^{(k)}) =-2\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)}). \end{aligned}$$

The Taylor expansion of ${\phi }_{(\mu ^{(k)}+\alpha \Delta \mu ^{(k)},\rho ^{(k)})}(w^{(k)}+\alpha \Delta w^{(k)})$ with respect to $\alpha$ at $\alpha = 0$ shows that

$$\begin{aligned} \begin{array}{rl} &{}{\phi }_{(\mu ^{(k)}+\alpha \Delta \mu ^{(k)},\rho ^{(k)})}(w^{(k)}+\alpha \Delta w^{(k)})-\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})\\ &{}\quad =\alpha \phi ^{'}_{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)};\Delta \mu ^{(k)},\Delta w^{(k)})+o(\alpha )\\ &{}\quad =-2\tau \alpha \phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})-2(1-\tau )\alpha \phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})+o(\alpha ). \end{array} \end{aligned}$$

Since $\tau < 1$ and $\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})>0$, the inequality (44) holds for all sufficiently small $\alpha > 0$. $\square$

Proof of Lemma 5.3

Due to Lemma 5.2, we have $\phi _{(\mu ^{(k)},\rho ^{(k)})}(w^{(k)})\le \phi _{(\mu ^{(0)},\rho ^{(0)})}(w^{(0)})$ for all $k = 1,2,\dots$ By the monotonicity of the merit function (27), we have

$$\begin{aligned} \frac{1}{2}\Vert Z^{(k)}-X^{(k)}\Vert _F^2\le \phi _{(\mu ^{(0)},\rho ^{(0)})}(w^{(0)}). \end{aligned}$$

which together with Theorem 5.1 implies that $\{Z^{(k)}\}$ is bounded. Suppose there exists an eigenvalue decomposition $S^{(k)}-\rho ^{(k)} X^{(k)}=\displaystyle \sum _{i=1}^n\lambda _i^{(k)} d_i^{(k)}(d_i^{(k)})^\top$, where $\Vert d_i^{(k)}\Vert = 1,i=1,\dots ,n$. Then

$$\begin{aligned} Y^{(k)}=\displaystyle \sum _{i=1}^n\dfrac{\sqrt{(\lambda _i^{(k)})^2+4\rho ^{(k)}\mu ^{(k)}}+\lambda _i^{(k)}}{2\rho ^{(k)}}d_i^{(k)}(d_i^{(k)})^{\top }. \end{aligned}$$

Therefore it is sufficient to prove that $\dfrac{\sqrt{(\lambda _i^{(k)})^2+4\rho ^{(k)}\mu ^{(k)}}+\lambda _i^{(k)}}{2\rho ^{(k)}}, i =1,\dots ,n$ are bounded. For any $i =1,\dots ,n$, we have

$$\begin{aligned} \begin{array}{rl} &{}\dfrac{\sqrt{(\lambda _i^{(k)})^2+4\rho ^{(k)}\mu ^{(k)}}+\lambda _i^{(k)}}{2\rho ^{(k)}}\\ &{}\quad \le \dfrac{\arrowvert \lambda _i^{(k)}\arrowvert }{\rho ^{(k)}} + \sqrt{\dfrac{\mu ^{(k)}}{\rho ^{(k)}}}\\ &{}\quad \le \dfrac{\Vert S^{(k)}-\rho ^{(k)} X^{(k)}\Vert _F}{\rho ^{(k)}} + \sqrt{\dfrac{\mu ^{(k)}}{\rho ^{(k)}}}\\ &{}\quad \le \dfrac{\Vert S^{(k)}\Vert _F}{\rho ^{(k)}}+\Vert X^{(k)}\Vert _F+ \sqrt{\dfrac{\mu ^{(k)}}{\rho ^{(k)}}}\\ &{}\quad \le \dfrac{\max \{1,\Vert X^{(k)}\Vert _F\}}{\sigma }+\Vert X^{(k)}\Vert _F+ \sqrt{\dfrac{\mu ^{(0)}}{\rho ^{(0)}}}. \end{array} \end{aligned}$$

Thus, $\dfrac{\sqrt{(\lambda _i^{(k)})^2+4\rho ^{(k)}\mu ^{(k)}}+\lambda _i^{(k)}}{2\rho ^{(k)}}, i=1,\dots ,n$ are bounded under Assumption 3.1, which implies that the sequence $\{Y^{(k)}\}$ is bounded.

Due to $\rho ^{(k)}Y^{(k)}Z^{(k)}=\mu ^{(k)}I$, we get $\Vert Y^{(k)}Z^{(k)}\Vert _F=\sqrt{n}\dfrac{\mu ^{(k)}}{\rho ^{(k)}}$. Combining with the boundedness of $\{Y^{(k)}\}$ and $\{Z^{(k)}\}$, one has the desired inequalities. $\square$