Abstract
We consider in this paper an infeasible predictor–corrector primal–dual path following interior point algorithm using the Nesterov–Todd search direction to solve semi-definite linear complementarity problems. Global convergence and polynomial iteration complexity of the algorithm are established. Two sufficient conditions are also given for superlinear convergence of iterates generated by the algorithm. Preliminary numerical results are finally provided when the algorithm is used to solve semi-definite linear complementarity problems.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
The class of semi-definite linear complementarity problems (SDLCPs), which contains the class of semi-definite programs (SDPs) as an important subclass, has many real life applications, for example, in optimal control, estimation and signal processing, communications and networks, statistics, finance [3]. Semi-definite programming has wide applications in NP-hard combinatorial problems [8] and global optimization, where it is used to find bounds on optimal values and to find approximate solutions. Interior point methods have been proven to be successful in solving linear programs, with many works in the literature devoted to its study since the 80s. Semi-definite programs are extensions of linear programs to the space of symmetric matrices, and interior point methods have been successfully extended from solving linear programs to solving semi-definite programs with the same polynomial complexity results—see for example [1, 14, 17, 45].
Among different interior point methods (IPMs), primal–dual path following interior point algorithms are the most successful and most widely studied. Due to the difficulty in maintaining symmetry in the linearized complementarity when using primal–dual path following interior point method to solve an SDP, researchers working in the IPM domain have proposed ways to overcome this problem, resulting in different symmetrized search directions [1, 11, 14, 17, 18, 20, 21, 42] being introduced. Among these search directions, the Alizadeh–Haeberly–Overton (AHO), Helmberg–Kojima–Monteiro (HKM) and Nesterov–Todd (NT) directions are better known, with the latter two being implemented in SDP solvers, such as SeDuMi, SDPT3. Various studies have been conducted to analyze primal–dual path following interior point algorithm, using the HKM search direction, such as [25], and the NT search direction, such as [16], to solve semi-definite programs. Works [19, 27, 30] have also been done to give an unified polynomial complexity analysis of interior point algorithms on semi-definite programs using a commutative class of search directions, which include the HKM and NT directions. More recently, a stream of research [10, 29, 43, 44] has appeared that derives polynomial complexity for a full NT step interior point method in solving linear programs, semi-definite programs and symmetric cone programs. Recent works on interior point methods include [2, 6, 28] that design new interior point algorithms, different from that considered in this paper, to solve symmetric cone programs.
The focus of this paper is on analyzing an infeasible predictor–corrector primal–dual path following interior point algorithm, using the NT search direction, to solve semi-definite linear complementarity problems. We consider an infeasible interior point algorithm since it is more practical than a feasible interior point algorithm, as it is usually difficult to find an initial interior point iterate which is also feasible. The algorithm considered in this paper was studied in [25] to solve semi-definite programs using the HKM search direction. The analysis in [25] to show global convergence and polynomial complexity cannot be carried over in a straightforward manner to analyze the algorithm using the search direction considered in this paper. Our contributions in this paper include showing polynomial complexity \({\mathcal {O}}(n \ln ( \max \{n\tau _0, \Vert r_0\Vert _2\}/\epsilon ))\) for the infeasible interior point algorithm using the NT search direction to solve a semi-definite linear complementarity problem. Our result complements the result obtained in [25] which considers the algorithm using the HKM search direction to solve semi-definite programs. Our iteration complexity bound is the best known iteration bound for infeasible interior point algorithms using the “narrow” neighborhood known so far in the literature. To the best of our knowledge, it is also the first time this polynomial complexity is derived for an infeasible predictor–corrector primal–dual path following interior point algorithm, using the NT search direction, on a semi-definite linear complementarity problem. Furthermore, under strict complementarity assumption, we provide local convergence results in Sect. 4 that are analogous to that in [34]. Note that superlinear convergence results using interior point methods are hard to obtain, to quote the opening sentences of [22]: “Local superlinear convergence is a natural and very desirable property of many methods in nonlinear optimization. However, for interior-point methods the corresponding analysis is not trivial”. It is worthwhile mentioning that among these local convergence results, we show that for the important class of linear semi-definite feasibility problems, only a suitably chosen initial iterate is needed for superlinear convergence, unlike what is generally believed to be needed to achieve superlinear convergence, which is for iterates to get close to the central path by repeatedly solving the corrector-step linear system in an iteration (see for example [13, 22]). We should also mention that although local convergence results using the NT search direction has been established in [16] by “narrowing” the central path neighborhood, the algorithm considered there generates feasible iterates, while here we consider an infeasible algorithm, which is more practical, and the analysis for the infeasible case is more complicated than that for the feasible case. Finally, our results in this paper indicate that using the NT search direction in an interior point algorithm to solve semi-definite linear complementarity problems is as good as using the HKM search direction both from the “polynomial complexity” and the “local convergence” point of view.
1.1 Facts, notations and terminology
The space of symmetric \(n \times n\) matrices is denoted by \(S^n\). The cone of positive semi-definite (resp., postive definite) symmetric matrices is denoted by \(S^n_+\) (resp., \(S^n_{++}\)). The identity matrix is denoted by \(I_{n \times n}\), where n stands for the size of the matrix. We omit the subscript when the size of the identity matrix is clear from the context. Given a symmetric matrix G, \(\lambda _{\min }(G)\) and \(\lambda _{\max }(G)\) are denoted to be the minimum and maximum eigenvalue of G respectively.
Given matrices G and K in \(\mathfrak {R}^{n_1 \times n_2}\), the inner product, \(G \bullet K\), between the two matrices is defined to be \(G \bullet K := {{\mathrm{Tr}}}(G^TK) = {{\mathrm{Tr}}}(GK^T)\), where \({{\mathrm{Tr}}}(\cdot )\) is the trace of a square matrix. \(\Vert \cdot \Vert _2\) for a vector in \(\mathfrak {R}^n\) refers to its Euclidean norm, and for a matrix in \(\mathfrak {R}^{n_1 \times n_2}\), it refers to its operator norm. On the other hand, \(\Vert G \Vert _F := \sqrt{G \bullet G}\), for \(G \in \mathfrak {R}^{n_1 \times n_2}\), refers to the Frobenius norm of G.
For a matrix \(G \in \mathfrak {R}^{n_1 \times n_2}\), we denote its component in the ith row and the jth column by \(G_{ij}\). \(G_{i \cdot }\) denotes the ith row of G and \(G_{\cdot j}\) the jth column of G. In the case when G is partitioned into blocks of submatrices, then \(G_{ij}\) refers to the submatrix in the corresponding (i, j) position.
Given square matrices \(G_i \in \mathfrak {R}^{n_i \times n_i}, i =1, \ldots , N\), \({\mathrm{Diag}}(G_1, \ldots , G_N)\) is a square matrix with \(G_i, i = 1, \ldots , N\), as its main diagonal blocks arranged in accordance to the way they are lined up in \({\mathrm{Diag}}(G_1, \ldots , G_N)\). All the other entries in \({\mathrm{Diag}}(G_1, \ldots , G_N)\) are zeroes.
Given \(X \in S^n\), \({\mathrm{svec}}(X)\) is defined to be
where \({\tilde{n}} := n(n+1)/2\). \({\mathrm{svec}}(\cdot )\) sets up a one-to-one correspondence between \(S^n\) and \(\mathfrak {R}^{{\tilde{n}}}\).
Note that for all \(X, Y \in S^n\), \(X \bullet Y = {\mathrm{svec}}(X)^T {\mathrm{svec}}(Y)\). Hence, \(\Vert X \Vert _F = \Vert {\mathrm{svec}}(X) \Vert _2\) for \(X \in S^n\).
Given \(G, K \in \mathfrak {R}^{n \times n}\), \(G \otimes _s K\) is a square matrix of size \({\tilde{n}}\) defined by
Fact 1
(Appendix of [39]) Let \(G, K, L \in \mathfrak {R}^{n \times n}\).
-
(a)
\(G \otimes _s K = K \otimes _s G\) and \((G \otimes _s K)^T = G^T \otimes _s K^T\).
-
(b)
\((G \otimes _s K)(L \otimes _s L) = (GL)\otimes _s(KL)\) and \((L \otimes _s L)(G \otimes _s K)= (LG) \otimes _s (LK)\).
-
(c)
If G and K are commuting symmetric matrices. Let \(\{x_i\}\) be their common basis of eigenvectors with corresponding eigenvalues \(\lambda _i^G\) and \(\lambda _i^K\). Then \(G \otimes _s K\) is symmetric and has the set of eigenvalues given by \(\left\{ \frac{1}{2}(\lambda _i^G \lambda _j^K + \lambda _j^G \lambda _i^K) \right\} \). Also, \({\mathrm{svec}}\left( \frac{1}{2}(x_i x_j^T + x_j x_i^T)\right) \) is an eigenvector corresponding to the eigenvalue \(\frac{1}{2} (\lambda _i^G \lambda _j^K + \lambda _j^G \lambda _i^K)\) of \(G \otimes _s K\).
Fact 2
([12]) For \(G, K \in \mathfrak {R}^{n \times n}\), \(\Vert GK \Vert _F \le \min \{\Vert G\Vert _F \Vert K\Vert _2, \Vert G\Vert _2 \Vert K\Vert _F\}\).
Fact 3
For \(x \in \mathfrak {R}, x \ge 0\), we have
Given functions \(f: \Omega \rightarrow E\) and \(g: \Omega \rightarrow \mathfrak {R}_{++}\), where \(\Omega \) is an arbitrary set and E is a normed vector space with norm \(\Vert \cdot \Vert \). For a subset \({\hat{\Omega }} \subseteq \Omega \), we write \(f(w) = {\mathcal {O}}(g(w))\) for all \(w \in {\hat{\Omega }}\) to mean that \(\Vert f(w) \Vert \le M g(w)\) for all \(w \in {\hat{\Omega }}\), where \(M > 0\) is a positive constant. Suppose \(E = S^n\). Then we write \(f(w) = \Theta (g(w))\) if for all \(w \in {\hat{\Omega }}\), \(f(w) \in S^n_{++}\), \(f(w) = {\mathcal {O}}(g(w))\) and \(f(w)^{-1} = {\mathcal {O}}(1/g(w))\). The subset \({\hat{\Omega }}\) should be clear from the context. For example, \({\hat{\Omega }} = (0,{\hat{w}})\) for some \({\hat{w}} > 0\) or \({\hat{\Omega }} = \{w_k\ ; \ k \ge 0\}\), where \(w_k \rightarrow 0\) as \(k \rightarrow \infty \). In the latter case, we write \(f(w_k) = o(g(w_k))\) to mean that \(\Vert f(w_k) \Vert /g(w_k) \rightarrow 0\), as \(k \rightarrow \infty \).
2 A primal–dual path following interior point algorithm on an SDLCP
We consider a semi-definite linear complementarity problem (SDLCP) which is the problem to find a solution, (X, Y), to the following system:
where \(q \in \mathfrak {R}^{{\tilde{n}}}\) and \({\mathcal {A}}, {\mathcal {B}}: S^n \rightarrow \mathfrak {R}^{{\tilde{n}}}\) are linear operators mapping \(S^n\) to the space \(\mathfrak {R}^{{\tilde{n}}}\), \({\tilde{n}} := n(n+1)/2\). \({\mathcal {A}}\) and \({\mathcal {B}}\) take the form
where \(A_i, B_i \in S^n\) for all \(i = 1, \ldots , {\tilde{n}}\). We also called the system (1)–(3) an SDLCP.
The following assumptions are assumed to hold for the system (1)–(3) in this and the next section, while we replace Assumption 1(b) by Assumption 2 in Sect. 4, although we still assume Assumptions 1(a), (c) in that section.
Assumption 1
-
(a)
System (1)–(3) is monotone. That is, \({\mathcal {A}}(X) + {\mathcal {B}}(Y) = 0\) for \(X, Y \in S^n \Rightarrow X \bullet Y \ge 0\).
-
(b)
There exists \((X^1,Y^1) \in S^n_{++} \times S^n_{++}\) such that \({\mathcal {A}}(X^1) + {\mathcal {B}}(Y^1) = q\).
-
(c)
\(\{ {\mathcal {A}}(X) + {\mathcal {B}}(Y)\ ; \ X, Y \in S^n \} = \mathfrak {R}^{{\tilde{n}}}\).
The first assumption [Assumption 1(a)] is satisfied for the class of semi-definite programs (SDPs), with equality for \(X \bullet Y\) instead of inequality. The second assumption ensures that (2) is satisfied for some positive definite matrix pair, while the last assumption is a technical assumption that can be satisfied for any SDP. Note that Assumption 1(b) is only used in this paper to ensure the existence of a solution to the SDLCP (1)–(3).
An SDP in its primal and dual form is given by
In the above formulation of an SDP, it is without loss of generality to assume that \(A_i, i = 1, \ldots , m\), are linearly independent.
An SDP is a special case of an SDLCP by letting \(A_i = 0\) for \(i = m+1, \ldots , {\tilde{n}}\), \(B_i = 0\) for \(i = 1, \ldots , m\), in (4). \(B_i, i = m+1, \ldots , {\tilde{n}}\), in (4) are chosen to be linearly independent and belong to the subspace in \(S^n\) orthogonal to the space spanned by \(A_i, i = 1, \ldots , m\).
Primal–dual path following interior point algorithms can be used to solve an SDLCP. We consider an infeasible predictor–corrector primal–dual path following interior point algorithm, as found in [25, 34], in this paper. In [25, 34], the search direction used is the Helmberg–Kojima–Monteiro (HKM) search direction [11, 14, 17], while in this paper, we consider the algorithm using the Nesterov–Todd (NT) search direction [20, 21]. The difference between the two search directions is the way “symmetrization” is being done on (1). For \(P \in \mathfrak {R}^{n \times n}\) an invertible matrix, the similarly transformed symmetrization operator \(H_P(\cdot )\), introduced in [45], is given by
where \(U \in \mathfrak {R}^{n \times n}\). Hence, \(H_P(\cdot )\) is a map from \(\mathfrak {R}^{n \times n}\) to \(S^n\). Different search directions correspond to different P, as will be explained later.
Infeasible primal–dual path following interior point algorithm works on the principle that iterates generated by the algorithm “follows” an (infeasible) central path \((X(\mu ),Y(\mu ))\), \(\mu > 0\), which is the unique solution to
where
for some \(X_0, Y_0 \in S^n_{++}\) and \(\mu _0 > 0\). Here, \(X_0, Y_0\) are such that \(X_0 Y_0 = \mu _0 I\). The existence and uniqueness of this central path follows from Theorem 2.3 in [35]. It also follows from Theorem 2.4 in [35] that there exists a solution \((X^*, Y^*)\) to the SDLCP (1)–(3). Although these theorems apply to the feasible central path when \(r_0 = 0\) in [35], they can be easily shown to hold for an infeasible central path when \(r_0 \not = 0\) by assuming Assumptions 1(a)–(c). We leave their proofs as exercises for the reader.
From now onwards, \((X^*,Y^*)\) denotes a solution to the SDLCP (1)–(3).
Using \(H_P(\cdot )\), the above central path \((X(\mu ),Y(\mu ))\), \(\mu > 0\), is also the unique solution to
since we have for \(X, Y \in S^n_{++}\) and \(\mu > 0\),
Below we describe the infeasible predictor–corrector primal–dual path following interior point algorithm considered in this paper. This algorithm is the same as that in [25], although there is a wider choice for \(\beta _1, \beta _2\) here. The key to the algorithm is solving the following system of linear equations:
for \(\Delta X, \Delta Y \in S^n\), where \(\tau > 0\), \(0 \le \sigma \le 1\) and \({\bar{r}} \in \mathfrak {R}^{{\tilde{n}}}\). Different choice of \(\tau , \sigma \) and \({\bar{r}}\) results in different step in Algorithm 1 described below.
The following (narrow) neighborhood of the central path is used in this paper:
where \(\tau > 0\) and \(0< \beta < 1\).
Note that P that appears in (5) and (7) is not chosen arbitrary, but is related to \(X, Y \in S^n_{++}\) as we will see next, after describing the algorithm we are analyzing in this paper.
Algorithm 1
Given \(\epsilon > 0\). Choose \(\beta _1 < \beta _2\), with \(\beta _2^2/(2(1-\beta _2)) \le \beta _1< \beta _2< \beta _2/(1-\beta _2) < 1\). Choose \((X_0,Y_0) \in {\mathcal {N}}_1(\beta _1,\tau _0)\) with \(\mu _0 = \tau _0 = X_0 \bullet Y_0/n\). For \(k = 0, 1, \ldots \), do (a1) through (a3):
- (a1):
-
If \(\max \{ X_k \bullet Y_k, \Vert r_k\Vert _2 \} \le \epsilon \), where \(r_k = {\mathcal {A}}(X_k) + {\mathcal {B}}(Y_k) - q\), then report \((X_k,Y_k)\) as an approximate solution to the system (1)–(3), and terminate.
- (a2):
-
(Predictor Step) Find the solution \((\Delta X_k^p, \Delta Y_k^p)\) of the linear system (5), (6), with \(X = X_k, Y = Y_k, P = P_k\), \(\sigma = 0\), \(\tau = \tau _k\) and \({\overline{r}} = r_k\). Define
$$\begin{aligned} {\hat{X}}_k = X_k + {\hat{\alpha }}_k \Delta X_k^p,\ \ {\hat{Y}}_k = Y_k + {\hat{\alpha }}_k \Delta Y_k^p, \end{aligned}$$where the steplength \({\hat{\alpha }}_k\) satisfies
$$\begin{aligned} \alpha _{k,1} \le {\hat{\alpha }}_k \le \alpha _{k,2}. \end{aligned}$$(8)Here,
$$\begin{aligned} \alpha _{k,1}= & {} \frac{2}{\sqrt{1 + 4 \delta _k/(\beta _2 - \beta _1)} + 1}, \end{aligned}$$(9)$$\begin{aligned} \delta _k= & {} \frac{1}{\tau _k} \Vert H_{P_k}(\Delta X_k^p \Delta Y_k^p) \Vert _F, \end{aligned}$$(10)and
$$\begin{aligned} \alpha _{k,2}= & {} \max \{ {\tilde{\alpha }} \in [0,1]\ ;\ (X_k + \alpha \Delta X_k^p, Y_k + \alpha \Delta Y_k^p)\nonumber \\&\in {\mathcal {N}}_1(\beta _2,(1-\alpha )\tau _k) \ \forall \ \alpha \in [0, {\tilde{\alpha }}] \}. \end{aligned}$$(11)If \({\hat{\alpha }}_k = 1\), then \(({\hat{X}}_k,{\hat{Y}}_k)\) solves the system (1)–(3) and terminate.
- (a3):
-
(Corrector Step) Find the solution \((\Delta X_k^c, \Delta Y_k^c)\) of the linear system (5), (6), with \(X = {\hat{X}}_k, Y = {\hat{Y}}_k, P = {\hat{P}}_k\), \(\sigma = (1 - {\hat{\alpha }}_k)\), \(\tau = \tau _k\) and \({\overline{r}} = 0\). Set
$$\begin{aligned} \begin{array}{c} X_{k+1} = {\hat{X}}_k + \Delta X_k^c,\ \ Y_{k+1} = {\hat{Y}}_k + \Delta Y_k^c, \\ \tau _{k+1} = (1 - {\hat{\alpha }}_k) \tau _k. \end{array} \end{aligned}$$Set \(k+1 \rightarrow k\) and go to Step (a1).
The above algorithm is an infeasible predictor–corrector primal–dual path following interior point algorithm. \(P_k\) in the algorithm is chosen such that \(P_k X_k Y_k P_k^{-1} \in S^n\). Examples of \(P_k\) that satisfy this are \(P_k = Y_k^{1/2}\), and \(P_k\) such that \(P_k^T P_k = W_k^{-1}\) with \(W_k Y_k W_k = X_k\). The former corresponds to the dual HKM search direction, while the latter corresponds to the NT search direction. \({\hat{P}}_k\) in the algorithm is also chosen to satisfy \({\hat{P}}_k {\hat{X}}_k {\hat{Y}}_k {\hat{P}}_k^{-1} \in S^n\).
We remark that \((\Delta X_k^p, \Delta Y_k^p)\) and \((\Delta X_k^c, \Delta Y_k^c)\) in Algorithm 1 exist and are unique, by observing that the left hand side of (5) and (6) together when written in matrix-vector product has the matrix invertible, which holds because \((X_k, Y_k) \in {\mathcal {N}}_1(\beta _1,\tau _k)\) (by Proposition 4) and \(({\hat{X}}_k,{\hat{Y}}_k) \in {\mathcal {N}}_1(\beta _2, (1 - {\hat{\alpha }}_k) \tau _k)\), respectively. (We leave the details to show the existence and uniqueness of \((\Delta X_k^p, \Delta Y_k^p)\) and \((\Delta X_k^c, \Delta Y_k^c)\) to the reader.) Furthermore, in the above algorithm, we note that there is a wider range of choice for \(\beta _1\) and \(\beta _2\) compared with the algorithm in [25].
Let us make an observation on our choice of \(P_k, {\hat{P}}_k\) in the following proposition:
Proposition 1
Suppose \(P \in \mathfrak {R}^{n \times n}\) is an invertible matrix with \(P X Y P^{-1} \in S^n\), where \(X, Y \in S^n_{++}\). Then \(PXP^{T}\) and \(P^{-T} Y P^{-1}\) have a common set of eigenvectors with corresponding real positive eigenvalues \(\lambda _i^X\) and \(\lambda _i^Y\), \(i = 1, \ldots , n\), respectively. Also, \(PXYP^{-1}\) has the same set of eigenvectors with corresponding eigenvalues \(\lambda _i^X \lambda _i^Y\), \(i = 1, \ldots , n\).
Proof
Since \(PXYP^{-1} \in S^n\), this implies that \(PXP^T, P^{-T}YP^{-1} \in S^n_{++}\) commute. Hence they share a common set of eigenvectors with corresponding real positive eigenvalues \(\lambda _i^X, \lambda _i^Y, i = 1, \ldots , n\). Furthermore, it is easy to see that \(PXYP^{-1} = (PXP^{T})(P^{-T}YP^{-1})\) has the same set of eigenvectors with corresponding eigenvalues \(\lambda _i^X \lambda _i^Y, i = 1, \ldots , n\). \(\square \)
From now onwards, we consider Algorithm 1 using the NT search direction in Steps (a2) and (a3) of the algorithm, which means that \({{P}}_{{k}}\) and \({\hat{{{P}}}_{{k}}}\) in these steps satisfy \({{P}}_{{k}}^{{T}} {{P}}_{{k}} = {{W}}_{{k}}^{-{{1}}}\) with \({{W}}_{{k}} {{Y}}_{{k}} {{W}}_{{k}} = {{X}}_{{k}}\), and \({\hat{{{P}}}}_{{k}}^{{T}} \hat{{{P}}}_{{k}} = \hat{{{W}}}_{{k}}^{-{1}}\) with \({\hat{{{W}}}}_{{k}} \hat{{{Y}}}_{{k}} \hat{{{W}}}_{{k}} = \hat{{{X}}}_{{k}}\) respectively. Proposition 1 then applies to \(P_k\) and \({\hat{P}}_k\) given in this way since they satisfy \(P_k X_k Y_k P_k^{-1}, {\hat{P}}_k {\hat{X}}_k {\hat{Y}}_k {\hat{P}}_k^{-1} \in S^n\), respectively. Furthermore, there are different ways in which \(P_k\) and \({\hat{P}}_k\) can be chosen to form the NT search direction in these steps of Algorithm 1. In Sect. 4, using a particular choice of \(P_k\), we establish two sufficient conditions for superlinear convergence using the algorithm on SDLCPs that satisfy the strict complementarity assumption.
We also require that P that appears in (7) to be related to X, Y in (7) by \({{{P}}^{{T}} {{P}} = {{W}}^{-\mathbf{1}}}\) with \({{WYW}} = {{X}}\).
The following are properties satisfied by \((X_k,Y_k)\) and \(({\hat{X}}_k,{\hat{Y}}_k)\) in Algorithm 1, which are useful in the analysis given in the paper on the convergence behavior of iterates generated by the algorithm.
Proposition 2
Let \((X,Y) \in {\mathcal {N}}_1(\beta ,\tau )\), where \(\tau > 0\), \(0< \beta < 1\), and P satisfies \(P^T P = W^{-1}\) with \(WYW = X\), then
Proof
The relation (12) follows immediately from \(P^T P = W^{-1}\), \(W Y W = X\) and then taking inverses.
Since \((X,Y) \in {\mathcal {N}}_1(\beta ,\tau )\), we have \(X, Y \in S^n_{++}\) and
By Proposition 1 and observing that \(H_P(XY) = PXYP^{-1}\), we have using the notations in Proposition 1 that
Hence, for all \(i = 1, \ldots , n\),
therefore (13) follows.
By (12), \(\lambda _i^X = \lambda _i^Y\) for all \(i = 1, \ldots , n\). The second inequality in (18) then implies (14), (15). Also, from (18) and \(\lambda _i^X = \lambda _i^Y\), we have
for all \(i = 1, \ldots , n\). Inequalities in (16) then follow.
Using Fact 1(c), \(I \otimes _s (PXP^T)\) is symmetric and
Therefore,
which shows (17). \(\square \)
The following technical result leads to Proposition 4, which ensures that the set in (11) is nonempty.
Proposition 3
Given \((X,Y) \in {\mathcal {N}}_1(\beta , \tau )\), where \(\tau > 0\), \(0< \beta < 1\), and P satisfies \(P^TP = W^{-1}\) with \(WYW = X\). Suppose \(U, V \in S^n\) are such that
then
Proof
Relation (20) can be written as
The latter can in turn be expressed as
by Fact 1(b).
Since P satisfies \(P^TP = W^{-1}\) with \(WYW=X\), by (12), we have \(PXP^{T} = P^{-T} Y P^{-1}\). Hence, from (21), taking the inverse of \(I \otimes _s (P X P^T)\), we get
Hence,
where the first inequality follows from (19) and Assumption 1(a), the first equality follows from (22), and the last inequality follows from Proposition 2. \(\square \)
The following result shows that iterates \((X_k,Y_k)\) generated by Algorithm 1 always belong to the narrow neighborhood of the central path (7).
Proposition 4
For all \(k \ge 0\), \((X_k, Y_k) \in {\mathcal {N}}_1(\beta _1,\tau _k)\).
Proof
We prove the proposition by induction. It is easy to see that \((X_0, Y_0) \in {\mathcal {N}}_1(\beta _1, \tau _0)\). Hence, the proposition holds for \(k = 0\). Suppose the proposition holds for \(k = k_0\), where \(k_0 \ge 0\), that is, \((X_{k_0},Y_{k_0}) \in {\mathcal {N}}_1(\beta _1, \tau _{k_0})\). We wish to show that
which then prove the proposition by induction. First observe that
where the first equality follows since \(P_{k_0+1} X_{k_0+1}Y_{k_0+1} P_{k_0+1}^{-1} \in S^n\), and the inequality follows from Lemma 2.2 in [25], again using \(P_{k_0+1} X_{k_0+1} Y_{k_0+1} P_{k_0+1}^{-1} \in S^n\). Next, observe that
where the third equality holds since \((\Delta X_{k_0}^c, \Delta Y_{k_0}^c)\) is the solution to the linear system (5), (6), in which \( X = {\hat{X}}_{k_0}, Y = {\hat{Y}}_{k_0}, P = {\hat{P}}_{k_0}, \sigma = 1, \tau =\tau _{k_0+1}\) and \({\bar{r}} = 0\). Hence,
where the second inequality follows from Fact 2 and \(\Vert U \Vert _2 \le \Vert U\Vert _F\) for \(U \in S^n\).
Since \((\Delta X_{k_0}^c, \Delta Y_{k_0}^c)\) is the solution to the linear system (5), (6), in which \(X = {\hat{X}}_{k_0}, Y = {\hat{Y}}_{k_0}, P = {\hat{P}}_{k_0}, \sigma = 1, \tau =\tau _{k_0+1}\) and \({\bar{r}} = 0\), where \(({\hat{X}}_{k_0},{\hat{Y}}_{k_0}) \in {\mathcal {N}}_1(\beta _2, \tau _{k_0+1})\), it follows from Proposition 3 that
Therefore, by the above inequality, (23) and (24) leads to
where the last inequality holds since \(({\hat{X}}_{k_0},{\hat{Y}}_{k_0}) \in {\mathcal {N}}_1(\beta _2, \tau _{k_0+1})\).
Since \(\beta _1, \beta _2 \in (0,1)\) are chosen such that \(\beta _2^2/(2(1-\beta _2)) \le \beta _1\), we have from above, \((X_{k_0+1},Y_{k_0+1}) \in {\mathcal {N}}_1(\beta _1,\tau _{k_0+1})\). By induction, the proposition is proved. \(\square \)
As mentioned earlier, the above proposition ensures that \(\alpha _{k,2}\) given by (11) is meaningful. The following result allows us to say more about \(\alpha _{k,2}\) and also shows that we can always find \({\hat{\alpha }}_k\) that satisfies (8) in Algorithm 1.
Proposition 5
We have \(\alpha _{k,1} \le \alpha _{k,2}\).
Proof
The proposition is proved by showing that for all \(0 \le \alpha \le \alpha _{k,1}\),
We have for all \(0 \le \alpha \le \alpha _{k,1}\)
where the second equality holds as \((\Delta X_k^p, \Delta Y_k^p)\) satisfies the linear system (5), (6), with \(X = X_k, Y = Y_k, P = P_k\), \(\sigma = 0\), \(\tau = \tau _k\) and \({\overline{r}} = r_k\), the second inequality holds as \((X_k, Y_k) \in {\mathcal {N}}_1(\beta _1,\tau _k)\), and the last inequality follows from (9), (10). Hence, since \(X_k, Y_k \in S^n_{++}\) and \(\beta _2 < 1\), we see that if \(0< \alpha \le \alpha _{k,1} < 1\), \(X_k + \alpha \Delta X_k^p, Y_k + \alpha \Delta Y_k^p \in S^n_{++}\).
Now, since \(X_k + \alpha \Delta X_k^p, Y_k + \alpha \Delta Y_k^p \in S^n_{++}\), the following holds.
where \(P_k(\alpha )\) is such that \(P_k(\alpha )^T P_k(\alpha ) = W_k(\alpha )^{-1}\) with \(W_k(\alpha )(Y_k + \alpha \Delta Y_k^p) W_k = X_k + \alpha \Delta X_k^p\) and hence \(P_k(\alpha )(X_k + \alpha \Delta X_k^p)(Y_k + \alpha \Delta Y_k^p)P_k(\alpha )^{-1} \in S^n\), so that the above inequality holds by Lemma 2.2 in [25].
Putting everything together, if \(0 \le \alpha \le \alpha _{k,1}\), we have \((X_k + \alpha \Delta X_k^p, Y_k + \alpha \Delta Y_k^p) \in {\mathcal {N}}_1(\beta _2, (1 - \alpha )\tau _k)\), as required. \(\square \)
We remark that the above proposition implies that \(\alpha _{k,2}\) given by (11) is always positive.
3 Global convergence and polynomial complexity of interior point algorithm
In this section, we show global convergence and iteration complexity results for iterates \(\{(X_k,Y_k)\}\) generated by Algorithm 1, by considering the “duality gap”, \(\mu _k := X_k \bullet Y_k/n\), and the “feasibility gap”, \(r_k = {\mathcal {A}}(X_k) + {\mathcal {B}}(Y_k) - q\). The following proposition relates \({\hat{\alpha }}_k\), \(\tau _k\) and \((X_k,Y_k)\) generated by Algorithm 1 with \(\mu _k\) and \(r_k\). First, we have the following definition.
Definition 1
Define for \(k \ge 0\), \(\psi _k = \Pi _{j=0}^{k} (1 - {\hat{\alpha }}_j)\). Also, define \(\psi _{-1} = 1\).
Proposition 6
For all \(k \ge 0\), \((1 - \beta _1)\tau _k \le \mu _k \le (1 + \beta _1)\tau _k\) and \({\mathcal {A}}(X_k) + {\mathcal {B}}(Y_k) - q = r_k = \psi _{k-1} r_0\).
Proof
By Proposition 4, for all \(k \ge 0\), \((X_k, Y_k) \in {\mathcal {N}}_1(\beta _1, \tau _k)\). Hence,
It then follows from Proposition 1 that
where \(\lambda _i^{X_k} \lambda _i^{Y_k}\), \(i = 1, \ldots , n\) are the eigenvalues of \(P_k X_k Y_k P_k^{-1}\). Therefore, for all \(i = 1, \ldots , n\),
The first result in the proposition then follows by noting that \(X_k \bullet Y_k = \sum _{i=1}^{n} \lambda _i^{X_k} \lambda _i^{Y_k}\).
Next, we show that
by induction on \(k \ge 0\).
Equality in (25) holds for \(k = 0\). Suppose (25) holds for \(k = k_0\) for some \(k_0 \ge 0\). Then
where the first equality follows from \(X_{k_0+1} = {\hat{X}}_{k_0} + \Delta X_{k_0}^c, Y_{k_0+1} = {\hat{Y}}_{k_0} + \Delta Y_{k_0}^c\) and \((\Delta X_{k_0}^c,\Delta Y_{k_0}^c)\) satisfying (6) with \({\bar{r}} = 0\), the third equality follows from \((\Delta X_{k_0}^p,\Delta Y_{k_0}^p)\) satistying (6) with \({\bar{r}} = r_{k_0}\), and the fifth equality follows by induction hypothesis. Hence, (25) holds for \(k = k_0 + 1\), and by induction, (25) holds for all \(k \ge 0\). \(\square \)
To show global convergence and iteration complexity results using \(\mu _k\) and \(r_k\), we only need to investigate the behavior of \(\psi _{k-1}\), since \(\tau _k = \psi _{k-1} \tau _0\) and \(\mu _k \le (1 + \beta _1) \tau _k\), and \(r_k = \psi _{k-1} r_0\). By definition of \(\psi _k\) in Definition 1, this is achieved by analyzing \({\hat{\alpha }}_j\). We consider \(\alpha _{j,1}\) instead, which is given by (9) since it is a lower bound to \({\hat{\alpha }}_j\). Since \(\alpha _{j,1}\) given by (9) is expressed in terms of \(\delta _j\), to analyze \(\alpha _{j,1}\), we only need to analyze \(\delta _j\). We have the following upper bound on \(\delta _j\).
Lemma 1
For all \(k \ge 0\), we have
where
Here,
We prove the above lemma in Sect. 3.1, as its proof is quite involved.
Based on Lemma 1, we have the following global convergence theorem using Algorithm 1.
Theorem 1
Given \((X_0,S_0) \in {\mathcal {N}}_1(\beta _1,\tau _0)\), we have \(\mu _k \rightarrow 0\) and \(r_k \rightarrow 0\) as \(k \rightarrow \infty \), and hence any accumulation point of the sequence \(\{(X_k,Y_k)\}\) generated by Algorithm 1 is a solution to the SDLCP (1)–(3).
Proof
Since \((X_0,S_0) \in S^n_{++}\) and for any \((X^*, Y^*)\), a solution to the SDLCP (1)–(3), it is easy to see that \(L_x\) and \(L_y\) given by (26), (27) respectively are positive constants, since \(\varsigma \), \(\varsigma _x\) and \(\varsigma _y\) are constants. Therefore, by the relation between \(\delta _j\) and \(L_x, L_y\) in Lemma 1, \(\delta _j\) is bounded above by a positive constant (that depends on \(X_0, Y_0, X^*, Y^*\)), say, L, independent of \(j \ge 0\). From (9), we therefore have
for all \(j \ge 0\). Since
\(\psi _k\) tends to zero as \(k \rightarrow \infty \).
Therefore, the theorem is proved by applying Proposition 6. \(\square \)
Let us now state an iteration complexity result using Algorithm 1.
Theorem 2
Let \((X_0, Y_0) \in {\mathcal {N}}_1(\beta _1,\tau _0)\) in Algorithm 1 be chosen such that there exists \((X^*, Y^*)\) a solution to the SDLCP (1)–(3) with \(\max \{ \Vert P_0 X^*P_0^{T} \Vert _F, \Vert P_0^{-T} Y^*P_0^{-1} \Vert _F \} \le \sqrt{\tau _0}\). Then given \(\epsilon > 0\), if Algorithm 1 does not stop at Step (a2) before the kth iteration with a solution to the SDLCP (1)–(3), it stops at the kth iteration in Step (a1), where \(k = {\mathcal {O}}(n \ln ( \max \{n\tau _0, \Vert r_0\Vert _2\}/\epsilon ))\), with an \(\epsilon \)-approximate solution to the SDLCP (1)–(3).
Proof
We are given \((X_0, Y_0)\) in \({\mathcal {N}}_1(\beta _1, \tau _0) \subset S^n_{++} \times S^n_{++}\), and \((X^*, Y^*)\) in \(S^n_+ \times S^n_+\) such that \(\max \{ \Vert P_0 X^*P_0^T \Vert _F,\)\(\Vert P_0^{-T} Y^*P_0^{-1} \Vert _F \} \le \sqrt{\tau _0}\).
Observe that
where the second inequality holds by (14) (since \((X_0, Y_0) \in {\mathcal {N}}_1(\beta _1,\tau _0)\)), \(\max \{ \Vert P_0 X^* P_0^{T} \Vert _F,\)\(\Vert P_0^{-T} Y^*P_0^{-1} \Vert _F \} \le \sqrt{\tau _0}\) and \(X_0 \bullet Y_0 = n \tau _0\).
Also,
using (16) and \(\max \{ \Vert P_0 X^*P_0^T \Vert _F, \Vert P_0^{-T} Y^*P_0^{-1} \Vert _F \} \le \sqrt{\tau _0}\).
Therefore, \(L_x, L_y\) given by (26), (27) respectively are less than or equal to \(L_0 n\), where \(L_0\) is a large enough number that depends only on \(\beta _1\). Hence, by Lemma 1, (8) and (9),
for all \(j \ge 0\).
Therefore,
where the last inequality holds by Fact 3.
Since \(\max \{X_k \bullet Y_k, \Vert r_k\Vert _2 \} \le \epsilon \) is needed for the algorithm to terminate at Step (a1), where \(X_k \bullet Y_k/n = \mu _k \le (1 + \beta _1) \tau _k = (1+ \beta _1) \psi _{k-1} \tau _0\) and \(r_k = \psi _{k-1} r_0\), a sufficient condition for termination at Step (a1) at the kth iteration is when k satisfies
That is,
and the result then follows. \(\square \)
3.1 Proof of Lemma 1
Note that
where the first inequality holds by Fact 2. To show Lemma 1, we analyze \(\Vert P_k \Delta X^p_k P^{T}_k \Vert _F\) and \(\Vert P^{-T}_k \Delta Y^p_k P^{-1}_k \Vert _F\) that appear in (28) further by bounding them from above as given in the following proposition.
Proposition 7
We have
where
Proof
We first observe that \((\Delta X^p_k, \Delta Y^p_k)\) satisfies
Now, by Proposition 6,
and with \({\mathcal {A}}(X^*) + {\mathcal {B}}(Y^*) = q\), from (31), we obtain
That is,
On the other hand,
where the last equality holds since \((X_k,Y_k)\) satisfies (5) with \(X = X_k, Y = Y_k, \Delta X = \Delta X^p_k, \Delta Y = \Delta Y^p_k, \sigma = 0\). Since \((X_k,Y_k) \in {\mathcal {N}}_1(\beta _1,\tau _k)\), Proposition 3 can be applied to (32), (33). We have from the proposition
The proposition then follows by applying triangle inequality to (34), (35) and upon algebraic manipulations. \(\square \)
Our objective to prove Lemma 1 is achieved by bounding \(t_x, t_y\) and t that appear in the upper bounds to \(\Vert P_k \Delta X_k^p P_k^T \Vert _F, \Vert P_k^{-T} \Delta Y_k^p P_k^{-1} \Vert _F\) in the above proposition appropriately. We need the following results to achieve this.
Proposition 8
For all \(k \ge 0\), \(Y_k \bullet X_0 + Y_0 \bullet X_k \le (2 + \beta _1 + \varsigma ) n \tau _0\).
Proof
By Proposition 6,
Hence,
Assumption 1(a) implies that
from which we have
Result then follows from the definition of \(\varsigma \), (13) (since \((X_k, Y_k) \in {\mathcal {N}}_1(\beta _1,\tau _k)\)), \(\tau _k = \psi _{k-1}\tau _0\), where \(\psi _{k-1} \le 1\), and \(X_0 \bullet Y_0 = n \mu _0 = n \tau _0\). \(\square \)
Remark 1
Since \((X_0,Y_0) \in S^n_{++} \times S^n_{++}\), we see easily from the above proposition that \(\{ (X_k, Y_k)\ ; \ k \ge 0 \}\) is bounded.
Using Proposition 8, the following holds.
Proposition 9
For all \(k \ge 0\),
Proof
It suffices to prove the first inequality since the proofs for the last three inequalities are similar.
We have
where the first inequality holds since \(P_k X_0 P_k^T \in S^n_{++}\), the second inequality holds by Fact 2, and the last inequality holds by Proposition 8. \(\square \)
With the above, we are ready to prove Lemma 1 by providing suitable upper bounds for \(t_x, t_y\) and t as given below.
Proposition 10
Proof
We first prove the upper bound on t. We have
Since \((X_k, Y_k) \in {\mathcal {N}}_1(\beta ,\tau _k)\),
Hence,
On the other hand,
Now,
where the third inequality follows by Proposition 9, and the last inequality follows from (15), (16). Similarly,
Putting everything together, we have (36).
In a similar way, we can show (37) and (38). \(\square \)
From (28), (29), (30), using Proposition 10, Lemma 1 is proved.
4 Local convergence study of interior point algorithm
In this section, we investigate the local convergence behavior of iterates \(\{ (X_k, Y_k) \}\) generated by Algorithm 1.
We first need an assumption as follows.
Assumption 2
There exists a strictly complementary solution \((X^*,Y^*)\) to the SDLCP (1)–(3). That is, \((X^*,Y^*)\) satisfies \(X^*+ Y^*\in S^n_{++}\).
Assumption 2 is usually applied when we study the local convergence behavior of interior point algorithms on semi-definite linear complementary problems [13, 16, 24, 25, 34]. Paper [5] consider asymptotic behavior of the central path for an SDP when this assumption is relaxed.
In this section, we consider an SDLCP (1)–(3) that satisfies Assumptions 1(a),(c) and Assumption 2.
\((X^*,Y^*)\) now denotes a strictly complementary solution to the SDLCP (1)–(3). Since \(X^*, Y^*\) commute, they are jointly diagonalizable by some orthogonal matrix Q. Applying this orthogonal similarity transformation on the matrices in the SDLCP (1)–(3), we may assume without loss of generality that
where \(\Lambda ^{X^*} = {\mathrm{Diag}}(\lambda _1^{X^*}, \ldots , \lambda _{k_0}^{X^*}) \in S^{k_0}_{++}\) and \(\Lambda ^{Y^*} = {\mathrm{Diag}}(\lambda _1^{Y^*}, \ldots , \lambda _{n-k_0}^{Y^*}) \in S^{n-k_0}_{++}\).
Henceforth, whenever we partition a matrix \(U \in S^n\), it is always understood that it is partitioned as \(\left[ \begin{array}{cc} U_{11} &{} U_{12} \\ U_{12}^T &{} U_{22} \end{array} \right] \), where \(U_{11} \in S^{k_0}\), \(U_{22} \in S^{n - k_0}\) and \(U_{12} \in \mathfrak {R}^{k_0 \times (n - k_0)}\).
We study local superlinear convergence using Algorithm 1 in the sense of
This is equivalent to
by Proposition 6. Note that we have \(\tau _k \rightarrow 0\) as \(k \rightarrow \infty \), by Theorem 1 and Proposition 6.
Superlinear convergence in the sense of (40) is intimately related to local convergence behavior of iterates, as investigated for example in [23]. The following can be verified easily.
Proposition 11
A sufficient condition for (41) to hold is
Proof
Note that (41) holds if \(\alpha _{k,1} \rightarrow 1\) as \(k \rightarrow \infty \). By (9), this sufficient condition is equivalent to \(\delta _k \rightarrow 0\) as \(k \rightarrow \infty \), where \(\delta _k\) is given by (10). Hence, the proposition is proved. \(\square \)
In the rest of this section, we are going to show that if certain condition holds for \((X_k, Y_k)\) and for certain subclass of SDLCPs, superlinear convergence using Algorithm 1 can be achieved. This is achieved by showing that the above sufficient condition (42) holds. Towards this end, we transform the system of equations, (5), (6), that relates \((\Delta X_k^p, \Delta Y_k^p)\) to \((X_k,Y_k)\) to an equivalent system of equations, and then analyze the resulting system.
From Step (a1) of Algorithm 1, where the system of Eqs. (5), (6) is used with \(\Delta X = \Delta X_k^p, \Delta Y = \Delta Y_k^p, X = X_k, Y = Y_k, P = P_k, \sigma = 0, \tau = \tau _k\) and \({\overline{r}} = r_k = (\tau _k/\tau _0)r_0\), we note that \((\Delta X_k^p, \Delta Y_k^p)\) satisfies
where \({\mathscr {A}}: \mathfrak {R}^{{\tilde{n}}} \rightarrow \mathfrak {R}^{{\tilde{n}}}\) and \({\mathscr {B}}: \mathfrak {R}^{{\tilde{n}}} \rightarrow \mathfrak {R}^{{\tilde{n}}}\) are defined by \({\mathscr {A}}{\mathrm{svec}}(U) := {\mathcal {A}}(U)\) and \({\mathscr {B}}{\mathrm{svec}}(U) := {\mathcal {B}}(U)\), respectively, for \(U \in S^n\). Here, \(({\mathscr {A}}\ {\mathscr {B}})\) has full row rank equal to \({\tilde{n}}\) by Assumption 1(c).
We observe that (43), (44) can be written in the following way.
by (12).
The task now is to transform (45) to an equivalent system of equations that allows us to show that if certain condition on iterates \((X_k,Y_k)\) as given in Theorem 3 is satisfied, and for certain subclass of SDLCPs as given in Theorem 4, superlinear convergence using Algorithm 1 can be ensured. First, we observe the following.
Proposition 12
\(X_k = \left[ \begin{array}{cc} \Theta (1) &{} {\mathcal {O}}(\sqrt{\tau _k}) \\ {\mathcal {O}}(\sqrt{\tau _k}) &{} \Theta (\tau _k) \end{array} \right] ,\ Y_k = \left[ \begin{array}{cc} \Theta (\tau _k) &{} {\mathcal {O}}(\sqrt{\tau _k}) \\ {\mathcal {O}}(\sqrt{\tau _k}) &{} \Theta (1) \end{array} \right] \).
Since the proof of the above proposition follows from proofs of similar results in [26, 36] and the relation between \(\mu _k\) and \(\tau _k\) as stated in Proposition 6, it will not be shown here.
The new system of equations that we are going to derive involves “iterate” \(({\bar{X}}_k, {\bar{Y}}_k)\) corresponding to \((X_k,Y_k)\), \({\bar{W}}_k\) corresponding to \(W_k\), and corresponding “predicted step” \((\Delta {\bar{X}}_k^p, \Delta {\bar{Y}}_k^p)\) to \((\Delta {X}_k^p, \Delta {Y}_k^p)\).
Define
and
Proposition 13
Also, any accumulation point of \(\{{\bar{X}}_k\}, \{{\bar{Y}}_k\}, \{{\bar{W}}_k\}\), as k tends to infinity, are symmetric, positive definite matrices.
Proof
Observe by Proposition 12 that
and by (13) (from which we get \(1 - \beta _1 \le \lambda _{\min }({\bar{X}}_k {\bar{Y}}_k) \le \lambda _{\max }({\bar{X}}_k {\bar{Y}}_k) \le 1 + \beta _1\)), any accumulation point of \(\{ {\bar{X}}_k \}, \{ {\bar{X}}_k^{-1}\}\) or \(\{ {\bar{Y}}_k \}, \{ {\bar{Y}}_k^{-1} \}\) are symmetric, positive definite as k tends to infinity, with
By definition of \({\bar{W}}_k\) in (46), we have \({\bar{W}}_k {\bar{Y}}_k {\bar{W}}_k = {\bar{X}}_k\), which implies that \({\bar{W}}_k = {\bar{X}}_k^{1/2}({\bar{X}}_k^{1/2} {\bar{Y}}_k {\bar{X}}_k^{1/2})^{-1/2} {\bar{X}}_k^{1/2}\), from which we see that \(\{ {\bar{W}}_k \}\) is a sequence of symmetric, positive definite matrices and has accumulation points which are symmetric, positive definite as k tends to infinity, since these are so for \(\{ {\bar{X}}_k \}\) and \(\{ {\bar{Y}}_k \}\). Therefore, (47) holds. \(\square \)
By partitioning matrices \(A_i\) and \(B_i\) in \({\mathcal {A}}\), \({\mathcal {B}}\), as appeared in (4), respectively into the 4 blocks format as discussed near the beginning of this section, we perform block Gaussian elimination on \(({\mathscr {A}}\ {\mathscr {B}})\) so that \({\mathscr {A}}, {\mathscr {B}}\) can be rewritten as
respectively. This technique has been used for example in [33, 36]. See also [15, 26]. We will take \({\mathscr {A}}, {\mathscr {B}}\) to be expressed as in (48) from now onwards. Note that \(({\mathscr {A}}\ {\mathscr {B}}) \in \mathfrak {R}^{{\tilde{n}} \times 2 {\tilde{n}}}\) has full row rank by Assumption 1(c), and \({\mathscr {A}}, {\mathscr {B}}\) also satisfy
The implication in (49) holds by Assumption 1(a).
Remark 2
In the case of an SDP, \({\mathscr {A}}\) and \({\mathscr {B}}\) are written as
where \({\mathscr {A}}_1\) consists of m rows and \({\mathscr {B}}_1\) consists of \({\tilde{n}} - m\) rows. As discussed in [34], by performing block Gaussian elimination, \({\mathscr {A}}_1\) and \({\mathscr {B}}_1\) are given by
respectively. Note that the way \({\mathscr {A}}\), \({\mathscr {B}}\) for an SDP are written in (50) is different from that for an SDLCP, see (48). We can however write them in the form of (48) by appropriately interchanging rows in \(({\mathscr {A}} \ {\mathscr {B}})\) for the SDP.
Now, in order to transform the equation system (45) to an equivalent system, let us define \(\bar{{\mathscr {A}}}(\tau ) \in \mathfrak {R}^{{\tilde{n}} \times {\tilde{n}}}\) and \(\bar{{\mathscr {B}}}(\tau ) \in \mathfrak {R}^{{\tilde{n}} \times {\tilde{n}}}\) to be
respectively, for \(\tau \ge 0\).
The following proposition, whose proof can be found for example in [32, 33, 36], relates \({\mathscr {A}}\) with \(\bar{{\mathscr {A}}}(\tau )\) and \({\mathscr {B}}\) with \(\bar{{\mathscr {B}}}(\tau )\):
Proposition 14
and
An important property of \(\bar{{\mathscr {A}}}(\tau ), \bar{{\mathscr {B}}}(\tau )\) is given below.
Proposition 15
For \(\tau \ge 0\), \((\bar{{\mathscr {A}}}(\tau )\ \bar{{\mathscr {B}}}(\tau ))\) has full row rank, and
Proof
We note that \(({\mathscr {A}}(\tau )\ {\mathscr {B}}(\tau ))\) has full row rank for all \(\tau \ge 0\) follows from Assumption 1(c) and the way block Gaussian elimination is performed on \(({\mathscr {A}}\ {\mathscr {B}})\) to obtain \({\mathscr {A}}, {\mathscr {B}}\) in the form (48)—see explanations for example in [36].
The implication in (54) holds for \(\tau > 0\) follows from Assumption 1(a) and (52), (53). For \(\tau = 0\), (54) holds by following the proof of Proposition 2.4 in [36] (see also the proof of Theorem 3.13 in [26]). \(\square \)
Because of the way \({\mathscr {A}}, {\mathscr {B}}\) are now structured, we have
Proposition 16
q has the following form
where \(q^1 \in \mathfrak {R}^{i_1}\).
Proof
By Proposition 6, the following equation holds
This equation is equivalent to
Since the left hand side of the above equation and \(\frac{\tau _k}{\tau _0} {\mathrm{Diag}}\Big ( I_{i_1 \times i_1}, \frac{1}{\sqrt{\tau _k}}I_{i_2 \times i_2}, \frac{1}{\tau _k}I_{({\tilde{n}} - i_1 - i_2) \times ({\tilde{n}} - i_1 - i_2)} \Big ) r_{0}\) are bounded as k tends to infinity, we conclude that q must take the form as given in the proposition. \(\square \)
Remark 3
From the proof of the above proposition, we see that \(({\bar{X}}_k, {\bar{Y}}_k)\) satisfies
where
Finally, by defining \(\Delta {\bar{X}}_k^p\) and \(\Delta {\bar{Y}}_k^p\) to be
we obtain from (45) the following new system of equations derived from the original system (45)
Let us take the inverse of the matrix on the left hand side of the above equation, which can be shown to exist for \(\tau _k > 0\). Define
and
Note that \({\mathcal {G}}_k\) and \(\bar{{\mathcal {G}}}_k\) are related by
We have from (58), by taking the inverse of the matrix on the left hand side in (58), the following
where to obtain the second equality, we use the identity
Given that we are using (59), which involves \(\Delta {\bar{X}}^p_k, \Delta {\bar{Y}}^p_k\), to derive meaningful conditions for superlinear convergence using Algorithm 1, let us now express (42) in terms of them. We know that \(P_k^T P_k = W_k^{-1}\) and by (46), we can let
In the following lemma, we provide the sufficient condition for superlinear convergence using Algorithm 1 in terms of \(\Delta {\bar{X}}_k^p, \Delta {\bar{Y}}_k^p\).
Lemma 2
If
then superlinear convergence in the sense of (40) using Algorithm 1 follows.
Proof
A sufficient condition for superlinear convergence in the sense of (40) is (42). Now observe that for (42) to hold, it is sufficient to have
By (47), \(P_k\) given by (60) satisfies
By (63), (62) holds if and only if
Hence, since (62) is sufficient for (40) to hold, the lemma follows by applying (56) and (57) on (64). \(\square \)
The system of equations in (59) relates \((X_k,Y_k)\) through \(({\bar{X}}_k,{\bar{Y}}_k)\) to \((\Delta {\bar{X}}_k^p, \Delta {\bar{Y}}_k^p)\), and allows us to have a way to validate conditions on \((X_k,Y_k)\) for superlinear convergence using Algorithm 1 by showing that (61) holds. Before we provide such a condition in Theorem 3 below, let us observe the following.
Proposition 17
Proof
We have
where the first equality holds by definition of \({\bar{r}}_{0,k}\) in (55), and the second equality holds by (52), (53) and the structure of q in Proposition 16. \(\square \)
Theorem 3
Suppose
then Algorithm 1 is a superlinearly convergent algorithm in the sense of (40).
Proof
By Proposition 12, (65) implies that
which further implies that
by Claim 1, given in “Appendix”.
Let \(({\bar{X}}^*, {\bar{Y}}^*)\) be any accumulation point of \(\{ ({\bar{X}}_k,{\bar{Y}}_k) \}\). Then (66) implies that
We also have \({\bar{W}}^*\), which is the corresponding accumulation point of \(\{ {\bar{W}}_k \}\), with \(({\bar{W}}^*)_{12} = 0\).
Hence,
Furthermore, we have
which follows from Proposition 17.
Note that
by Remark 3. From this equality, it then follows from (67), (68), \(({\bar{X}}^*)_{12} = ({\bar{Y}}^*)_{12} = 0\), the structure of q and that of \(\bar{{\mathscr {A}}}(0), \bar{{\mathscr {B}}}(0)\) that
Therefore,
Hence, in the limit as k tends to infinity, \(\bar{{\mathcal {G}}}_k^{-1} \left( q - \frac{\tau _k}{\tau _0} {\bar{r}}_{0,k} \right) \) tends to
Therefore, by (59),
In a similar way, we can show that
Since (61) holds in this case, the theorem is proved. \(\square \)
In Theorem 3, we provide a sufficient condition for superlinear convergence using Algorithm 1 on any SDLCP that satisfies Assumptions 1(a), (c) and Assumption 2. This sufficient condition is similar to that found in [24, 34] using Algorithm 1 with the HKM search direction. We have shown in the above theorem that the same condition is also sufficient for superlinear convergence using Algorithm 1 with the NT search direction. Superlinear convergence result has been establised in [16] using the NT search direction by “narrowing” the neighborhood of the central path, although in [16], a feasible algorithm is considered, while here, we consider an infeasible algorithm, with more involved analysis.
In the following theorem, we give another sufficient condition for superlinear convergence using Algorithm 1 on SDLCPs that have certain structure
Theorem 4
Let \({\mathscr {A}}, {\mathscr {B}}\) be such that for all \(1 \le i \le {\tilde{n}}\), if \({\mathscr {A}}_{i \cdot } \not = 0\), then \({\mathscr {B}}_{i \cdot } = 0\) (or equivalently, if \({\mathscr {B}}_{i \cdot } \not = 0\), then \({\mathscr {A}}_{i \cdot } = 0\)). Furthermore, let q satisfies either one of the following two conditions:
-
1.
For all \(1 \le i \le i_1\), if \({\mathscr {B}}_{i \cdot } \not = 0\), then \(q_i = 0\).
-
2.
For all \(1 \le i \le i_1\), if \({\mathscr {A}}_{i \cdot } \not = 0\), then \(q_i = 0\).
Suppose \((X_0,Y_0)\) is chosen such that
if q satisfies the first condition, and
if q satisfies the second condition, then iterates generated by Algorithm 1 converge superlinearly in the sense of (40).
Proof
We only need to show the theorem when the first condition (condition 1) on q is satisfied. The proof of the theorem when the second condition on q holds is similar.
By Remark 3 and \({\bar{W}}_k {\bar{Y}}_k {\bar{W}}_k = {\bar{X}}_k\), we have
Let \(({\bar{X}}^*, {\bar{Y}}^*)\) be any accumulation point of \(\{ ({\bar{X}}_k,{\bar{Y}}_k) \}\) as k tends to infinity, with \({\bar{W}}^*\) the corresponding accumulation point of \(\{ {\bar{W}}_k \}\). Then,
On the other hand,
which follows from Proposition 17.
By conditions imposed on \(({\mathscr {A}},{\mathscr {B}},q)\) and \((X_0,Y_0)\) in the theorem, (51), and the structure of q in Proposition 16, it follows from (69) and (70) that
Hence, in the limit as k tends to infinity, \(\bar{{\mathcal {G}}}_k^{-1}\left( q - \frac{\tau _k}{\tau _0} {\bar{r}}_{0,k} \right) \) tends to \(- {\mathrm{svec}}({\bar{Y}}^*)\) and \(({\bar{W}}_k \otimes {\bar{W}}_k)\bar{{\mathcal {G}}}_k^{-1}\left( q - \frac{\tau _k}{\tau _0} {\bar{r}}_{0,k} \right) \) tends to \(- {\mathrm{svec}}({\bar{X}}^*)\). Therefore, (59) implies that
from which we conclude that (61) holds which then implies that Algorithm 1 is a superlinearly convergent algorithm. \(\square \)
Theorem 4 tells us that for certain subclass of SDLCPs, if a suitable starting point \((X_0,Y_0)\) is chosen, then we have superlinear convergence using Algorithm 1 on the SDLCP. We can apply this theorem to an important subclass of semi-definite programs known as linear semi-definite feasibility problems. A linear semi-definite feasibility problem is a semi-definite program where \(C = 0\) in (\({\mathcal {P}}\)) or \(b_j = 0\), \(j = 1, \ldots , m\), in (\({\mathcal {D}}\)).
Corollary 1
For a linear semi-definite feasibility problem, when \(C = 0\) in (\({\mathcal {P}}\)), if \(X_0\) is chosen such that for all \(i_1 + i_2 + 1 \le i \le {\tilde{n}}\), \({\mathscr {A}}_{i \cdot } {\mathrm{svec}}(X_0) = 0\), and when \(b_j = 0\), \(j = 1, \ldots , m\), in (\({\mathcal {D}}\)), if \(Y_0\) is chosen such that for all \(i_1 + i_2 + 1 \le i \le {\tilde{n}}\), \({\mathscr {B}}_{i \cdot }{\mathrm{svec}}(Y_0) = 0\), then Algorithm 1 is a superlinearly convergent algorithm on the linear semi-definite feasibility problem.
Proof
When \(C = 0\) in (\({\mathcal {P}}\)), it is easy to check that \(({\mathscr {A}}, {\mathscr {B}}, q)\) satisfies the conditions in Theorem 4, with q satisfying the first condition there. When \(b_j = 0\), \(j = 1, \ldots , m\), in (\({\mathcal {D}}\)), it is also easy to check that \(({\mathscr {A}}, {\mathscr {B}}, q)\) satisfies the conditions in Theorem 4, with q satisfying the second condition there. Corollary then follows from Theorem 4. \(\square \)
Corollary 1 states a similar result as Theorem 5.1 in [34]. The latter holds for the HKM search direction, while Corollary 1 applies to the NT search direction. It is worthwhile to note that to solve the linear semi-definite feasibility problem, in the literature [4, 7, 9, 38, 41] the assumption that the interior of dual feasible region is nonempty is usually made. In the above corollary, we do not need such an assumption to show superlinear convergence using Algorithm 1. Only strict complementarity assumption and a suitable initial iterate are needed. In fact, we see from Corollary 1 that if the linear semi-definite feasibility problem has primal feasible region with nonempty relative interior in the case when \(C = 0\) in (\({\mathcal {P}}\)), and has dual feasible region with nonempty interior in the case when \(b_j = 0\), \(j = 1, \ldots , m\), in (\({\mathcal {D}}\)), then Algorithm 1 is always a superlinearly convergent algorithm, irrespectively of starting point \((X_0, Y_0)\). This is so because the conditions in the corollary are satisfied trivially, as \(k_0 = n\) in the former case and \(k_0 = 0\) in the latter case. That is, in these cases, the matrices that we considered are no longer partitioned into 4 blocks, and we have \(i_1 = {\tilde{n}}\) and \(i_2 = 0\) in both cases.
5 Numerical study
In this section, we report on the numerical results we obtained upon applying Algorithm 1 to solve instances of SDLCP (1)–(3). The algorithm is implemented by writing Matlab R2018a scripts and is run on a personal computer with an Intel(R) Core(TM) i5-4460 CPU and 8 GB of memory. In existing SDP solvers, such as SeDuMi [37], SDPT3 [40], there is an option for the equation system to find the corrector step in the interior point algorithm to have a second order term. Having the second order term tends to enhance practical efficiency of the algorithm. However, we decide to perform our numerical experiments without introducing a second order term in the equation system to find the corrector step in Algorithm 1. This is mainly due to more observed numerical warnings when we run our Matlab programs as iterates get closer to an optimal solution for instances of SDLCP (1)–(3) we tested when the algorithm is used with the second order term introduced. It is also worthwhile to note that with or without the second order term in the equation system to find the corrector step, the number of iterations to solve an instance of SDLCP (1)–(3), namely, a linear semi-definite feasibility problem (LSDFP) with
using our implemented algorithm differs only by 1 iteration, when tolerance \(\epsilon \) is set to \(10^{-10}\).
In all our numerical experiments, we set \(\beta _1 = 0.3, \beta _2 = 0.45\) and the tolerance \(\epsilon \) to be \(10^{-10}\) in Algorithm 1.
We generate random instances of SDLCP (1)–(3) by first generating diagonal matrices \(D_{\hat{{\mathscr {A}}}}, D_{\hat{{\mathscr {B}}}} \in \mathfrak {R}^{{\tilde{n}} \times {\tilde{n}}}\), where \({\tilde{n}} = n(n+1)/2\), such that the main diagonal entries of \(D_{\hat{{\mathscr {B}}}}\) are randomly taken from the uniform distribution between \(-5\) and \(-1\), and the main diagonal entries of \(D_{\hat{{\mathscr {A}}}}\) are zero or nonzero with equal probability. If a main diagonal entry of \(D_{\hat{{\mathscr {A}}}}\) is nonzero, then it is randomly assigned a value from the uniform distribution between 0 and 4. We obtain the matrices \(\hat{{\mathscr {A}}}, \hat{{\mathscr {B}}} \in \mathfrak {R}^{{\tilde{n}} \times {\tilde{n}}}\) by
where \(U, V \in \mathfrak {R}^{{\tilde{n}} \times {\tilde{n}}}\) are randomly generated orthogonal matrices, see [31]. We have \({\mathscr {A}}, {\mathscr {B}} \in \mathfrak {R}^{{\tilde{n}} \times {\tilde{n}}}\) to be obtained from \(\hat{{\mathscr {A}}}, \hat{{\mathscr {B}}}\) by interchanging corresponding columns in the latter matrices when a random number generated from the uniform (0, 1) distribution is less than 0.5, and keeping these columns when the random number is greater than or equal to 0.5. \({\mathscr {A}}, {\mathscr {B}}\) thus obtained satisfy Assumption 1(a),(c). On the other hand, Assumption 1(b) is satisfied by setting q to be
Hence, \(X^1, Y^1\) in Assumption 1(b) are both equal to the identity matrix. We set
where
This choice of initial iterate \((X_0,Y_0)\) is motivated by [40].
For each n from 5 to 15, we attempt to solve, using Algorithm 1, 100 instances of SDLCP (1)–(3) of size n, randomly generated with given initial iterate \((X_0,Y_0)\) as described in the last paragraph. For each n, we compute the average number of iterations taken and average runtime for the algorithm to terminate for those instances that give real valued \((X_k,Y_k)\) upon termination of the algorithm. We denote \(I_n\) to be the number of these instances. Our results are given in Table 1.
Comparing our implemented algorithm with existing solvers, using our implemented algorithm on the LSDFP with data given by (71), we need 12 iterations and a runtime of 0.02 secs before termination, while SDPT3 needs 7 iterations with a runtime of 0.13 secs as reported by the software (OPTIONS.gaptol in SDPT3 is set to \(1e-10\)), and SeDuMi needs 5 iterations with a runtime of 1.28 secs as reported by the software (pars.eps in SeDuMi is set to \(1e-14\)). The initial iterate \((X_0,Y_0)\) used for our implemented algorithm on this LSDFP is
The same initial iterate is also used when solving this problem using SDPT3.
We now report on our numerical investigation on the local convergence of Algorithm 1 when it is used to solve the LSDFP with data given by (71). It is easy to check that the LSDFP satisfies Assumption 2 with
where \(x_1, x_2, y_1, y_2, y_3\) are constrained for these matrices to be positive semi-definite. It is also easy to check that when the initial iterate \((X_0,Y_0)\) in Algorithm 1 is chosen to be given by (72), the condition in Corollary 1 is satisfied. As predicted by our theoretical result in the corollary, our numerical results given in Table 2 shows that superlinear convergence of iterates generated by the implemented algorithm when solving this LSDFP takes place. It is interesting to note that with other choices of initial iterate \((X_0,Y_0)\), we also observe superlinear convergence of iterates generated by the implemented algorithm.
6 Conclusion
In this paper, we consider an infeasible predictor–corrector primal–dual path following interior point algorithm, using the Nesterov–Todd (NT) search direction, to solve a semi-definite linear complementarity problem (SDLCP). Global convergence is shown using the algorithm to solve an SDLCP and an iteration complexity bound which is polynomial in n, the size of the matrices involved, is also provided. This complexity bound is the best known so far for infeasible interior point algorithms when the “narrow” neighborhood is used for solving SDPs. Furthermore, we study superlinear convergence using the algorithm under strict complementarity assumption. Two sufficient conditions are provided for this to occur. The first sufficient condition is on the behavior of iterates generated by the algorithm, while the second sufficient condition is on the structure of SDLCPs. We finally report on preliminary numerical results we obtained upon implementing the interior point algorithm and using it to solve SDLCPs that are not necessarily SDPs.
Notes
The proof follows that of Proposition 4.4 in [34].
References
Alizadeh, F., Haeberly, J.A., Overton, M.: Primal–dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results. SIAM J. Optim. 8, 746–768 (1998)
Alzalg, B.: A primal–dual interior-point method based on various selections of displacement step for symmetric optimization. Comput. Optim. Appl. 72, 363–390 (2019)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
Chua, S.K., Toh, K.-C., Zhao, G.: An analytic center cutting plane method with deep cuts for semidefinite feasibility problems. J. Optim. Theory Appl. 123, 291–318 (2004)
da Cruz Neto, J.X., Ferreira, O.P., Monteiro, R.D.C.: Asymptotic behavior of the central path for a special class of degenerate SDP problems. Math. Program. Ser. A 103, 487–514 (2005)
Faybusovich, L., Zhou, C.: Long-step path-following algorithm for solving symmetric programming problems with nonlinear objective functions. Comput. Optim. Appl. 72, 769–795 (2019)
Gahinet, P., Nemirovski, A.: The projective method for solving linear matrix inequalities. Math. Program. Ser. B 77, 163–190 (1997)
Goemans, M., Williamson, D.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM 42, 1115–1145 (1995)
Goffin, J.-L., Luo, Z., Ye, Y.: Complexity analysis of an interior cutting plane method for convex feasibility problems. SIAM J. Optim. 6, 638–652 (1996)
Gu, G., Zangiabadi, M., Roos, C.: Full Nesterov–Todd step infeasible interior-point method for symmetric optimization. Eur. J. Oper. Res. 214, 473–484 (2011)
Helmberg, C., Rendl, F., Vanderbei, R.J., Wolkowicz, H.: An interior-point method for semidefinite programming. SIAM J. Optim. 6, 342–361 (1996)
Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)
Kojima, M., Shida, M., Shindoh, S.: Local convergence of predictor–corrector infeasible-interior-point algorithms for SDPs and SDLCPs. Math. Program. Ser. A 80, 129–160 (1998)
Kojima, M., Shindoh, S., Hara, S.: Interior-point methods for the monotone semidefinite linear complementarity problem in symmetric matrices. SIAM J. Optim. 7, 86–125 (1997)
Lu, Z., Monteiro, R.D.C.: Error bounds and limiting behavior of weighted paths associated with the SDP map \({X}^{1/2}{S}{X}^{1/2}\). SIAM J. Optim. 15, 348–374 (2004)
Luo, Z.-Q., Sturm, J.F., Zhang, S.: Superlinear convergence of a symmetric primal–dual path following algorithm for semidefinite programming. SIAM J. Optim. 8, 59–81 (1998)
Monteiro, R.D.C.: Primal–dual path following algorithms for semidefinite programming. SIAM J. Optim. 7, 663–678 (1997)
Monteiro, R.D.C., Tsuchiya, T.: Polynomial convergence of a new family of primal–dual algorithms for semidefinite programming. SIAM J. Optim. 9, 551–577 (1999)
Monteiro, R.D.C., Zhang, Y.: A unified analysis for a class of long-step primal–dual path-following interior-point algorithms for semidefinite programming. Math. Program. 81, 281–299 (1998)
Nesterov, Y., Todd, M.: Self-scaled barriers and interior-point methods for convex programming. Math. Oper. Res. 22, 1–42 (1997)
Nesterov, Y., Todd, M.: Primal–dual interior-point methods for self-scaled cones. SIAM J. Optim. 8, 324–364 (1998)
Nesterov, Y., Tunçel, L.: Local superlinear convergence of polynomial-time interior-point methods for hyperbolicity cone optimization problems. SIAM J. Optim. 26, 139–170 (2016)
Potra, F.A.: Q-superlinear convergence of the iterates in primal–dual interior-point methods. Math. Program. Ser. A 91, 99–115 (2001)
Potra, F.A., Sheng, R.: Superlinear convergence of interior-point algorithms for semidefinite programming. J. Optim. Theory Appl. 99, 103–119 (1998)
Potra, F.A., Sheng, R.: A superlinearly convergent primal–dual infeasible-interior-point algorithm for semidefinite programming. SIAM J. Optim. 8, 1007–1028 (1998)
Preiß, M., Stoer, J.: Analysis of infeasible-interior-point paths arising with semidefinite linear complementarity problems. Math. Program. Ser. A 99, 499–520 (2004)
Rangarajan, B.: Polynomial convergence of infeasible-interior-point methods over symmetric cones. SIAM J. Optim. 16, 1211–1229 (2006)
Rigó, P.R., Darvay, Z.: Infeasible interior-point method for symmetric optimization using a positive-asymptotic barrier. Comput. Optim. Appl. 71, 483–508 (2018)
Roos, C.: A full-Newton step \({\cal{O}}(n)\) infeasible interior-point algorithm for linear optimization. SIAM J. Optim. 16, 1110–1136 (2006)
Schmieta, S., Alizadeh, F.: Extension of primal–dual interior point algorithm to symmetric cones. Math. Program. 96, 409–438 (2003)
Shilon, O.: RandOrthMat. https://uk.mathworks.com/matlabcentral/fileexchange/11783-randorthmat (2006)
Sim, C.-K.: On the analyticity of underlying HKM paths for monotone semidefinite linear complementarity problems. J. Optim. Theory Appl. 141, 193–215 (2009)
Sim, C.-K.: Asymptotic behavior of underlying NT paths in interior point methods for monotone semidefinite linear complementarity problems. J. Optim. Theory Appl. 148, 79–106 (2011)
Sim, C.-K.: Superlinear convergence of an infeasible predictor–corrector path-following interior point algorithm for a semidefinite linear complementarity problem using the Helmberg–Kojima–Monteiro direction. SIAM J. Optim. 21, 102–126 (2011)
Sim, C.-K., Zhao, G.: Underlying paths in interior point methods for the monotone semidefinite linear complementarity problem. Math. Program. Ser. A 110, 475–499 (2007)
Sim, C.-K., Zhao, G.: Asymptotic behavior of Helmberg–Kojima–Monteiro (HKM) paths in interior point methods for monotone semidefinite linear complementarity problem: general theory. J. Optim. Theory Appl. 137, 11–25 (2008)
Sturm, J.F.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones (Updated for Version 1.05) (1998–2001)
Sun, J., Toh, K.-C., Zhao, G.: An analytic center cutting plane method for semidefinite feasibility problems. Math. Oper. Res. 27, 332–346 (2002)
Todd, M.J., Toh, K.C., Tütüncü, R.H.: On the Nesterov–Todd direction in semidefinite programming. SIAM J. Optim. 8, 769–796 (1998)
Toh, K.-C., Todd, M.J., Tütüncü, R.H.: On the implementation and usage of SDPT3—a Matlab software package for semidefinite-quadratic-linear programming, Version 4.0. In: Anjos, M.F., Lasserre, J.B. (eds.) Handbook on Semidefinite, Conic and Polynomial Optimization, Vol. 166 of International Series in Operations Research & Management Science, pp. 715–754. Springer, Berlin (2012)
Toh, K.-C., Zhao, G., Sun, J.: A multiple-cut analytic center cutting plane method for semidefinite feasibility problems. SIAM J. Optim. 12, 1126–1146 (2002)
Tseng, P.: Search directions and convergence analysis of some infeasible path-following methods for the monotone semi-definite LCP. Optim. Methods Softw. 9, 245–268 (1998)
Wang, G.Q., Bai, Y.Q., Gao, X.Y., Wang, D.Z.: Improved complexity analysis of full Nesterov–Todd step interior-point methods for semidefinite optimization. J. Optim. Theory Appl. 165, 242–262 (2015)
Wang, G.Q., Kong, L.C., Tao, J.Y., Lesaja, G.: Improved complexity analysis of full Nesterov–Todd step feasible interior-point method for symmetric optimization. J. Optim. Theory Appl. 166, 588–604 (2015)
Zhang, Y.: On extending some primal–dual interior-point algorithms from linear programming to semidefinite programming. SIAM J. Optim. 8, 365–386 (1998)
Acknowledgements
We would like to thank the editor and the associate editor for handling the paper and the two anonymous reviewers for providing valuable comments to improve the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Claim 1
We have
if and only if
Proof
Footnote 1 We only need to show the if direction, since the only if direction follows trivially using Proposition 12.
Suppose \((X_k Y_k)_{12} = o(\sqrt{\tau _k})\). Then \(({\bar{X}}_k {\bar{Y}}_k)_{12} = o(1)\). Let \(({\bar{X}}^*,{\bar{Y}}^*)\) be any accumulation point of \(({\bar{X}}_k,{\bar{Y}}_k)\) as k tends to infinity. We have \(({\bar{X}}^*{\bar{Y}}^*)_{12} = 0\). If we can show that \(({\bar{X}}^*)_{12} = ({\bar{Y}}^*)_{12} = 0\), then we show that \(({\bar{X}}_k)_{12} = o(1), ({\bar{Y}}_k)_{12} = o(1)\), and hence \((X_k)_{12} = o(\sqrt{\tau _k}),\ (Y_k)_{12} = o(\sqrt{\tau _k})\).
Now, if we premultiply and postmultiply \(({\bar{X}}^*{\bar{Y}}^*)_{12}\) by \(({\bar{Y}}^*)^T_{12}\) and \(({\bar{Y}}^*)^{-1}_{22}\) respectively, and express \(({\bar{X}}^*{\bar{Y}}^*)_{12}\) in terms of block components of \({\bar{X}}^*\) and \({\bar{Y}}^*\), we obtain
Therefore, we have
On the other hand, by Remark 3, \(({\bar{X}}_k,{\bar{Y}}_k)\) satisfies
In the limit as k tends to infinity, from (74), we obtain
where we apply the result in Proposition 17.
That is,
Using the structure of q in Proposition 16 and that of \(\bar{{\mathscr {A}}}(0), \bar{{\mathscr {B}}}(0)\), the following holds:
By the monotonicity result in Proposition 15, we have
Hence, \(({\bar{X}}^*)_{12} \bullet ({\bar{Y}}^*)_{12} \ge 0\). Combined with (73), we conclude from \(\Vert ({\bar{X}}^*)^{1/2}_{11} ({\bar{Y}}^*)_{12} ({\bar{Y}}^*)^{-1/2}_{22} \Vert _F^2 = 0\) that \(({\bar{Y}}^*)_{12} = 0\).
In a similar fashion, we can show that \(({\bar{X}}^*)_{12} = 0\). \(\square \)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Sim, CK. Interior point method on semi-definite linear complementarity problems using the Nesterov–Todd (NT) search direction: polynomial complexity and local convergence. Comput Optim Appl 74, 583–621 (2019). https://doi.org/10.1007/s10589-019-00110-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-019-00110-z