Facial Reduction Algorithms for Conic Optimization Problems

Abstract

In the conic optimization problems, it is well-known that a positive duality gap may occur, and that solving such a problem is numerically difficult or unstable. For such a case, we propose a facial reduction algorithm to find a primal–dual pair of conic optimization problems having the zero duality gap and the optimal value equal to one of the original primal or dual problems. The conic expansion approach is also known as a method to find such a primal–dual pair, and in this paper we clarify the relationship between our facial reduction algorithm and the conic expansion approach. Our analysis shows that, although they can be regarded as dual to each other, our facial reduction algorithm has ability to produce a finer sequence of faces of the cone including the feasible region. A simple proof of the convergence of our facial reduction algorithm for the conic optimization is presented. We also observe that our facial reduction algorithm has a practical impact by showing numerical experiments for graph partition problems; our facial reduction algorithm in fact enhances the numerical stability in those problems.

Appendix: Details of Example 4.2

We show that our FRA may generate a finer sequence of faces from a given SDP. We give such an example in this appendix. In this example, our FRA does not terminate even if it finds the minimal cone of the given SDP. In fact, solutions of the homogenized dual systems generated in our FRA do not satisfy the criterion for the termination. For this, we need more iterations in our FRA to terminate our FRA.

For simplicity, we apply FRA-SDP instead of our FRA to our problem. In fact, as we have mentioned in Sect. 5.1, FRA-SDP is essentially equivalent to our FRA. Hence, we obtain the same faces as those by our FRA.

Consider the following POP:

$$ \inf_{x, y}\bigl\{x^2y^2 : (x,y) \in\mathbb{R}^2\bigr\} $$

(25)

Let ℕ be the set of nonnegative integers. For a given nonnegative integer p, we define $\mathbb{N}^{2}_{p}:=\{\alpha=(\alpha_{1}, \alpha_{2})\in \mathbb{N}^{2} : \alpha_{1}+\alpha_{2}\le p \}$. For $\alpha\in \mathbb{N}^{2}_{4}$, we set $E_{\alpha}\in\mathbb{S}^{6}$ and real values b _α as follows:

Here E _α is the constant matrix indexed by $\mathbb{N}^{2}_{2}\times \mathbb{N}^{2}_{2}$ and b _α is the constant vector indexed by $\mathbb{N}^{2}_{2}$ for all $\alpha\in\mathbb{N}^{2}_{4}$. For example, E _(2,1) is written as follows:

where blanks in these matrices mean zero. Then the following SDP problem is an SDP relaxation of POP (25) in the sense of [28, 29]:

$$ \sup_{X\in\mathbb{S}^{6}_+}\bigl\{ -E_{(0,0)}\boldsymbol{\cdot} X : E_{\alpha}\boldsymbol{\cdot} X = b_{\alpha} \ \bigl(\alpha\in \mathbb{N}^2_4\setminus\bigl\{(0,0)\bigr\}\bigr) \bigr\}. $$

(26)

We apply FRA-SDP into (26). Then from the definition of E _α and b _α , we need to find a nonzero solution of the following homogenized dual system (9):

$$ b^Ty = y_{(2,2)} \le 0,\quad\quad W=-\sum _{\alpha\in\mathbb{N}^2_{4}\setminus\{(0, 0)\}} y_{\alpha}E_{\alpha},\quad\quad W\in \mathbb {S}^6_+. $$

(27)

Here, W is of the form

Note that because (26) is feasible, we can assume that b ^T y=y _(2,2)=0. In fact, the following matrix $\tilde{X}$ is feasible in (26):

$$\tilde{X} = \bordermatrix{ &(0,0)&(1,1)&(1,0)&(0,1)&(2,0)&(0,2)\cr (0,0)&&&&&&\cr (1,1)&&1&&&&\cr (1,0)&&&&&&\cr (0,1)&&&&&&\cr (2,0)&&&&&&\cr (0,2)&&&&&&\cr} $$

The first two equalities of (27) mean that

Since the first and the second diagonal elements of W are zero, the positive semidefiniteness of W implies that all elements in the first and second rows and columns must be zero. Then we obtain y _(2,0)=y _(0,2)=0, and by the same reasoning, we see that the third and fourth rows and columns are also zero. Therefore, W in (27) must have the form

(28)

where blanks stand for zero, and y _(4,0)≤0 and y _(0,4)≤0. Any element of the above form satisfies (27).

If we choose $\bar{W}^{1}$ as y _(4,0)<0 and y _(0,4)<0, then $\bar{W}^{1}$ lies in the relative interior of the homogenized dual system, and then the first face $\mathcal{F}_{1}$ is calculated by

$$\mathcal{F}_1 = \left \{ X : X = \left ( \begin{array}{c@{\quad}c} X_1 & O\\ O & O_2 \end{array} \right ) \mbox{ for some } X_1\in\mathbb{S}^4_+ \right \}. $$

As we have observed in Sect. 5.1, we can eliminate the unnecessary columns and rows to reduce the size of SDP (26) by restricting the variable to X ₁. However, for the moment, we just applied the original FRA to see exactly how it works.

The dual $\mathcal{F}_{1}^{*}$ is

$$\mathcal{F}_1^* = \left \{ W : W = \left ( \begin{array}{c@{\quad}c} W_1 & W^{\prime}\\ W^{\prime T}& W^{\prime\prime} \end{array} \right ) \mbox{ for some } W_1\in\mathbb{S}^4_+, W^{\prime}\in \mathbb{R}^{4\times 2} \mbox{ and } W^{\prime\prime}\in \mathbb{S}^{2} \right \}. $$

Then the homogenized dual system obtained by $\mathcal{F}_{1}$ is as follows:

$$ b^Ty = y_{(2,2)} = 0,\quad\quad W=-\sum _{\alpha\in\mathbb{N}^2_{4}\setminus\{(0, 0)\}} y_{\alpha}E_{\alpha}, \quad\quad W\in \mathcal{F}^*_1. $$

(29)

Any solution W ² of the homogenized dual system (29) has the form

(30)

We can prove that W ² with y _(2,0)<0 and y _(0,2)<0 is in the relative interior of the homogenized dual system (29) and that $W^{2}\not\in\mbox{span}(W^{1})$. Then the second face $\mathcal{F}_{2}$ is

$$\mathcal{F}_2 = \left \{ X : X = \left ( \begin{array}{c@{\quad}c} X_2 &O\\ O& O_4 \end{array} \right ) \mbox{ for some } X_2\in\mathbb{S}^2_+ \right \}, $$

and it is not difficult to verify that the second face $\mathcal{F}_{2}$ is the minimal cone for SDP (26) by Lemma 3.3. From Theorem 4.1, $\mathcal{F}_{1}^{*}$ and $\mathcal{F}_{2}^{*}$ are equivalent to the cone $\varGamma_{\mathcal{B}}(\mathcal{K}^{*})$ and $\varGamma_{\mathcal{B}}^{2}(\mathcal{K}^{*})$, respectively, because we choose W ¹ and W ² from the relative interior of the homogenized dual systems (27) and (29), respectively.

If we apply the FRA-CE and reduce the size of SDP problems at each iteration as we have shown in Sect. 5.1, we will have the following equivalent problem to SDP (26):

$$\sup_{\tilde{X}\in\mathbb{S}^2}\left \{ -\left (\begin{array}{c@{\quad}c}1 & 0 \\ 0 & 0\end{array} \right ) \boldsymbol{\cdot} \tilde{X} : \left (\begin{array}{c@{\quad}c}0 & 1 \\ 1 & 0\end{array} \right ) \boldsymbol{\cdot} \tilde{X} = 0, \ \left (\begin{array}{c@{\quad}c}0 & 0 \\ 0 & 1\end{array} \right ) \boldsymbol{\cdot} \tilde{X} = 1, \ \tilde{X} \in \mathbb {S}^2_+ \right \}. $$

Note that this problem has an interior feasible solution, and thus we can no longer apply our FRA.

We remark that when we work in the original space, although we see that $\mathcal{F}_{2}$ is the minimal cone for SDP (26) by Lemma 3.3, our FRA does not terminate. In fact, any solution W in the homogenized dual system constructed from $\mathcal{F}_{2}$ satisfies

(31)

Then any W with y _(1,0)≠0 is a solution of (31). However, the solution is not included in span(W ¹,W ²), so that our FRA finds W ³ and generates the third face $\mathcal{F}_{3}$, which is the same as the minimal cone $\mathcal{K}_{\min}$. Although all faces generated after $\mathcal{F}_{2}$ are the same as the minimal cone $\mathcal{K}_{\min}$, our FRA must find W ⁱ until $\ker A\cap H_{c}^{-}\cap \mathcal{F}_{i}^{*}\subseteq\mbox{span}(W^{1}, \ldots, W^{i})$. This fact shows that if we add the condition $\mathcal{F}_{i}\subseteq(\ker A\cap\ker c^{T}\cap \mathcal{F}_{i}^{*})^{\perp}$ in Step 2 of our FRA, the algorithm may terminate in fewer iterations than the original FRA and returns the minimal cone.

We next show that our FRA can generate a finer sequence of faces for SDP (26) in this example. If we choose W ⁱ from $(\mathcal{B}\cap \mathcal{F}_{i}^{*})\setminus \operatorname{relint}(\mathcal{B}\cap \mathcal{F}_{i}^{*})$, our FRA may provide a different sequence of faces from FRA-CE, i.e., the conic expansion approach. For example, if we choose W ¹ with y _(0,4)=0 at (28), W ¹ is not in the relative interior of the homogenized dual system (27), and the first face $\mathcal{G}_{1}$ by FRA and the dual $\mathcal{G}^{*}_{1}$ are

Any solution $\tilde{W}^{2}$ of the homogenized dual system obtained by $\mathcal{G}_{1}$ has the form

If we choose $\tilde{W}^{2}$ with y _(0,2)<0 and y _(4,0)<0, $\tilde{W}^{2}$ is in the relative interior and we obtain the second face $\mathcal{G}_{2}$:

$$\mathcal{G}_2 = \left \{ X : X = \left ( \begin{array}{c@{\quad}c} X_2 & O_3\\ O_3& O_{3} \end{array} \right ) \mbox{ for some } X_2\in\mathbb{S}^3_+ \right \}. $$

Also the dual of $\mathcal{G}_{2}^{*}$ is

$$\mathcal{G}_2^* = \left \{ W : W = \left ( \begin{array}{c@{\quad}c} W_2 & W^{\prime}\\ W^{\prime T}& W^{\prime\prime} \end{array} \right ) \mbox{ for some } W_2\in\mathbb{S}^3_+, W^{\prime}\in \mathbb{R}^{3\times 3} \mbox{ and } W^{\prime\prime}\in \mathbb{S}^{3} \right \}. $$

Any solution $\tilde{W}^{3}$ of the homogenized dual system by $\mathcal{G}_{2}$ has the form

The matrix $\tilde{W}^{3}$ with y _(2,0)<0 is in the relative interior of the homogenized dual system by $\mathcal{G}_{2}$ and the third face $\mathcal{G}_{3}$ generated by $\tilde{W}^{3}$ is

$$\mathcal{G}_3 = \left \{ X : X = \left ( \begin{array}{c@{\quad}c} X_3 & O\\ O& O_4 \end{array} \right ) \mbox{ for some } X_3\in\mathbb{S}^2_+ \right \}, $$

and is equivalent to the minimal cone for SDP (26). As we have already seen, our FRA does not terminate because $\ker A\cap\ker c^{T}\cap\mathcal{G}_{3}^{*}\not\subseteq\mbox{span}(\tilde{W}^{1}, \tilde{W}^{2}, \tilde{W}^{3})$ and we need to find $\tilde{W}^{i+1}\in(\ker A\cap\ker c^{T}\cap\mathcal{G}_{i}^{*})\setminus\mbox{span}(\tilde{W}^{1}, \ldots, \tilde{W}^{i})$ until $\ker A\cap\ker c^{T}\cap\mathcal{G}_{i}^{*}\subseteq\mbox{span}(\tilde{W}^{1}, \ldots, \tilde{W}^{i})$.

Figure 2 shows the relationships among faces $\mathcal{F}_{i}$ and $\mathcal{G}_{i}$.