Computational Optimization and Applications

, Volume 53, Issue 2, pp 619–648

Strong duality and minimal representations for cone optimization

Authors

  • Levent Tunçel
    • Department of Combinatorics & OptimizationUniversity of Waterloo
    • Department of Combinatorics & OptimizationUniversity of Waterloo
Article

DOI: 10.1007/s10589-012-9480-0

Cite this article as:
Tunçel, L. & Wolkowicz, H. Comput Optim Appl (2012) 53: 619. doi:10.1007/s10589-012-9480-0

Abstract

The elegant theoretical results for strong duality and strict complementarity for linear programming, LP, lie behind the success of current algorithms. In addition, preprocessing is an essential step for efficiency in both simplex type and interior-point methods. However, the theory and preprocessing techniques can fail for cone programming over nonpolyhedral cones. We take a fresh look at known and new results for duality, optimality, constraint qualifications, CQ, and strict complementarity, for linear cone optimization problems in finite dimensions. One theme is the notion of minimal representation of the cone and the constraints. This provides a framework for preprocessing cone optimization problems in order to avoid both the theoretical and numerical difficulties that arise due to the (near) loss of the strong CQ, strict feasibility. We include results and examples on the surprising theoretical connection between duality gaps in the original primal-dual pair and lack of strict complementarity in their homogeneous counterpart. Our emphasis is on results that deal with Semidefinite Programming, SDP.

Keywords

Cone optimizationDualityPreprocessingConstraint qualificationDuality gapSemidefinite programmingStrict complementarityNice conesDevious conesFacially dual complete cones

1 Introduction

In this paper we study duality, optimality conditions, and preprocessing for the conic optimization problem, i.e., the problem of optimizing a linear objective function over the intersection of a convex cone with an affine space. We include both the linear transformation and the subspace forms for the formulation of our optimization problems; we study known and new characterizations of optimality that hold without any constraint qualification, CQ; and, we collect needed technical results on the cone facial structure.

In addition, as new results, we derive new CQs including a universal constraint qualification, UCQ, i.e., a CQ that holds independent of the data b,c; and, we study the geometry of nice and devious cones and the relationship that lack of closure has to both strong duality and zero duality gaps, including characterizations for a zero duality gap, and a surprising new connection between duality gaps and the failure of strict complementarity in the homogeneous problems.

One theme is the notion of minimal representation of the cone and the constraints in order to preprocess and regularize the problem and thus avoid both the theoretical and numerical difficulties that arise due to (near) loss of strict feasibility, i.e., we see that loss of this strong CQ is a modeling issue rather than inherent to the problem instance. As it is well-known that the existence of a strong CQ of the Mangasarian-Fromovitz or Robinson type is equivalent to stability of the problem, e.g., [66], this justifies the pleasing paradigm: efficient modeling provides for a stable program.

1.1 Background and motivation

The cone programming model has been studied for a long time, e.g., [23], as a generalization of the classical linear program, LP. More recently, many important applications have arisen for more general nonpolyhedral cones. This includes the case when K is the cone of positive semidefinite matrices, \(\mathbb {S}^{n}_{+}\); then we get semidefinite programming, SDP, e.g., [73]. Another important case is second order cone programming, SOCP, where \(K=\mathrm{SOC}_{1} \oplus\cdots\oplus\mathrm{SOC}_{k}\), a direct sum of second order (Lorentz) cones, e.g., [4, 45]. These research areas remain very active, see e.g., [2, 33, 38, 41, 49, 67, 73, 77] and URL: www-user.tu-chemnitz.de/~helmberg/semidef.html. Optimality conditions and CQs have been studied in e.g., [23, 31, 62] and more recently for both linear and nonlinear problems in e.g., [66]. (See the historical notes in [66, Sect. 4.1.5].) Optimality conditions and strong duality results without a CQ have appeared in e.g., [1316, 35, 36, 53, 57, 58].

Both strong duality and strict complementarity behave differently for general cone optimization problems, compared to the LP case. First, strong duality for a cone program can fail in the absence of a CQ, i.e., there may not exist a dual optimal solution and there may be a nonzero duality gap. In addition, the (near) failure of the Slater CQ (strict feasibility) has been used in complexity measures, [60, 61]. Moreover, numerical difficulties are well correlated with (near) failure of the Slater CQ, see [25, 26]. Similarly, unlike the LP case, [28], there are general cone optimization problems for which there does not exist a primal-dual optimal solution that satisfies strict complementarity, see e.g., [73] for examples. Theoretical difficulties arise, e.g., for local convergence rate analysis. Again, we have that numerical difficulties are well correlated with loss of strict complementarity, see [71]. An algorithm for generating SDP problems where strict complementarity fails, independent of whether the Slater CQ holds or not, is also given in [71].

Connections between weakest CQs and the closure of the sum of a subspace and a cone date back to e.g., [31]. We present a surprising theoretical connection between strict complementarity of the homogeneous problem and duality gaps, as well as show that both loss of strict complementarity and strong duality are connected to the lack of closure of the sum of a cone and a subspace.

Examples where no CQ holds arise in surprisingly many cases. For example, Slater’s CQ fails for many SDP relaxations of hard combinatorial problems, see e.g., [5, 74, 75]. A unifying approach to remedy this situation is given in [68]. Another instance is the SDP that arises from relaxations of polynomial optimization problems, e.g., [69]. Exploiting the absence of Slater’s CQ is done in [42]. Current public domain codes for SDP are based on interior-point methods and do not take into account loss of Slater’s CQ (strict feasibility) or loss of strict complementarity. As discussed above, both of these conditions can result in theoretical and numerical problems, e.g., [25, 69, 71]. Contrary to the LP case, e.g., [29, 39, 48], current SDP codes do not perform extensive preprocessing to avoid these difficulties. (Though some preprocessing is done to take advantage of sparsity, e.g., [27]. A projection technique for the cases where Slater’s CQ fails is studied in [18].)

1.2 Outline

In Sect. 2 we present the notation and preliminary results. We introduce: the subspace forms for the cone optimization in Sect. 2.1.1 and the complementarity partition and minimal sets in Sect. 2.1.2. Technical facial properties are presented in Sects. 3.1 and 3.2. The notions of nice and devious cones are described in Sect. 3.3. We include many relationships for the facial structure of the cone optimization problems.

The strong duality results, with and without CQs, and the CQs and UCQ, are presented in Sect. 4, see e.g., Theorem 4.10. We use both the minimal cone known in the literature and introduce the minimal subspace in order to obtain a regularization that guarantees that Slater’s CQ holds, rather than the weaker generalized Slater CQ given in the literature, see (4.9) in Theorem 4.10. We study the failure of strong duality and strict complementarity in Sect. 5. This includes a characterization for a zero duality gap in Sect. 5.1. The surprising relation between duality gaps and the failure of the strict complementarity property for the homogeneous problem, is given in Sect. 5.1.2, see e.g., Theorems 5.9 and 5.7. Our concluding remarks are in Sect. 6.

2 Notation and preliminary results

The set K is a convex cone if it is a cone, i.e., it is closed under nonnegative scalar multiplication, λKK,∀λ≥0, and, it is also closed under addition K+KK. The cone K is a proper cone if it is closed, pointed, and has nonempty interior. We let uKv (respectively, uKv) denote the partial order induced by K, i.e., vuK (respectively, \(v-u \in\operatorname{int}K\)).

We use \(\overline{S}\) to denote closure, \(\operatorname{precl}S = \overline{S} \backslash S\) to denote the preclosure of a set S. We let \(\operatorname{conv}S\) denote the convex hull of the setS and \(\operatorname{cone}S\) denote the convex cone generated by S. (By abuse of notation, we use \(\operatorname{cone}s := \operatorname{cone}\{s\}\), for a single element s. This holds similarly for, e.g., s:={s} and other operations that act on single element sets.) The dual or nonnegative polar cone of a set S is S:={x:〈x,s〉≥0, ∀sS}. In particular, for the space of n×n symmetric matrices, \(\mathbb{S}^{n}\), we use the trace inner-product \(\langle x, s\rangle= \operatorname{trace}xs\), i.e., the trace of the product of the matrices x and s. We use \(\mathcal{R}(\cdot)\) and \(\mathcal{N}(\cdot)\) to denote range space and nullspace, respectively. We let \(\mathbb{S}^{n}_{+}\subset\mathbb{S}^{n}\) denote the cone of positive semidefinite matrices. In particular, setting \(K=\mathbb{S}^{n}_{+}\) yields the partial order induced by the cone of positive semidefinite matrices in \(\mathbb {S}^{n}\), i.e., the so-called Löwner partial order, \(x \preceq_{\mathbb{S}^{n}_{+}} s\).

We let ei denote the ith unit vector of appropriate dimension, and Eij denote the (i,j)th unit matrix in \(\mathbb{S}^{n}\), i.e., \(E_{ii}=e_{i}e_{i}^{T}\) and if ij, \(E_{ij}=e_{i}e_{j}^{T}+e_{j}e_{i}^{T}\). By abuse of notation, we let xij denote the ij element of \(x \in\mathbb{S}^{n}\).

The subset FK is a face of the coneK, denoted \(F \unlhd K\), if
$$ ( s \in F, 0\preceq_K u \preceq_K s )\quad\mbox{implies}\quad(\operatorname{cone}u \subseteq F ).$$
(2.1)
If \(F \unlhd K\) but is not equal to K, we write FK. If {0}⊂FK, then F is a proper face of K. (Similarly, S1S2 denotes a proper subset, i.e., S1S2,S1S2.) For SK, we let \(\operatorname{face}S\) denote the smallest face of K that contains S; equivalently \(\operatorname{face}S\) is the intersection of all faces containing S. A face \(F \unlhd K\) is an exposed face if it is the intersection of K with a hyperplane. The cone K is facially exposed if every face \(F\unlhd K\) is exposed. If \(F \unlhd K\), then the conjugate face is Fc:=KF. Note that if the conjugate face Fc is a proper face, then it is exposed using any \(s \in\operatorname{relint}F\), i.e., \(F^{c} = K \cap s^{\perp}, \forall s \in\operatorname{relint}F\).
We study the following pair of dual conic optimization problems in standard form:
$$ (\mathbb{P})\quad v_P:= \sup_y \bigl\{\langle b, y\rangle: \mathcal{A}^* y \preceq_K c\bigr\}$$
(2.2)
$$ (\mathbb{D}) \quad v_D:= \inf_x \bigl\{\langle c, x\rangle: \mathcal{A}x = b,\ x \succeq_{K^*} 0\bigr\},$$
(2.3)
where: \(\mathcal{A}: \mathcal{V}\rightarrow\mathcal{W}\) is a (onto) linear transformation between two finite dimensional inner-product spaces; \(\mathcal{A}^{*}\) denotes the adjoint transformation; and K is a convex cone. Throughout, we assume that the optimal value vP is finite. Weak duality holds for any pair of primal-dual feasible solutions y,x, i.e., if \(s=c-\mathcal{A}^{*}y \succeq_{K} 0, \mathcal{A}x=b, x\succeq_{K^{*}}0\), then we get
$$\langle b, y\rangle = \langle\mathcal{A}x, y\rangle = \bigl\langle\mathcal{A}^*y, x\bigr\rangle = \langle c-s, x\rangle \leq\langle c, x\rangle\quad (\mathrm{Weak}\ \mathrm{Duality}).$$
The usual constraint qualification, CQ, used for the primal (2.2) is the Slater condition, i.e., strict feasibility \(\mathcal{A}^{*}\hat{y} \prec c\). If we assume Slater’s CQ holds and the primal optimal value is finite, then strong duality holds, i.e., we have a zero duality gap and attainment of the dual optimal value,
$$v_P= v_D= \bigl\langle c, x^*\bigr\rangle,\quad\mbox{for some dual feasible}\ x^*\quad (\mathrm{Strong}\ \mathrm{Duality}).$$
Denote the primal-dual feasible sets of (2.2) and (2.3) by
$$ \mathcal{F}^y_P=\mathcal{F}^y_P(c) = \bigl\{y: \mathcal{A}^* y\preceq_K c\bigr\}, \qquad \mathcal{F}_D^x =\mathcal{F}_D^x(b)= \{x: \mathcal{A}x=b,\ x\succeq_{K^*} 0\},$$
(2.4)
respectively. The set of feasible slacks for (2.2) is
$$ \mathcal{F}^s_P=\mathcal{F}^s_P(c) = \bigl\{s: s=c-\mathcal{A}^* y\succeq_K 0,\ \mbox{for some}\ y\bigr\}.$$
(2.5)
We allow for the dependence on the parameters b and c. Similarly, the optimal solution sets are denoted by \(\mathcal{O}^{s}_{P},\mathcal{O}^{y}_{P},\mathcal{O}^{x}_{D}\). Moreover, the pair of feasible primal-dual solutions s,x are said to satisfy strict complementarity, SC if
$$ (\mathbb{SC})\quad \begin{array}{@{}l}s \in\operatorname{relint}F_P\ \mbox{and}\ x \in\operatorname{relint}F_P^c,\quad \mbox{for some}\ F_P \unlhd K, \\\mbox{or} \\s \in\operatorname{relint}F_D^c\ \mbox{and}\ x \in\operatorname{relint}F_D,\quad \mbox{for some}\ F_D \unlhd K^*.\end{array}$$
(2.6)
(Note that this implies \(s+x \in\operatorname{int}(K+K^{*})\), see Proposition 3.3, part 1, below.)

2.1 Subspace form, complementarity partitions, and minimal sets

2.1.1 Subspace form for primal-dual pair (\(\mathbb{P}\)) and (\(\mathbb{D}\))

Suppose that \(\tilde{s}, \tilde{y}\), and \(\tilde{x}\) satisfy
$$ \mathcal{A}^* \tilde{y} +\tilde{s} = c, \qquad\mathcal{A}\tilde{x} = b.$$
(2.7)
Then, for any feasible primal-dual triple (x,y,s), where s is, as usual, the primal slack given by \(s = c - \mathcal{A}^{*} y\), we have \(\langle c, \tilde{x}\rangle = \langle\mathcal{A}^{*} y +s, \tilde{x}\rangle = \langle b, y\rangle + \langle s, \tilde{x}\rangle \). Therefore, the objective in (2.2) can be rewritten as
$$\sup_y \langle b, y\rangle = \sup_s \bigl(\langle c,\tilde{x}\rangle - \langle s, \tilde{x}\rangle \bigr) = \langle c,\tilde{x}\rangle -\inf_s \langle s,\tilde{x}\rangle .$$
We let \(\mathcal{L}\) denote the nullspace \(\mathcal{N}(\mathcal{A})\) of the operator \(\mathcal{A}\). Then
$$ \mathcal{F}_P^s =\mathcal{F}_P^s(c) = \bigl( c+\mathcal{L}^{\perp}\bigr) \cap K = \bigl( \tilde{s}+\mathcal{L}^{\perp } \bigr) \cap K.$$
(2.8)
In addition, for \(x \in\tilde{x} + \mathcal{L}\), we get \(\langle c, x\rangle =\langle\mathcal{A}^{*} \tilde{y} +\tilde{s},x\rangle = \langle\tilde{s}, x\rangle + \langle\mathcal{A}^{*}\tilde{y}, \tilde{x}\rangle = \langle\tilde{s}, x\rangle + \langle\tilde{y}, b\rangle \). We can now write the primal and dual conic pair, (2.2) and (2.3), in the so-called subspace form (see e.g., [50, Sect. 4.1]):
$$ v_P= \langle c,\tilde{x}\rangle -\inf_s \bigl\{ \langle s, \tilde{x}\rangle : s \in\bigl(\tilde{s} +\mathcal{L}^\perp\bigr) \cap K\bigr\},$$
(2.9)
$$ v_D= \langle\tilde{y}, b\rangle +\inf_x \bigl\{ \langle\tilde{s}, x\rangle : x \in(\tilde{x} +\mathcal{L}) \cap K^* \bigr\}.$$
(2.10)
The symmetry means that we can directly extend results proved for (2.9) to (2.10). Note that we have much flexibility in the choice of \(\tilde{s}\) and \(\tilde{x}\). In particular, if (2.9) and (2.10) are feasible, we may choose \(\tilde{s} \in\mathcal{F}_{P}^{s}\) and \(\tilde{x} \in\mathcal{F}_{D}^{x}\), and in the case that the optimal values are attained, we may choose \(\tilde{s} \in\mathcal{O}^{s}_{P}\) and \(\tilde{x} \in \mathcal{O}_{D}^{x}\).

Proposition 2.1

Let\(\tilde{s}, \tilde{y}\), and\(\tilde{x}\)satisfy (2.7). Then (2.9) and (2.10) are a dual pair of cone optimization problems equivalent to (2.2) and (2.3), respectively. Moreover, (2.2) (resp. (2.3)) is feasible if, and only if, \(\tilde{s} \in K+\mathcal{L}^{\perp}\) (resp. \(\tilde{x} \in K^{*}+\mathcal{L}\)).

2.1.2 Complementarity partitions, CP, and minimal sets

Denote the minimal faces for the homogeneous problems (recession directions) by
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equ11_HTML.gif
(2.11)
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equ12_HTML.gif
(2.12)
For connections between recession directions and optimality conditions, see e.g., [1, 6, 11]. Note that \(f_{P}^{0} \subseteq(f_{D}^{0})^{c}\) (equivalently, \(f_{D}^{0} \subseteq (f_{P}^{0})^{c}\)).

Definition 2.2

The pair of faces \(F_{1}\unlhd K\), \(F_{2} \unlhd K^{*}\) form a complementarity partition of K,K, denoted CP, if \(F_{1} \subseteq F_{2}^{c}\). (Equivalently, \(F_{2} \subseteq F_{1}^{c}\).) The partition is proper if both F1 and F2 are proper faces. The partition is strict if (F1)c=F2 or (F2)c=F1.

It is well known that
$$ F\unlhd K, G\unlhd K, F\subseteq G \implies F\unlhd G.$$
(2.13)
Therefore, we can assume \(F_{1} \unlhd F_{2}^{c}\) and \(F_{2} \unlhd F_{1}^{c}\) in Definition 2.2. Moreover, for every linear subspace \(\mathcal{L}\), the pair of faces
$$ \bigl(f_P^0, f_D^0\bigr)\ \text{form a complementarity partition, CP}$$
(2.14)
of K,K; and, if K is a polyhedral cone, then the partition is strict. The minimal face of (2.2) is the face of K generated by the feasible slack vectors; while the minimal face for (2.3) is the face of K generated by the feasible set of the dual problem, i.e., we denote \(f_{P} := \operatorname{face}\mathcal{F}_{P}^{s}\), \(f_{D} := \operatorname{face}\mathcal{F}^{x}_{D}\). Note that both fP and fD depend implicitly on the points \(\tilde{s}, \tilde{x}\) in the subspace formulations (2.9) and (2.10). Sometimes, to use this dependence more explicitly, we write \(f_{P}(\tilde{s})\) for fP and \(f_{D}(\tilde{x})\) for fD.

Given a set S, \(\mathrm{span}(S)\) denotes the set of all linear combinations of the elements in S.

An immediate important property of the complementarity partitions is:
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equ15_HTML.gif
(2.15)
We now obtain the following complementarity partitions and the corresponding theorems of the alternative from (2.15).

Proposition 2.3

Let\(\tilde{y},\tilde{s},\tilde{x}\)satisfy (2.7). Then the five pairs of faces
$$ \left[\begin{array} {@{}c@{\quad}c@{}}f_P^0 & f_D^0\\[2mm]\operatorname{face} \bigl(\bigl(\mathcal{L}^\perp+ \mathrm{span}\,\tilde{s}\bigr) \cap K \bigr)& \operatorname{face} \bigl(f_D^0 \cap\tilde{s}^\perp \bigr)\\[2mm]\operatorname{face} \bigl(f_P^0 \cap\tilde{x}^\perp \bigr) & \operatorname{face} \bigl((\mathcal{L}+ \mathrm{span}\,\tilde{x}) \cap K^*\bigr)\\[2mm]f_P & \operatorname{face} \bigl(f_D^0 \cap\tilde{s}^\perp \bigr)\\[2mm]\operatorname{face} \bigl(f_P^0 \cap\tilde{x}^\perp \bigr) & f_D \end{array} \right] $$
(2.16)
form complementarity partitions ofK,K.

Proof

That the first pair in (2.16) forms a complementarity partition follows from the definitions. The result for the second and third pairs follow from replacing \(\mathcal{L}^{\perp}\) with \(\mathcal{L}^{\perp}+ \operatorname{span}\tilde{s}\), and from replacing \(\mathcal{L}\) with \(\mathcal{L}+ \operatorname{span}\tilde{x}\), respectively. The result for the final two pairs follows from
$$\mathcal{F}_P^s \subseteq \bigl(\mathcal{L}^\perp+\mathrm{span}\, \tilde{s} \bigr) \cap K, \qquad \mathcal{F}_D^x\subseteq (\mathcal{L}+ \mathrm{span}\,\tilde{x} ) \cap K^*.$$
 □
If \(\operatorname{int}K^{*} \neq\emptyset\) (respectively, \(\operatorname{int}K\neq\emptyset\)), then the first pair in Proposition 2.3 is related to the following characterization for Slater’s CQ when c=0 (respectively, b=0):
$$ \begin{array}{@{}l}f_P^0 = \{0\}\quad\iff\quad\mathcal{L}\cap\operatorname{int}K^* \neq\emptyset\\[2mm]\bigl(\text{respectively},\ f_D^0 = \{0\}\quad\iff\quad\mathcal{L}^\perp\cap\operatorname{int}K \neq \emptyset\bigr).\end{array}$$
(2.17)
Equivalent characterizations for strict feasibility are related to the remaining four pairs in Proposition 2.3.
The primal and dual minimal subspace representations of \(\mathcal {L}^{\perp}\) and of \(\mathcal{L}\), respectively, are given by
$$ \mathcal{L}^\perp_{PM}:= \mathcal{L}^\perp \cap(f_P-f_P), \qquad \mathcal{L}_{DM}:=\mathcal{L}\cap(f_D-f_D).$$
(2.18)
The cone of feasible directions at\(\hat{y} \in\mathcal {F}_{P}^{y}\) is
$$ \mathcal{D}^{\leq}_P(\hat{y}) =\operatorname{cone} \bigl( \mathcal {F}_P^y -\hat{y} \bigr).$$
(2.19)
We similarly define the cones \(\mathcal{D}^{\leq}_{P}(\hat{s}),\mathcal {D}^{\leq }_{D}(\hat{x})\). For these three cones, we assume that \(\hat{y}, \hat{s},\hat{x}\) are suitable feasible points in \(\mathcal{F}_{P}^{y},\mathcal{F}_{P}^{s},\mathcal {F}_{D}^{x}\), respectively. The closures of these cones of feasible directions yield the standard tangent cones, denoted \(\mathbb{T}_{P}(\hat{y}), \mathbb{T}_{P}(\hat{s}), \mathbb{T}_{D}(\hat{x})\), respectively. (See e.g., [9, 21].) Note that if the primal feasible set is simply K, the cone of feasible directions corresponds to the so-called radial cone.

Proposition 2.4

[65, 70]

LetKbe closed. ThenKis a polyhedral cone if, and only if, at every point\(\hat{s}\in K\), the radial cone ofK, \(\operatorname{cone}(K-\hat{s})\), at\(\hat{s}\)is closed.

Example 2.5

We now look at two examples that illustrate the lack of closure for nonpolyhedral cones, e.g., in each instance we get
$$ K +\operatorname{span}f_P^0\subset\overline{K +\operatorname{span}f_P^0} = K +\bigl(\bigl(f_P^0\bigr)^c\bigr)^\perp.$$
(2.20)
The lack of closure in (2.20) can be used to find examples with both finite and infinite positive duality gaps; see e.g., the following example in item 1.
  1. 1.
    First, let n=2 and \(\mathcal{L}\) in (2.9) and (2.10) be such that \(\mathcal{L}^{\perp}= \operatorname{span}\{ E_{22} \}\). Then \(f_{P}^{0}=\operatorname{cone}\{E_{22}\}\) and \(f_{D}^{0}=\operatorname{cone}\{E_{11}\}\). Therefore, \(f_{P}^{0} = (f_{D}^{0})^{c}\) and \(f_{D}^{0} = (f_{P}^{0})^{c}\), i.e., this is a strict complementarity partition. Moreover, (2.20) holds e.g., \(E_{12} \in(f_{P}^{0})^{\perp}\cap(f_{D}^{0})^{\perp}\) and
    https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Eque_HTML.gif
    We can now choose \(\tilde{s} = E_{12}, \tilde{x}=E_{11}\). Then the primal is infeasible, e.g., Proposition 2.1, while the dual optimal value vD=0.
     
  2. 2.

    Now, let n=3 and suppose that \(\mathcal{L}^{\perp}= \operatorname{span}\{E_{33},E_{22}+E_{13}\}\). Then \(f_{P}^{0}=\operatorname{cone}\{E_{33}\}\) and \(f_{D}^{0}=\operatorname{cone}\{E_{11}\}\). Therefore, \(f_{P}^{0} \subset(f_{D}^{0})^{c}\) and \(f_{D}^{0} \subset(f_{P}^{0})^{c}\), i.e., this is not a strict complementarity partition. In addition, note that (2.20) holds and moreover, if we choose \(\tilde{s} =\tilde{x}= E_{22} \in(f_{P}^{0})^{c} \cap(f_{D}^{0})^{c}\), then \(E_{22} \in \operatorname{precl}(\mathcal{L}+ (\operatorname{face}\tilde{s})^{c}) \cap \operatorname{precl}(\mathcal{L}^{\perp}+ (\operatorname{face}\tilde{s})^{c})\). This means that \(0\neq\bar{s} = \bar{x} = E_{22}\) is both primal and dual optimal, see Proposition 5.2, below. Since \(\langle\bar{s},\bar{x}\rangle > 0\), we have obtained a finite positive duality gap.

     

Example 2.6

We can use \(K=K^{*}=\mathbb{S}^{n}_{+}\) and the algorithm in [71] to generate \(\mathcal{A}\) so that we have \((f_{P}^{0})^{c}\cap(f_{D}^{0})^{c} \neq\{0\}\). Here the linear transformation \(\mathcal{A}^{*}y =\sum_{i=1}^{m} y_{i}A_{i}\) for given \(A_{i} \in\mathbb{S}^{n}\), i=1,…,m. The main idea is to start with [QP QG QD] an orthogonal matrix; and then we construct one of the m matrices representing \(\mathcal {A}\) as
$$A_1 = [Q_P\quad Q_G\quad Q_D]\left[\begin{array} {@{}c@{\quad}c@{\quad}c@{}} 0 & 0 &Y_2^T\cr 0 & \fbox{$Y_{1}$} &Y_3^T\cr Y_2 & Y_3 &Y_4 \end{array} \right] [Q_P\quad Q_G\quad Q_D]^T,$$
where Y1≻0, Y4 symmetric, and QDY2≠0. The other matrices \(A_{i} \in\mathbb{S}^{n}\) are chosen so that the set {A1QP,…,AmQP} is linearly independent. Then we get the partition in positions given by
$$\left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}}f_P^0 & \cdot& \cdot\cr \cdot& G &\cdot\cr \cdot& \cdot& f_D^0\end{array}\right],$$
where G represents the gap in the partition.
All instances in the above examples have the facial block structure
$$\left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}}f_D^0 & 0 &0 \\0 & 0 & 0 \\0 & 0& f_P^0\end{array}\right].$$
Viz., the matrices in \(f_{D}^{0}\) are nonzero only in the (1,1) block, and the matrices in \(f_{P}^{0}\) are nonzero only in the (3,3) block. We now formalize the concept of such block structure for \(\mathbb{S}^{n}\) in the following definition and lemma. (These may be extended to more general cones using appropriate bases.)

Definition 2.7

The support of\(x \in\mathbb{S}^{n}\) is \(\mathcal{S}(x) := \{ (ij) : x_{ij}\neq0 \}\).

In the next lemma, as we formalize the concept of the above-mentioned block structure, kD corresponds to the size of the block \(f_{D}^{0}\), and (nkP+1) corresponds to the size of the block \(f_{P}^{0}\).

Lemma 2.8

Let\(K:=\mathbb{S}^{n}_{+}\).
  1. 1.
    There exists an orthogonal matrixQand integers 0≤kD<kPn+1 such that
    $$ x\in f_D^0, (ij)\in\mathcal{S}\bigl(Q^TxQ\bigr)\quad \implies\quad\max\{i,j\} \leq k_D,$$
    (2.21)
    and
    $$ s\in f_P^0, (ij)\in\mathcal{S}\bigl(Q^TsQ\bigr) \quad\implies\quad\min\{i,j\} \geq k_P.$$
    (2.22)
     
  2. 2.

    Letn≥3 and suppose the subspace\(\mathcal{L}\)is such that the complementarity partition\((f^{0}_{P},f^{0}_{D})\)is not strict. Then, there exists an orthogonal matrixQand integers 1≤kD<kP−1≤n−1 such that (2.21) and (2.22) hold.

     

Proof

We can choose \(x=Q_{x} D_{x} Q_{x}^{T} \in\operatorname{relint}f_{D}^{0}\) and \(s =Q_{s} D_{s} Q_{s}^{T}\in\operatorname{relint}f_{P}^{0}\), where Qx,Qs has orthonormal columns (of eigenvectors) and both Dx and Ds are diagonal positive definite. Choose Qr so that Q:=[Qx Qr Qs] is an orthogonal matrix. Then this Q does what we want, since \(f_{P}^{0} f_{D}^{0} = 0\). □

3 Facial properties

We now collect some interesting though technical facial properties for general convex cones K. These results are particularly useful for SDP. We include the notions of nice and devious cones. Further results are given in e.g., [7, 8, 17, 51].

3.1 Faces of general cones

Recall that a nonempty face \(F\unlhd K\) is exposed if F=ϕK, for some ϕK. Note that the faces of K are closed if K is closed.

Proposition 3.1

LetKbe closed and\(\emptyset\neq F \unlhd K\). Then:
  1. 1.

    \((F-F)\cap K =(\operatorname{span}F)\cap K = F\).

     
  2. 2.

    Fcc=Fif, and only if, Fis exposed.

     
  3. 3.

    \(\overline{K^{*}+\operatorname{span}F^{c}} \subseteq\overline{K^{*} +F^{\perp}}\). Moreover, ifKis facially exposed, then\(\overline{K^{*}+\operatorname{span}F^{c}} = \overline{K^{*} + F^{\perp}}\).

     

Proof

  1. 1.

    That \(F-F=\operatorname{span}F\) follows from the definition of a cone. Further, suppose s=f1f2 with sK and fiF,i=1,2. Then s+f2=f1F. Therefore, sF, by the definition of a face.

     
  2. 2.

    The result follows from the facts: the conjugate of G:=Fc is exposed by any \(x \in\operatorname{relint}G\); and, every exposed face is exposed by any point in the relative interior of its conjugate face.

     
  3. 3.

    That \({K^{*}+\operatorname{span}F^{c}} \subseteq{K^{*} + F^{\perp}}\) is clear from the definition of the conjugate face Fc. To prove equality, suppose that K is facially exposed and that \(w =(x+f)\in(K^{*}+F^{\perp}) \backslash\overline{K^{*} + \mathop {\mathrm{span}}F^{c}}\), with xK,fF. Then there exists ϕ such that \(\langle\phi, w\rangle < 0 \leq\langle\phi, (K^{*}+\mathop{\mathrm {span}}F^{c})\rangle \). This implies that ϕK∩(Fc)=KF, since K is facially exposed. This in turn implies 〈ϕ,w〉=〈ϕ,x+f〉≥0, a contradiction.

     
 □

Proposition 3.2

Let\(s \in\operatorname{relint}S\)andSKbe a convex set. Then:
  1. 1.

    \(\operatorname{face}s = \operatorname{face}S \),

     
  2. 2.

    \(\operatorname{cone}(K-s) =\operatorname{cone}(K-S) = K-\operatorname{face}s =K-\operatorname{face}S = K+\operatorname{span}\operatorname{face}s= K+\operatorname{span}\operatorname{face}S\).

     

Proof

  1. 1.

    That \(\operatorname{face}s \subseteq\operatorname{face}S\) is clear. To prove the converse inclusion, suppose that zSK,zs. Since \(s \in\operatorname{relint}S\), there exists wS, 0<θ<1, such that s=θw+(1−θ)z, i.e., s∈(w,z). Since \(s \in \operatorname{face}s\), we conclude that both \(w,z \in\operatorname{face}s\).

     
  2. 2.

    That \(\operatorname{cone}(K-s) \subseteq K- \operatorname{cone}s \subseteq K-\operatorname{face}s\subseteq K-\operatorname{span}\operatorname{face}s\) is clear. The other inclusions follow from part 1 and \(\operatorname{cone}(K-s) \supseteq\operatorname{cone}(\operatorname{face}(s)-s) = \operatorname{span}\operatorname{face}s\).

     
 □

We can combine Proposition 2.4 and Proposition 3.2, and conclude that K is polyhedral if, and only if, \(K+ \mathop {\mathrm{span}}F\) is closed, for all \(F \unlhd K\). The following Proposition 3.3 illustrates some technical properties of faces, conjugates, and closure.

Proposition 3.3

LetTbe a convex cone and\(F\unlhd T\).
  1. 1.
    Suppose that\(\bar{s} \in\operatorname{relint}F\)and\(\bar{x} \in \operatorname{relint}F^{c}\). Then
    $$\bar{s} + \bar{x} \in\operatorname{int}\bigl(T+T^*\bigr).$$
     
  2. 2.
    Suppose that\(\bar{s} \in\operatorname{relint}F\). Then
    $$ \begin{array}{@{}l} \overline{\operatorname{cone}(T-\bar{s})}= \bigl(F^c\bigr)^*;\\[2mm]\operatorname{cone}(T-\bar{s}) \supseteq\operatorname{relint}\bigl(\bigl(F^c\bigr)^* \bigr).\end{array}$$
    (3.1)
     

Proof

  1. 1.
    First, note that if \(\operatorname{int}(T+T^{*}) = \emptyset\), then we have
    $$0\neq\bigl(T+T^*\bigr)^\perp\subseteq T^{**}\cap T^*=\overline{T}\cap T^*\subseteq T+T^*,$$
    a contradiction, i.e., this shows that \(\operatorname{int}(T+T^{*}) \neq \emptyset\).

    Now suppose that \(\bar{s} + \bar{x} \notin\operatorname{int}(T+T^{*})\). Then we can find a supporting hyperplane ϕ so that \(\bar{s} + \bar{x} \in(T+T^{*})\cap\phi^{\perp}\lhd T+T^{*}\) and \(0 \neq\phi\in(T+T^{*})^{*}=\overline{T} \cap T^{*}\). Therefore, we conclude \(\langle\phi, \bar{s} + \bar{x}\rangle =0\) implies that both \(\langle\phi, \bar{s}\rangle =0\) and \(\langle\phi, \bar{x}\rangle =0\). This means that \(\phi\in T^{*}\cap\bar{s}^{\perp}= F^{c}\) and \(\phi\in\bar{T} \cap\bar{x}^{\perp}= (F^{c})^{c}\), giving 0≠ϕ∈(Fc)∩(Fc)c={0}, which is a contradiction.

     
  2. 2.

    The first result follows from: \(\overline{\operatorname{cone}(T-\bar{s})}= {(T-\bar{s})}^{**}= {(T^{*} \cap \bar{s}^{\perp})}^{*}\).

    For the second result, we use the first result and Theorem 6.3 of Rockafellar [63] to deduce
    $$\operatorname{relint}\bigl(\operatorname{cone}(T-\bar{s}) \bigr)= \mathrm {relint} \bigl(\overline{\operatorname{cone}(T-\bar{s})} \bigr)=\operatorname{relint}\bigl[\bigl(F^c \bigr)^* \bigr]$$
    which implies the desired conclusion.
     
 □

3.2 Faces for primal-dual pair (\(\mathbb{P}\)) and (\(\mathbb{D}\))

We now present facial properties specific to the primal-dual pair (2.2) and (2.3). In particular, this includes relationships between the minimal faces fP,fD and the minimal faces for the homogeneous problems, \(f_{P}^{0},f_{D}^{0}\).

Proposition 3.4

Suppose that both (2.2) and (2.3) are feasible, i.e., equivalently\(\tilde{s} \in K + \mathcal{L}^{\perp}\)and\(\tilde{x}\in K^{*} +\mathcal{L}\). Let\(\hat{s} \in\mathcal{F}_{P}(\tilde{s})\)and\(\hat{x} \in\mathcal {F}_{D}(\tilde{x})\). Then the following hold.
  1. 1.
    $$ f_P^0 \subseteq\operatorname{face}\bigl(\hat{s} + f_P^0\bigr) \subseteq f_P(\tilde {s}), \qquad f_D^0 \subseteq\operatorname{face}\bigl(\hat{x} + f_D^0\bigr) \subseteq f_D(\tilde{x}).$$
    (3.2)
     
  2. 2.
    $$\tilde{s} \in\bigl(f_D^0\bigr)^c+\mathcal{L}^\perp\Leftrightarrow f_P(\tilde{s}) \subseteq \bigl(f_D^0\bigr)^c; \qquad \tilde{x} \in \bigl(f_P^0\bigr)^c + \mathcal{L}\Leftrightarrow f_D(\tilde{x}) \subseteq\bigl(f_P^0\bigr)^c.$$
     

Proof

Since both problems are feasible, we can assume, without loss of generality, that \(\hat{s}=\tilde{s} \in K, \hat{x}=\tilde{x} \in K^{*}\).

1. Since \(\operatorname{cone}\tilde{s}\) and \(f_{P}^{0}\) are convex cones containing the origin, \(\operatorname{cone}\tilde{s} + f_{P}^{0} = \mathop{\mathrm {conv}}(\operatorname{cone}\tilde{s} \cup f_{P}^{0})\); see e.g., [63, Theorem 3.8]. Hence,
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equm_HTML.gif
This proves the first inclusion. It is clear that \(\tilde{s} + (\mathcal{L}^{\perp}\cap K) \subseteq(\tilde{s} + \mathcal {L}^{\perp}) \cap K\). This yields the second inclusion. The final two inclusions follow similarly.

2. Suppose that \(\tilde{s} \in(f_{D}^{0})^{c}+\mathcal{L}^{\perp}\) and \(\tilde{s} + r \in K\) with \(r\in\mathcal{L}^{\perp}\). Then, for all \(\ell\in\mathcal{L}\cap K^{*}\subseteq f_{D}^{0}\), we have \(\langle\tilde{s} + r, \ell\rangle =\langle\tilde{s}, \ell\rangle =0\), since \(\tilde{s} \in(f_{D}^{0})^{c}+\mathcal{L}^{\perp}\). This implies that \(f_{P}(\tilde{s})=\operatorname{face}( (\tilde{s} +\mathcal{L}^{\perp})\cap K)\) is orthogonal to \(f_{D}^{0}=\operatorname{face}(\mathcal{L}\cap K^{*})\), i.e., the first implication holds. (This also follows from part 1, using \(f_{P}(\tilde{s})\subseteq[f_{D}(\tilde{x})]^{c}\).)

For the converse implication, since (2.2) is feasible, we have \(\tilde{s} \in f_{P}(\tilde{s})\). So if \(f_{P}(\tilde{s}) \subseteq(f_{D}^{0})^{c}\), then \(\tilde{s} \in(f_{D}^{0})^{c}\).

The second equivalence follows similarly. □

Additional relationships between the faces follow. First, we need a lemma that is of interest in its own right.

Lemma 3.5

Let\(\tilde{s} \in f_{P}^{0}\), and suppose that\(s = \tilde{s} + \ell\)is feasible for (2.2) with\(\ell\in\mathcal{L}^{\perp}\). Then\(\ell\in\operatorname{span}f_{P}^{0}\).

Proof

Let \(v\in\operatorname{relint}(\mathcal{L}^{\perp}\cap K)\). Then \(v \in \operatorname{relint}f_{P}^{0}\), and since \(\tilde{s} \in f_{P}^{0}\), we have \(v - \epsilon\tilde{s} \in f_{P}^{0} \) for some ϵ>0. Now if is such that \(s = \tilde{s} + \ell\) is feasible for (2.2), then \(\tilde{s} + \ell\in K\), and
$$\frac{1}{\epsilon}v + \ell = \frac{1}{\epsilon} (v - \epsilon\tilde{s}) + (\tilde{s} + \ell) \in f_P^0 + K = K.$$
For convenience, define α:=1/ϵ. Since \(\alpha v \in\mathcal{L}^{\perp}\) and \(\ell\in\mathcal{L}^{\perp}\), we in fact have
$$ \alpha v + \ell\in K \cap\mathcal{L}^\perp\subseteq f_P^0,$$
(3.3)
which implies \(\ell\in f_{P}^{0} -f_{P}^{0}\). □

Proposition 3.6

  1. 1.

    \(\tilde{s} \in f_{P}^{0}\cup\mathcal{L}^{\perp}\Rightarrow f_{P}(\tilde{s}) = f_{P}^{0}\)and\(\tilde{x} \in f_{D}^{0}\cup\mathcal{L}\Rightarrow f_{D}(\tilde{x}) = f_{D}^{0}\).

     
  2. 2.

    Let\(f_{D}^{0} \lhd K^{*}\). Then there exists\(0 \neq\phi\in K\cap\mathcal{L}^{\perp}\).

     

Proof

1. We begin by proving the first statement. If \(\tilde{s} \in\mathcal{L}^{\perp}\), then \(\tilde{s} + \mathcal {L}^{\perp}= \mathcal{L}^{\perp}\), so the desired result holds. If instead \(\tilde{s} \in f_{P}^{0}\), then it follows from Lemma 3.5 that \(\ell\in\mathop {\mathrm{span}}f_{P}^{0}\) for all feasible points of the form \(s = \tilde{s} + \ell\). Hence all feasible s lie in the set \(\mathrm{span}\,(f_{P}^{0}) \cap K\), which by Proposition 3.1, part 1, equals \(f_{P}^{0}\). So \(f_{P}(\tilde{s}) \subseteq f_{P}^{0}\); but, the reverse inclusion holds by Proposition 3.4, part 1.

The second statement for \(f_{D}^{0}\) in proven in a similar way.

2. Existence is by the theorem of the alternative for the Slater CQ; see (2.15) and the related Proposition 2.3. □

We conclude this subsection with a new result indicating that the failure of strict complementarity in \((f_{P}^{0}, f_{D}^{0})\) can be related to the lack of closure in the sum of the cone and the subspace.

Proposition 3.7

LetKbe closed. If there exists a nonzerox∈−(KK) such that
$$x \in\bigl(K \cap\mathcal{L}^\perp\bigr)^\perp\cap\bigl(K^*\cap \mathcal {L}\bigr)^\perp, $$
(3.4)
then\(x \in\operatorname{precl}(K+\mathcal{L}^{\perp}) \cap\mathrm {precl}(K^{*}+\mathcal{L})\). Hence, neither\(K+\mathcal{L}^{\perp}\)nor\(K^{*}+\mathcal{L}\)is closed.

Proof

To obtain a contradiction, suppose that (3.4) holds for a nonzero x∈−(KK), but \(x \in K + \mathcal{L}^{\perp}\). Then there exists wK such that \(x-w \in\mathcal{L}^{\perp}\). Moreover, xw∈−KK=−K, so \(x-w \in-(K \cap\mathcal{L}^{\perp})\). It follows from (3.4) that 〈x,xw〉=0. However, 〈x,xw〉=〈x,x〉+〈−x,w〉>0, where we have used the fact that x∈−K. Hence, \(x \notin K + \mathcal{L}^{\perp}\). A similar argument shows that \(x \notin K^{*} + \mathcal{L}\).

Since K and \(\mathcal{L}^{\perp}\) are closed convex cones, we have \((K \cap\mathcal{L}^{\perp})^{*} = \overline{K^{*} + (\mathcal{L}^{\perp})^{*}}= \overline{K^{*} + \mathcal{L}}\). It follows from (3.4) that \(x \in\overline{K^{*} + \mathcal{L}}\). Similarly, \(x \in\overline{K + \mathcal{L}^{\perp}}\). This completes the proof. □

The reader might naturally ask: when does such an x satisfying the conditions of the last proposition exist? As an answer, consider the case K=K and a linear subspace \(\mathcal{L}\) such that \((f_{P}^{0}, f_{D}^{0})\) is a proper partition but is not strictly complementary. Let \(G:=(f_{P}^{0})^{c} \cap(f_{D}^{0})^{c}\). Then every x∈−G∖{0} satisfies the assumption of the above proposition.

3.3 Nice cones, devious cones, and SDP

Definition 3.8

A face \(F\unlhd K\) is called nice if K+F is closed. A closed convex cone K is called a nice cone or a facially dual-complete cone, FDC, if
$$ K^*+F^\perp\ \mbox{is closed for all}\ F\unlhd K.$$
(3.5)

The condition in (3.5) was used in [13] to allow for extended Lagrange multipliers in \(f_{P}^{*}\) to be split into a sum using K and \(f_{P}^{\perp}\). This allowed for restricted Lagrange multiplier results with the multiplier in K. The condition (3.5) was also used in [52] where the term nice cone was introduced. In addition, it was shown by Pataki (forthcoming paper) that a FDC cone must be facially exposed.

Moreover, the FDC property has an implication for Proposition 3.1, part 3. We now see that this holds for SDP.

Lemma 3.9

[58, 73]

Suppose thatFis a proper face of\(\mathbb{S}^{n}_{+}\), i.e., \(\{0\}\neq F \lhd \mathbb{S}^{n}_{+}\). Then:
$$F^* = \mathbb{S}^n_++ F^\perp= \overline{\mathbb{S}^n_++ \mathop {\mathrm{span}}F^c},$$
$$\mathbb{S}^n_++ \operatorname{span}F^c\ \mathit{is\ not\ closed}.$$

From Lemma 3.9, we see that \(\mathbb{S}^{n}_{+}\) is a nice cone. In fact, as pointed out in [52], many other classes of cones are nice cones, e.g., polyhedral and p-cones. However, the lack of closure property in Lemma 3.9 is not a nice property. In fact, from Proposition 3.2, part 2, this corresponds to the lack of closure for radial cones, see [65] which can result in duality problems. Therefore, we add the following.

Definition 3.10

A face \(F\unlhd K\) is called devious if the set \(K+\operatorname{span}F\) is not closed. A cone K is called devious if
$$\mathrm{the\ set}\ K+\operatorname{span}F\ \mathrm{is}\ \mathbf{not}\ \mathrm{closed\ for\ all}\ \{0\} \neq F \lhd K.$$

By Lemma 3.9, \(\mathbb{S}^{n}_{+}\) is a nice but devious cone. On the other hand, polyhedral cones are nice and not devious, since sums of polyhedral cones and subspaces are closed, e.g., [63, Chap. 9].

The facial structure of \(\mathbb{S}^{n}_{+}\) is well known, e.g., [58, 73]. Each face \(F \unlhd\mathbb{S}^{n}_{+}\) is characterized by a unique subspace \(S\subseteq\mathbb{R}^{n}\):
$$F = \bigl\{ x \in\mathbb{S}^n_+: \mathcal{N}(x) \supseteq S \bigr\}; \qquad \operatorname{relint}F = \bigl\{ x \in\mathbb{S}^n_+:\mathcal{N}(x) = S \bigr\}.$$
The conjugate face satisfies
$$F^c = \bigl\{ x \in\mathbb{S}^n_+: \mathcal{N}(x)\supseteq S^\perp \bigr\}; \qquad \operatorname{relint}F = \bigl\{ x \in \mathbb{S}^n_+: \mathcal{N}(x) = S^\perp \bigr\}.$$
The description of \(\operatorname{span}F\) for \(F\unlhd\mathbb {S}^{n}_{+}\) is now clear.

Another useful property of SDPs (and the Löwner partial order) is given by the following lemma. This lemma played a critical role in the explicit description of a dual SDP problem for which strong duality holds.

Lemma 3.11

[57]

Let\(\tilde{K} \subseteq\mathbb{S}^{n}_{+}\)be a closed convex cone. Then
$$\bigl[(\operatorname{face}\tilde{K} )^c \bigr]^\perp =\left\{ W +W^T: W \in\mathbb{R}^{n \times n},\left[\begin{array}{@{}c@{\quad}c@{}} I & W^T\cr W & U \end{array} \right] \succeq0,\ \mathit{for\ some}\ U \in\tilde{K} \right\}.$$

Properties 3.12

The following three properties of the cone\(\mathbb{S}^{n}_{+}\)are needed for the strong duality approach in Ramana [57]. The first two also make the Borwein-Wolkowicz approach in [15] behave particularly well:
  1. 1.

    Kis facially exposed.

     
  2. 2.

    Kis FDC.

     
  3. 3.

    Lemma 3.11.

     

Suppose that the cone K describing the problem \((\mathbb{P})\) is SDP-representable. (That is, there exists d and a linear subspace \(V \subset\mathbb{S}^{d}\) such that \(V \cap\mathbb{S}^{d}_{++} \neq\emptyset\) and K is isomorphic to \((V \cap\mathbb{S}^{d}_{+})\).) Then by [20, Corollary 1, Proposition 4], K is facially exposed and FDC, since \(\mathbb{S}^{d}_{+}\) is. Moreover, by [20, Proposition 3], every proper face of K is a proper face of \(\mathbb{S}^{d}_{+}\) intersected with the subspace V. Hence, assuming that a suitable representation of K is given, an analogue of Lemma 3.11 is also available in this case. Therefore, SDP-representable cones (which strictly include homogeneous cones, due to a result of Chua [19] and Faybusovich [24]) satisfy all three of the above-mentioned Properties 3.12. For related recent results on homogeneous cones and strong duality, see Pólik and Terlaky [53].

4 Duality and minimal representations

In this section, we see that minimal representations of the problem guarantee strong duality and stability results, i.e., combining the minimal cone and the minimal subspace together reduces both the dimension of the problem and the number of constraints, and also guarantees Slater’s constraint qualification. We first use the minimal subspaces and extend the known strong duality results without any constraint qualification, see e.g., [1315, 72]. Equivalent strong duality results based on an extended Lagrangian are given in [56, 57]. (See [58, 73] for comparison and summaries of the two types of duality results.) By strong duality for (2.2), we mean that there is a zero duality gap, vP=vD, and the dual optimal value vD in (2.3) is attained.

4.1 Strong duality and constraint qualifications

We now present strong duality results that hold with and without CQs. We also present: a weakest constraint qualification (WCQ), i.e., a CQ at a given feasible point \(\bar{y} \in\mathcal{F}_{P}^{y}(c)\) that is independent of b; and a universal constraint qualification, (UCQ), i.e., a CQ that is independent of both b and c. Following is the classical, well-known, strong duality result for (2.2) under the standard Slater CQ.

Theorem 4.1

(See, e.g., [46, 66])

Suppose that Slater’s CQ (strict feasibility) holds for (2.2). Then strong duality holds for (2.2), i.e., vP=vDand the dual valuevDin (2.3) is attained. Equivalently, there exists\(\bar{x} \in K^{*}\)such that
$$\langle b,y\rangle + \bigl\langle c-\mathcal{A}^*y, \bar{x}\bigr\rangle \geq v_P, \quad\forall y\in\mathbb{R}^m.$$
Moreover, ifvPis attained at\(\bar{y} \in\mathcal{F}_{P}^{y}\), then\(\langle c-\mathcal{A}^{*} \bar{y}, \bar{x}\rangle =0\) (complementary slackness holds).

A nice compact formulation follows.

Corollary 4.2

Suppose that Slater’s CQ (strict feasibility) holds for (2.2) and\(\bar{y} \in\mathcal{F}_{P}^{y}\). Then, \(\bar{y}\)is optimal for (2.2) if, and only if,
$$ b \in\mathcal{A} \bigl[ (K-\bar{s})^* \bigr],$$
(4.1)
where\(\bar{s} = c-\mathcal{A}^{*} \bar{y}\).

Proof

The result follows from the observation that \((\operatorname{face}\bar{s})^{c}=K^{*} \cap\bar{s}^{\perp}= (K-\bar{s})^{*}\), i.e., (4.1) is equivalent to dual feasibility and complementary slackness. □

Strong duality can fail if Slater’s CQ does not hold. In [1315], an equivalent regularized primal problem that is based on the minimal face,
$$ v_{RP}:=\sup\bigl\{\langle b, y\rangle : \mathcal{A}^* y\preceq_{f_P} c\bigr\}$$
(4.2)
is considered. Its Lagrangian dual is given by
$$ v_{DRP}:=\inf\bigl\{\langle c, x\rangle : \mathcal{A}x = b,\ x \succeq_{f_P^*} 0\bigr\}.$$
(4.3)

Theorem 4.3

[13]

Strong duality holds for the pair (4.2) and (4.3), or equivalently, for the pair (2.2) and (4.3); i.e., vP=vRP=vDRPand the dual optimal valuevDRPis attained. Equivalently, there existsx∈(fP)such that
$$\langle b, y\rangle + \bigl\langle c-\mathcal{A}^*y, x^*\bigr\rangle \geq v_P, \quad\forall y \in\bigl(\mathcal{A}^*\bigr)^{-1}(f_P-f_P).$$
Moreover, ifvPis attained at\(\bar{y} \in\mathcal{F}_{P}^{y}\), then\(\langle c-\mathcal{A}^{*} \bar{y}, x^{*}\rangle =0\) (complementary slackness holds).

Corollary 4.4

Let\(\bar{y} \in\mathcal{F}_{P}^{y}\). Then\(\bar{y}\)is optimal for (2.2) if, and only if,
$$b \in\mathcal{A} \bigl[ (f_P-\bar{s})^* \bigr],$$
where\(\bar{s} = c-\mathcal{A}^{*} \bar{y}\).

Proof

As above, in the proof of Corollary 4.2, the result follows from the observation that \(f_{P}^{*} \cap\bar{s}^{\perp}= (f_{P}-\bar{s})^{*}\). □

The next result uses the minimal subspace representation of \(\mathcal{L}^{\perp}\), introduced in (2.18), \(\mathcal{L}^{\perp}_{PM}= \mathcal{L}^{\perp}\cap(f_{P}-f_{P})\).

Corollary 4.5

Let\(\tilde{y}, \tilde{s}\), and\(\tilde{x}\)satisfy (2.7) with\(\tilde{s} \in f_{P}-f_{P}\)and let
$$ K^*+(f_P)^\perp= (f_P)^*.$$
(4.4)
Consider the following pair of dual programs.
$$ v_{RP_M} = \langle c, \tilde{x}\rangle -\inf_s \bigl\{ \langle s, \tilde{x}\rangle : s \in\bigl(\tilde{s} +\mathcal{L}_{PM}^\perp\bigr) \cap K\bigr\},$$
(4.5)
$$ v_{DRP_M}= \langle b, \tilde{y}\rangle +\inf_x \bigl\{ \langle\tilde{s}, x\rangle : x \in(\tilde{x} +\mathcal{L}_{PM}) \cap K^*\bigr\}.$$
(4.6)
Then, \(v_{RP_{M}} = v_{RP}= v_{P}= v_{DRP_{M}}= v_{DRP}\), and strong duality holds for (4.5) and (4.6), or equivalently, for the pair (2.2) and (4.6).

Proof

That \(v_{P}=v_{RP_{M}}=v_{RP}\) follows from the definition of the minimal subspace representation in (2.18):
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equx_HTML.gif
For the regularized dual, we see that
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equy_HTML.gif
 □

Remark 4.6

The condition in (4.4) is equivalent to K+(fP) being closed, and is clearly true for every face of K, if K is a FDC cone.

Remark 4.7

Using the minimal subspace representations of \(\mathcal{L}\) in (2.3), i.e., replacing \(\mathcal{L}\) in (2.3) by \(\mathcal{L}_{DM}\) in (2.18), we may obtain a result similar to Corollary 4.5.

Note that if the Slater CQ holds, then the minimal sets (face and subspace) satisfy fP=K and (2.18). We now see that if at least one of these conditions holds, then strong duality holds.

Corollary 4.8

Suppose that\(\operatorname{int}K = \emptyset\)but the generalized Slater CQ (relative strict feasibility) holds for (2.2), i.e.,
$$ \hat{s} := c - \mathcal{A}^* \hat{y} \in \operatorname{relint}K,\quad\mathit{for\ some}\ \hat{y} \in \mathcal{W}\quad(\mathit{Generalized\ Slater\ CQ}).$$
(4.7)
(Equivalently, suppose that the minimal face satisfiesfP=K.) Then strong duality holds for (2.2).

Proof

The proof follows immediately from Theorem 4.3 after noting that K=fP. □

The following corollary illustrates strong duality for a variation of the generalized Slater constraint qualification, i.e., for the case that the minimal subspace satisfies (2.18).

Corollary 4.9

Let\(\tilde{s} \in f_{P}-f_{P}\)andKbe\(\mathrm{FDC}\). Suppose that
$$ \mathcal{L}^\perp\cap(K-K)\subseteq f_P - f_P\quad (\mathit{Subspace\ CQ}).$$
(4.8)
(Equivalently, suppose that\(\mathcal{L}_{PM}^{\perp}= \mathcal {L}^{\perp}\cap(K-K)\).) Then strong duality holds for (2.2).

Proof

Follows directly from Corollary 4.5. □

We now summarize the results in the special case that K is FDC (a nice cone). The first item presents a regularized problem that satisfies Slater’s CQ. This is the approach used in [18]. Note that early results on weakest constraint qualifications for general nonlinear problems are given in e.g., [31].

Theorem 4.10

Let\(\tilde{s}, \tilde{x}\)satisfy linear feasibility (2.7) with\(\tilde{s} \in f_{P}-f_{P}\)and letKbe\(\mathrm{FDC}\). Then we have the following conclusions.
  1. 1.
    The primal optimal values are all equal, \(v_{P}=v_{RP}=v_{RP_{M}}\). Moreover, strong duality holds for the primal, where the primal is chosen from the set
    $$\bigl\{(2.9),\ (4.2),\ (4.5) \bigr \}\quad(\mathit{set\ of\ primal\ programs})$$
    and the dual is chosen from the set
    $$\bigl\{(4.3),\ (4.6)\bigr\}\quad (\mathit{set\ of\ dual\ programs})$$
    i.e., the optimal values are all equal and the dual optimal value is attained.
     
  2. 2.
    Furthermore, let\(\mathcal{T}: \mathbb{R}^{t} \rightarrow\mathcal{V}\)be a one-one linear transformation with\(\mathcal{R}(\mathcal{T})=f_{P}-f_{P}\). Then Slater’s CQ holds for the regularized problem
    $$ v_{RP_M} = \langle c, \tilde{x}\rangle -\inf_{v \in\mathbb{R}^t} \bigl\{ \bigl\langle v, \mathcal{T}^* \tilde{x}\bigr\rangle : v \in \bigl(\mathcal{T}^\dagger\tilde{s} + \mathcal{T}^\dagger \bigl(\mathcal{L}_{PM}^\perp \bigr) \bigr) \cap \mathcal{T}^\dagger(f_P)\bigr\}.$$
    (4.9)
     
  3. 3.
    The following are CQs for (2.2):
    1. (a)

      fP=K (equivalently generalized Slater CQ (4.7));

       
    2. (b)

      \(\mathcal{L}^{\perp}\cap(K-K)\subseteq f_{P}-f_{P} \) (equivalently\(\mathcal{L}_{PM}^{\perp}= \mathcal{L}^{\perp}\cap(K-K)\));

       
     
  4. 4.
    Let\(\bar{y} \in\mathcal{F}_{P}^{y}(c)\)and\(\bar{s} = c-\mathcal{A}^{*}\bar{y}\). Then,
    $$ \mathcal{D}^\leq_P(\bar{y})^* = -\mathcal{A}\bigl((K-\bar{s})^*\bigr)\ \mathit{is\ a\ WCQ}\ \mathit{for}\ (2.2)\ \mathit{at}\ (\bar{y},\bar{s}).$$
    (4.10)
    Equivalently,
    $$ \mathcal{A}\bigl[(f_P-\bar{s})^*\bigr] = \mathcal{A}\bigl((K-\bar{s})^*\bigr)\ \mathit{is\ a\ WCQ}\ \mathit{for}\ (2.2)\ \mathit{at}\ (\bar{y}, \bar{s}).$$
    (4.11)
     

Proof

  1. 1.

    These results summarize Theorem 4.3 and Corollary 4.5.

     
  2. 2.

    From the definitions, we know that there exists \(\ell\in \mathcal{L}^{\perp}_{PM}\) with \(\hat{s}= \tilde{s} +\ell\in\operatorname{relint}f_{P}\). Therefore, \(v:=\mathcal{T}^{\dagger}\hat{s}= \mathcal{T}^{\dagger}(\tilde{s} + \ell ) \in\operatorname{int}\mathcal{T}^{\dagger}(f_{P})\).

     
  3. 3.

    The results follow from Corollaries 4.8, 4.9, respectively.

     
  4. 4.

    The so-called Rockafellar-Pshenichnyi condition, e.g., [55], [31, Theorem 1], states that \(\bar{y}\) is optimal if, and only if, \(b \in-\mathcal{D}^{\leq}_{P}(\bar{y})^{*}\). From Theorem 4.3, \(\bar{y}\) is optimal if, and only if, \(\mathcal{A}\bar{x} = b, \langle \bar{s}, \bar{x}\rangle = 0\), for some \(\bar{x} \in f_{P}^{*}\); equivalently, if, and only if, \(b \in \mathcal{A}((f_{P}-\bar{s})^{*})\). The result follows from the fact that strong duality holds at an optimal \(\bar{y}\) if, and only if, \(\mathcal{A}\bar{x} = b, \langle\bar{s}, \bar{x}\rangle = 0\), for some \(\bar{x}\in K^{*}\); equivalently, \(\mathcal{A}\bar{x} = b\), for some \(\bar{x} \in(K-\bar{s})^{*}\).

     
 □

Remark 4.11

The WCQ in (4.10) follows the approach in e.g., [22, 31, 43, 44, 76]. Moreover, since for any set S, \(\mathcal{A}(S)\) is closed if, and only if, \(S+\mathcal{L}\) is closed (e.g., [10, 34]), we conclude that a necessary condition for the WCQ to hold at a feasible \(\bar{s} \in\mathcal{F}_{P}^{s}\) is that
$$ (K-\bar{s} )^* + \mathcal{L}\ \mbox{is closed}.$$
(4.12)
(For a recent detailed study of when the linear image of a closed convex set is closed, see e.g., [52]. For related perturbation results, see [12].)

4.1.1 Universal constraint qualifications

A universal CQ, denoted UCQ, is a CQ that holds independent of the data b,c, i.e., as in LP, strong duality holds for arbitrary perturbations of the data b,c as long as feasibility is not lost.

Theorem 4.12

Suppose thatKis\(\mathrm{FDC}\), and\(\tilde{s} \in K, \tilde{x} \in K^{*}\)in the primal-dual subspace representation in (2.9) and (2.10). Then
$$\mathcal{L}^\perp\subseteq f_P^0- f_P^0$$
is a UCQ, i.e., a universal CQ for (2.2).

Proof

The result follows from Corollary 4.9 and the fact that \(\tilde{s} \in K, \tilde{x} \in K^{*}\) implies \(f_{P}^{0} \subseteq f_{P}\) and \(f_{D}^{0} \subseteq f_{D}\), see Proposition 3.4. □

Corollary 4.13

Suppose that\(K=\mathbb{S}^{n}_{+}\), bothvPandvDare finite, andn≤2. Then strong duality holds for at least one of (2.2) or (2.3).

Proof

We have both \(\mathcal{F}_{P}^{s} \neq\emptyset\) and \(\mathcal{F}_{D}^{x}\neq\emptyset\). By going through the possible cases, we see that one of the CQs \(\mathcal{L}^{\perp}\subseteq f_{P}^{0}- f_{P}^{0}\) or \(\mathcal{L}\subseteq f_{D}^{0}- f_{D}^{0}\) must hold. □

5 Failure of Strong Duality and Strict Complementarity

As discussed above, the absence of a CQ for (2.2) can result in the failure of strong duality, i.e., we have theoretical difficulties. In addition, it has been shown that near loss of Slater’s CQ is closely correlated with an increase in the expected number of iterations in interior-point methods both in theory [59, 61] and empirically, [25, 26]. Therefore, a regularization step should be an essential preprocessor for SDP solvers.

It is also known that the lack of strict complementarity for SDP may result in theoretical difficulties. For example, superlinear and quadratic convergence results for interior-point methods depend on the strict complementarity assumption, e.g., [3, 37, 40, 47, 54]. This is also the case for convergence of the central path to the analytic center of the optimal face, [32]. In addition, it is shown empirically in [71] that the loss of strict complementarity is closely correlated with the expected number of iterations in interior-point methods. However, one can generate problems where strict complementarity fails independent of whether or not Slater’s CQ holds for the primal and/or the dual, [71]. Therefore, we see a connection between the theoretical difficulty from an absence of Slater’s CQ and numerical algorithms, and a similar connection for the absence of strict complementarity. We see below that duality and strict complementarity of the homogeneous problem have a surprising theoretical connection as well.

Strong duality for (2.2) means a zero duality gap, vP=vD and dual attainment, vD=〈c,x〉, for some \(x^{*}\in \mathcal {F}_{D}^{x}\). The CQs (resp. UCQs), introduced above in Sect. 4, guarantee that strong duality holds independent of the data b (resp. b and c). Under our assumption that vP is finite valued, there are three cases of failure to consider: (i) a zero duality gap but no dual attainment; (ii) an infinite duality gap (dual infeasibility); (iii) a finite positive duality gap.

5.1 Finite positive duality gaps

5.1.1 Positive gaps and cones of feasible directions

We present characterizations for a finite positive duality gap under attainment assumptions in Proposition 5.2. We first give sufficient conditions for a positive duality gap using well known optimality conditions based on feasible directions.

Proposition 5.1

Let\(\tilde{s} \in\mathcal{F}_{P}^{s}\), \(\tilde{x} \in\mathcal{F}_{D}^{x}\), and\(\langle\tilde{s}, \tilde{x}\rangle >0\). Suppose that\(\tilde{s} \in\mathcal{D}^{\leq}_{D}(\tilde{x})^{*}\)and\(\tilde{x} \in\mathcal{D}^{\leq}_{P}(\tilde{s})^{*}\). Then\(\tilde{s}\)is optimal for (2.2), \(\tilde{x}\)is optimal for (2.3), and −∞<vP<vD<∞.

Proof

The optimality of \(\tilde{s}\) and \(\tilde{x}\) follows immediately from the definition of the cones of feasible directions and the Rockafellar-Pshenichnyi condition, see e.g., the proof of Theorem 4.10. The finite positive duality gap follows from the hypotheses that both (2.2) and (2.3) are feasible, and that \(\langle\tilde{s}, \tilde{x}\rangle >0\). □

A well-known characterization for a zero duality gap can be given using the perturbation function. For example, define
$$v_P(\epsilon) := \sup_y \bigl\{\langle b,y\rangle :\mathcal{A}^* y \preceq_K c +\epsilon \bigr\},\quad\mbox{where}\ \epsilon\in\mathcal{V}.$$
The connection with the dual functional \(\phi(x):=\sup_{y} \langle b,y\rangle + \langle x, c-\mathcal{A}^{*}y\rangle \) is given in e.g., [46]. Then the geometry shows that the closure of the epigraph of vP characterizes a zero duality gap. We now use representations of the cones of feasible directions and the optimal solution sets \(\mathcal{O}^{s}_{P}, \mathcal{O}^{x}_{D}\), to present a characterization for a finite positive duality gap in the case of attainment of the primal and dual optimal values.

Proposition 5.2

Suppose thatKis closed, \(\tilde{y}\)is feasible for (2.2), with corresponding slack\(\tilde{s}\), and that\(\tilde{x}\)is feasible for (2.3). Then
$$\tilde{s} \in\mathcal{O}_P^s, \tilde{x} \in\mathcal{O}_D^x,\quad\langle\tilde{s}, \tilde{x}\rangle > 0,$$
if, and only if,
$$\tilde{x} \in\mathcal{D}^{\leq}_P(\tilde{s})^* \setminus(K-\tilde{s})^*,\quad \mathit{and}\quad \tilde{s} \in\mathcal{D}^{\leq}_P(\tilde{x})^* \setminus\bigl(K^*-\tilde{x}\bigr)^*.$$

Proof

Using the subspace problem formulations, the Rockafellar-Pshenichnyi condition implies that
$$\tilde{s} \in\mathcal{O}_P^s\ \bigl(\mbox{resp.}\ \tilde{x} \in\mathcal{O}_D^x\bigr)\quad \iff\quad \tilde{x} \in \mathcal{D}^{\leq}_P(\tilde{s})^*\ \bigl(\mbox{resp.}\ \tilde{x}\in\mathcal{D}^{\leq}_P(\tilde{s})^*\bigr).$$
However, \(\tilde{x} \in(K-\tilde{s})^{*}\) (or \(\tilde{s} \in(K^{*}-\tilde{x})^{*})\) holds if, and only if, \(\langle\tilde{s}, \tilde{x}\rangle =0\). □

5.1.2 Positive gaps and strict complementarity

In this section, we study the relationships between complementarity partitions and positive duality gaps. In particular, we consider cases where the complementarity partition for the pair of faces \((f_{P}^{0},f_{D}^{0})\) fails to be strict. An instance with a finite positive gap is given in Example 2.5, item 2. We provide another example to illustrate the application of the optimality conditions that use the minimal sets.

Example 5.3

We let \(K = K^{*} = \mathbb{S}_{+}^{6}\), with \(\mathcal{A}^{*} y = \sum_{i}A_{i} y_{i}\) and
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equag_HTML.gif
The primal and dual recession cones (first complementarity partition) are
$$f_P^0 =\left[\begin{array}{@{}c@{\quad}c@{}}\mathbb{S}^2_+ & 0\\0 & 0\end{array}\right]\unlhd K,\qquad f_D^0 = Q \mathbb{S}^2_+ Q^T \unlhd K^*,\quad \mbox{where}\ Q = [e_3\quad e_4];$$
and positionwise
$$\left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}}f_P^0 & 0 & 0\cr0 & f_D^0 & 0 \cr0 & 0 & G\end{array}\right],$$
where G=02 represents the gap in strict complementarity. We note that \(f_{D}^{0} \cap\{c\}^{\perp}= f_{D}^{0}\). We apply (2.15) with the second (or fourth) complementarity partition in Proposition 2.3. We choose
$$x =\left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}}0_2 & 0 & 0\cr0 & I_2 & 0\cr0 & 0 & 0_2\end{array}\right]\in\operatorname{relint}f_D^0 = \operatorname{relint}\bigl(f_D^0 \cap\{c\}^\perp\bigr).$$
We get fPf1:=K∩{x} and the equivalent problem to (P)
$$\mbox{(P)}\quad v_P= \sup\bigl\{\langle b,y\rangle : \mathcal{A}^* y\preceq_{f_1} c\bigr\}.$$
However, we need one more step to find fP. We again apply (2.15) and choose
$$0\neq x =E_{55}\in\operatorname{relint}\bigl(f_1 \cap\{c\}^\perp\bigr).$$
This yields \(f_{P} = Q \mathbb{S}^{3}_{+} Q^{T}, \mbox{where } Q = [e_{1}\;e_{2}\; e_{6}]\).
Similarly, we can work on the dual problem. In summary, we get that the faces and recession cones of the primal and dual are
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equam_HTML.gif
The optimal values are vP=0 and vD=1.

A connection between optimality and complementarity can be seen in the following proposition.

Proposition 5.4

Suppose that (2.2) has optimal solution\(\tilde{y}\)with corresponding optimal slack\(\tilde{s}\), and that (2.3) has optimal solution\(\tilde{x}\). Then
$$\langle\tilde{s}, \tilde{x}\rangle = \inf \bigl\{\langle s,x\rangle : s \in \mathcal{F}^s_P, x \in \mathcal {F}^x_D\bigr\}.$$

Proof

Since \(\tilde{s}\) is feasible, there exists \(\ell' \in\mathcal{L}^{\perp}\) such that \(c=\tilde{s}+\ell'\). Then for every feasible solution x of (2.3) and every feasible solution (y,s) of (2.2), we have
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equao_HTML.gif
The second equation above uses the facts \(\ell' \in\mathcal {L}^{\perp}\), \((\tilde{x} - x) \in\mathcal{L}\). Now, using the subspace form (2.9), (2.10) of the primal-dual pair, we conclude the desired result. □

Example 5.5

(S. Schurr [64])

It is possible to have a finite positive duality gap even if the complementarity partition for the pair of faces \((f_{P}^{0},f_{D}^{0})\) is strict. Let \(K = K^{*} = \mathbb{S}_{+}^{5}\), and
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equap_HTML.gif
Then
$$f_P^0 = \left[\begin{array}{@{}c@{\quad}c@{}}\mathbb{S}^2_+ & 0\\ 0 & 0\end{array}\right],\qquad f_D^0 = \left[\begin{array}{@{}c@{\quad}c@{}}0 & 0\\0 & \mathbb{S}^3_+\end{array}\right],\qquad f_P = Q \mathbb{S}^4_+ Q^T,\qquad f_D = \left[\begin{array}{@{}c@{\quad}c@{}}0 & 0\\ 0 &\mathbb{S}^4_+\end{array}\right],$$
where Q=[e1e2e4e5]. The primal optimal value is zero and the dual optimal value is \((\sqrt{5}-1)/2\), and both are attained. This can be seen using the optimal \(\tilde{s}=c\) for (2.2), and \(\tilde{x}\) optimal for (2.3) (optimal \(x^{*}=\tilde{x}\) has values \(1/\sqrt{5}\) and \((3-\sqrt{5})/(2\sqrt{5})\) for the diagonal (5,5) and (4,4) elements, respectively).

We also have an example without the attainment of the optimal values.

Example 5.6

Consider the SDP with data \(K = K^{*} = \mathbb{S}_{+}^{5}\), and
https://static-content.springer.com/image/art%3A10.1007%2Fs10589-012-9480-0/MediaObjects/10589_2012_9480_Equar_HTML.gif
The primal optimal value is zero and the dual optimal value is 1, but neither value is attained.

We now consider cases when the assumption that the complementarity partition for the pair of faces \((f_{P}^{0},f_{D}^{0})\) fails to be strict implies a finite positive duality gap. Our main result for the relationship between the failure of strict complementarity and finite nonzero duality gaps follows. We focus on the SDP case.

Theorem 5.7

Let\(K=\mathbb{S}^{n}_{+}\), suppose that the subspace\(\mathcal{L}\subset \mathbb{S}^{n}\)is such that the complementarity partition for the pair of faces\((f_{P}^{0},f_{D}^{0})\)minimally fails to be strict (dimG=1, where the face\(G:=(f_{P}^{0})^{c}\cap(f_{D}^{0})^{c} \unlhd K\)). Then for everyM>0, there exist\(\tilde{x}, \tilde{s}\in\mathrm {relint}\,G\)such that the underlying problems (2.9) and (2.10) have a duality gap of exactlyM.

Proof

First note that \(\mathcal{L}\cap K = \{0\}\) if, and only if, \(\mathcal {L}^{\perp}\cap \operatorname{int}K \neq\emptyset\) if, and only if, \(f_{P}^{0}=K\). Using this and a similar result for \(\mathcal{L}^{\perp}\cap K = \{0\}\), we conclude that both \(f_{P}^{0}\) and \(f_{D}^{0}\) are proper faces of K. Let M>0 be arbitrary and \(\tilde{s} \in\operatorname{relint}G\) such that \(\langle\tilde{s}, \tilde{s}\rangle =M\). Let \(\tilde{x}:= \tilde{s}\). We claim that \(\tilde{s}\) is optimal in \((\mathbb{P})\) and \((\mathbb{D})\). We prove the optimality claim by contradiction. Suppose \(\tilde{s}\) is not optimal in \((\mathbb{P})\). Since it is feasible, with objective value M, there must exist another feasible solution of \((\mathbb{D})\) with strictly better objective value. The latter implies, there exists \(u \in\mathbb{S}^{n}\) such that \(u \in\mathcal{L}^{\perp}\), \(u \succeq-\tilde{s}\) and \(\langle\tilde{s}, u\rangle < 0\). Under an orthogonal similarity transformation, we have the following representation of the faces:
$$\left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}}G & 0 & 0\cr0 & f_P^0 & 0 \cr0 & 0 &f_D^0\end{array}\right].$$
Then, \(\tilde{s}= \sqrt{M} E_{11}\), and \(u \succeq-\tilde{s}\) implies that the 3×3 block of u is zero. Thus,
$$u=\left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}}\alpha& \bar{v}^T & 0\cr\bar{v} & V & 0 \cr0 & 0 & 0\end{array}\right],$$
where α<0. Now let \(\hat{s} \in\operatorname{relint}f_{P}^{0}\), that is,
$$\hat{s}=\left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}}0 & 0 & 0\cr0 & \xi& 0 \cr0 & 0 & 0\end{array}\right],$$
where ξ≻0. For every \(\beta\in\mathbb{R}\), \((\beta \hat{s}-u) \in\mathcal{L}^{\perp}\). Moreover,
$$\beta\hat{s}-u = \left[\begin{array}{@{}c@{\quad}c@{\quad}c@{}} -\alpha&-\bar{v}^T & 0\cr -\bar{v} & \beta\xi-V & 0\cr 0 & 0 & 0\end{array} \right]\succeq0\quad\iff\quad (\beta\xi-V ) +\frac{1}{\alpha}\bar{v}\bar{v}^T \succeq0,$$
and,
$$\left[\begin{array}{@{}c@{\quad}c@{}} -\alpha& -\bar{v}^T\cr -\bar{v} & \beta\xi-V \end{array} \right]\succ0\quad\iff\quad (\beta\xi-V ) +\frac{1}{\alpha}\bar{v}\bar{v}^T \succ0.$$
The latter is true for all sufficiently large β>0. Hence, for all sufficiently large β>0, \((\beta\hat{s}-u) \in\mathcal{L}^{\perp} \cap\mathbb{S}^{n}_{+}\) with \(\mathop{\mathrm{rank}}(\beta\hat{s}-u)=\mathop {\mathrm{rank}}(\hat{s}) +1\), a contradiction. This proves, \(\tilde{s}\) is optimal in \((\mathbb{P})\). Similarly, \(\tilde{x}\) is optimal in \((\mathbb{D})\). The duality gap is \(\langle\tilde{x}, \tilde{s}\rangle = \langle\tilde{s},\tilde{s}\rangle =M\), as claimed. □

Example 5.8

We now see that choosing one of \(\tilde{s}, \tilde{x}\) in \(\operatorname{relint}G\) may not result in a positive duality gap. Consider the SDP with data \(K = K^{*} = \mathbb{S}_{+}^{4}\), and
$$A_1 = E_{44},\qquad A_2 = E_{24} + E_{33},\qquad A_3 = E_{13} + E_{22}.$$
Then
$$f_P^0 = \left[\begin{array}{@{}c@{\quad}c@{}}0 & 0\\ 0 & \mathbb{R}_+\end{array}\right] \unlhd K,\qquad f_D^0 = \left[\begin{array}{@{}c@{\quad}c@{}}\mathbb{R}_+ & 0\\ 0 & 0\end{array}\right] \unlhd K^*,$$
and
$$G := \bigl(f_P^0\bigr)^c \cap \bigl(f_D^0\bigr)^c = Q \mathbb{S}^2_+Q^T,$$
where \(Q = [e_{2}\; e_{3}]\). If \(\tilde{s}\) and \(\tilde{x}\) are chosen such that \(\tilde{s} \in\operatorname{relint}(G)\) and \(\tilde{x} \in G\), with \(\tilde{x}_{33} > 0\), then the optimal values are both \(x_{33}(s_{33}-s_{23}^{2}/s_{22})\). However, there exist matrices \(\tilde{s}, \tilde{x}\in G\) that are singular on G such that (2.2) and (2.3) admit a positive duality gap. For example, if \(\tilde{s}=\tilde{x}\) is the diagonal matrix \(\tilde{s} = \tilde{x}= \mathop{\mathrm{Diag}}((0~0~s_{33}~0))\), then the primal optimal value is zero and the dual optimal value is x33s33.1 Both values are attained at \(\tilde{s}=\tilde{x}\).

Note that the construction in the proof of Theorem 5.7 resulted in a primal-dual pair for which every feasible solution is optimal. We further investigate this connection in the next two results.

Theorem 5.9

LetKbe a closed convex cone. Suppose that the partition\((f_{P}^{0}, f_{D}^{0})\)is strictly complementary and that the following condition holds:
$$\tilde{s} \in(f_D^0)^c + \mathcal{L}^\perp, \qquad \tilde{x} \in(f_P^0)^c + \mathcal{L}. $$
(5.1)
Then every feasible solution in\((\mathbb{P})\)and every feasible solution in\((\mathbb{D})\)is optimal and strong duality holds for both\((\mathbb{P})\)and\((\mathbb{D})\).

Proof

Suppose that (5.1) holds. Then by Proposition 3.4, part 2, \(f_{P} \subseteq(f_{D}^{0})^{c}\) and \(f_{D} \subseteq(f_{P}^{0})^{c}\). Further suppose that \((f_{P}^{0},f_{D}^{0})\) forms a strict complementarity partition. Then \((f_{D}^{0})^{c} = f_{P}^{0}\). Since for all feasible problems, \(f_{P}^{0} \subseteq f_{P}\) and \(f_{D}^{0} \subseteq f_{D}\) (Proposition 3.4, part 1), we actually have \(f_{P}^{0} = f_{P}\) and \(f_{D}^{0} = f_{D}\). Then, 〈fP,fD〉=0, i.e. every feasible point is optimal (and we have no duality gap). □

Corollary 5.10

Suppose that both (2.9) and (2.10) are feasible but strong duality fails either problem. In addition, suppose that all feasible points for (2.2) and (2.3) are optimal. Then the complementarity partition for the pair of faces\((f_{P}^{0},f_{D}^{0})\)fails to be strict.

Proof

Suppose that all feasible points for (2.2) are optimal. Then the primal objective function is constant along all primal recession directions. That is, \(\langle\tilde{x}, \mathcal{L}^{\perp}\cap K \rangle= \{0\}\), i.e., \(\tilde{x} \in(\mathcal{L}^{\perp}\cap K)^{\perp}\). Now by construction, \(\tilde{x}\) is dual feasible, i.e., \(\tilde{x} \in(\mathcal{L}^{\perp}\cap K)^{\perp}\cap K^{*}= (f_{P}^{0})^{\perp}\cap K^{*} = (f_{P}^{0})^{c}\). Finally, as argued previously, translating \(\tilde{x}\) by a point in \(\mathcal{L}\) leaves the dual problem unchanged, giving the condition on \(\tilde{x}\) in (5.1). In a similar way we can show that if all feasible points for (2.3) are optimal, then the condition on \(\tilde{s}\) in (5.1) holds. The desired result now follows from Theorem 5.9. □

5.2 Infinite duality gap and devious faces

As we have already noted, (2.3) is feasible if, and only if, \(\tilde{x} \in K^{*}+\mathcal{L}\). Moreover the feasibility of (2.3) is equivalent to a finite duality gap (possibly zero), recalling our assumption that the primal optimal value vP is finite. We now see that if a nice cone has a devious face, then it is easy to construct examples with an infinite duality gap.

Proposition 5.11

Suppose thatKis a nice, proper cone andFis a devious face ofK, i.e.,
$$K^*+\bigl(F^c\bigr)^\perp=\overline{K^*+\operatorname{span}F}\quad\mathit{and}\quad {\bigl(K^*+\operatorname{span}F\bigr)}\ \mathit{is\ not\ closed}.$$
Let\(\mathcal{L}=\operatorname{span}F\)and choose\(c=\tilde{s}=0\)and\(\tilde{x} \in (K^{*}+(F^{c})^{\perp}) \backslash(K^{*}+\mathcal{L})\). Then\((\tilde{x} + \mathcal{L})\cap K^{*}=\emptyset\)and we getvD=+∞. Moreover, \(\mathcal{L}^{\perp}= F^{\perp}\)and, for every feasiblesFK,
$$\langle\tilde{x}, s\rangle = \langle\tilde{x}_{K^*}+ \tilde{x}_{(F^c)^\perp}, s\rangle \geq0,$$
i.e., 0=vP<vD=∞.

Proof

The proof follows from the definitions. □

Proposition 5.11 can be extended to choosing any \(\mathcal{L}\) that satisfies \(K^{*} + \mathcal{L}\) is not closed and \(K^{*}+\mathcal {L}\subset K^{*}+(F^{c})^{\perp}\).

Example 5.12

Let \(K=\mathbb{S}^{2}_{+}\), and suppose that (2.2) and (2.3) admit a nonzero duality gap. Then Slater’s CQ fails for both primal and dual, i.e., \(\{0\} \neq f_{P}^{0}\subset\mathbb{S}^{2}_{+}\) and \(\{0\} \neq f_{D}^{0}\subset\mathbb{S}^{2}_{+}\). After a rotation (see Lemma 2.8) we can assume the problem has the structure
$$\left[\begin{array}{@{}c@{\quad}c@{}}f_D^0 & 0 \\[1mm]0 & f_P^0\end{array}\right],$$
viz., the matrices in \(f_{D}^{0}\) are nonzero only in the (1,1) position, and the matrices in \(f_{P}^{0}\) are nonzero only in the (2,2) position. There are only three possible options for \(\mathcal{L}\): \(\mathrm {span}\{E_{11}\}\), \(\mathrm{span}\{E_{22}\}\), \(\mathrm{span}\{E_{11},E_{12}\}\), or \(\mathrm{span}\{E_{22},E_{12}\}\). In each case, either \(\mathcal{L}\) is one-dimensional and \(\mathcal{L}^{\perp}\) is two-dimensional, or vice versa. So without loss of generality, we may choose \(\mathcal{L}= \mathrm {span}\{E_{11}\}\). Now take \(\tilde{x} = E_{12} \in\mathbb{S}_{+}^{2}+(f_{P}^{0})^{\perp}\). Then
$$ \tilde{x} \notin\mathbb{S}_+^2+\mathcal{L}=\mathbb{S}_+^2+\mathop {\mathrm{span}}f_D^0\subset \mathbb{S}_+^2+\bigl(f_P^0\bigr)^\perp,$$
(5.2)
and the dual program (2.3) is infeasible. But choosing \(c=\tilde{s} = E_{22}\) implies that the primal optimal value \(v_{P}= \langle c, \tilde{x}\rangle -y_{1}\langle E_{22}, \tilde{x}\rangle =0<v_{D}=+\infty\).

Corollary 5.13

If\(K=\mathbb{S}_{+}^{2}\), then a finite positive duality gap cannot occur.

Proof

See Corollary 4.13. □

The above Corollary 5.13 also follows from [65, Proposition 4], i.e., it states that a finite positive duality gap cannot happen if \(\dim\mathcal{W}\leq3\).

5.3 Regularization for Slater condition

In terms of worst-case performance, deciding whether the Slater condition holds for a given SDP problem seems no easier than solving an SDP problem. However, in many applications, there may be enough structure or a priori information which allows the user to regularize for the Slater condition. For example, if we consider applications of SDP arising as relaxations of nonconvex optimization problems, there are general methods (see [68]) that can be used for regularization so that the new problem has Slater points. The methods of [68] only require the prior knowledge of the affine hull (or linear span depending on the SDP relaxation used) of the nonconvex solution set.

In practice, due to the special structure present, heuristics can be effective (see [42]).

5.4 Regularization for strict complementarity

Suppose that strong duality holds for both the primal and dual SDPs, but strict complementarity fails for every primal-dual optimal solution \((\bar{s}, \bar{x}) \in\mathbb {S}^{n}_{+}\oplus\mathbb{S}^{n}_{+}\). Following [71], \((\bar{s},\bar{x} )\) is called a maximal complementary solution pair if the pair maximizes the sum \(\mathop{\mathrm{rank}}(s)+\mathop{\mathrm{rank}}(x)\) over all primal-dual optimal (s,x). The strict complementarity nullity, \(g:=n-\mathop{\mathrm{rank}}(\bar{s})-\mathop{\mathrm{rank}}(\bar{x})\).

Let \(\mathcal{U}= \mathcal{N}(\bar{s}) \cap\mathcal{N}(\bar{x})\) be the common nullspace of dimension g, and U be the n×g matrix with orthonormal columns satisfying \(\mathcal{R}(U) = \mathcal{U}\). Let [U Q] be an orthogonal matrix. Then we can regularize so that strict complementarity holds by replacing both primal-dual variables s,x by QsQT,QxQT, respectively. This is equivalent to replacing the matrices C,Ai,i=1,…,m that define \(\mathcal {A}\) by QTCQ,QTAiQ,i=1,…,m. Note that we would then have to check for possible linear dependence of the new matrices QTAiQ, as well as possible loss of Slater CQ. Checking for linear dependence is indeed easy. Even though the loss of Slater condition seems to be more problematic, as we discussed in the previous subsection, in many applications this too may be relatively easy. Finding the common nullspace can be done dynamically during the solution process. This is done by checking the ratios of eigenvalues of s and x between iterates to see if the convergence is to 0 or to O(1). (In the case of LP, this corresponds to identifying nonbasic variables using the so-called Tapia indices, see e.g., [30].)

6 Conclusion

In this paper we have looked at known and new, duality and optimality results for the cone optimization problem (2.2). We have used the subspace formulations of the primal and dual problems, (2.9), (2.10), to provide new CQs and new optimality conditions that hold without any CQ. This includes a UCQ, i.e., a CQ that holds independent of both data vectors b and c. In particular, the optimality characterizations show that a minimal representation of the cone and/or the linear transformation of the problem results in regularization, i.e., efficient modeling for the cone K and for the primal and dual constraints results in a stable formulation of the problem. In addition, we have discussed conditions for a zero duality gap and the surprising relations to the lack of strict complementarity in the homogeneous problem and to the closure of sums of cones. The (near) failure of Slater’s CQ relates to both theoretical and numerical difficulties. The same holds true for the failure of strict complementarity. We have discussed regularization procedures for both failures. We hope that these results will lead to preprocessing for current cone optimization software packages.

Footnotes
1

Similarly, we can use the (2,2) position rather than the (3,3) position.

 

Acknowledgements

Research supported by The Natural Sciences and Engineering Research Council of Canada. Part of this paper is based on work presented at: ICCOPT II & MOPTA-07, McMaster University, Hamilton, 2007; 9th SIAM Conference on Optimization, Boston, May, 2008; and HPOPT 10, Tilburg, 2008. The authors would like to acknowledge Simon Schurr (Univ. of Waterloo) for many helpful conversations, as well as for other contributions to this paper. URL for paper: orion.math.uwaterloo.ca/~hwolkowi/henry/reports/ABSTRACTS.html

Copyright information

© Springer Science+Business Media, LLC 2012