Dimension reduction for semidefinite programs via Jordan algebras

Abstract

We propose a new method for simplifying semidefinite programs (SDP) inspired by symmetry reduction. Specifically, we show if an orthogonal projection map satisfies certain invariance conditions, restricting to its range yields an equivalent primal–dual pair over a lower-dimensional symmetric cone—namely, the cone-of-squares of a Jordan subalgebra of symmetric matrices. We present a simple algorithm for minimizing the rank of this projection and hence the dimension of this subalgebra. We also show that minimizing rank optimizes the direct-sum decomposition of the algebra into simple ideals, yielding an optimal “block-diagonalization” of the SDP. Finally, we give combinatorial versions of our algorithm that execute at reduced computational cost and illustrate effectiveness of an implementation on examples. Through the theory of Jordan algebras, the proposed method easily extends to linear and second-order-cone programming and, more generally, symmetric cone optimization.

Appendix

1.1 Proof of Theorem 2.1

We now prove Theorem 2.1, which stated that a subspace ${\mathcal {S}} \subseteq {\mathbb {S}}^n$ is a Jordan subalgebra if and only if its orthogonal projection $P_{{\mathcal {S}}}$ is unital and positive. Analogues for complex Jordan algebras are well known; see [38, 39] and also the thesis [24]. One direction is also shown in [28]. The converse direction is shown in part by translating an argument of [38] from the complex to real case. Since they are short and self-contained, we give full proofs of both directions.

To begin, we need the following lemma relating invariance under squaring to eigenvalue decompositions.

Lemma 7.1

For a non-zero $X \in {\mathbb {S}}^n$, let $E_X \subset {\mathbb {S}}^n$ be the set of pairwise orthogonal idempotent matrices for which

$$\begin{aligned} X = \sum _{ E \in E_X} \lambda _E E, \end{aligned}$$

where the range of $E \in E_X$ is an eigenspace of X and ${\{\lambda _E\}}_{E \in E_X}$ is the set of non-zero (distinct) eigenvalues of X. For a subspace ${\mathcal {S}} \subseteq {\mathbb {S}}^n$, the following are equivalent.

• 1. ${\mathcal {S}}$ contains the set $E_X$ for all non-zero $X \in {\mathcal {S}}$.

• 2. ${\mathcal {S}}$ is invariant under squaring, i.e., ${\mathcal {S}} \supseteq \{ X^2 : X \in {\mathcal {S}} \}$.

Proof

That statement one implies two is immediate given that $X^2 = \sum _{ E \in E_X } \lambda ^2_E E$. Conversely, suppose X has non-zero eigenvalue $\lambda$ of maximum magnitude. Then, if statement two holds, the idempotent ${\hat{E}} = \lim _{n \rightarrow \infty }{(|\lambda |^{-1} X )}^{2n}$ is contained in ${\mathcal {S}}$ and has range equal to an eigenspace or, if $\pm |\lambda |$ are both eigenvalues, the sum of two eigenspaces. Replacing X with $X-\lambda {\hat{E}}$ and iterating yields a set of idempotents whose span contains $E_X$; moreover, this set is contained in ${\mathcal {S}}$. $\square$

We now use this lemma and the mentioned argument of [38] to prove Theorem 2.1

To prove ($2 \Rightarrow 1)$, consider $X \succeq 0$ and suppose $P_{{\mathcal {S}}}(X)$ is non-zero. For a non-zero eigenvalue $\lambda _E$ of $P_{{\mathcal {S}}}(X)$, let $E \in {\mathbb {S}}^n$ denote the idempotent with range equal to the associated eigenspace. If (2) holds, then Lemma 7.1 implies $P_{{\mathcal {S}}}(E) = E$. Hence,

$$\begin{aligned} 0 \le E\cdot X = P_{{\mathcal {S}}}(E)\cdot X = E\cdot P_{{\mathcal {S}}}(X) = \lambda _E ||E ||^2. \end{aligned}$$

We conclude the eigenvalues of $P_{{\mathcal {S}}}(X)$ are non-negative, i.e., that $P_{{\mathcal {S}}}(X) \succeq 0$. To show the unitality condition, let Z be a matrix in ${\mathcal {S}}$ of maximum rank and let

$$\begin{aligned} {\hat{E}}= \sum _{ E \in E_Z} E. \end{aligned}$$

For all $X \in {\mathcal {S}}$, it holds that $t {\hat{E}} \succeq X^2$ for some $t > 0$. This shows the range of ${\hat{E}}$ contains the range of $X^2$ and hence the range of X. It follows ${\hat{E}} X=X$.

To prove ($1 \Rightarrow 2)$, suppose the unit element E has rank r. Then we can find an orthogonal matrix $Q = (Q_1, Q_2) \in {\mathbb {R}}^{n \times n}$ for which $E=Q_{1} Q_{1}^T$ and

$$\begin{aligned} {\mathcal {S}} = \left\{ Q \left( \begin{array}{cc} X &{} 0 \\ 0 &{} 0 \end{array} \right) Q^T : X \in \mathcal {{\hat{S}}} \subseteq {\mathbb {S}}^{r}\right\} , \end{aligned}$$

where $\mathcal {{\hat{S}}} := Q_{1}^T {\mathcal {S}} Q_{1}$. Further, the projection $P_{{\mathcal {S}}}$ satisfies

$$\begin{aligned} P_{{\mathcal {S}}}(X) = Q_{1} Q_{1}^T P_{\mathcal {{\hat{S}}} }(X) Q_{1} Q_{1}^T \end{aligned}$$

where $P_{\mathcal {{\hat{S}}} } : {\mathbb {S}}^{r} \rightarrow {\mathbb {S}}^{r}$ is the orthogonal projection onto $\mathcal {{\hat{S}}}$. It follows that if $\mathcal {{\hat{S}}}$ is invariant under squaring, so is ${\mathcal {S}}$, and if $P_{{\mathcal {S}}}$ is positive, so is $P_{\mathcal {{\hat{S}}} }$. Hence, Statement 2 follows by showing $\mathcal {{\hat{S}}}$ is invariant under squaring.

We show this applying the argument from [38, Theorem 2.2.2] and using the fact $\mathcal {{\hat{S}}}$ contains the identity matrix of order r. Dropping the subscript $\mathcal {{\hat{S}}}$ from $P_{\mathcal {{\hat{S}}}}$, we first note since P is positive and $P(I)=I$, it satisfies the Kadison inequality, which states $P(X^2) -P(X)P(X) \succeq 0$ for all $X \in {\mathbb {S}}^r$ (e.g., Theorem 2.3.4 of [4]). Hence, for X in the range of P

$$\begin{aligned} P(X^2) -X^2 \succeq 0. \end{aligned}$$

Letting $Z=P(X^2) -X^2$ and taking the trace shows

$$\begin{aligned} {{\,\mathrm{Tr}\,}}Z= & {} I\cdot Z = P(I)\cdot Z = I\cdot P(Z) = {{\,\mathrm{Tr}\,}}\big (P^2(X^2) -P(X^2)\big )\\&\quad = {{\,\mathrm{Tr}\,}}\big ( P(X^2) -P(X^2) \big ) = 0. \end{aligned}$$

Since $Z \succeq 0$, we conclude $Z=0$, i.e., that $P(X^2)=X^2$. Therefore $X^2$ is in the range of P.

1.2 Invariant affine sets of projections

Recall Condition 2.1-(b) and Condition 2.1-(c) require invariance of the affine sets $Y+{\mathcal {L}}$ and $C+\mathcal {L}^{\perp }$ under the projection $P_{{\mathcal {S}}}$. We now prove the characterization of these conditions provided by Lemma 3.1.

Lemma 7.2

For an affine set $Y + {\mathcal {L}}$, let $Y_{{\mathcal {L}}^{\perp }} \in {\mathbb {S}}^n$ denote the projection of $Y\in {\mathbb {S}}^n$ onto the subspace ${\mathcal {L}}^{\perp }$. Let $P_{{\mathcal {S}}} : {\mathbb {S}}^n \rightarrow {\mathbb {S}}^n$ denote the orthogonal projection onto a subspace ${\mathcal {S}}$ of ${\mathbb {S}}^n$. The following statements are equivalent.

• 1. $P_{{\mathcal {S}}}(Y + {\mathcal {L}}) \subseteq Y + {\mathcal {L}}$

• 2. $P_{{\mathcal {S}}}(Y_{{\mathcal {L}}^{\perp }}) = Y_{{\mathcal {L}}^{\perp }}$ and $P_{{\mathcal {S}}}({\mathcal {L}}) \subseteq {\mathcal {L}}$

Proof

To begin, first note $P_{{\mathcal {S}}}$—being an orthogonal projection—is a contraction with respect to the Frobenius norm $\Vert X \Vert _{F}$ (recalling our use of the trace inner-product); further, $Y_{{\mathcal {L}}^{\perp }}$ is the unique minimizer of this norm over $Y + {\mathcal {L}}$. Hence, if $P_{{\mathcal {S}}}(Y + {\mathcal {L}}) \subseteq Y + {\mathcal {L}}$, then $P_{{\mathcal {S}}}(Y_{{\mathcal {L}}^{\perp }}) = Y_{{\mathcal {L}}^{\perp }}$; in addition, since $Y + {\mathcal {L}} = Y_{{\mathcal {L}}^{\perp }} + {\mathcal {L}}$,

$$\begin{aligned} Y_{{\mathcal {L}}^{\perp }} + P_{{\mathcal {S}}}({\mathcal {L}}) = P_{{\mathcal {S}}}( Y_{{\mathcal {L}}^{\perp }} +{\mathcal {L}}) \subseteq Y_{{\mathcal {L}}^{\perp }} + {\mathcal {L}}, \end{aligned}$$

which implies $P_{{\mathcal {S}}}({\mathcal {L}}) \subseteq {\mathcal {L}}$. The converse direction is obvious given that $Y + {\mathcal {L}} = Y_{{\mathcal {L}}^{\perp }} + {\mathcal {L}}$. $\square$

If we apply the previous lemma to both the primal and dual affine sets we obtain the conditions $P_{{\mathcal {S}}}({\mathcal {L}}) \subseteq {\mathcal {L}}$ and $P_{{\mathcal {S}}}({\mathcal {L}}^{\perp }) \subseteq {\mathcal {L}}^{\perp }$. However, Lemma 3.1 only contains one of these conditions, since they turn out to be equivalent. Consider the following.

Lemma 7.3

[15, Proposition 3.8] Let $P_{{\mathcal {L}}} : {\mathbb {S}}^n \rightarrow {\mathbb {S}}^n$ and $P_{{\mathcal {S}}}: {\mathbb {S}}^n \rightarrow {\mathbb {S}}^n$ denote the orthogonal projections onto subspaces ${\mathcal {L}}$ and ${\mathcal {S}}$ of ${\mathbb {S}}^n$. The following four statements are equivalent.

• ${\mathcal {L}}$ is an invariant subspace of $P_{{\mathcal {S}}}$

• ${\mathcal {L}}^{\perp }$ is an invariant subspace of $P_{{\mathcal {S}}}$

• ${\mathcal {S}}$ is an invariant subspace of $P_{{\mathcal {L}}}$

• ${\mathcal {S}}^{\perp }$ is an invariant subspace of $P_{{\mathcal {L}}}$

Combining these two lemmas proves Lemma 3.1.

1.3 Linear images of self-dual cones

The following was used to prove Proposition 2.6.

Lemma 7.4

Let ${\mathcal {W}}$ and ${\mathcal {V}}$ be inner-product spaces and $\mathcal {C}\subseteq {\mathcal {V}}$ and ${\mathcal {K}}\subseteq {\mathcal {W}}$ self-dual convex cones. Let $T : {\mathcal {V}} \rightarrow {\mathcal {W}}$ be a injective linear map with adjoint $T^* : {\mathcal {W}} \rightarrow {\mathcal {V}}$. If ${\mathcal {K}}= T(\mathcal {C})$, then $T^*T(\mathcal {C}) = \mathcal {C}$.

Proof

For all $x,y \in \mathcal {C}$,

$$\begin{aligned} \langle T^*T(x), y \rangle = \langle T(x), T(y) \rangle \ge 0 \end{aligned}$$

by self-duality of ${\mathcal {K}}$. By self-duality of $\mathcal {C}$, we conclude $T^*T(x) \in \mathcal {C}$. On the other hand, since $T^*$ is surjective, we have for any $x \in \mathcal {C}$ existence of $w\in {\mathcal {V}}$ for which $x=T^*w$. Further, for all $y \in \mathcal {C}$,

$$\begin{aligned} 0 \le \langle T^*w, y\rangle = \langle w, T y\rangle \end{aligned}$$

which, since ${\mathcal {K}}= T(\mathcal {C})$, shows $w \in {\mathcal {K}}$. Hence, $w=Tz$ for $z \in \mathcal {C}$, showing $x = T^*Tz$. $\square$