Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding

Abstract

We introduce a first-order method for solving very large convex cone programs. The method uses an operator splitting method, the alternating directions method of multipliers, to solve the homogeneous self-dual embedding, an equivalent feasibility problem involving finding a nonzero point in the intersection of a subspace and a cone. This approach has several favorable properties. Compared to interior-point methods, first-order methods scale to very large problems, at the cost of requiring more time to reach very high accuracy. Compared to other first-order methods for cone programs, our approach finds both primal and dual solutions when available or a certificate of infeasibility or unboundedness otherwise, is parameter free, and the per-iteration cost of the method is the same as applying a splitting method to the primal or dual alone. We discuss efficient implementation of the method in detail, including direct and indirect methods for computing projection onto the subspace, scaling the original problem data, and stopping criteria. We describe an open-source implementation, which handles the usual (symmetric) nonnegative, second-order, and semidefinite cones as well as the (non-self-dual) exponential and power cones and their duals. We report numerical results that show speedups over interior-point cone solvers for large problems, and scaling to very large general cone programs.

Appendix: Nonexpansivity

In this appendix we show that the mapping consisting of one iteration of the algorithm (17) is nonexpansive, i.e., if we denote the mapping by $\phi$, then we shall show that

$$\begin{aligned} \Vert \phi (u, v) - \phi (\hat{u}, \hat{v}) \Vert _2 \le \Vert (u ,v) - (\hat{u}, \hat{v}) \Vert _2, \end{aligned}$$

for any (u, v) and $(\hat{u}, \hat{v})$.

From (17) we can write the mapping as the composition of two operators, $\phi = P \circ L$, where

$$\begin{aligned} P(x) = (\Pi _\mathcal {C}(x), -\Pi _{-\mathcal {C}^*}(x)), \end{aligned}$$

and

$$\begin{aligned} L(u,v) = (I+Q)^{-1}(u+v) - v. \end{aligned}$$

To show that $\phi$ is nonexpansive, we only need to show that both P and L are nonexpansive.

To show that P is nonexpansive, we proceed as follows

$$\begin{aligned} \Vert x - \hat{x}\Vert _2^2&= \Vert \Pi _\mathcal {C}(x) + \Pi _{-\mathcal {C}^*}(x) - \Pi _\mathcal {C}(\hat{x}) - \Pi _{-\mathcal {C}^*}(\hat{x}) \Vert _2^2 \\&= \Vert \Pi _\mathcal {C}(x)- \Pi _\mathcal {C}(\hat{x})\Vert _2^2 + \Vert \Pi _\mathcal {-C^*}(x)- \Pi _\mathcal {-C^*}(\hat{x})\Vert _2^2 \\&\qquad -2 \Pi _\mathcal {C}(\hat{x})^\mathrm{T} \Pi _\mathcal {-C^*}(x) - 2 \Pi _\mathcal {C}(x)^\mathrm{T} \Pi _\mathcal {-C^*}(\hat{x}) \\&\ge \Vert \Pi _\mathcal {C}(x)- \Pi _\mathcal {C}(\hat{x})\Vert _2^2 + \Vert \Pi _\mathcal {-C^*}(x)- \Pi _\mathcal {-C^*}(\hat{x})\Vert _2^2 \\&= \Vert (\Pi _\mathcal {C}(x)- \Pi _\mathcal {C}(\hat{x}) ) , - (\Pi _\mathcal {-C^*}(x)- \Pi _\mathcal {-C^*}(\hat{x}) )\Vert _2^2 \\&= \Vert P(x) - P(\hat{x}) \Vert _2^2, \end{aligned}$$

where the first equality is from the Moreau decompositions of x and $\hat{x}$ with respect to the cone $\mathcal {C}$, the second follows by expanding the norm squared and the fact that $\Pi _\mathcal {C}(x) \perp \Pi _\mathcal {-C^*}(x)$ for any x, and the inequality follows from $\Pi _\mathcal {C}(\hat{x})^\mathrm{T} \Pi _\mathcal {-C^*}(x) \le 0$ by the definition of dual cones.

Similarly for L we have

$$\begin{aligned} \Vert L(u,v) - L(\hat{u}, \hat{v}) \Vert _2&= \left\| (I+Q)^{-1}(u - \hat{u} +v - \hat{v}) - v + \hat{v}\right\| _2 \\&= \left\| \begin{bmatrix} (I+Q)^{-1}&-(I-(I+Q)^{-1}) \end{bmatrix} (u - \hat{u}, v - \hat{v}) \right\| _2 \\&\le \Vert (u - \hat{u}, v - \hat{v}) \Vert _2 = \Vert (u,v) - (\hat{u}, \hat{v})\Vert _2, \end{aligned}$$

where the inequality can be seen from the fact that

$$\begin{aligned} \begin{bmatrix} (I+Q)^{-1}&-(I-(I+Q)^{-1}) \end{bmatrix} \begin{bmatrix} (I+Q)^{-1}&-(I-(I+Q)^{-1}) \end{bmatrix}^\mathrm{T} = I \end{aligned}$$

by the skew symmetry of Q, and so $\left\| \begin{bmatrix} (I+Q)^{-1}&-(I-(I+Q)^{-1}) \end{bmatrix}\right\| _2 = 1$.