Radial duality part I: foundations

Abstract

Renegar (SIAM J Optim 26(4):2649–2676, https://doi.org/10.1137/15M1027371 2016) introduced a novel approach to transforming generic conic optimization problems into unconstrained, uniformly Lipschitz continuous minimization. We introduce radial transformations generalizing these ideas, equipped with an entirely new motivation and development that avoids any reliance on convex cones or functions. Of practical importance, this facilitates the development of new families of projection-free first-order methods applicable even in the presence of nonconvex objectives and constraint sets. Our generalized construction of this radial transformation uncovers that it is dual (i.e., self-inverse) for a wide range of functions including all concave objectives. This gives a new duality relating optimization problems to their radially dual problem. For a broad class of functions, we characterize continuity, differentiability, and convexity under the radial transformation as well as develop a calculus for it. This radial duality provides a foundation for designing projection-free radial optimization algorithms, which is carried out in the second part of this work.

Proofs computing some radial set transformations

1.1 Proof of Proposition 1

It suffices to show S being convex implies $\varGamma S$ is convex, since the duality of the radial set transformation (3) will then imply the reverse direction. Consider any $(y,v),(y',v')\in \varGamma S$. Let $(x,u) = \varGamma (y,v)$ and $(x',u')=\varGamma (y',v')$. Then $\lambda (x,u) + (1-\lambda )(x',u') \in S$ for any $0\le \lambda \le 1$. Therefore the line segment between (y, v) and $(y',v')$ lies in $\varGamma S$ as

$$\begin{aligned} \varGamma S&\ni \frac{(\lambda x + (1-\lambda )x', 1)}{\lambda u + (1-\lambda )u'} =\frac{\lambda /v}{\lambda /v + (1-\lambda )/v'}(y,v) + \frac{(1-\lambda )/v'}{\lambda /v + (1-\lambda )/v'}(y',v'). \end{aligned}$$

1.2 Proof of Proposition 2

It suffices to show S being a halfspace implies $\varGamma S$ is a halfspace, since the duality of the radial set transformation (3) will then imply the reverse direction. By definition, we have

$$\begin{aligned} \varGamma S&=\left\{ \frac{(x',1)}{u'}\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} \zeta \\ \delta \end{bmatrix}^T\begin{bmatrix} x'-x \\ u'-u \end{bmatrix}\le 0\right\} \\&= \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} \zeta \\ \delta \end{bmatrix}^T\begin{bmatrix} y'/v'-x \\ 1/v'-u \end{bmatrix}\le 0 \right\} \\&= \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} \zeta \\ \delta \end{bmatrix}^T\begin{bmatrix} y'-v'x \\ 1-v'u \end{bmatrix}\le 0 \right\} \\&= \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} \zeta \\ -(\zeta ,\delta )^T(x,u) \end{bmatrix}^T\begin{bmatrix} y' \\ v' \end{bmatrix} +\delta \le 0 \right\} \\&= \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} \zeta \\ -(\zeta ,\delta )^T(x,u) \end{bmatrix}^T\begin{bmatrix} y'-y \\ v'-v \end{bmatrix} \le 0 \right\} . \end{aligned}$$

1.3 Proof of Proposition 3

It suffices to show S being an ellipsoid in $\mathcal {E}\times \mathbb {R}_{++}$ implies $\varGamma S$ is an ellipsoid $\mathcal {E}\times \mathbb {R}_{++}$, since the duality of the radial set transformation (3) will then imply the reverse direction. Denote the blocks of H by $\begin{bmatrix} H_{11} &{} H_{12}\\ H_{12}^T &{} H_{22} \end{bmatrix}$ and define the following matrix

$$\begin{aligned} G = \begin{bmatrix} H_{11} &{} -\begin{bmatrix} H_{11} \\ H^T_{12} \end{bmatrix}^T\begin{bmatrix} x \\ u\end{bmatrix}\\ -\begin{bmatrix} x \\ u\end{bmatrix}^T\begin{bmatrix} H_{11} \\ H^T_{12} \end{bmatrix} &{} \begin{bmatrix} x \\ u \end{bmatrix}^T \begin{bmatrix} H_{11} &{} H_{12}\\ H_{12}^T &{} H_{22} \end{bmatrix} \begin{bmatrix} x \\ u \end{bmatrix} -1 \end{bmatrix}, \end{aligned}$$

related to the radially dual ellipsoid. For any ellipsoid $S\subseteq \mathcal {E}\times \mathbb {R}_{++}$ defined by (7), we claim that $\varGamma S$ is the following ellipsoid in $\mathcal {E}\times \mathbb {R}_{++}$ with center $\begin{bmatrix}y \\ v\end{bmatrix} = G^{-1}\begin{bmatrix}-H_{12}\\ H_{12}^Tx + H_{22}u \end{bmatrix}$

$$\begin{aligned} \varGamma S = \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} y'-y \\ v'-v \end{bmatrix}^T \left( \frac{G}{\begin{bmatrix}y \\ v\end{bmatrix}^T G \begin{bmatrix}y \\ v\end{bmatrix} - H_{22}}\right) \begin{bmatrix} y'-y \\ v'-v \end{bmatrix} \le 1\right\} . \end{aligned}$$

First, we observe that G is indeed positive definite. Since H is positive definite, considering its Schur complements ensures $H_{11}$ is positive definite and $H_{22} - H_{12}^TH_{11}^{-1}H_{12}>0$. Likewise, G is positive definite if $H_{11}$ is positive definite and

$$\begin{aligned} \left( \begin{bmatrix} x \\ u \end{bmatrix}^T \begin{bmatrix} H_{11} &{} H_{12}\\ H_{12}^T &{} H_{22} \end{bmatrix} \begin{bmatrix} x \\ u \end{bmatrix} -1\right) - \left( \begin{bmatrix} H_{11} \\ H^T_{12} \end{bmatrix}^T\begin{bmatrix} x \\ u\end{bmatrix}\right) ^TH_{11}^{-1}\left( \begin{bmatrix} H_{11} \\ H^T_{12} \end{bmatrix}^T\begin{bmatrix} x \\ u\end{bmatrix}\right) > 0. \end{aligned}$$

Simplifying this condition for G to be positive definite yields the equivalent inequality $u^2(H_{22} - H_{12}^TH_{11}^{-1}H_{12}) >1$. To see why this is the case, recall $S\subseteq \mathcal {E}\times \mathbb {R}_{++}$. Computing the minimum height of a point in S gives $\min _{({{\bar{x}}},{{\bar{u}}})\in S} {{\bar{u}}} = u - 1/\sqrt{H_{22} - H_{12}^TH_{11}^{-1}H_{12}} >0$. Applying $\varGamma$ to S and then completing the square in the final line gives the claim as

$$\begin{aligned} \varGamma S&= \left\{ \frac{(x',1)}{u'}\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} x'-x \\ u'-u \end{bmatrix}^T \begin{bmatrix} H_{11} &{} H_{12}\\ H_{12}^T &{} H_{22} \end{bmatrix} \begin{bmatrix} x'-x \\ u'-u \end{bmatrix} \le 1\right\} \\&= \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} y'/v'-x \\ 1/v'-u \end{bmatrix}^T \begin{bmatrix} H_{11} &{} H_{12}\\ H_{12}^T &{} H_{22} \end{bmatrix} \begin{bmatrix} y'/v'-x \\ 1/v'-u \end{bmatrix} \le 1\right\} \\&= \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} y'-v'x \\ 1-v'u \end{bmatrix}^T \begin{bmatrix} H_{11} &{} H_{12}\\ H_{12}^T &{} H_{22} \end{bmatrix} \begin{bmatrix} y'-v'x \\ 1-v'u \end{bmatrix} \le v'^2\right\} \\&= \left\{ (y',v')\in \mathcal {E}\times \mathbb {R}_{++} \mid \begin{bmatrix} y' \\ v' \end{bmatrix}^T G \begin{bmatrix} y' \\ v' \end{bmatrix} +2H_{12}^Ty'-2(H_{12}^Tx+H_{22}u)v'\le -H_{22}\right\} . \end{aligned}$$