Implementation of an oracle-structured bundle method for distributed optimization

Abstract

We consider the problem of minimizing a function that is a sum of convex agent functions plus a convex common public function that couples them. The agent functions can only be accessed via a subgradient oracle; the public function is assumed to be structured and expressible in a domain specific language (DSL) for convex optimization. We focus on the case when the evaluation of the agent oracles can require significant effort, which justifies the use of solution methods that carry out significant computation in each iteration. To solve this problem we integrate multiple known techniques (or adaptations of known techniques) for bundle-type algorithms, obtaining a method which has a number of practical advantages over other methods that are compatible with our access methods, such as proximal subgradient methods. First, it is reliable, and works well across a number of applications. Second, it has very few parameters that need to be tuned, and works well with sensible default values. Third, it typically produces a reasonable approximate solution in just a few tens of iterations. This paper is accompanied by an open-source implementation of the proposed solver, available at https://github.com/cvxgrp/OSBDO.

Appendix A: Convergence proof

In this section we give a proof of convergence of the bundle method for oracle-structured optimization. Our proof uses well known ideas, and borrows heavily from Belloni (2005). We will make one additional (and traditional) assumption, that f and g are Lipschitz continuous on $\mathop \textbf{dom}g$.

We say that the update was accepted in iteration k if $x^{k+1}={\tilde{x}}^{k+1}$. Suppose this occurs in iterations $k_1< k_2< \cdots$. We let $K=\{k_1, k_2, \ldots \}$ denote the set of iterations where the update was accepted. We distinguish two cases: $|K| = \infty$ and $|K| < \infty$.

1.1 Infinite updates

We assume $|K| = \infty$. First we establish that $\delta ^{k_s}\rightarrow 0$ as $s\rightarrow \infty$. Since $k=k_s$ is an accepted step, from step 6 of the algorithm we have

$$\begin{aligned} \eta \delta ^{k_s} \le h(x^{k_s}) - h(x^{k_{s}+1}) = h(x^{k_s}) - h(x^{k_{s+1}}). \end{aligned}$$

Summing this inequality from $s=1$ to $s=l$ and dividing by $\eta$ gives

$$\begin{aligned} \sum _{s=1}^l\delta ^{k_s} \le \frac{h(x^{k_1}) - h(x^{k_{l+1}})}{\eta } \le \frac{h(x^0) - h^\star }{\eta }, \end{aligned}$$

which implies that $\delta ^{k_s}$ is summable, and so converges to zero as $s \rightarrow \infty$.

Since ${\tilde{x}}^{k_s+1}$ minimizes ${\hat{h}}^{k_s}(x)+(\rho /2)\Vert x-x^{k_s}\Vert _2^2$, we have

$$\begin{aligned} \partial \hat{h}^{k_s}\left( {\tilde{x}}^{k_s+1} \right) + \rho ( {\tilde{x}}^{k_s+1}-x^{k_s}) \ni 0. \end{aligned}$$

Using ${\tilde{x}}^{k_s+1} = x^{k_s+1} = x^{k_{s+1}}$, we have

$$\begin{aligned} \rho (x^{k_s}-x^{k_{s+1}}) \in \partial \hat{h}^{k_s}\left( x^{k_{s+1}} \right) . \end{aligned}$$

It follows that

$$\begin{aligned} h^\star = h(x^\star ) \ge {\hat{h}}^{k_s}(x^\star ) \ge {\hat{h}}^{k_s}(x^{k_{s+1}}) + \rho (x^{k_s}-x^{k_{s+1}}) ^T (x^\star - x^{k_{s+1}}). \end{aligned}$$

We first rewrite this as

$$\begin{aligned} \frac{h^\star - {\hat{h}}^{k_s}(x^{k_{s+1}})}{\rho }\ge & {} (x^{k_s}-x^{k_{s+1}}) ^T (x^\star - x^{k_s}) + (x^{k_s}-x^{k_{s+1}}) ^T (x^{k_s}- x^{k_{s+1}}))\\= & {} (x^{k_s}-x^{k_{s+1}}) ^T (x^\star - x^{k_s}) + \Vert x^{k_s}-x^{k_{s+1}}\Vert _2^2, \end{aligned}$$

and then in the form we will use below,

$$\begin{aligned} 2(x^{k_s}-x^{k_{s+1}}) ^T (x^\star - x^{k_s}) \le (2/\rho )\left( h^\star - {\hat{h}}^{k_s}(x^{k_{s+1}})\right) - 2\Vert x^{k_s}-x^{k_{s+1}}\Vert _2^2. \end{aligned}$$

Now we use a standard subgradient algorithm argument. We have

$$\begin{aligned} \Vert x^{k_{s+1}} - x^\star \Vert _2^2= & {} \Vert x^{k_s} - x^\star \Vert _2^2 + \Vert x^{k_{s+1}} - x^{k_s}\Vert _2^2 + 2 (x^{k_s} - x^{k_{s+1}})^T (x^\star - x^{k_s}) \\\le & {} \Vert x^{k_s} - x^\star \Vert _2^2+(2/\rho ) \left( h^\star - \hat{h}^{k_s}(x^{k_{s+1}}) \right) - \Vert x^{k_{s+1}} - x^{k_s}\Vert _2^2 \\= & {} \Vert x^{k_s} - x^\star \Vert _2^2+(2/\rho ) \left( h^\star - h(x^{k_s}) +\delta ^{k_s} \right) . \end{aligned}$$

Summing this inequality from $s=1$ to $s=l$ and re-arranging yields

$$\begin{aligned} (2/\rho ) \sum _{s=1}^l \left( h(x^{k_s}) - h^\star \right)\le & {} \Vert x^{k_1} - x^\star \Vert _2^2 - \Vert x^{k_{l+1}} - x^\star \Vert _2^2 + (2/\rho )\sum _{s=1}^l\delta ^{k_s} \\\le & {} \Vert x^{k_1} - x^\star \Vert _2^2 +2(h(x^0) - h^\star )/\eta \rho . \end{aligned}$$

It follows that the nonnegative series $h(x^{k_s}) - h^\star$ is summable, and therefore, $h(x^{k_s})\rightarrow h^\star$ as $s \rightarrow \infty$.

1.2 Finite updates

We assume $|K| < \infty$, with $p = \max K$ its largest entry. It follows that for any $k>p$, we have $h(x^k) - h\left( \tilde{x}^{k+1}\right) <\eta \delta ^k$. Note that $x^k=x^p$ for all $k\ge p+1$. Moreover, using

$$\begin{aligned} \Vert \tilde{x}^{k+2} - x^p \Vert _2^2 = \Vert \tilde{x}^{k+2} - \tilde{x}^{k+1} \Vert _2^2 + \left\| \tilde{x}^{k+1} - x^p\right\| _2^2 - 2\left( x^p - \tilde{x}^{k+1}\right) ^T(\tilde{x}^{k+2} -\tilde{x}^{k+1}) \end{aligned}$$

with $\rho (x^p - \tilde{x}^{k+1}) \in \partial \hat{h}^k\left( \tilde{x}^{k+1} \right)$ and $\hat{h}^{k+1}(\tilde{x}^{k+2})\ge \hat{h}^k(\tilde{x}^{k+2})$, we get

$$\begin{aligned} \delta ^k - \delta ^{k+1}\ge & {} \hat{h}^{k+1}(\tilde{x}^{k+2})-\hat{h}^k\left( \tilde{x}^{k+1} \right) - \rho \left( x^p - \tilde{x}^{k+1}\right) ^T(\tilde{x}^{k+2} -\tilde{x}^{k+1})\\{} & {} + (\rho /2)\left\| \tilde{x}^{k+2} -\tilde{x}^{k+1}\right\| _2^2\\\ge & {} (\rho /2)\left\| \tilde{x}^{k+2} -\tilde{x}^{k+1}\right\| _2^2. \end{aligned}$$

Therefore, $\delta ^k \ge \delta ^{k+1}+ (\rho /2)\left\| \tilde{x}^{k+2} - \tilde{x}^{k+1}\right\| _2^2$ for all $k \ge p+1$. Then from

$$\begin{aligned} \hat{h}^k(x^p)\ge & {} \hat{h}^k\left( \tilde{x}^{k+1}\right) + \rho (x^p - \tilde{x}^{k+1})^T(x^p - \tilde{x}^{k+1})\\= & {} h(x^k)-\delta ^k+(\rho /2)\left\| x^p - \tilde{x}^{k+1}\right\| _2^2, \end{aligned}$$

it follows that $\left\| x^p - \tilde{x}^{k+1}\right\| _2^2\le 2\delta ^k/\rho \le 2\delta ^p/\rho$.

Now we use the assumption that f and g are Lipschitz continuous with Lipschitz constant L for all $x \in \mathop \textbf{dom}g$. Every $q\in \partial \hat{f}^k(x)$ has the form $q= \sum _{t \le k} \theta _t q^t$, with $\theta _t \ge 0$ and $\sum _t \theta _t=1$, a convex combination of normal vectors of active constraints at x, where $q^t \in \partial f(x^t)$. Therefore, $\hat{h}^k(x)=\hat{f}^k (x)+g(x)$ is 2L-Lipschitz continuous.

Combining this with

$$\begin{aligned} \delta ^k \le h(x^k) - \hat{h}^k\left( \tilde{x}^{k+1}\right) , \qquad -\eta \delta ^k \le {h}\left( \tilde{x}^{k+1}\right) - h(x^k), \end{aligned}$$

we have

$$\begin{aligned} (1-\eta )\delta ^k \le {h}\left( \tilde{x}^{k+1}\right) - h\left( \tilde{x}^k\right) + \hat{h}^k\left( \tilde{x}^k\right) - \hat{h}^k\left( \tilde{x}^{k+1}\right) \le 4L \left\| \tilde{x}^k - \tilde{x}^{k+1} \right\| _2. \end{aligned}$$

Therefore, from

$$\begin{aligned} \frac{(1-\eta )^2\rho }{32L^2} \sum _{k\ge p} \left( \delta ^k\right) ^2 \le \sum _{k\ge p} \left( \delta ^k-\delta ^{k+1}\right) \le \delta ^{p}, \end{aligned}$$

we can establish that $\delta ^{k}$ converges to zero as $k \rightarrow \infty$. This implies

$$\begin{aligned} \underset{k\rightarrow \infty }{\lim }\left( \hat{h}^k\left( \tilde{x}^{k+1}\right) +(\rho /2) \left\| \tilde{x}^{k+1} - x^p\right\| _2^2 \right) = h(x^p). \end{aligned}$$

Also from $\underset{k\rightarrow \infty }{\lim }\ \left( h(x^p) - h\left( \tilde{x}^{k+1}\right) \right) = 0$ and $\left\| {x}^{p} - \tilde{x}^{k+1}\right\| _2^2 \le \frac{2\delta ^k}{\rho }$, it follows that

$$\begin{aligned} \underset{k\rightarrow \infty }{\lim }\ \hat{h}^k\left( \tilde{x}^{k+1} \right) = h(x^p), \qquad \underset{k\rightarrow \infty }{\lim }\ \left\| {x}^{p} - \tilde{x}^{k+1}\right\| _2^2 = 0. \end{aligned}$$

Hence, we get $0 \in \partial h(x^p)$, which implies $h(x^p)=h^\star$.