Distributed Optimization in Multi-agent Networks Using One-bit of Relative State Information

Abstract

This chapter is concerned with the design of distributed discrete-time algorithms to cooperatively solve an additive cost optimization problem in multi-agent networks. The striking feature of our distributed algorithms lies in the use of only the sign of relative state information between neighbors, which substantially differentiates our algorithms from others in the existing literature. Moreover, the algorithm does not require the interaction matrix to be doubly-stochastic. We first interpret the proposed algorithms in terms of the penalty method in optimization theory and then perform non-asymptotic analysis to study convergence for static network graphs. Compared with the celebrated distributed subgradient algorithms, which however use the exact relative state information, the convergence speed is essentially not affected by the loss of information. We also extend our results to the cases of deterministically and randomly time-varying graphs. Finally, we validate the theoretical results by simulations.

Parts of the results in this chapter have previously been appeared in [26,27,28].

Appendix: Proof of Theorem 4

We first show that $\widetilde{d}(\rho )<\infty$. Since $\widetilde{f}_\lambda (x)$ is convex, $\widetilde{\mathscr {X}}(\rho )$ is convex and $\mathscr {X}^\star \subseteq \widetilde{\mathscr {X}}(\rho )$ for any $\rho >0$. One can verify that $\widetilde{\mathscr {X}}(\rho )-\mathscr {X}^\star$ is bounded. If $\widetilde{\mathscr {X}}(\rho )-\mathscr {X}^\star$ is empty, then $\widetilde{d}(\rho )=0$, otherwise $0\le \widetilde{d}(\rho )=\max _{x\in \widetilde{\mathscr {X}}(\rho )}\min _{x^\star \in \mathscr {X}^\star }|x-x^\star |=\max _{x\in \widetilde{\mathscr {X}}(\rho )-\mathscr {X}^\star }\min _{x^\star \in \mathscr {X}^\star }|x-x^\star |<\infty$.

Then, we claim the following.

Claim 1: If $\Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert >c_\rho$ for all $x^\star \in \mathscr {X}^\star$, then $\widetilde{f}_\lambda (\mathbf {x}^k)-f^\star >{\rho c_a^2}/{2}$.

Recall from (15) that

$$ \begin{aligned}&\widetilde{f}_\lambda (\mathbf {x}^k)-f^\star \ge f(\bar{x}^k)-f^\star +(\lambda a_\text {min}^{(l)}-\frac{1}{2} cn)v(\mathbf {x}^k), \forall k. \end{aligned} $$

This implies that if either $f(\bar{x}^k)-f^\star >{\rho c_a^2}/{2}$ or $v(\mathbf {x}^k)>\frac{\rho c_a^2}{2\lambda a_\text {min}^{(l)}- cn}$, then $\widetilde{f}_\lambda (\mathbf {x}^k)-f^\star >{\rho c_a^2}/{2}$. Let

$$c_\rho :=2\sqrt{n}\max \{\widetilde{d}(\rho ),\frac{\rho c_a^2}{2\lambda a_\text {min}^{(l)}- cn}\}.$$

Since

$$ \begin{aligned} c_\rho&<\Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert \le \Vert \mathbf {x}^{k}-\bar{x}^k{\mathbbm {1}}\Vert +\Vert \bar{x}^k{\mathbbm {1}}-x^\star {\mathbbm {1}}\Vert \\&\le \sqrt{n}v(\mathbf {x}^k)+\sqrt{n}|\bar{x}^k-x^\star | \end{aligned} $$

we obtain that $v(\mathbf {x}^k)>c_\rho /(2\sqrt{n})\ge \frac{\rho c_a^2}{2\lambda a_\text {min}^{(l)}- cn}$ or $|\bar{x}^k-x^\star |>c_\rho /(2\sqrt{n})\ge \widetilde{d}(\rho )$. For the former case we have $\widetilde{f}_\lambda (\mathbf {x}^k)-f^\star >{\rho c_a^2}/{2}$. For the latter case, $\bar{x}^k\notin \widetilde{\mathscr {X}}(\rho )$, which by the definition of $\widetilde{\mathscr {X}}(\rho )$ implies $\widetilde{f}_\lambda (\mathbf {x}^k)-f^\star >{\rho c_a^2}/{2}$.

Claim 2: There is $x_0^\star \in \mathscr {X}^\star$ such that $\liminf _{k\rightarrow \infty } \Vert \mathbf {x}^{k}-x_0^\star {\mathbbm {1}}\Vert \le c_\rho$.

Otherwise, there exists $k_0>0$ such that

$$\Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert> c_\rho , \forall x^\star \in \mathscr {X}^\star , \forall k>k_0.$$

By Claim 1, there exists some $\varepsilon >0$ such that $\widetilde{f}_\lambda (\mathbf {x}^k)-f^\star >{\rho c_a^2}/{2}+\varepsilon$ for all $k>k_0$. Together with (18), it yields that

$$\begin{aligned} \Vert \mathbf {x}^{k+1}-x^\star {\mathbbm {1}}\Vert ^2&\le \Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert ^2-2\rho (\widetilde{f}_\lambda (\mathbf {x}^k)-f^\star )+\rho ^2c_a^2 \\&\le \Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert ^2-2\rho (\frac{\rho c_a^2}{2}+\varepsilon )+\rho ^2c_a^2 \nonumber \\&=\Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert ^2-2\rho \varepsilon .\nonumber \end{aligned}$$

(38)

Summing this relation implies that for all $k>k_0$,

$$\begin{aligned}&\Vert \mathbf {x}^{k+1}-x^\star {\mathbbm {1}}\Vert ^2\le \Vert \mathbf {x}^{k_0}-x^\star {\mathbbm {1}}\Vert ^2-2(k+1-k_0)\rho \varepsilon , \end{aligned}$$

which clearly cannot hold for a sufficiently large k. Thus, we have verified Claim 2.

Claim 3: There is $x^\star \in \mathscr {X}^\star$ such that $\limsup _{k\rightarrow \infty } \Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert \le c_\rho +\rho c_a$.

Otherwise, for any $x^\star \in \mathscr {X}^\star$, there must exist a subsequence $\{\mathbf {x}^k\}_{k\in \mathscr {K}}$ (which depends on $x^\star$) such that for all $k\in \mathscr {K}$,

$$\begin{aligned} \Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert >c_\rho +\rho c_a. \end{aligned}$$

(39)

Notice that the penalty function $h(\mathbf {x})$ can be represented as

$$\begin{aligned} h(\mathbf {x})&=\sum _{e=1}^m a_e |\mathbf {b}_e^\mathsf {T}\mathbf {x}|. \end{aligned}$$

where $a_e$ is the weight of edge e. The subdifferential of $h(\mathbf {x})$ is then given by

$$\begin{aligned} \partial h(\mathbf {x})=\sum _{e=1}^ma_e \text {SGN}(\mathbf {b}_e^\mathsf {T}\mathbf {x})\mathbf {b}_e=BA_e\text {SGN}(B^\mathsf {T}\mathbf {x}) \end{aligned}$$

(40)

where $A_e=\text {diag}\{a_1,...,a_m\}$. Then, it follows from (40) that

where the second inequality follows from

$$ \begin{aligned} \Vert BA_e\text {sgn}(B^\mathsf {T}\mathbf {x}^k)\Vert&\le \sqrt{n}\Vert BA_e\text {sgn}(B^\mathsf {T}\mathbf {x}^k)\Vert _\infty \\&\le \sqrt{n}\Vert BA_e\Vert _\infty \Vert \text {sgn}(B^\mathsf {T}\mathbf {x}^k)\Vert _\infty \\&\le \sqrt{n}\max _{i}\sum _{j=1}^na_{ij}=\sqrt{n}\Vert A\Vert _\infty . \end{aligned} $$

Thus, we obtain that for all $k\in \mathscr {K}$,

$$\begin{aligned} \Vert \mathbf {x}^{k-1}-x^\star {\mathbbm {1}}\Vert \ge \Vert \mathbf {x}^{k}-x^\star {\mathbbm {1}}\Vert -\rho c_a>c_\rho . \end{aligned}$$

(41)

By Claim 2, there must exist some $k_1\in \mathscr {K}$ and $k_1>k_0$ such that

$$\begin{aligned}&\Vert \mathbf {x}^{k_1-1}-x_0^\star {\mathbbm {1}}\Vert \le c_\rho +\rho c_a. \end{aligned}$$

Together with (41), it implies that

$$\begin{aligned} c_\rho <\Vert \mathbf {x}^{k_1-1}-x_0^\star {\mathbbm {1}}\Vert \le c_\rho +\rho c_a. \end{aligned}$$

(42)

Hence, it follows from Claim 1 that $\widetilde{f}_\lambda (\mathbf {x}^{k_1-1})-f^\star >{\rho c_a^2}/{2}$, which together with (38) and (42) yields that

$$\begin{aligned} \Vert \mathbf {x}^{k_1}-x_0^\star {\mathbbm {1}}\Vert \le \Vert \mathbf {x}^{k_1-1}-x_0^\star {\mathbbm {1}}\Vert \le c_\rho +\rho c_a. \end{aligned}$$

(43)

Setting $x^\star =x_0^\star$ in (39), we have $\Vert \mathbf {x}^{k_1}-x_0^\star {\mathbbm {1}}\Vert >c_\rho +\rho c_a.$ This contradicts (43), and hence verifies Claim 3.

In view of (19), the proof is completed. $\square$