Combining Lagrangian decomposition and excessive gap smoothing technique for solving large-scale separable convex optimization problems

Abstract

A new algorithm for solving large-scale convex optimization problems with a separable objective function is proposed. The basic idea is to combine three techniques: Lagrangian dual decomposition, excessive gap and smoothing. The main advantage of this algorithm is that it automatically and simultaneously updates the smoothness parameters which significantly improves its performance. The convergence of the algorithm is proved under weak conditions imposed on the original problem. The rate of convergence is $O(\frac {1}{k})$, where k is the iteration counter. In the second part of the paper, the proposed algorithm is coupled with a dual scheme to construct a switching variant in a dual decomposition framework. We discuss implementation issues and make a theoretical comparison. Numerical examples confirm the theoretical results.

Appendix: The proofs of technical lemmas

This appendix provides the proofs of two technical lemmas stated in the previous sections.

Proof of Lemma 4

The proof of this lemma is very similar to Lemma 3 in [27].

Proof

Let $\hat{y} := y^{*}(\hat{x};\beta_{2}) := \frac{1}{\beta_{2}}(A\hat {x}-b)$. Then it follows from (18) that:

(77)

By using the expression f(x;β ₂)=ϕ(x)+ψ(x;β ₂), the definition of $\bar{x}$, the condition (26) and (77) we have:

which is indeed the condition (21). □

Proof of Lemma 7

Let us define $\xi(t) := \frac{2}{\sqrt{1+4/t^{2}}+1}$. It is easy to show that ξ is increasing in (0,1). Moreover, τ _k+1=ξ(τ _k ) for all k≥0. Let us introduce u:=2/t. Then, we can show that $\frac {2}{u+2} < \xi(\frac{2}{u}) < \frac{2}{u+1}$. By using this inequalities and the increase of ξ in (0,1), we have:

$$ \frac{\tau_0}{1+2\tau_0k} \equiv\frac{2}{u_0 + 2k} < \tau_k < \frac{2}{u_0 + k} \equiv\frac{2\tau_0}{2+\tau_0k}. $$

(78)

Now, by the update rule (58), at each iteration k, we only either update $\beta_{1}^{k}$ or $\beta_{2}^{k}$. Hence, it implies that:

$$ \begin{aligned}[c] \beta_1^k& = (1-\tau_0) (1- \tau_2)\cdots(1-\tau_{2\lfloor {k/2\rfloor}})\beta_1^0, \\ \beta_2^k& = (1-\tau_1) (1- \tau_3)\cdots(1-\tau_{2\lfloor {k/2\rfloor}-1})\beta_2^0, \end{aligned} $$

(79)

where ⌊x⌋ is the largest integer number which is less than or equal to the positive real number x. On the other hand, since τ _i+1<τ _i for i≥0, for any l≥0, it implies:

$$ \begin{aligned}[c] &(1-\tau_0)\prod_{i=0}^{2l}(1- \tau_i) < \bigl[(1-\tau_0) (1-\tau_2) \cdots(1-\tau_{2l}) \bigr]^2 < \prod _{i=0}^{2l+1}(1-\tau_i),\quad\mbox{and} \\ &\prod_{i=0}^{2l-1}(1- \tau_i) < \bigl[(1-\tau_1) (1-\tau_3) \cdots(1-\tau_{2l-1}) \bigr]^2 < (1-\tau_0)^{-1} \prod_{i=0}^{2l}(1-\tau_i). \end{aligned} $$

(80)

Note that $\prod_{i=0}^{k}(1-\tau_{i}) = \frac{(1-\tau_{0})}{\tau _{0}^{2}}\tau_{k}^{2}$, it follows from (79) and (80) for k≥1 that:

By combining these inequalities and (78), and noting that τ ₀∈(0,1), we obtain (59). □