A hybrid Bregman alternating direction method of multipliers for the linearly constrained difference-of-convex problems

Abstract

In this paper, we propose a hybrid Bregman alternating direction method of multipliers for solving the linearly constrained difference-of-convex problems whose objective can be written as the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous concave function. At each iteration, we choose either subgradient step or proximal step to evaluate the concave part. Moreover, the extrapolation technique was utilized to compute the nonsmooth convex part. We prove that the sequence generated by the proposed method converges to a critical point of the considered problem under the assumption that the potential function is a Kurdyka–Łojasiewicz function. One notable advantage of the proposed method is that the convergence can be guaranteed without the Lischitz continuity of the gradient function of concave part. Preliminary numerical experiments show the efficiency of the proposed method.

Appendix

Proof of Proposition 2

Proof

Clearly, problem (51) is the special case of problem (1)–(2) with $f_1(x)= \rho \Vert x\Vert _1$, $f_2(x)=\rho \Vert x\Vert _2$ and $g(y)=\frac{1}{2}\Vert Hy-y^0\Vert ^2$, $A={\mathcal {I}}$, $B=- K$ and $b=\mathbf {0}$. Moreover, $f_1$, $f_2$, g, A and B satisfy Assumption 1 (a)–(d). It follows from the choice of $\phi$ and $\psi$ that $L_{\phi }=r_1$, $L_{\psi }=r_2$, $\upsilon _{\phi }=r_1$ and $\upsilon _{\psi }=r_2$. By simple computations, we have that $b_1>0$ and $b_2>0$. It follows from $\sigma < \frac{1}{2\Vert H\Vert ^2}$ that

$$\begin{aligned} g(y)- \sigma \Vert \nabla g(y)\Vert ^2&= \frac{1}{2}\Vert Hy-y^0\Vert ^2 - \sigma \Vert H^{*}(Hy-y^{0})\Vert ^2\nonumber \\&\ge (\frac{1}{2} -\sigma \Vert H\Vert ^2) \Vert Hy-y^0\Vert ^2\ge 0. \end{aligned}$$

(53)

It follows that for any $k\ge 1$,

$$\begin{aligned}&\varTheta (\xi _{1},x_{1},x_{0},y_{1}, y_0, \lambda _{1}) \nonumber \\&\quad \ge \rho (\Vert x_k \Vert _1 - \Vert x_k\Vert _2) + g(y_k)- \sigma \Vert \nabla g(y_{k})\Vert ^2+ \frac{\eta _2}{2\beta } \Vert y_{k} -y_{k-1}\Vert ^2\nonumber \\&\qquad + (\sigma -\frac{1}{\beta \eta _0}) \Vert \nabla g(y_k)\Vert ^2 + \frac{\beta }{2}\Vert x_k -K y_k - \frac{\lambda _k}{\beta }\Vert ^2,\nonumber \\&\quad \ge a_{1} t_0 \Vert y_k - z\Vert ^2 + \frac{\beta }{2}\Vert x_k -K y_k - \frac{\lambda _k}{\beta }\Vert ^2 + t_1, \end{aligned}$$

(54)

where $a_{1} =(\frac{1}{2} -\sigma \Vert H\Vert ^2)$, $z=\frac{1}{t_0}H^* y^{0}$ and $t_1= a_1 (\Vert y_0\Vert ^2 - t_0\Vert z\Vert ^2 )$ with $t_0=\lambda _{\min }(H^*H)$, the first inequality follows from (48), the second inequality is from $\inf _{x} \{\Vert x\Vert _1 - \Vert x\Vert _2\} \ge 0$ and (53). Since H has full column rank, it follows from (54) that the sequences $\{ y_k \}_{k\in {\mathbb {N}}}$ and $\{ x_k -K y_k - \frac{\lambda _k}{\beta }\}_{k\in {\mathbb {N}}}$ are bounded, which together with (47) implies the sequences $\{ \lambda _k \}_{k\in {\mathbb {N}}}$ and $\{ x_k \}_{k\in {\mathbb {N}}}$ are bounded. Thus, the sequence $\{\omega _k\}_{k\in {\mathbb {N}}}$ is bounded. Now, we point that for this problem, the potential function $\varTheta (\xi ,x,{\tilde{x}},y,{\tilde{y}},\lambda )$, defined in (10), is a KL function. In fact, it follows from the definition of $f_1$, $f_2$ and $\varTheta$ that

$$\begin{aligned}&\varTheta (\xi ,x,{\tilde{x}},y,{\tilde{y}},\lambda ) \\&\quad =\rho \Vert x\Vert _1 + I_{\varOmega }(\xi )-\langle \xi , x\rangle + \frac{1}{2}\Vert Hy-y^{0}\Vert ^2 -\langle \lambda , x-Ky-b\rangle \\&\qquad +\frac{\beta }{2}\Vert x-Ky-b\Vert ^2+ \frac{\theta _1}{2}\Vert x-{\tilde{x}}\Vert ^2 + \frac{\eta _2}{\beta } \Vert y-{\tilde{y}}\Vert ^2, \end{aligned}$$

where $I_{\varOmega }(\xi )$ is the indicator function of closed convex set $\varOmega =\{\xi \in {\mathbb {R}}^{n_1}\mid \Vert \xi \Vert ^2 \le \rho ^2\}$. Clearly, $\varOmega$ is a semi-algebraic set. By [10], we know that indicator function of semi-algebraic set is semi-algebraic, and that $\Vert \cdot \Vert _{p}$ is semi-algebraic whenever p is rational, i.e., $p=\frac{p_1}{p_2}$ where $p_1$ and $p_2$ are positive integers. Using the fact that finite sums of semi-algebraic functions is a semi-algebraic function, it yields that $\varTheta$ is a semi-algebraic function, and hence it is a KL function. Note that all assumptions in Theorem 2 hold, which means the conclusion holds. $\square$

Proposition 7

Consider the total variation image restoration problem [45]:

$$\begin{aligned} \min _{x,y}\,\, \rho \Vert x\Vert _1 - \rho \Vert x\Vert _2 + \frac{1}{2}\Vert Hy-y^{0}\Vert ^2, \quad s.t. \quad x - K y= \mathbf {0}, \end{aligned}$$

(55)

where $\rho >0$ is a regularization parameter, H is a blurred operator, $K:{\mathbb {R}}^{n}\rightarrow {\mathbb {R}}^{2n}$ is the discrete gradient operator. Suppose that $({\bar{x}}, {\bar{y}})$ is a local minimum of problem (55), then there exists ${\bar{\lambda }}$ such that $({\bar{x}}, {\bar{y}}, {\bar{\lambda }})$ is a critical point of problem (55), i.e., $({\bar{x}}, {\bar{y}}, {\bar{\lambda }})$ satisfy the inclusion (3).

Proof

We note that problem (55) is equivalent to the following problem

$$\begin{aligned} \min _{x,y}\,\, \rho \Vert Ky\Vert _1 - \rho \Vert Ky\Vert _2 + \frac{1}{2}\Vert Hy-y^{0}\Vert ^2. \end{aligned}$$

(56)

Since $({\bar{x}}, {\bar{y}})$ is a local minimum of problem (55), it yields that ${\bar{x}}=K{\bar{y}}$, and that ${\bar{y}}$ is a local minimum of problem (56). It follows from Lemma 1 (b) that

$$\begin{aligned} \mathbf {0}\in \partial ((\rho \Vert \cdot \Vert _1 -\rho \Vert \cdot \Vert _2)\circ K) ({\bar{y}}) + H^{*}(H{\bar{y}}-y^0), \end{aligned}$$

(57)

where $H^*$ is the adjoint operator of H. Since $\Vert \cdot \Vert _1$ is a proper convex function and $\Vert \cdot \Vert _2$ is a continuous convex function, it follows from the Corollary 3.4 in [37] and (57) that

$$\begin{aligned} \mathbf {0}\in \partial (\rho \Vert \cdot \Vert _1\circ K) ({\bar{y}}) - \partial (\rho \Vert \cdot \Vert _2\circ K) ({\bar{y}}) + H^{*}(H{\bar{y}}-y^0). \end{aligned}$$

Note that $\partial (\rho \Vert \cdot \Vert _1\circ K) ({\bar{y}}) = \rho K^{*}\partial \Vert K{\bar{y}}\Vert _1$ and $\partial (\rho \Vert \cdot \Vert _2\circ K) ({\bar{y}}) = \rho K^{*}\partial \Vert K{\bar{y}}\Vert _2$, where $K^*$ is the adjoint operator of K. Take ${\bar{\xi }}_1 \in \rho \partial \Vert {\bar{x}}\Vert _1$ and ${\bar{\xi }}_2 \in \rho \partial \Vert {\bar{x}}\Vert _2$, such that $K^{*}({\bar{\xi }}_1 -{\bar{\xi }}_2) + H^{*}(H{\bar{y}}-y^0)= \mathbf {0}$. Letting ${\bar{\lambda }}= {\bar{\xi }}_1 -{\bar{\xi }}_2$, it yields that

$$\begin{aligned} {\left\{ \begin{array}{ll} {\bar{\lambda }}\in \partial f_1 ({\bar{x}}) - \partial f_2 ({\bar{x}}),\\ -K^{*}{\bar{\lambda }}= \nabla g({\bar{y}}),\\ {\bar{x}} - K{\bar{y}} = \mathbf {0}. \end{array}\right. } \end{aligned}$$

This completes the proof. $\square$