Primal-dual splittings as fixed point iterations in the range of linear operators

Abstract

In this paper we study the convergence of the relaxed primal-dual algorithm with critical preconditioners for solving composite monotone inclusions in real Hilbert spaces. We prove that this algorithm define Krasnosel’skiĭ-Mann (KM) iterations in the range of a particular monotone self-adjoint linear operator with non-trivial kernel. Our convergence result generalizes (Condat in J Optim Theory Appl 158: 460–479, 2013, Theorem 3.3) and follows from that of KM iterations defined in the range of linear operators, which is a real Hilbert subspace under suitable conditions. The Douglas–Rachford splitting (DRS) with a non-standard metric is written as a particular instance of the primal-dual algorithm with critical preconditioners and we recover classical results from this new perspective. We implement the algorithm in total variation reconstruction, verifying the advantages of using critical preconditioners and relaxation steps.

6 Appendix

Proof of Lemma 2.4:

Let $\varvec{x} \in \mathrm{Fix}\,{\varvec{S}}.$ Since ${\varvec{S}}={\varvec{S}}\circ \varvec{Q}$ we have

$$\begin{aligned} {\varvec{S}}\varvec{x} = \varvec{x}\quad&\Leftrightarrow \quad {\varvec{S}}(\varvec{Q} \varvec{x}) = \varvec{x}\\&\Rightarrow \quad \varvec{Q}\circ {\varvec{S}}(\varvec{Q}\varvec{x}) = \varvec{Q} \varvec{x}, \end{aligned}$$

which yields $\varvec{Q}\varvec{x} \in \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}})$. Thus, $\varvec{x}={\varvec{S}}(\varvec{Q} \varvec{x}) \in {\varvec{S}}(\mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}}))$ and we conclude $\mathrm{Fix}\,{\varvec{S}}\subset {\varvec{S}}( \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}}))$. Conversely, let $\varvec{x} \in \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}})$. Since ${\varvec{S}}={\varvec{S}}\circ \varvec{Q}$, we have

$$\begin{aligned} \varvec{Q} ({\varvec{S}}\varvec{x})=\varvec{x}\quad&\Rightarrow \quad {\varvec{S}}(\varvec{Q} ({\varvec{S}}\varvec{x}))={\varvec{S}}\varvec{x}\\&\Rightarrow \quad {\varvec{S}}({\varvec{S}}\varvec{x})={\varvec{S}}\varvec{x}\\&\Rightarrow \quad {\varvec{S}}\varvec{x} \in \mathrm{Fix}\,{\varvec{S}}. \end{aligned}$$

Thus ${\varvec{S}}( \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}})) \subset \mathrm{Fix}\,{\varvec{S}}$ and the result follows. $\square$

Proof of Proposition 3.1:

1: It is a direct consequence of [13, Proposition 2.1]. 2.: (2a $\Rightarrow$ 2b). Let $(v_n)_{n \in {\mathbb {N}}}$ be sequence in $\mathrm{ran }(\varSigma ^{-1} - L \varUpsilon L^*)$ such that $v_n \rightarrow v.$ Therefore, for each $n \in {\mathbb {N}},$ there exists $u_n \in {\mathcal {G}}$ such that $v_n=\varSigma ^{-1} u_n - L \varUpsilon L^* u_n$. Note that ${\varvec{V}}(\varUpsilon L^* u_n , u_n)=(0, v_n)\rightarrow (0, v)$. Since $\mathrm{ran }\, {\varvec{V}}$ is closed, there exists some $(x,u) \in {\mathcal {H}}\times {\mathcal {G}}$ such that ${\varvec{V}}(x,u)=(0, v)$, i.e.,

$$\begin{aligned} {\varvec{V}}(x,u)=(0,v) \quad&\Leftrightarrow \quad {\left\{ \begin{array}{ll} \varUpsilon ^{-1}x- L^* u=0\\ \varSigma ^{-1} u- L x = v \end{array}\right. }\\&\Rightarrow \quad \varSigma ^{-1} u- L \varUpsilon L^* u = v. \end{aligned}$$

Then $v \in \mathrm{ran }(\varSigma ^{-1} - L \varUpsilon L^*)$, and therefore $\mathrm{ran }(\varSigma ^{-1}- L \varUpsilon L^*)$ is closed.

(2b $\Rightarrow$ 2a). Let $\big ((y_n,v_n)\big )_{n \in {\mathbb {N}}}$ be a sequence in $\mathrm{ran }\, {\varvec{V}}$ such that $(y_n,v_n) \rightarrow (y,v).$ Then, for every $n \in {\mathbb {N}}$, there exists $(x_n,u_n)$ such that $(y_n,u_n)={\varvec{V}}(x_n,u_n)$, or equivalently,

$$\begin{aligned} {\left\{ \begin{array}{ll} y_n=\varUpsilon ^{-1}x_n- L^*u_n\\ v_n=\varSigma ^{-1} u_n- Lx_n. \end{array}\right. } \end{aligned}$$

(6.1)

By applying $L \varUpsilon$ to the first equation in (6.1) and adding it to the second equation, by the continuity of $\varUpsilon$ and L, we obtain

$$\begin{aligned} (\varSigma ^{-1} - L \varUpsilon L^*)u_n= L \varUpsilon y_n + v_n\,\rightarrow \, L \varUpsilon y + v. \end{aligned}$$

(6.2)

Hence, since $\mathrm{ran }(\varSigma ^{-1} - L \varUpsilon L^*)$ is closed, there exists $u \in {\mathcal {G}}$ such that $L \varUpsilon y+ v =(\varSigma ^{-1} - L \varUpsilon L^*)u$. We deduce ${\varvec{V}}\left( \varUpsilon (L^*u+y),u \right) =(y,v)$, and therefore $\mathrm{ran }\, {\varvec{V}}$ is closed.

(2a $\Leftrightarrow$ 2c). Define ${\tilde{{\varvec{V}}}} : {\mathcal {G}}\oplus {\mathcal {H}}\rightarrow {\mathcal {G}}\oplus {\mathcal {H}}: (u,x) \mapsto (\varSigma ^{-1} u-Lx,\varUpsilon ^{-1} x-L^*u)$. By the equivalence 2a $\Leftrightarrow$ 2b $\mathrm{ran }{\tilde{{\varvec{V}}}}$ is closed if and only if $\mathrm{ran }(\varUpsilon ^{-1} - L^* \varSigma L)$ is closed. Consider the isometric map $\varvec{\varLambda } : {\mathcal {H}}\oplus {\mathcal {G}}\rightarrow {\mathcal {G}}\oplus {\mathcal {H}}: (x,u) \mapsto (u,x)$. Since $\varvec{\varLambda }\circ {\varvec{V}}={\tilde{{\varvec{V}}}}$, ${\mathrm{ran }\, {\varvec{V}}}$ is closed if and only if $\mathrm{ran }{\tilde{{\varvec{V}}}}$ is closed and the result follows. $\square$

Proposition 6.1

In the context of Problem 1.1, set $L=\mathrm{Id}$, let $\varUpsilon :{\mathcal {H}}\rightarrow {\mathcal {H}}$ be a strongly monotone self adjoint linear bounded operator, set $\varLambda :{\mathcal {H}}\times {\mathcal {H}}\rightarrow {\mathcal {H}}:(x,u)\mapsto x-\varUpsilon u$, let ${\varvec{V}}$, ${\varvec{W}}$, and $G_{\varUpsilon ,B,A}$ be the operators defined in (1.8), (3.2), and (3.15), respectively. Then, $\varLambda (\mathrm{Fix}\,(P_{{\mathrm{ran }\, {\varvec{V}}}}\circ J_{{\varvec{W}}}))=\mathrm{Fix}\,G_{\varUpsilon ,B,A}$.

Proof

The inclusion $\subset$ is proved in (3.23). Conversely, since $\varLambda ^*:z\mapsto (z,-\varUpsilon z)$, we have $\varLambda \circ \varLambda ^*=\mathrm{Id}+\varUpsilon ^2$ and [3, Proposition 3.30 & Example 3.29] yields $P_{{\mathrm{ran }\, {\varvec{V}}}}=P_{\mathrm{ran }\varLambda ^*}=\varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1} \varLambda$. Therefore, if ${\hat{z}}\in \mathrm{Fix}\,G_{\varUpsilon ,B,A}$, by setting $({\hat{x}},{\hat{u}}):=\varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1}{\hat{z}}$, we have ${\hat{z}}=\varLambda ({\hat{x}},{\hat{u}})$ and we deduce from (3.22) that

$$\begin{aligned} P_{{\mathrm{ran }\, {\varvec{V}}}}\circ J_{{\varvec{W}}}({\hat{x}},{\hat{u}})&=\varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1} \varLambda (J_{{\varvec{W}}}({\hat{x}},{\hat{u}})) \nonumber \\&= \varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1}G_{\varUpsilon ,B,A} (\varLambda ({\hat{x}},{\hat{u}}))\nonumber \\&= \varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1}G_{\varUpsilon ,B,A}{\hat{z}}\nonumber \\&=\varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1} {\hat{z}}\nonumber \\&=({\hat{x}},{\hat{u}}). \end{aligned}$$

(6.3)

Consequently, $({\hat{x}},{\hat{u}})\in \mathrm{Fix}\,(P_{{\mathrm{ran }\, {\varvec{V}}}}\circ J_{{\varvec{W}}})$ and ${\hat{z}}=\varLambda ({\hat{x}},{\hat{u}})\in \varLambda (\mathrm{Fix}\,(P_{{\mathrm{ran }\, {\varvec{V}}}}\circ J_{{\varvec{W}}}))$. $\square$