Enforcing Geometrical Priors in Deep Networks for Semantic Segmentation Applied to Radiotherapy Planning

Abstract

Incorporating prior knowledge into a segmentation process, whether it is geometrical constraints such as volume penalisation, (partial) convexity enforcement, or topological prescriptions to preserve the contextual relations between objects, proves to improve accuracy in medical image segmentation, in particular when addressing the issue of weak boundary definition. Motivated by this observation, the proposed contribution aims to provide a unified variational framework including geometrical constraints in the training of convolutional neural networks in the form of a penalty in the loss function. These geometrical constraints take several forms and encompass level curve alignment through the integration of the weighted total variation, an area penalisation phrased as a hard constraint in the modelling, and an intensity homogeneity criterion based on a combination of the standard Dice loss with the piecewise constant Mumford–Shah model. The mathematical formulation yields a non-smooth non-convex optimisation problem, which rules out conventional smooth optimisation techniques and leads us to adopt a Lagrangian setting. The application falls within the scope of organ-at-risk segmentation in CT (computed tomography) images, in the context of radiotherapy planning. Experiments demonstrate that our method provides significant improvements (i) over existing non-constrained approaches both in terms of quantitative criteria, such as the measure of overlap, and qualitative assessment (spatial regularisation/coherency, fewer outliers), (ii) over in-layer constrained deep convolutional networks, and shows a certain degree of versatility.

Appendices

Proof of Theorem 2

The sets ${\mathcal {C}}_1$ and ${\mathcal {C}}_2$ are closed and convex, while the objective function is continuous and coercive. Indeed, $\forall l \in \left\{ 1,\cdots ,L\right\}$,

$$\begin{aligned} \left\| \begin{array}{ccc} \dfrac{\mu }{2}\,\Vert u^l-s^l(\theta )-w^l\Vert ^2&{}\ge \dfrac{\mu }{4}\,\Vert u^l\Vert ^2-\dfrac{\mu }{2}\,\Vert s^l(\theta )+w^l\Vert ^2,\\ \dfrac{1}{2}\,\Vert u^l-v^l\Vert ^2&{}\ge \dfrac{1}{2}\,\Vert u^l\Vert ^2+\dfrac{1}{2}\,\Vert v^l\Vert ^2-\Vert u^l\Vert \,\Vert v^l\Vert . \end{array}\right. \end{aligned}$$

Using Young’s inequality with $\epsilon$ (valid for $\epsilon >0$) and stated by $ab\le \dfrac{a^2}{2\epsilon }+\dfrac{\epsilon b^2}{2}$, we get:

$$\begin{aligned} {\mathcal {J}}(u,v) \ge \left( \dfrac{\mu +2}{4}-\dfrac{1}{2\epsilon }\right) \,\displaystyle {\sum _{l=1}^{L}}\,\Vert u^l\Vert ^2-\dfrac{\mu }{2}\,\displaystyle {\sum _{l=1}^{L}}\,\Vert s^l(\theta )+w^l\Vert ^2\\ +\left( \dfrac{1}{2}-\dfrac{\epsilon }{2}\right) \,\displaystyle {\sum _{l=1}^{L}}\,\Vert v^l\Vert ^2, \end{aligned}$$

or equivalently,

$$\begin{aligned} {\mathcal {J}}(u,v) \ge \left( \dfrac{\mu +2}{4}-\dfrac{1}{2\epsilon }\right) \,\Vert u\Vert ^2-\dfrac{\mu }{2}\,\Vert s(\theta )+w\Vert ^2\\ +\left( \dfrac{1}{2}-\dfrac{\epsilon }{2}\right) \,\Vert v\Vert ^2. \end{aligned}$$

(To lighten the notations, when there is no ambiguity about the dimension of the mathematical objects we handle, we omit the lower index making this dimension explicit in the definition of the Euclidean norm as well as on the related scalar product.)

Taking $\epsilon$ such that $\dfrac{2}{\mu +2}<\epsilon <1$ yields the desired result.

To conclude, functional ${\mathcal {J}}$ is strictly convex, due to the strict convexity of functional ${\mathcal {H}}$ defined by ${\mathcal {H}}(u,v)=\dfrac{\mu }{2}\,\Vert u-s(\theta )-w\Vert ^2+\dfrac{1}{2}\,\Vert u-v\Vert ^2$, u denoting the concatenation of the $u^l$’s, similarly for $s(\theta ), w$ and v. A straightforward computation gives that $\forall (u_1,v_1)\in {\mathcal {C}}_1 \times {\mathcal {C}}_2$, $\forall (u_2,v_2)\in {\mathcal {C}}_1 \times {\mathcal {C}}_2$,

$$\begin{aligned} \langle \nabla {\mathcal {H}}(u_1,v_1)-&\nabla {\mathcal {H}}(u_2,v_2),\begin{pmatrix}u_1-u_2\\ v_1-v_2 \end{pmatrix}\rangle \\&=\mu \,\Vert u_1-u_2\Vert ^2+\Vert (u_1-u_2)-(v_1-v_2)\Vert ^2, \end{aligned}$$

this quantity vanishing if and only if $u_1=u_2$ and $v_1=v_2$.

Proof of Theorem 3

For every $p\in {\mathcal {B}}, (u,v) \mapsto {\mathcal {L}}(u,v,p)$ is strictly convex, owing to the strict convexity of functional ${\mathcal {H}}$. For each $p\in {\mathcal {B}}$, functional $(u,v) \mapsto {\mathcal {L}}(u,v,p)$ is continuous and coercive. To establish such a coercivity inequality, denoting by $\kappa =\Vert {\text{ div }}\Vert =\displaystyle {\sup _{\Vert p\Vert _{Y^L} \le 1}}\,\Vert {\text{ div }}\,p\Vert _{X^L}$, we first observe that with the convention $(p^1)^1_{0,j}=(p^1)^1_{N,j}=(p^1)^2_{i,0}=(p^1)^2_{i,N}=0$ (similarly for $p^l$ with $l\in \left\{ 2,\cdots ,L\right\}$) and by applying twice the inequality $(a+b)^2\le 2(a^2+b^2)$,

$$\begin{aligned} \Vert {\text{ div }}\,p\Vert _{X^L}^2\le 8\,\Vert p\Vert ^2. \end{aligned}$$

Thus $\kappa \le 2\sqrt{2}$ and with suitable $\epsilon$,

$$\begin{aligned} {\mathcal {L}}(u,v,p)\ge&\left( \dfrac{\mu +2}{4}-\dfrac{1}{2\epsilon }\right) \,\Vert u\Vert ^2-\dfrac{\mu }{2}\,\Vert s(\theta )+w\Vert ^2\\&+\left( \dfrac{1}{2}-\dfrac{\epsilon }{2}\right) \,\Vert v\Vert ^2-\Vert {\text{ div }}\Vert \Vert p\Vert \Vert u\Vert ,\\ \ge&\left( \dfrac{\mu +2}{4}-\dfrac{1}{2\epsilon }\right) \,\Vert u\Vert ^2-\dfrac{\mu }{2}\,\Vert s(\theta )+w\Vert ^2\\&+\left( \dfrac{1}{2}-\dfrac{\epsilon }{2}\right) \,\Vert v\Vert ^2-2\sqrt{2L}N\Vert u\Vert , \end{aligned}$$

since $p\in {\mathcal {B}}$, entailing that $\Vert p\Vert _{Y^L}^2\le LN^2$.

Applying again Young’s inequality with $\epsilon '>0$,

$$\begin{aligned} {\mathcal {L}}(u,v,p)\ge&\left( \dfrac{\mu +2}{4}-\dfrac{1}{2\epsilon }-\dfrac{\epsilon '}{2}\right) \,\Vert u\Vert ^2+\left( \dfrac{1}{2}-\dfrac{\epsilon }{2}\right) \,\Vert v\Vert ^2\\&-\dfrac{\mu }{2}\,\Vert s(\theta )+w\Vert ^2-\dfrac{4LN^2}{\epsilon '}. \end{aligned}$$

This latter inequality shows that by choosing $\epsilon '$ suitable— which is always possible, the coercivity property is ensured. Also, the quantity ${\mathcal {L}}(u,v,p)$ is bounded below (independently of p) and one remarks that if one takes ${\tilde{u}}$ such that ${\tilde{u}}^1 \equiv 1$ and $\forall l\in \left\{ 2,\cdots ,L\right\} , {\tilde{u}}^l\equiv 0$, and ${\tilde{v}}$ such that $\forall l \in \left\{ 1,\cdots ,L\right\}$, $\forall (i,j)\in {\mathcal {G}}$, ${\tilde{v}}_{i,j}^l=\dfrac{\alpha ^l}{N^2}$, ${\mathcal {L}}({\tilde{u}},{\tilde{v}},p)$ is independent of p, showing that the infimum is finite.

Then for each $p\in {\mathcal {B}}$, functional ${\mathcal {L}}(\cdot ,\cdot ,p)$ is continuous, coercive and strictly convex, so it admits a unique minimiser in ${\mathcal {C}}_1 \times {\mathcal {C}}_2$—${\mathcal {C}}_1$ and ${\mathcal {C}}_2$ being closed convex sets—denoted by $(e_1(p),e_2(p))$. We denote this minimum by f(p), i.e.

$$\begin{aligned} f(p)=\displaystyle {\min _{(u,v)\in {\mathcal {C}}_1\times {\mathcal {C}}_2}}\,{\mathcal {L}}(u,v,p)={\mathcal {L}}(e_1(p),e_2(p),p). \end{aligned}$$

Function $p \mapsto f(p)$ is concave as the pointwise infimum of concave functions ($\forall (u,v) \in {\mathcal {C}}_1 \times {\mathcal {C}}_2, p \mapsto {\mathcal {L}}(u,v,p)$ is concave since in fact linear). This can be proved using the hypograph of f. Also, f is upper semicontinuous as the pointwise infimum of continuous functions. It is therefore bounded above and attains its upper bound as the set ${\mathcal {B}}$ is compact at a point denoted by ${\bar{p}}$. Thus

$$\begin{aligned} f({\bar{p}})=\displaystyle {\max _{p\in {\mathcal {B}}}}\,f(p)=\displaystyle {\max _{p\in {\mathcal {B}}}}\,\displaystyle {\min _{(u,v)\in {\mathcal {C}}_1 \times {\mathcal {C}}_2}}\,{\mathcal {L}}(u,v,p). \end{aligned}$$

Additionally, as $f({\bar{p}})=\displaystyle {\min _{(u,v)\in {\mathcal {C}}_1 \times {\mathcal {C}}_2}}\,{\mathcal {L}}(u,v,{\bar{p}})$, one has, $\forall (u,v)\in {\mathcal {C}}_1 \times {\mathcal {C}}_2$,

$$\begin{aligned} f({\bar{p}}) \le {\mathcal {L}}(u,v,{\bar{p}}). \end{aligned}$$

(19)

By concavity of ${\mathcal {L}}$ with respect to the third argument (in fact, linearity), $\forall (u,v)\in {\mathcal {C}}_1 \times {\mathcal {C}}_2, \forall p \in {\mathcal {B}}, \forall \lambda \in ]0,1[$,

$$\begin{aligned} {\mathcal {L}}(u,v,(1-\lambda )\,{\bar{p}}+\lambda p)=(1-\lambda )\,{\mathcal {L}}(u,v,{\bar{p}})+\lambda \,{\mathcal {L}}(u,v,{p}). \end{aligned}$$

Taking as particular value $(u,v)=(e_1((1-\lambda )\,{\bar{p}}+\lambda p),e_2((1-\lambda )\,{\bar{p}}+\lambda p))=(e_{\lambda }^1,e_{\lambda }^2)$, it yields, using again that $f({\bar{p}})=\displaystyle {\max _{p\in {\mathcal {B}}}}\,f(p)$ and the concavity (even linearity) of ${\mathcal {L}}$ with respect to the third argument,

$$\begin{aligned} f({\bar{p}})&\ge f((1-\lambda )\,{\bar{p}}+\lambda p)={\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,(1-\lambda )\,{\bar{p}}+\lambda p),\\&\ge (1-\lambda )\,{\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,{\bar{p}})+\lambda \,{\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,p). \end{aligned}$$

As ${\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,{\bar{p}}) \ge f({\bar{p}})=\displaystyle {\min _{(u,v)\in {\mathcal {C}}_1 \times {\mathcal {C}}_2}}\,{\mathcal {L}}(u,v,{\bar{p}})$, the latter inequality implies that $\forall p \in {\mathcal {B}}$,

$$\begin{aligned} f({\bar{p}}) \ge {\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,{p}). \end{aligned}$$

(20)

By virtue of the coercivity property established previously, one has, $\forall p \in {\mathcal {B}}$ (parameters $\epsilon$ and $\epsilon '$ being suitably chosen),

$$\begin{aligned}&\scriptstyle {\left( \dfrac{\mu +2}{4}-\dfrac{1}{2\epsilon }-\dfrac{\epsilon '}{2}\right) \,\Vert e_{\lambda }^1\Vert ^2+\left( \dfrac{1}{2}-\dfrac{\epsilon }{2}\right) \,\Vert e_{\lambda }^2\Vert ^2-\dfrac{\mu }{2}\,\Vert s(\theta )+w\Vert ^2-\dfrac{4LN^2}{\epsilon '}}\\&\le {\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,(1-\lambda )\,{\bar{p}}+\lambda p)\le {\mathcal {L}}({\tilde{u}},{\tilde{v}},(1-\lambda )\,{\bar{p}}+\lambda p), \end{aligned}$$

with ${\tilde{u}}\in {\mathcal {C}}_1$ and ${\tilde{v}} \in {\mathcal {C}}_2$ as defined before, making the right-hand side independent of $p, {\bar{p}}$ and $\lambda$—and constituting thus a uniform bound, and showing that $e_{\lambda }^1$ is uniformly bounded (this was already known owing to the definition of ${\mathcal {C}}_1$) as well as $e_{\lambda }^2$. One can thus extract a subsequence (common extracting mapping) $e_{\lambda _n}^1$ and $e_{\lambda _n}^2$ with $\lambda _n \underset{n \rightarrow +\infty }{\rightarrow } 0$ converging to some limits ${\bar{u}}$ and ${\bar{v}}$. We show next that ${\bar{u}}=e_1({\bar{p}})$ and ${\bar{v}}=e_2({\bar{p}})$.

As by definition, ${\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,(1-\lambda )\,{\bar{p}}+\lambda p)= \min _{(u,v)\in \displaystyle {{\mathcal {C}}_1\times {\mathcal {C}}_2}}\,{\mathcal {L}}(u,v,(1-\lambda )\,{\bar{p}}+\lambda p), \forall (u,v)\in {\mathcal {C}}_1\times {\mathcal {C}}_2$,

$$\begin{aligned}&{\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,(1-\lambda )\,{\bar{p}}+\lambda p)\\&\quad \le {\mathcal {L}}(u,v,(1-\lambda )\,{\bar{p}}+\lambda p), \end{aligned}$$

and by linearity of ${\mathcal {L}}$ with respect to the third argument, it follows that $\forall (u,v)\in {\mathcal {C}}_1\times {\mathcal {C}}_2$,

$$\begin{aligned}&(1-\lambda )\,{\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,{\bar{p}})+\lambda \,{\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,p)\\&\quad \le {\mathcal {L}}(u,v,(1-\lambda )\,{\bar{p}}+\lambda p). \end{aligned}$$

The quantity ${\mathcal {L}}(e_{\lambda }^1,e_{\lambda }^2,p)$ is bounded below by f(p) so that passing to the limit in the previous inequality when $\lambda _n$ tends to 0 yields, using the continuity of ${\mathcal {L}}$,

$$\begin{aligned} {\mathcal {L}}({\bar{u}},{\bar{v}},{\bar{p}})\le {\mathcal {L}}(u,v,{\bar{p}}), \end{aligned}$$

this being true for all $(u,v)\in {\mathcal {C}}_1\times {\mathcal {C}}_2$. By uniqueness of the minimiser of $\displaystyle {\min _{(u,v)\in {\mathcal {C}}_1\times {\mathcal {C}}_2}}\,{\mathcal {L}}(u,v,{\bar{p}})$, we deduce that $({\bar{u}},{\bar{v}})=(e_1({\bar{p}}),e_2({\bar{p}}))$.

At last, passing to the limit in (20) yields ${\mathcal {L}}({\bar{u}},{\bar{v}},p)\le f({\bar{p}}), \forall p \in {\mathcal {B}}$, which combines with (19) and the invocation of [16, Chapter VI, Proposition 1.3] enables one to conclude that $({\bar{u}},{\bar{v}},{\bar{p}})$ is a saddle point of ${\mathcal {L}}$.

Comment on the Fact that Fixed Points of Algorithm 3 are Saddle Points of the Associated Lagrangian

First of all, one remarks that

$$\begin{aligned} {\hat{x}}&={{\,\mathrm{\mathrm{arg \, min}}\,}}_{x}\,g(x)+\langle \nabla f({\bar{x}})+K^{*}{\tilde{y}},x\rangle +\dfrac{1}{2\tau }\,\Vert x-{\bar{x}}\Vert ^2,\\&={{\,\mathrm{\mathrm{arg \, min}}\,}}_{x}\,g(x)+\dfrac{1}{2\tau }\,\Vert x-\left( {\bar{x}}-\tau \,\left( \nabla f({\bar{x}})+K^{*}{\tilde{y}}\right) \right) \Vert ^2,\\&={\text{ prox }}_{\tau \,g}({\bar{x}}-\tau \,(\nabla f({\bar{x}})+K^{*}{\tilde{y}})). \end{aligned}$$

The first step of the algorithm can thus be rephrased as

$$\begin{aligned} x^{n+1}={\text{ prox }}_{\tau \,g}(x^n-\tau \,(\nabla f(x^n)+K^{*}y^n)). \end{aligned}$$

(21)

Similarly,

$$\begin{aligned} {\hat{y}}&={{\,\mathrm{\mathrm{arg \, min}}\,}}_{y}\,h^{*}(y)-\langle K{\tilde{x}},y\rangle +\dfrac{1}{2\sigma }\,\Vert y-{\bar{y}}\Vert ^2,\\&={{\,\mathrm{\mathrm{arg \, min}}\,}}_{y}\,h^{*}(y)+\dfrac{1}{2\sigma }\,\Vert y-\left( {\bar{y}}+\sigma \,K{\tilde{x}}\right) \Vert ^2,\\&={\text{ prox }}_{\sigma \,h^{*}}({\bar{y}}+\sigma \,K{\tilde{x}}). \end{aligned}$$

The second step of the algorithm thus reads as

$$\begin{aligned} y^{n+1}={\text{ prox }}_{\sigma \,h^{*}}(y^n+\sigma \,K(2x^{n+1}-x^n)). \end{aligned}$$

(22)

Now, considering a fixed point $(x^{*},y^{*})$ of the algorithm and owing to the fact that

$$\begin{aligned} r={\text{ prox }}_f(s)\,\,&\Longleftrightarrow \,\,s-r\in \partial f(r)\,\,\\&\Longleftrightarrow \,\,\forall t,\,f(t)\ge f(r)+\langle s-r,t-r\rangle , \end{aligned}$$

the relation (21) gives that $\forall x \in {\mathcal {X}}$,

$$\begin{aligned} g(x) \ge g(x^{*})-\langle x-x^{*},\nabla f(x^{*})+K^{*}y^{*}\rangle , \end{aligned}$$

while the relation (22) leads to $\forall y \in {\mathcal {Y}}$,

$$\begin{aligned} h^{*}(y)\ge h^{*}(y^{*})+\langle y-y^{*},Kx^{*}\rangle . \end{aligned}$$

By summing both inequalities, it yields

$$\begin{aligned} g(x^{*})-h^{*}(y)-\langle x-x^{*},\nabla f(x^{*})\rangle -\langle Kx,y^{*}\rangle \\\le g(x)-h^{*}(y^{*})-\langle Kx^{*},y\rangle . \end{aligned}$$

But by convexity of $f,f(x^{*})+\langle \nabla f(x^{*}),x-x^{*}\rangle \le f(x)$, which implies that

$$\begin{aligned}&g(x^{*})-h^{*}(y)+f(x^{*})+\langle Kx^{*},y\rangle \\&\le g(x)-h^{*}(y^{*})+\langle Kx,y^{*}\rangle +f(x). \end{aligned}$$

Then $\forall x \in {\mathcal {X}}, \forall y \in {\mathcal {Y}}$, ${\mathcal {L}}(x^{*},y)\le {\mathcal {L}}(x,y^{*})$, showing that $(x^{*},y^{*})$ is a saddle point of the associated Lagrangian denoted by ${\mathcal {L}}$ here.

Proof of Theorem 4

We recall that the considered saddle-point structure reads as

and that the general iteration of the algorithm is given by

$$\begin{aligned} ({\hat{x}},{\hat{y}})=PD_{\tau ,\sigma }({\bar{x}},{\bar{y}},{\tilde{x}},{\tilde{y}}), \end{aligned}$$

with

$$\begin{aligned} \left\{ \begin{array}{ccc} {\hat{x}}&{}=&{}\displaystyle {{{\,\mathrm{\mathrm{arg \, min}}\,}}_{x}} \,f({\bar{x}})+\langle \nabla f({\bar{x}}),x-{\bar{x}}\rangle +g(x)+\langle Kx,{\tilde{y}}\rangle \\ &{}&{}+\dfrac{1}{\tau }\,D_x(x,{\bar{x}}),\\ {\hat{y}}&{}=&{}\displaystyle {{{\,\mathrm{\mathrm{arg \, min}}\,}}_{y}} \,h^{*}(y)-\langle K{\tilde{x}},y\rangle +\dfrac{1}{\sigma }\,D_y(y,{\bar{y}}). \end{array}\right. \end{aligned}$$

The inputs are thus the points $({\bar{x}},{\bar{y}})$ as well as the intermediate points $({\tilde{x}},{\tilde{y}})$, while the outputs are the generated points $({\hat{x}},{\hat{y}})$.

Here $D_x$ and $D_y$ are Bregman proximity/distance functions (see [10, p. 256] for further details) chosen to be $D_x(x,{\bar{x}})=\frac{1}{2}\,\Vert x-{\bar{x}}\Vert ^2$ (respectively, $D_y(y,{\bar{y}})=\frac{1}{2}\,\Vert y-{\bar{y}}\Vert ^2$) in our setting, this choice being the most common one. Also, from an algorithmic viewpoint, an iteration is applied with ${\bar{x}}=x^n$, ${\bar{y}}=y^n, {\tilde{x}}=2x^{n+1}-x^n$ and ${\tilde{y}}=y^n$. Lemma 1 from [10, Lemma 1, p. 257 with proof] states that, provided the previous general iteration holds, then for any $x\in {\mathcal {X}}$ and for any $y\in {\mathcal {Y}}$, one has

$$\begin{aligned}&{\mathcal {L}}({\hat{x}},y)-{\mathcal {L}}(x,{\hat{y}})\\&\quad \le \frac{1}{\tau }\,D_x(x,{\bar{x}})-\frac{1}{\tau }\,D_x(x,{\hat{x}})-\frac{1}{\tau }\,D_x({\hat{x}},{\bar{x}})+\frac{L_f}{2}\,\Vert {\hat{x}}-{\bar{x}}\Vert ^2\\&\qquad +\frac{1}{\sigma }\,D_y(y,{\bar{y}})-\frac{1}{\sigma }\,D_y(y,{\hat{y}}) -\frac{1}{\sigma }\,D_y({\hat{y}},{\bar{y}})\\&\qquad +\langle K(x-{\hat{x}}),{\tilde{y}}-{\hat{y}}\rangle -\langle K({\tilde{x}}-{\hat{x}}),y-{\hat{y}}\rangle . \end{aligned}$$

Applying [10, Lemma 1] with ${\hat{x}}=x^{n+1}, {\hat{y}}=y^{n+1}$ and $D_x, D_y$ as defined above, gives that $\forall x \in {\mathcal {X}}$ and $\forall y \in {\mathcal {Y}}$, one has:

$$\begin{aligned}&{\mathcal {L}}(x^{n+1},y)-{\mathcal {L}}(x,y^{n+1}) \\&\quad \le \,\, \dfrac{1}{2\tau }\,\Vert x-x^n\Vert ^2-\dfrac{1}{2\tau }\,\Vert x-x^{n+1}\Vert ^2-\dfrac{1}{2\tau }\,\Vert x^{n+1}-x^n\Vert ^2\\&\qquad +\dfrac{L_f}{2}\,\Vert x^{n+1}-x^n\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y-y^n\Vert ^2-\dfrac{1}{2\sigma }\,\Vert y-y^{n+1}\Vert ^2\\&\qquad -\dfrac{1}{2\sigma }\,\Vert y^{n+1}-y^n\Vert ^2+\langle K(x-x^{n+1}),y^n-y^{n+1}\rangle \\&\qquad -\langle K(x^{n+1}-x^{n}),y-y^{n+1}\rangle . \end{aligned}$$

Remarking then that $\left\{ \begin{array}{ccc}x-x^{n+1}&{}=&{}x-x^n+x^n-x^{n+1}\\ x^{n+1}-x^n&{}=&{}x^{n+1}-x+x-x^n \end{array}\right.$, and thus

$$\begin{aligned}&\langle K(x-x^{n+1}),y^n-y^{n+1}\rangle -\langle K(x^{n+1}-x^{n}),y-y^{n+1}\rangle \\&\quad =\langle K(x-x^{n}),y^n-y^{n+1}\rangle +\langle K(x^n-x^{n+1}),y^n-y^{n+1}\rangle \\&\qquad -\langle K(x^{n+1}-x),y-y^{n+1}\rangle -\langle K(x-x^n),y-y^{n+1}\rangle ,\\&\quad =-\langle K(x-x^n),y-y^n\rangle +\langle K(x^{n+1}-x^n),y^{n+1}-y^n\rangle \\&\qquad +\langle K(x-x^{n+1}),y-y^{n+1}\rangle , \end{aligned}$$

it yields

$$\begin{aligned}&{\mathcal {L}}(x^{n+1},y)-{\mathcal {L}}(x,y^{n+1}) \nonumber \\&\quad \le \left[ \dfrac{1}{2\tau }\,\Vert x-x^n\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y-y^n\Vert ^2-\langle K(x-x^n),y-y^n\rangle \right] \nonumber \\&\qquad -\scriptstyle {\left[ \dfrac{1}{2\tau }\,\Vert x-x^{n+1}\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y-y^{n+1}\Vert ^2-\langle K(x-x^{n+1}),y-y^{n+1}\rangle \right] }\nonumber \\&\qquad -\scriptstyle {\left[ \dfrac{1}{2\tau }\,\Vert x^{n+1}-x^{n}\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y^{n+1}-y^{n}\Vert ^2-\langle K(x^{n+1}-x^{n}),y^{n+1}-y^{n}\rangle \right. }\nonumber \\&\qquad \left. -\dfrac{L_f}{2}\,\Vert x^{n+1}-x^n\Vert ^2 \right] . \end{aligned}$$

(23)

Owing to hypothesis (H) according to which $\left( \frac{1}{\tau }-L_f\right) \frac{1}{\sigma } >\Vert K\Vert ^2$, the quantities in brackets are non-negative. To show this, let us focus on the last quantity in brackets. The same reasoning applies to the two former ones. Using Cauchy–Schwarz inequality combined with Young’s inequality with parameter $\varepsilon >0$, one has $\langle K(x^{n+1}-x^n),y^{n+1}-y^n\rangle \le \Vert K\Vert \,\Vert x^{n+1}-x^n\Vert \,\Vert y^{n+1}-y^n\Vert \le \dfrac{\Vert K\Vert }{2\varepsilon }\Vert x^{n+1}-x^n\Vert ^2+\dfrac{\Vert K\Vert \varepsilon }{2}\Vert y^{n+1}-y^n\Vert ^2$, so that setting $\varepsilon =\frac{1}{{\sqrt{\sigma \,\left( \frac{1}{\tau }-L_f\right) }}}$ leads to:

$$\begin{aligned}&\left( \dfrac{1}{2\tau }-\dfrac{L_f}{2}\right) \,\Vert x^{n+1}-x^n\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y^{n+1}-y^n\Vert ^2\\&\qquad -\langle K(x^{n+1}-x^n),y^{n+1}-y^n\rangle \\&\quad \ge \left( \dfrac{1}{2\tau }-\dfrac{L_f}{2}-{\sqrt{\sigma \,\left( \frac{1}{\tau }-L_f\right) }}\,\dfrac{\Vert K\Vert }{2}\right) \,\Vert x^{n+1}-x^n\Vert ^2 \\&\qquad +\left( \dfrac{1}{2\sigma }-\frac{\Vert K\Vert }{{2\sqrt{\sigma \,\left( \frac{1}{\tau }-L_f\right) }}}\right) \,\Vert y^{n+1}-y^n\Vert ^2. \end{aligned}$$

Hypothesis (H) enables one to conclude that the weights balancing $\Vert x^{n+1}-x^n\Vert ^2$ and $\Vert y^{n+1}-y^n\Vert ^2$ are positive, or equivalently that there exists $\xi >0$ so that

$$\begin{aligned}&\left( \dfrac{1}{2\tau }-\dfrac{L_f}{2}\right) \,\Vert x^{n+1}-x^n\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y^{n+1}-y^n\Vert ^2\nonumber \\&\quad -\langle K(x^{n+1}-x^n),y^{n+1}-y^n\rangle \ge \xi \,\left( \Vert x^{n+1}-x^n\Vert ^2\right. \nonumber \\&\quad \left. +\,\Vert y^{n+1}-y^n\Vert ^2\right) . \end{aligned}$$

(24)

An immediate consequence is that equation (23) reduces to

$$\begin{aligned}&{\mathcal {L}}(x^{n+1},y)-{\mathcal {L}}(x,y^{n+1}) \nonumber \\&\quad \le \left[ \dfrac{1}{2\tau }\,\Vert x-x^n\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y-y^n\Vert ^2-\langle K(x-x^n),y-y^n\rangle \right] \nonumber \\&\qquad -\scriptstyle {\left[ \dfrac{1}{2\tau }\,\Vert x-x^{n+1}\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y-y^{n+1}\Vert ^2-\langle K(x-x^{n+1}),y-y^{n+1}\rangle \right] }. \end{aligned}$$

(25)

By taking as particular (x, y) a saddle point $(x^{*},y^{*})$ of the Lagrangian ${\mathcal {L}}$ (whose existence is ensured in our case by Theorem 3), entailing, by definition of a saddle point, that ${\mathcal {L}}(x^{n+1},y^{*})-{\mathcal {L}}(x^{*},y^{n+1})\ge 0$, inequality (25) gives:

$$\begin{aligned}&\scriptstyle {\dfrac{1}{2\tau }\,\Vert x^{*}-x^{n+1}\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y^{*}-y^{n+1}\Vert ^2-\langle K(x^{*}-x^{n+1}),y^{*}-y^{n+1}\rangle }\\&\quad \scriptstyle {-\dfrac{1}{2\tau }\,\Vert x^{*}-x^n\Vert ^2-\dfrac{1}{2\sigma }\,\Vert y^{*}-y^n\Vert ^2+\langle K(x^{*}-x^n),y^{*}-y^n\rangle \le 0}. \end{aligned}$$

By summing from $n=0$ to $N-1$, one gets:

$$\begin{aligned}&\scriptstyle {\dfrac{1}{2\tau }\,\Vert x^{*}-x^{N}\Vert ^2-\dfrac{1}{2\tau }\,\Vert x^{*}-x^{0}\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y^{*}-y^{N}\Vert ^2-\dfrac{1}{2\sigma }\,\Vert y^{*}-y^{0}\Vert ^2}\\&\quad \scriptstyle {+\langle K(x^{*}-x^0),y^{*}-y^0\rangle -\langle K(x^{*}-x^{N}),y^{*}-y^{N}\rangle \le 0}, \end{aligned}$$

and using an estimate as in (24), it follows, still using Cauchy–Schwarz inequality and Young’s inequality, that

$$\begin{aligned}&\xi \,\left( \Vert x^{*}-x^{N}\Vert ^2+\Vert y^{*}-y^{N}\Vert ^2\right) \\&\quad \le \left( \dfrac{1}{2\tau }+\dfrac{\Vert K\Vert }{2}\right) \Vert x^{*}-x^{0}\Vert ^2+\left( \dfrac{1}{2\sigma }+\dfrac{\Vert K\Vert }{2}\right) \,\Vert y^{*}-y^{0}\Vert ^2, \end{aligned}$$

showing that the sequence $(x^n,y^n)$ is a bounded sequence. One can thus extract a subsequence $(x^{\Psi (n)},y^{\Psi (n)})$ that (strongly) converges to $(\hat{{\hat{x}}},\hat{{\hat{y}}})$ (since we work in finite dimension). Let us now come back to inequality (23). Proceeding as before with $(x,y)=(x^{*},y^{*})$ and summing the inequalities from $n=0$ to $N-1$ coupled with estimation (24) shows that

$$\begin{aligned}&\xi \,\left( \displaystyle {\sum _{n=0}^{N-1}}\,\Vert x^{n+1}-x^n\Vert ^2+\displaystyle {\sum _{n=0}^{N-1}}\,\Vert y^{n+1}-y^n\Vert ^2\right) \nonumber \\&\quad \le \dfrac{1}{2\tau }\,\Vert x^{*}-x^0\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y^{*}-y^0\Vert ^2-\langle K(x^{*}-x^0),y^{*}-y^0\rangle \nonumber \\&\qquad -\scriptstyle {\dfrac{1}{2\tau }\,\Vert x^{*}-x^N\Vert ^2-\dfrac{1}{2\sigma }\,\Vert y^{*}-y^N\Vert ^2+\langle K(x^{*}-x^N),y^{*}-y^N\rangle }, \end{aligned}$$

(26)

the latter line containing a negative quantity according again to hypothesis (H). Thus

$$\begin{aligned}&\xi \,\left( \displaystyle {\sum _{n=0}^{N-1}}\,\Vert x^{n+1}-x^n\Vert ^2+\displaystyle {\sum _{n=0}^{N-1}}\,\Vert y^{n+1}-y^n\Vert ^2\right) \nonumber \\&\quad \le \dfrac{1}{2\tau }\,\Vert x^{*}-x^0\Vert ^2+\dfrac{1}{2\sigma }\,\Vert y^{*}\\&\qquad -y^0\Vert ^2-\langle K(x^{*}-x^0),y^{*}-y^0\rangle . \end{aligned}$$

The sequence $({\mathcal {S}}(N))_{N\in {\mathbb {N}}^{*}}$ with general term ${\mathcal {S}}(N)=$ $\displaystyle {\sum _{n=0}^{N-1}}\,\Vert x^{n+1}-x^n\Vert ^2$ is thus increasing, bounded above, so it converges and $\displaystyle {\lim _{n \rightarrow +\infty }}\,(x^{n+1}-x^n)=0$ (similarly, $\displaystyle {\lim _{n \rightarrow +\infty }}\,(y^{n+1}-y^n)=0$), implying that $(x^{\Psi (n)-1},y^{\Psi (n)-1})$ also converges to $(\hat{{\hat{x}}},\hat{{\hat{y}}})$ (take $n:=\Psi (n)-1$ in the previous result), which is thus a fixed point of Algorithm 3 iteration, hence a saddle point of the Lagrangian ${\mathcal {L}}$ from Appendix Appendix C.

For the last time, we come back to inequality (25) with $(x,y)=(\hat{{\hat{x}}},\hat{{\hat{y}}})$ and sum the inequalities from $n=\Psi (n)$ to $N-1$ with $N>\Psi (n)$. It yields

$$\begin{aligned}&\xi \,\left( \Vert \hat{{\hat{x}}}-x^N\Vert ^2+\Vert \hat{{\hat{y}}}-y^N\Vert ^2\right) \\&\quad \le \left( \dfrac{1}{2\tau }+\dfrac{\Vert K\Vert }{2}\right) \,\Vert \hat{{\hat{x}}}-x^{\Psi (n)}\Vert ^2\\&\qquad +\left( \dfrac{1}{2\sigma }+\dfrac{\Vert K\Vert }{2}\right) \,\Vert \hat{{\hat{y}}}-y^{\Psi (n)}\Vert ^2, \end{aligned}$$

which proves that $x^N \rightarrow \hat{{\hat{x}}}$ and $y^N \rightarrow \hat{{\hat{y}}}$ as N tends to $+\infty$.