Whiteness Constraints in a Unified Variational Framework for Image Restoration

Abstract

We propose a robust variational model for the restoration of images corrupted by blur and the general class of additive white noises. The key idea behind our proposal relies on a novel hard constraint imposed on the residual of the restoration, namely we characterize a residual whiteness set to which the restored image must belong. As the feasible set is unbounded, solution existence results for the proposed variational model are given. Moreover, based on theoretical derivations as well as on Monte Carlo simulations, we provide well-founded guidelines for setting the whiteness constraint limits. The solution of the non-trivial optimization problem, due to the non-smooth non-convex proposed model, is efficiently obtained by an alternating directions method of multipliers, which in particular reduces the solution to a sequence of convex optimization subproblems. Numerical results show the potentiality of the proposed model for restoring blurred images corrupted by several kinds of additive white noises.

Appendix

1.1 Proof of Lemma 1

Proof

First, we notice that the whiteness set $\mathcal {W}_{\alpha } \subset {{\mathbb {R}}}^{d^2}$ defined in (17)–(18) is given by the intersection of $\big ( d^2 - 1 \big )$ different sets, namely

$$\begin{aligned} \mathcal {W}_{\alpha } \,\;{=}\; \bigcap _{[l,m] \in \overline{{\Theta }}_0} \, \mathcal {W}_{\alpha }^{(l,m)} \, , \end{aligned}$$

(58)

where $\overline{{\Theta }}_0 \subset \mathbb {Z}^2$ is defined in (8) and

$$\begin{aligned} \mathcal {W}_{\alpha }^{(l,m)}= & {} \Big \{ u \in {{\mathbb {R}}}^{d^2} : \,\,\, -w_{\alpha } d^2 \nonumber \\&\le (Ku-g)^\mathrm{T} \,\, \overline{S}^{(l,m)} \, (Ku-g) \,\;{\le }\, +w_{\alpha } d^2 \Big \} \, . \end{aligned}$$

(59)

Any matrix $\,\overline{S}^{(l,m)} \in {{\mathbb {R}}}^{d^2 \times d^2}$ in (59) is given by the symmetric part of the matrix $\,S^{(l,m)} \in {{\mathbb {R}}}^{d^2 \times d^2}$ representing the 2D circular (l, m)-shift operator, that is, the operator which circularly shifts the elements of a $d \times d$ matrix of l rows and m columns. It is easy to verify that matrices $\overline{S}^{(l,m)}$ are all indefinite; hence, the associated sets $\mathcal {W}_{\alpha }^{(l,m)}$ in (59) are closed and can be non-convex. It clearly follows that the whiteness set $\mathcal {W}_{\alpha }$ in (58) is closed (since intersection of closed sets) and can be non-convex.

According to Definition 5, in order to prove that $\mathcal {W}_{\alpha }$ is unbounded it is sufficient to demonstrate that there exists an unbounded sequence $\big \{u^{(k)}\big \} \subset \mathcal {W}_{\alpha }$. After recalling that the blur matrix K can be strongly ill-conditioned or even numerically singular but in purely mathematical sense it is invertible, we consider the following sequences:

$$\begin{aligned}&\big \{u^{(k)}\big \} := \big \{ K^{-1} \big ( g + s^{(k)} \big ) \big \} \, , \quad \nonumber \\&\big \{s^{(k)}\big \} := \big \{ \nu ^{(k)} e_m \big \} \, , \quad \big \{\nu ^{(k)}\big \} \subset {{\mathbb {R}}}\; \mathrm {unbounded}. \end{aligned}$$

(60)

Such sequences are unbounded, in fact

$$\begin{aligned} \lim _{k \rightarrow +\infty } \big \Vert u^{(k)} \big \Vert _2= & {} \lim _{k \rightarrow +\infty } \big \Vert K^{-1} \big ( g + s^{(k)} \big ) \big \Vert _2 \nonumber \\= & {} \lim _{k \rightarrow +\infty } \big \Vert K^{-1} g + \nu ^{(k)} K^{-1} e_m \big \} \big \Vert _2 \nonumber \\= & {} \lim _{k \rightarrow +\infty } \left\| \nu ^{(k)} \left( \frac{1}{\nu ^{(k)}} \, K^{-1} g \;{+}\; K^{-1} e_m \right) \right\| _2 \nonumber \\= & {} \lim _{k \rightarrow +\infty } \left( \big | \nu ^{(k)} \big | \, \big \Vert K^{-1} e_m \big \Vert _2 \right) \nonumber \\= & {} \big \Vert K^{-1} e_m \big \Vert _2 \, \lim _{k \rightarrow +\infty } \big | \nu ^{(k)} \big | \, = \, + \infty . \nonumber \\ \, \end{aligned}$$

(61)

To prove that the unbounded sequences in (60) belong to the whiteness set in (17)–(18), we derive the expression of $r_{Ku^{(k)}-g}$, that is the sample autocorrelation of the residue image associated with the generic term of the sequences:

$$\begin{aligned} r_{Ku^{(k)}-g}= & {} \frac{1}{d^2} \, \big ( Ku^{(k)}-g \big ) \,{\star }\, \big ( Ku^{(k)}-g \big ) \;{=}\; \frac{1}{d^2} \, s^{(k)} \,{\star }\, s^{(k)} \\= & {} \frac{1}{d^2} \, \big ( \nu ^{(k)} e_m \big ) \,{\star }\, \big ( \nu ^{(k)} e_m \big ) \;{=}\; \frac{\big ( \nu ^{(k)} \big )^2}{d^2} \, e_m \,{\star }\, e_m \\= & {} \frac{\big ( \nu ^{(k)} \big )^2}{d^2} \,\, \mathrm {mat}(e_1) \, , \end{aligned}$$

that is,

$$\begin{aligned} r_{Ku^{(k)}-g}[l,m] \; = \left\{ \begin{array}{ll} \big ( \nu ^{(k)} \big )^2 / d^2 &{} \;\;\mathrm {for} \;\; [l,m] = [0,0] \\ 0 &{} \;\;\mathrm {for} \;\; [l,m] \in \, \overline{{\Theta }}_0 \end{array} \right. \quad \forall \, k \in \mathbb {N} \, . \end{aligned}$$

(62)

It follows from (61), (62) and (17)–(18) that any sequence $\big \{u^{(k)}\big \}$ defined as in (60) is unbounded and belongs to the whiteness set $\mathcal {W}_{\alpha }$ for any real $\alpha \ge 0$. This implies that $\mathcal {W}_{\alpha }$ is unbounded.

Finally, in order to demonstrate that $\mathcal {W}_{\alpha }$ in (17)–(18) is non-convex, it is sufficient to prove that, for any given real $\alpha \ge 0$, there always exist two images $u_{\alpha }, v_{\alpha } \in {{\mathbb {R}}}^{d^2}$ belonging to $\mathcal {W}_{\alpha }$ and a scalar $\gamma \in (0,1)$ such that the image $z_{\alpha } := \gamma \, u_{\alpha } + (1-\gamma ) \, v_{\alpha }$ does not belong to $\mathcal {W}_{\alpha }$. By taking

$$\begin{aligned} \left\{ \, \begin{array}{ll} u_{\alpha } &{}\;{:=}\; K^{-1} \big (\, g + 2 \, d \, \sqrt{\rho _{\alpha }} \,\, e_p \big ) \\ v_{\alpha } &{}\;{:=}\; K^{-1} \big (\, g + 2 \, d \, \sqrt{\rho _{\alpha }} \,\, e_q \big ) \end{array} \right. \quad \mathrm {with}\;\;\; \nonumber \\ p,q \in \{1,\ldots ,d^2\}, \; p \ne q, \quad \rho _{\alpha } > w_{\alpha }, \end{aligned}$$

(63)

and, according to the choice $\gamma = 1/2$,

$$\begin{aligned} z_{\alpha } \;{:=}\; \frac{1}{2} \, u_{\alpha } + \frac{1}{2} \, v_{\alpha } = K^{-1} \left( g + d \, \sqrt{\rho _{\alpha }} \,\, \left( e_p + e_q \right) \right) , \end{aligned}$$

(64)

we have

$$\begin{aligned} r_{K u_{\alpha } - g}[l,m] \;{=}\; r_{K v_{\alpha } - g}[l,m] \;{=}\; 0 \;\; \forall \, (l,m) \in \, \overline{{\Theta }}_0 \end{aligned}$$

(65)

and

$$\begin{aligned}&r_{K z_{\alpha }-g}[l,m] \nonumber \\&\quad = \frac{1}{d^2} \, \left( \big ( K z_{\alpha }-g \big ) \,{\star }\, \big ( K z_{\alpha }-g \big ) \right) [l,m] \nonumber \\&\quad = \frac{1}{d^2} \, \left( \left( d \, \sqrt{\rho _{\alpha }} \left( e_p + e_q \right) \right) \,{\star }\, \left( d \, \sqrt{\rho _{\alpha }} \left( e_p + e_q \right) \right) \right) [l,m] \nonumber \\&\quad = \rho _{\alpha } \, \left( \left( e_p + e_q \right) \,{\star }\, \left( e_p + e_q \right) \right) [l,m] \nonumber \\&\quad = \rho _{\alpha } \, \left( e_p \,{\star }\;\, e_p + e_q \,{\star }\;\, e_q + e_p \,{\star }\;\, e_q + e_q \,{\star }\;\, e_p \right) [l,m] \nonumber \\&\quad = \rho _{\alpha } \, \big ( \mathrm {mat}(e_1) + \mathrm {mat}(e_1) + e_p \,{\star }\;\, e_q + e_q \,{\star }\;\, e_p \big )[l,m]\nonumber \\&\quad = \left\{ \begin{array}{ll} 2 \, \rho _{\alpha } &{} \;\,\mathrm {for} \;\; [l,m] = [0,0] \\ \rho _{\alpha } &{} \;\,\mathrm {for} \;\; [l,m] \in \left\{ [\bar{l},\bar{m}],[-\bar{l},-\bar{m}]\right\} \\ &{}\quad \quad \;\,\mathrm {for}\;\,\mathrm {some}\;[\bar{l},\bar{m}] \,{\ne }\, [0,0] \\ 0 &{} \;\,\mathrm {otherwise} \end{array} \right. . \end{aligned}$$

(66)

Since $\rho _{\alpha } > w_{\alpha }$ by assumption, $z_{\alpha }$ does not belong to $\mathcal {W}_{\alpha }$, and the proof is completed. $\square$

1.2 Proof of Lemma 2

Proof

The fact that the discrete TV semi-norm function is proper, continuous, convex and bounded from below by zero is well known and immediate to verify. It is also well known that the TV function is not coercive over its entire domain ${{\mathbb {R}}}^{d^2}$. To prove that the TV function is coercive over the unbounded whiteness set $\mathcal {W}_{\alpha } \subset {{\mathbb {R}}}^{d^2}$ defined in (17)–(18), first we outline the set of all the unbounded sequences $\big \{u^{(k)}\big \} \subset {{\mathbb {R}}}^{d^2}$ for which the TV function is not coercive, that is for which $\displaystyle {\lim _{k \rightarrow \infty }} \mathrm {TV}\big (u^{(k)}\big ) < +\infty$, then we demonstrate that such sequences are not contained into $\mathcal {W}_{\alpha }$.

Let us define the matrix $D := (D_h^T,D_v^T)^\mathrm{T} \in {{\mathbb {R}}}^{2d^2 \times d^2}$ with $D_h,D_v \in {{\mathbb {R}}}^{d^2}$ the coefficient matrices of linear finite difference operators approximating the horizontal and vertical partial derivatives of image u, respectively. Then, the TV semi-norm of u defined in (2) can be regarded as the composition of a linear map $\mathcal {D}: {{\mathbb {R}}}^{d^2} \rightarrow {{\mathbb {R}}}^{2d^2}$ with coefficient matrix D and a suitable nonlinear function $\mathcal {G}: {{\mathbb {R}}}^{2d^2} \rightarrow {{\mathbb {R}}}$, that is

$$\begin{aligned} \mathrm {TV}(u)= & {} \mathcal {G}(\mathcal {D}(u)) \, , \quad \quad \mathcal {D}(u) \,:=\, Du, \nonumber \\ \mathcal {G}(v):= & {} \sum _{i=1}^{d^2} \left\| \left( v_i,v_{i+d^2} \right) \right\| _2 \, . \end{aligned}$$

(67)

We now study coerciveness of the function $\mathcal {G}$ above. As it is immediate to verify that

$$\begin{aligned} \mathcal {G}(v)= & {} \sum _{i=1}^{d^2} \left\| \left( v_i,v_{i+d^2} \right) \right\| _2 \;{\ge }\;\, \big \Vert \left( v_1,v_2,\ldots ,v_{2d^2} \right) \big \Vert _2 \nonumber \\= & {} \Vert v \Vert _2 \, , \end{aligned}$$

(68)

for any unbounded sequence $\big \{v^{(k)}\big \} \subseteq {{\mathbb {R}}}^{2d^2}$—that is, according to Definition 4, any sequence satisfying $\big \Vert v^{(k)} \big \Vert _2 \xrightarrow {k \rightarrow \infty } +\infty$—we have that

$$\begin{aligned}&\lim _{\left\| v^{(k)} \right\| _2 \xrightarrow {k \rightarrow \infty } +\infty } \mathcal {G}\big (v^{(k)}\big ) \nonumber \\&\quad \ge \lim _{\left\| v^{(k)} \right\| _2 \xrightarrow {k \rightarrow \infty } +\infty } \big \Vert v^{(k)} \big \Vert _2 \;{=}\; +\infty \, , \end{aligned}$$

(69)

that is the function $\mathcal {G}$ is coercive over its entire domain ${{\mathbb {R}}}^{2d^2}$.

For what concerns the linear operator $\mathcal {D}$, clearly the kernel of the coefficient matrix D has dimension 1 and is given by

$$\begin{aligned} \mathrm {ker}(D) \;{=}\; \left\{ u \in {{\mathbb {R}}}^{d^2}: \; u = \nu \, {\mathbb {1}}, \;\, \nu \in {{\mathbb {R}}}\, \right\} \, . \end{aligned}$$

(70)

Since for any vector $u \in {{\mathbb {R}}}^{d^2}$ there always exists only one pair of vectors $u_1 \in \mathrm {ker}(D)$, $u_2 \in (\mathrm {ker}(D))^{\perp }$ such that $u = u_1 + u_2$, then any unbounded sequence $\big \{u^{(k)}\big \} \subseteq {{\mathbb {R}}}^{d^2}$ can be additively split as follows

$$\begin{aligned} \big \{u^{(k)}\big \}= & {} \big \{u_1^{(k)}\big \} + \big \{u_2^{(k)}\big \}, \quad \big \{u_1^{(k)}\big \} \subseteq \mathrm {ker}(D), \nonumber \\ \big \{u_2^{(k)}\big \}\subseteq & {} \left( \mathrm {ker}(D)\right) ^{\perp } \, , \end{aligned}$$

(71)

where either $\big \{u_1^{(k)}\big \}$ or $\big \{u_2^{(k)}\big \}$ is unbounded. In case that $\big \{u_2^{(k)}\big \}$ is unbounded, then clearly $\big \{D u_2^{(k)}\big \}$ is also unbounded and, due to coerciveness of the function $\mathcal {G}$, $\mathrm {TV}\big ( u_2^{(k)} \big ) = \mathcal {G}\big (D u_2^{(k)}\big )$ tends to $\infty$. On the other hand, any unbounded $\big \{u_1^{(k)}\big \}$ is mapped by D into a null (bounded) sequence. It follows that all the unbounded sequences for which the TV function is not coercive are of the form

$$\begin{aligned} \big \{u^{(k)}\big \} \;{=}\; \big \{ u_2^{(k)} \;{+}\; \nu ^{(k)} \, {\mathbb {1}} \big \} \, , \end{aligned}$$

(72)

with $\big \{u_2^{(k)}\big \} \subset {{\mathbb {R}}}^{d^2}$ any bounded sequence and $\big \{\nu ^{(k)}\big \} \subset {{\mathbb {R}}}$ any (scalar) unbounded sequence.

We now prove that no unbounded sequence of the form (72) belongs to the whiteness set $\mathcal {W}_{\alpha }$ in (17)–(18), for any real $\alpha \ge 0$. According to Definition (18) of the residual sample autocorrelation, we have:

$$\begin{aligned} r_{Ku^{(k)}-g}= & {} \frac{1}{d^2} \, \big ( Ku^{(k)}-g \big ) \,{\star }\, \big ( Ku^{(k)}-g \big ) \nonumber \\= & {} \frac{1}{d^2} \, \Big ( K \big ( u_1^{(k)} \;{+}\; \nu ^{(k)} \, {\mathbb {1}} \big ) - g \Big ) \,{\star }\, \Big ( K \big ( u_1^{(k)}\nonumber \\&+\, \nu ^{(k)} \, {\mathbb {1}} \big ) - g \Big ) \nonumber \\= & {} \frac{1}{d^2} \, \big ( \nu ^{(k)} K {\mathbb {1}} + K u_1^{(k)} - g \big ) \,{\star }\, \big ( \nu ^{(k)} K {\mathbb {1}} \nonumber \\&+\, K u_1^{(k)} - g \big ) \, . \end{aligned}$$

(73)

Since the blur PSF is typically energy-preserving, which implies that the sum of the elements of each row of the blur matrix K is equal to one, then $K {\mathbb {1}} = {\mathbb {1}}$. Moreover, since K is bounded, the sequence $c^{(k)} := K u_1^{(k)} - g$ is bounded. From (73), we have

$$\begin{aligned}&r_{Ku^{(k)}-g}[l,m] \nonumber \\&\quad = \frac{1}{d^2} \, \Big ( \big ( \nu ^{(k)} {\mathbb {1}} + c^{(k)} \big ) \,{\star }\, \big ( \nu ^{(k)} {\mathbb {1}} + c^{(k)} \big ) \Big )[l,m] \nonumber \\&\quad = \frac{1}{d^2} \left( \big (\nu ^{(k)}\big )^2 \, {\mathbb {1}} \,{\star }\, {\mathbb {1}} + \nu ^{(k)} \, {\mathbb {1}} \,{\star }\, c^{(k)} + \nu ^{(k)} \, c^{(k)} \,{\star }\, {\mathbb {1}}\right. \nonumber \\&\qquad \left. +\, c^{(k)} \,{\star }\; c^{(k)} \right) [l,m] \nonumber \\&\quad = \frac{1}{d^2} \left( \big (\nu ^{(k)}\big )^2 d^2 \, \mathrm {mat}({\mathbb {1}}) \,+ 2 \, \nu ^{(k)} \left( \sum _{i=1}^{d^2} c^{(k)}_i \right) \mathrm {mat}({\mathbb {1}}) \right. \nonumber \\&\qquad \left. +\, c^{(k)} \,{\star }\, c^{(k)} \right) [l,m] \nonumber \\&\quad = \frac{1}{d^2} \left( \big (\nu ^{(k)}\big )^2 d^2 \,+ 2 \, \nu ^{(k)} \left( \sum _{i=1}^{d^2} c^{(k)}_i \right) + \left( c^{(k)} \,{\star }\, c^{(k)} \right) [l,m] \right) \nonumber \\&\quad = \big (\nu ^{(k)}\big )^{2} \left( 1 \,{+}\, \frac{2}{d^2 \nu ^{(k)}} \left( \sum _{i=1}^{d^2} c^{(k)}_i \right) +\, \frac{\left( c^{(k)} \,{\star }\, c^{(k)} \right) [l,m]}{d^2 \big (\nu ^{(k)}\big )^2} \right) . \end{aligned}$$

(74)

As the sequence $c^{(k)} \subset {{\mathbb {R}}}^{d^2}$ is bounded and the sequence $\nu ^{(k)} \subset {{\mathbb {R}}}$ is unbounded, the second and third terms within parentheses in (74) both represent bounded sequences in ${{\mathbb {R}}}$ (more precisely, they both tend to zero as k approaches $+\infty$). Hence, we have that

$$\begin{aligned} \lim _{k \rightarrow +\infty } r_{Ku^{(k)}-g}[l,m]= & {} \lim _{k \rightarrow +\infty } \big (\nu ^{(k)}\big )^2 \nonumber \\= & {} + \infty \quad \forall \, [l,m] \in \, \overline{{\Theta }}_0 . \end{aligned}$$

(75)

It thus follows from (75) that unbounded sequences of the form (72) do not belong to the whiteness set $\mathcal {W}_{\alpha }$ in (17) for any real $\alpha \ge 0$, at least for k greater than a certain value. This implies coercivity of the TV function over $\mathcal {W}_{\alpha }$ and concludes the proof. $\square$