Image Completion and Blind Deconvolution: Model and Algorithm

Abstract

In this paper, we study a model for recovering edges in an underlying image from a single blurred image whose entries are only partially known on randomly distributed indices. In the proposed model, blurred image, the underlying image and convolution kernel are all unknowns to be solved. Besides the classical convolution-type data fitting term for image deblurring, our model incorporates nuclear norm prior for blurred image, a total variation (TV) regularization prior for recovering edges, and Tikhonov regularization prior for the blur kernel. We develop a proximal alternating minimization (PAM) iterative method to solve the model and establish its convergence. Efficient implementations are proposed for solving the subproblems arising from PAM iterations. Numerical results are reported to show the performance of our proposed approach is better than the method using TV regularization prior on the blur kernel.

Appendices

Appendix A: Some Preliminaries for Proving Theorem 3.1

Definition A.1

([22]) Let V be a finite dimensional real vector space and let $\phi :V\rightarrow {\mathbb {R}}^{n}$ be a linear isomorphism.

• (a) A subset $A\subset {\mathbb {R}}^n$ is called a semi-algebraic set if and only if there exists a finite number of real polynomial functions $P_{ij}, Q_{ij}:{\mathbb {R}}^n\rightarrow {\mathbb {R}}$ such that

  $$\begin{aligned} A=\bigcup \limits _{j=1}^{p}\bigcap \limits _{i=1}^{q}\{x\in {\mathbb {R}}^n|P_{ij}(x)=0,\quad Q_{ij}(x)<0\}. \end{aligned}$$

• (b) A subset $B\subset V$ is said to be semi-algebraic if and only if $\phi (B)$ is a semi-algebraic set in ${\mathbb {R}}^{n}$.

• (c) A function $f:A_1\subset {\mathbb {R}}^n\rightarrow A_2\subset {\mathbb {R}}^m$ is said to be semi-algebraic if and only if its graph $\{(x,f(x))\in {\mathbb {R}}^{n+m}|x\in A_1\}$ is a semi-algebraic set in ${\mathbb {R}}^{n+m}$.

• (d) A function $f:B_1\subset V\rightarrow B_2\subset {\mathbb {R}}^{m}$ is said to be semi-algebraic if and only if the composition $f\circ \phi ^{-1}:\phi (B_1)\rightarrow B_2$ is semi-algebraic.

We call a function f is semi-algebraic on A if and only if $\{(x,f(x))|x\in A\}$ is a semi-algebraic set.

Lemma A.1

([7, 8, 23]) A semi-algebraic real valued function f is a KL function, i.e., f satisfies Kurdyka-ojasiewicz property at each $x\in \mathrm{dom}(f)$.

Proposition A.1

([4, 6, 15, 22, 42])

• (i) A subset of ${\mathbb {R}}$ is semi-algebraic if and only if it is finite union of points and open intervals (bounded or unbounded).

• (ii) Complements, finite unions, finite intersections, finite Cartesian products of semi-algebraic sets are semi-algebraic.

• (iii) Finite sums and finite products of semi-algebraic functions are semi-algebraic.

• (iv) The composition $g\circ f$ of semi-algebraic mappings $f:A_1\rightarrow A_2$ and $g:A_2\rightarrow A_3$ is semi-algebraic.

• (v) If $f:A_1\rightarrow A_2$ is semi-algebraic and $A_3\subset A_1$ is a semi-algebraic subset, then $f(A_3)$ is a semi-algebraic set.

• (vi) Polynomials defined on ${\mathbb {R}}^n$ are semi-algebraic functions.

• (vii) The absolute value function $f(x):=|x|$ is semi-algebraic on ${\mathbb {R}}$.

• (viii) The nuclear norm function $f_*:w\in {\mathbb {R}}^{M_1}\mapsto ||{\mathbb {V}}_1^{-1}(w)||_{*}\in [0,+\infty )$ is semi-algebraic.

• (IX) If $f_{i}:A\rightarrow {\mathbb {R}}$ is semi-algebraic for each $i=1,2,\ldots ,n$, then the function $f:x\in A\mapsto (f_1(x);f_2(x);\cdots ;f_n(x))\in {\mathbb {R}}^{n}$ is semi-algebraic.

Proposition A.2

If $f:A_1\rightarrow A_2$ is semi-algebraic and $A_3\subset A_1$ is a semi-algebraic subset, then f is semi-algebraic on $A_3$.

Proof

By Proposition A.1$\mathbf{(ii)}$, $\mathbf{(v)}$, $A_3\times f(A_3)$ is semi-algebraic. Notice that

$$\begin{aligned} \{(x,f(x))|x\in A_3\}=\{(x,f(x))|x\in A_1\}\cap (A_3\times f(A_3)), \end{aligned}$$

which together with Proposition A.1$\mathbf{(ii)}$ proves the proposition. $\square$

For positive integer n, define a coordinate projection $\Pi _{n+1}^{n}:{\mathbb {R}}^{n+1}\rightarrow {\mathbb {R}}^{n}$ such that

$$\begin{aligned} \Pi _{n+1}^{n}(\mathbf{w}):=(w_1;w_2;\ldots ;w_n)\in {\mathbb {R}}^{n} \mathrm{~for~each~}\mathbf{w}=(w_1;w_2;\ldots ;w_n;w_{n+1})\in {\mathbb {R}}^{n+1}. \end{aligned}$$

Lemma A.2

([18, 37]) If A is a semi-algebraic set in ${\mathbb {R}}^{n+1}$, then $\Pi _{n+1}^{n}(A)$ is a semi-algebraic set in ${\mathbb {R}}^{n}$.

Lemma A.3

${\tilde{F}}$ is semi-algebraic on ${\hat{S}}$. Hence, ${\tilde{F}}$ satisfies the Kurdyka-ojasiewicz property at each $v\in {\hat{S}}$.

Proof

On ${\hat{S}}$, ${\tilde{F}}$ can be expressed as

$$\begin{aligned} {\tilde{F}}(v):=G(v)+g_1(v_1)+g_3(v_3),\quad v\in {\hat{S}}. \end{aligned}$$

From the definition of ${\hat{S}}$, it is clear that

$$\begin{aligned} {\hat{S}}={\hat{S}}_1\times {\hat{S}}_2\times {\hat{S}}_3, \end{aligned}$$

where

$$\begin{aligned}&{\hat{S}}_1=\left\{ v_1\in {\mathbb {R}}^{M_1}\Big | v_1(i)=[{\mathbb {V}}_1(\mathbf{M})](i)\mathrm{~for~}i\in {\mathbb {I}}_1(\Omega ),~ v_1(i)\in [0,1]\mathrm{~for~}i\notin {\mathbb {I}}_1(\Omega )\right\} , \end{aligned}$$

(A.1)

$$\begin{aligned}&{\hat{S}}_2=\{v_2\in {\mathbb {R}}^{M_2}|v_2(i)\ge 0\mathrm{~for~each~}i,\quad \mathbf{1}^{\mathrm{T}}v_2=1\}, \end{aligned}$$

(A.2)

$$\begin{aligned}&{\hat{S}}_3=\{v_3\in {\mathbb {R}}^{M_3}|v_3(i)\in [0,1] \mathrm{~for~each~}i\}. \end{aligned}$$

(A.3)

By Proposition A.1$\mathbf{(i)}$, $\mathbf{(ii)}$, it is clear that ${\hat{S}}_1$ and ${\hat{S}}_3$ are both semi-algebraic sets.

Denote ${\mathbb {R}}_{+}^{M_2}=[0,+\infty )^{M_2}$. Define a linear polynomial $\theta :{\mathbb {R}}^{M_2}\rightarrow {\mathbb {R}}$ such that

$$\begin{aligned} \theta (w):=\mathbf{1}^{\mathrm{T}}w,\quad w\in {\mathbb {R}}^{M_2}. \end{aligned}$$

Then, it is easy to verify that ${\hat{S}}_2=\Pi _{M_2+1}^{M_2}({\hat{S}}_{2,1}\cap {\hat{S}}_{2,2})$, where

$$\begin{aligned} {\hat{S}}_{2,1}=\{(w,\theta (w))|w\in {\mathbb {R}}_{+}^{M_2}\},\quad {\hat{S}}_{2,2}={\mathbb {R}}_{+}^{M_2}\times (-\infty ,1]. \end{aligned}$$

Notice that ${\mathbb {V}}_i^{-1}$ ($i=1,2,3$) are linear mappings between finite-dimension spaces. Hence, each element of ${\mathbb {V}}_i^{-1}(v_i)$ ($i=1,2,3$) is actually a linear polynomial of $v:=(v_1;v_2;v_3)$. Additionally, by the definition of $||\cdot ||_{F}$ and the bi-linearity of ‘$\otimes$’, we know that G(v) is a polynomial of v. Moreover, $D_{1}$ and $D_{2}$ are linear mappings between finite-dimension spaces. Hence, each element of $D_{1}(v_3)$ and $D_{2}(v_3)$ can be regarded as a linear polynomial of $v:=(v_1;v_2;v_3)$. Therefore, $g_3(v_3)$ is simply finite sums of compositions of absolute value function and linear polynomial of v. Define $G_3(v):=G(v)+g_3(v_3)$ for any $v=(v_1;v_2;v_3)\in {\mathbb {R}}^{M}$ with $v_3\in {\mathbb {R}}^{M_3}$. To summarize, $G_3$ is semi-algebraic on ${\mathbb {R}}^{M}$ by Proposition A.1$\mathbf{(iii)}$, $\mathbf{(iv)}$, $\mathbf{(vi)}$, $\mathbf{(vii)}$.

For any $v=(v_1;v_2;v_3)\in {\mathbb {R}}^{M}$ with $v_i\in {\mathbb {R}}^{M_i}~(i=1,2,3)$, define

$$\begin{aligned} {\mathbb {M}}(v):=(m_1(v);m_2(v);\cdots ;m_1(v))\in {\mathbb {R}}^{M_1}\mathrm{~such~that~}m_i(v)=v_1(i),\quad i\in \Xi (M_1). \end{aligned}$$

Then, $m_{i}(v)$ is a linear polynomial of v for each $i\in \Xi (M_1).$ Thus, by Proposition A.1$\mathbf{(vi)}$, $\mathbf{(IX)}$, ${\mathbb {M}}:{\mathbb {R}}^{M}\rightarrow {\mathbb {R}}^{M_1}$ is semi-algebraic. Moreover, by Proposition A.1$\mathbf{(viii)}$, the nuclear norm function $f_*$ is semi-algebraic. The composition $f_*\circ {\mathbb {M}}:{\mathbb {R}}^{M}\rightarrow [0,+\infty )$ is semi-algebraic by Proposition A.1$\mathbf{(iv)}$. Notice that for any $v=(v_1;v_2;v_3)\in {\mathbb {R}}^{M}$ with $v_1\in {\mathbb {R}}^{M_1}$, $g_1(v_1)=\mu (f_{*}\circ {\mathbb {M}})(v)$. In other words, ${\tilde{F}}(v)=G_3(v)+\mu (f_{*}\circ {\mathbb {M}})(v)$. Hence, ${\tilde{F}}$ as summation of two semi-algebraic functions is semi-algebraic on ${\mathbb {R}}^{M}$.

By Proposition A.1$\mathbf{(i)}$, $\mathbf{(ii)}$, ${\hat{S}}_{2,2}$ is a semi-algebraic set in ${\mathbb {R}}^{M_2+1}$. $\theta :{\mathbb {R}}^{M_2}\rightarrow {\mathbb {R}}$ as a linear polynomial is semi-algebraic. And ${\mathbb {R}}_{+}^{M_2}\subset {\mathbb {R}}^{M_2}$ is a semi-algebraic subset. Then, by Proposition A.2, $\theta$ is semi-algebraic on ${\mathbb {R}}_{+}^{M_2}$, i.e., ${\hat{S}}_{2,1}$ is a semi-algebraic subset in ${\mathbb {R}}^{M_2+1}$. Then, by Proposition A.1$\mathbf{(ii)}$, ${\hat{S}}_{2,1}\cap {\hat{S}}_{2,2}$ is semi-algebraic. Hence, from Lemma A.2, ${\hat{S}}_2$ is semi-algebraic set. Hence, ${\hat{S}}$ as Cartesian products of ${\hat{S}}_i~(i=1,2,3)$ is semi-algebraic.

Then, by Proposition A.2, ${\tilde{F}}$ is semi-algebraic on ${\hat{S}}$. The proof is complete. $\square$

Appendix B: Proof of Theorem 3.1

Proof

For two subsets $A_1$, $A_2$ and an element x of the same vector space, denote

$$\begin{aligned} A_1+A_2:=\{x_1+x_2|x_1\in A_1,~x_2\in A_2\},\quad x+A_1=A_1+x:=\{x+y|y\in A_1\}. \end{aligned}$$

It has been mentioned above that ${\tilde{F}}$ is a PLSC function on ${\mathbb {R}}^{M}$. From (3.2)–(3.4), we see that

$$\begin{aligned}&{\tilde{F}}(v_1^{i+1};v_2^{i};v_3^{i})+\frac{\rho _1}{2}||v_1^{i+1}-v_1^i||_2^2\le {\tilde{F}}(v_1^{i};v_2^{i};v_3^{i}),\quad i\in {\mathbb {N}},\\&{\tilde{F}}(v_1^{i+1};v_2^{i+1};v_3^{i})+\frac{\rho _2}{2}||v_2^{i+1}-v_2^{i}||_2^2\le {\tilde{F}}(v_1^{i+1};v_2^{i};v_3^{i}),\quad i\in {\mathbb {N}},\\&{\tilde{F}}(v_1^{i+1};v_2^{i+1};v_3^{i+1})+\frac{\rho _3}{2}||v_3^{i+1}-v_3^{i}||_2^2\le {\tilde{F}}(v_1^{i+1};v_2^{i+1};v_3^{i}),\quad i\in {\mathbb {N}}. \end{aligned}$$

Summing over the three inequalities above, we obtain

$$\begin{aligned} {\tilde{F}}(v^{i+1})+\frac{{\check{\rho }}}{2}||v^{i+1}-v^{i}||_2^2\le {\tilde{F}}(v^{i}),\quad i\in {\mathbb {N}}, \end{aligned}$$

(B.1)

where ${\check{\rho }}=\min \{\rho _1,\rho _2,\rho _3\}$. Hence, $\mathbf{H1}$ (sufficient decrease condition) is satisfied with $a=\frac{{\check{\rho }}}{2}$.

Since ${\hat{S}}$ is a bounded set and it is clear that ${v^i}_{i\in {\mathbb {N}}}\subset {\hat{S}}$, ${v^i}_{i\in {\mathbb {N}}}$ is bounded, i.e.,

$$\begin{aligned} \sup \limits _{i\in {\mathbb {N}}}||v^i||_2<+\infty . \end{aligned}$$

Let $\partial _{v_i}$ and $\nabla _{v_i}$ denote sub-differential and gradient with respect to variable $v_i$, respectively, for $i=1,2,3$. It has been mentioned in proof of Lemma A.3 that G is a polynomial function and thus is infinitely differentiable. Then, (3.2) and Proposition 3.1 imply that

$$\begin{aligned} 0&\in \partial _{v_1}\Big [{\tilde{F}}(\cdot ;v_2^i;v_3^i)+\frac{\rho _1}{2}||\cdot -v_1^{i}||_2^2\Big ]\Big |_{v_1=v_1^{i+1}}\nonumber \\&=\partial _{v_1}{\tilde{F}}(v_1^{i+1};v_2^i;v_3^i)+\rho _1(v_1^{i+1}-v_1^{i})\nonumber \\&=\nabla _{v_1}G(v_1^{i+1};v_2^i;v_3^i)+\partial _{v_1}\delta _{{\hat{S}}}(v_1^{i+1};v_2^i;v_3^i)+\partial g_1(v_1^{i+1})+\rho _1(v_1^{i+1}-v_1^{i}),\quad i\in {\mathbb {N}}. \end{aligned}$$

(B.2)

Similarly, (3.3)–(3.4) and Proposition 3.1 imply that

$$\begin{aligned}&0\in \nabla _{v_2}G(v_1^{i+1};v_2^{i+1};v_3^{i})+\partial _{v_2}\delta _{{\hat{S}}}(v_1^{i+1};v_2^{i+1};v_3^i)+\rho _2(v_2^{i+1}-v_2^{i}),\quad i\in {\mathbb {N}}, \end{aligned}$$

(B.3)

$$\begin{aligned}&0\in \nabla _{v_3}G(v_1^{i+1};v_2^{i+1};v_3^{i+1})+\partial g_3(v_3^{i+1})+\partial _{v_3}\delta _{{\hat{S}}}(v_1^{i+1};v_2^{i+1};v_3^{i+1})+\rho _3(v_3^{i+1}-v_3^{i}),\quad i\in {\mathbb {N}}. \end{aligned}$$

(B.4)

(B.2)–(B.4) imply that there exists $w_1^{i+1}\in \partial _{v_1}\delta _{{\hat{S}}}(v_1^{i+1};v_2^i;v_3^i)+\partial g_1(v_1^{i+1})$, $w_2^{i+1}\in \partial _{v_2}\delta _{{\hat{S}}}(v_1^{i+1};v_2^{i+1};v_3^i)$, $w_{3}^{i+1}\in \partial g_{3}(v_{3}^{i+1})+\partial _{v_3}\delta _{{\hat{S}}}(v_1^{i+1};v_2^{i+1};v_3^{i+1})$ such that

$$\begin{aligned} -{\tilde{w}}^{i+1}=u^{i}+D(v^{i+1}-v^{i}),\quad i\in {\mathbb {N}}, \end{aligned}$$

where

$$\begin{aligned}&{\tilde{w}}^{i+1}=(w_1^{i+1};w_2^{i+1};w_3^{i+1}),\\&u^{i}=(\nabla _{v_1}G(v_1^{i+1};v_2^i;v_3^i);\nabla _{v_2}G(v_1^{i+1};v_2^{i+1};v_3^{i});\nabla _{v_3}G(v_1^{i+1};v_2^{i+1};v_3^{i+1})),\\&D=\mathrm{blockdiag}(\rho _1I_{M_1},\rho _2I_{M_2},\rho _3I_{M_3}), \end{aligned}$$

$I_k$ denotes $k\times k$ identity matrix. Denote $w^{i+1}={\tilde{w}}^{i+1}+\nabla G(v^{i+1})$. Notice that

$$\begin{aligned} \delta _{{\hat{S}}}(v_1,v_2,v_3)=\sum \limits _{j=1}^{3}\delta _{{\hat{S}}_j}(v_j), \end{aligned}$$

with ${\hat{S}}_j~(j=1,2,3)$ defined in (A.1)–(A.3). Moreover, since $v_{j}^{i}\in {\hat{S}}_{j}$ for all $i\in {\mathbb {N}}$ and $j=1,2,3$, we have

$$\begin{aligned}&\partial _{v_1}\delta _{{\hat{S}}}(v_1^{i+1};v_2^i;v_3^i)=[\partial \delta _{{\hat{S}}_1}(v_1^{i+1});\mathbf{0}_{M_2};\mathbf{0}_{M_3}],\quad \partial _{v_2}\delta _{{\hat{S}}}(v_1^{i+1};v_2^{i+1};v_3^i)=[\mathbf{0}_{M_1};\partial \delta _{{\hat{S}}_2}(v_2^{i+1});\mathbf{0}_{M_3}],\\&\partial _{v_3}\delta _{{\hat{S}}}(v_1^{i+1};v_2^{i+1};v_3^{i+1})=[\mathbf{0}_{M_1};\mathbf{0}_{M_2};\partial \delta _{{\hat{S}}_3}(v_3^{i+1})], \end{aligned}$$

where $\mathbf{0}_{m}$ denotes $m\times 1$ zero vector. Then, it is clear that $w^{i+1}\in \partial {\tilde{F}}(v^{i+1})$ and

$$\begin{aligned} -w^{i+1}=u^{i}-\nabla G(v^{i+1})+D(v^{i+1}-v^{i}),\quad i\in {\mathbb {N}}. \end{aligned}$$

(B.5)

Denote $E=\{v^{i}|i\in {\mathbb {N}}\}$. For $v=(v_1;v_2;v_3)\in {\mathbb {R}}^{M}$ with $v_i\in {\mathbb {R}}^{M_i}$ ($i=1,2,3$), define the coordinate projections by

$$\begin{aligned} \Pi _i(v):=v_i,\quad i=1,2,3. \end{aligned}$$

Denote ${\hat{E}}=\Pi _1(E)\times \Pi _2(E)\times \Pi _3(E)$. Since E is bounded, it is trivial to see that ${\hat{E}}\subset {\mathbb {R}}^{M}$ is also bounded. Since G is a polynomial, it is easy to prove that $\nabla G$ is Lipschitz-continuous on any bounded subset of ${\mathbb {R}}^{M}$. Hence, there exists a constant $c>0$ such that

$$\begin{aligned} ||\nabla G(v)-\nabla G(w)||_2\le c||v-w||_2,\quad v,w\in {\hat{E}}. \end{aligned}$$

Therefore,

$$\begin{aligned} ||u^{i}-\nabla G(v^{i+1})||_2=&[||\nabla _{v_1}G(v_1^{i+1};v_2^i;v_3^i)-\nabla _{v_1}G(v^{i+1})||_2^2\nonumber \\&+||\nabla _{v_2}G(v_1^{i+1};v_2^{i+1};v_3^i)-\nabla _{v_2}G(v^{i+1})||_2^2]^{\frac{1}{2}}\nonumber \\ \le&[c^2(||v_2^{i}-v_2^{i+1}||_2^2+||v_3^{i}-v_3^{i+1}||_2^2)+c^2||v_3^i-v_3^{i+1}||_2^2]^{\frac{1}{2}}\nonumber \\ \le&\sqrt{2}c||v^{i+1}-v^{i}||_2,\quad i\in {\mathbb {N}}, \end{aligned}$$

which together with (B.5) implies that

$$\begin{aligned} ||w^{i+1}||_2\le ||u^{i}-\nabla G(v^{i+1})||_2+{\hat{\rho }}||v^{i+1}-v^{i}||_2\le (\sqrt{2}c+{\hat{\rho }})||v^{i+1}-v^{i}||_2,\quad i\in {\mathbb {N}}, \end{aligned}$$

where ${\hat{\rho }}=\max \{\rho _1,\rho _2,\rho _3\}$. Hence, $\mathbf{H2}$ (Relative error condition) is satisfied with $b=\sqrt{2}c+{\hat{\rho }}$.

Moreover, since $\{v^i\}_{i\in {\mathbb {N}}}\subset {\mathbb {R}}^{M}$ is bounded and thus relative compact, there exists a subsequence $\{v^{i_j}\}_{j\in {\mathbb {N}}}$ and $\bar{v}\in {\mathbb {R}}^{M}$ such that $v^{i_j}\rightarrow \bar{v}$ as $j\rightarrow +\infty$. As mentioned above $\{v_1^i|i\in {\mathbb {N}}\}\subset {\hat{S}}$, it holds that $\{v^i\}_{i\in {\mathbb {N}}}\subset {\hat{S}}$. Since ${\hat{S}}$ is closed, $\bar{v}\in {\hat{S}}$. Additionally, it is easy to see that ${\tilde{F}}$ is continuous on ${\hat{S}}$. Therefore, ${\tilde{F}}(v^{i_j})\rightarrow {\tilde{F}}(\bar{v})$ as $j\rightarrow +\infty$. Hence, $\mathbf{H3}$ (Continuity condition) is fulfilled.

By Lemma A.3, ${\tilde{F}}$ is semi-algebraic on ${\hat{S}}$. Hence, by Lemma A.1, ${\tilde{F}}$ satisfies the Kurdyka-ojasiewicz property at $\bar{v}\in {\hat{S}}$. Then, the proof is complete by applying Lemma 3.1. $\square$