Inexact alternating direction method based on Newton descent algorithm with application to Poisson image deblurring

Abstract

The recovery of images from the observations that are degraded by a linear operator and further corrupted by Poisson noise is an important task in modern imaging applications such as astronomical and biomedical ones. Gradient-based regularizers involving the popular total variation semi-norm have become standard techniques for Poisson image restoration due to its edge-preserving ability. Various efficient algorithms have been developed for solving the corresponding minimization problem with non-smooth regularization terms. In this paper, motivated by the idea of the alternating direction minimization algorithm and the Newton’s method with upper convergent rate, we further propose inexact alternating direction methods utilizing the proximal Hessian matrix information of the objective function, in a way reminiscent of Newton descent methods. Besides, we also investigate the global convergence of the proposed algorithms under certain conditions. Finally, we illustrate that the proposed algorithms outperform the current state-of-the-art algorithms through numerical experiments on Poisson image deblurring.

Appendix: Proof of Theorem 1

The proof of Theorem 1 is similar to the previous literature [19, 20]. Only some changes are needed due to the introduction of the Newton descent algorithm. The entire proof is not reproduced here due to limited space. Just enough is sketched to make the changes clear.

Proof

Denote $L_{k}=(\omega ^{k})^{-1}\left( \delta ^{k} K^{T}K + \alpha \nabla ^{T}\nabla \right)$. According to the iterative formula with respect to u, we know that $u^{k+1}$ is the solution of the minimization problem

$$\begin{aligned} \small { \begin{aligned} \min \limits _{u\in U} \left\{ \langle \nabla D_{f}(u^{k}) + \alpha \text {div}(d^{k}-\nabla u^{k}) + \text {div} p^{k}, \right. \\ \left. u-u^{k}\rangle + \frac{1}{2}(u - u^{k})^{T}L_{k}(u - u^{k}) \right\} . \end{aligned} } \end{aligned}$$

(19)

Therefore, the sequence $\{u^{k}, d^{k}, p^{k}\}$ generated by the IADMNDA algorithm satisfies

$$\begin{aligned} \left\{ \begin{array}{lll}\nabla D_{f}(u^{k}) + L_{k}(u^{k+1}-u^{k})+\alpha \nabla ^{T}(\nabla u^{k}-d^{k}-\alpha ^{-1}p^{k}) + \partial \iota _{U}(u^{k+1})\ni 0, &{}~ &{}~ ~\\ \lambda \partial \Vert d^{k+1}\Vert _{1} + \alpha (d^{k+1}-\nabla u^{k+1}+\alpha ^{-1}p^{k})\ni 0, &{}~ &{}~ ~\\ p^{k+1}=p^{k} + \alpha (d^{k+1}-\nabla u^{k+1})&{} ~ &{}~ \end{array} \right. \end{aligned}$$

(20)

Because $\left( u^{*}, d^{*}, p^{*}\right)$ is one solution of (16), it is also the KKT point that satisfies (17).

Similarly to [19, 20], denote the errors by $u^{k}_{e} = u^{k}-u^{*}$, $d^{k}_{e} = d^{k}-d^{*}$, and $p^{k}_{e} = p^{k}-p^{*}$. Utilizing the subtraction between (20) and (17), taking the inner product with $u^{k+1}_{e}$, $d^{k+1}_{e}$ and $p^{k}_{e}$, and removing the non-negative terms, we further obtain that

$$\begin{aligned} \begin{aligned}&\frac{1}{\alpha }\Vert p^{k+1}_{e}\Vert ^{2}_{2} + \alpha \Vert d^{k+1}_{e}\Vert ^{2}_{2} + \left( \Vert u^{k+1}_{e}\Vert ^{2}_{L_{k}}+ \Vert u^{k+1}_{e}\Vert ^{2}_{\nabla ^{2}D_{f}(\zeta ^{k})} \right. \\&\left. \quad +\,\Vert u^{k}_{e}\Vert ^{2}_{\nabla ^{2}D_{f}(\zeta ^{k})} -\alpha \Vert \nabla u^{k+1}_{e}\Vert ^{2}_{2}\right) + \alpha \Vert d^{k} - \nabla u^{k+1}\Vert _{2}^{2}\\&\quad + \left( \Vert u^{k+1}-u^{k}\Vert ^{2}_{L_{k}}-\Vert u^{k+1}-u^{k}\Vert ^{2}_{\nabla ^{2}D_{f}(\zeta ^{k})} -\alpha \Vert \nabla (u^{k+1}-u^{k})\Vert ^{2}_{2}\right) \\&\le \frac{1}{\alpha }\Vert p^{k}_{e}\Vert ^{2}_{2} + \alpha \Vert d^{k}_{e}\Vert ^{2}_{2} + \Vert u^{k}_{e}\Vert ^{2}_{L_{k}}-\alpha \Vert \nabla u^{k}_{e}\Vert ^{2}_{2}. \end{aligned} \end{aligned}$$

(21)

where $\zeta ^{k}\in [u^{*}, u^{k}]$, with $[u^{*}, u^{k}]$ denoting the line segment between $u^{*}$ and $u^{k}$. Since $\delta _{\min }\ge \overline{\gamma _{D}}$, we can easily deduce that the last term of the left side of the inequality (21) is non-negative.

According to the definition of $\omega ^{k}$ in Algorithm 1, we know that $\omega ^{k}$ is monotone non-increasing, and hence, there exists $\omega ^{*}$ such that $\lim \limits _{k\rightarrow +\infty }\omega ^{k}=\omega ^{*}$.

Denote $\small {\tilde{L}_{k}=(\omega ^{*})^{-1}\left( \delta ^{k} K^{T}K + \alpha \nabla ^{T}\nabla \right) }$. By the boundedness of $\delta ^{k}$ and $\lim \limits _{k\rightarrow +\infty }\omega ^{k}=\omega ^{*}$ we have that

$$\begin{aligned} \lim \limits _{k\rightarrow +\infty } \tilde{L}_{k}-L_{k}=\mathbf 0 . \end{aligned}$$

(22)

Since $\delta ^{k+1}-\delta ^{k}\le \underline{\gamma _{D}}$, we know that $\delta ^{k}K^{T}K + \nabla ^{2}D_{f}(\zeta ^{k})\succeq \delta ^{k+1}K^{T}K$ according to the condition (18). Therefore, by the definition of $\tilde{L}_{k}$ we further have

$$\begin{aligned} \begin{aligned}&\Vert u^{k+1}_{e}\Vert ^{2}_{\tilde{L}_{k}}+ \Vert u^{k+1}_{e}\Vert ^{2}_{\nabla ^{2}D_{f}(\zeta ^{k})}-\alpha \Vert \nabla u^{k+1}_{e}\Vert ^{2}_{2}\\&\quad \ge \Vert u^{k+1}_{e}\Vert ^{2}_{\tilde{L}_{k+1}}-\alpha \Vert \nabla u^{k+1}_{e}\Vert ^{2}_{2}\ge 0. \end{aligned} \end{aligned}$$

(23)

According to (22), we can easily conclude that there exists some $k_{0}$ such that

$$\begin{aligned} \sum _{k=k_{0}}^{+\infty }\left( \Vert u^{k}_{e}\Vert ^{2}_{\frac{1}{2}\nabla ^{2}D_{f}(\zeta ^{k})} + \Vert u^{k+1}_{e}\Vert ^{2}_{L_{k}-\tilde{L}_{k}}-\Vert u^{k}_{e}\Vert ^{2}_{L_{k}-\tilde{L}_{k}}\right) >0. \end{aligned}$$

In the next, summing (21) from some $k_{0}$ to $+\infty$, utilizing (23), and removing the other non-negative terms, we obtain that

$$\begin{aligned} \lim _{k\rightarrow +\infty }\Vert u^{k}-u^{*}\Vert ^{2}_{\frac{1}{2}\nabla ^{2}D_{f}(\zeta ^{k})}=0. \end{aligned}$$

(24)

Due to $\Vert u^{k}-u^{*}\Vert ^{2}_{\frac{1}{2}\nabla ^{2}D_{f}(\zeta ^{k})}\ge \frac{1}{2}\underline{\gamma _{D}} \Vert Ku^{k}-Ku^{*}\Vert ^{2}_{2}$, we get

$$\begin{aligned} \lim \limits _{k\rightarrow +\infty }\Vert Ku^{k}-Ku^{*}\Vert _{2}=0. \end{aligned}$$

Due to $\Vert K^{T}K\Vert _{2}>0$, we further have $\lim \limits _{k\rightarrow +\infty }\Vert u^{k}-u^{*}\Vert _{2}=0$, which implies that $\{u^{k}\}$ converges to a solution of the minimization problem (16). $\square$