Cluster Sparsity Field: An Internal Hyperspectral Imagery Prior for Reconstruction

Abstract

Hyperspectral images (HSIs) have significant advantages over more traditional image types for a variety of computer vision applications dues to the extra information available. The practical reality of capturing and transmitting HSIs however, means that they often exhibit large amounts of noise, or are undersampled to reduce the data volume. Methods for combating such image corruption are thus critical to many HSIs applications. Here we devise a novel cluster sparsity field (CSF) based HSI reconstruction framework which explicitly models both the intrinsic correlation between measurements within the spectrum for a particular pixel, and the similarity between pixels due to the spatial structure of the HSI. These two priors have been shown to be effective previously, but have been always considered separately. By dividing pixels of the HSI into a group of spatial clusters on the basis of spectrum characteristics, we define CSF, a Markov random field based prior. In CSF, a structured sparsity potential models the correlation between measurements within each spectrum, and a graph structure potential models the similarity between pixels in each spatial cluster. Then, we integrate the CSF prior learning and image reconstruction into a unified variational framework for optimization, which makes the CSF prior image-specific, and robust to noise. It also results in more accurate image reconstruction compared with existing HSI reconstruction methods, thus combating the effects of noise corruption or undersampling. Extensive experiments on HSI denoising and HSI compressive sensing demonstrate the effectiveness of the proposed method.

Appendices

Appendix A ${\varvec{\Theta }}$-subproblem

In this appendix, we give the detailed derivation of solving ${\varvec{\Theta }}$-subproblem with the alternative minimization scheme, which reduces the ${\varvec{\Theta }}$-subproblem into four simpler subproblems on ${\varvec{\gamma }}_k$,${\varvec{\varpi }}_k$,${\varvec{\eta }}_k$, and ${\varvec{\nu }}_k$, respectively.

1.1 A.1 Optimization for ${\varvec{\gamma }}_k$

Removing the irrelevant variables, we can obtain the subproblem over ${\varvec{\gamma }}_k$ as

$$\begin{aligned} \begin{aligned}&\min \limits _{{\varvec{\gamma }}_k} \Vert \mathbf{{Y}}_k\Vert ^2_{{\varvec{\Gamma }}_k} - n_k\log |{\varvec{\Lambda }}_k| + n_k\log |{\varvec{\Gamma }}_k| + \sum \limits _{j} \varpi _{ik}\gamma _{jk},\\&\quad = \min \limits _{{\varvec{\gamma }}_k} \Vert \mathbf{{Y}}_k\Vert ^2_{{\varvec{\Gamma }}_k} + n_k\log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A}{\varvec{\Phi }}}|\\&\qquad + n_k\log |{\varvec{\Gamma }}_k| + \sum \limits _{j} \varpi _{jk}\gamma _{jk}. \end{aligned} \end{aligned}$$

(28)

Since $f({\varvec{\gamma }}^{-1}_k) = \log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A}{\varvec{\Phi }}}|$ over ${\varvec{\gamma }}^{-1}_k = [\gamma ^{-1}_{1k},\ldots ,\gamma ^{-1}_{n_dk}]^T$ results in Eq. (28) to be nonconvex, we turn to find a strict upper bound of $f({\varvec{\gamma }}^{-1}_k)$ as

$$\begin{aligned} \begin{aligned} f({\varvec{\gamma }}^{-1}_k) \le \mathbf{z}^T{\varvec{\gamma }}^{-1}_k - f^*(\mathbf{z}), \end{aligned} \end{aligned}$$

(29)

where $f^*(\mathbf{z})$ is the concave conjugate function of $f({\varvec{\gamma }}^{-1}_k)$ and $\mathbf{z} = [z_1,\ldots ,z_{n_d}]^T$. The equality of Eq. (29) holds iff

$$\begin{aligned} \begin{aligned} \mathbf{z}&= \nabla _{{\varvec{\gamma }}^{-1}_k}\log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A}{\varvec{\Phi }}}| \\&= \mathbf {diag}[({\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A} {\varvec{\Phi }}})^{-1}]. \end{aligned} \end{aligned}$$

(30)

Substituting the upper bound in Eq. (29) into Eq. (28) and removing the irrelevant variables, the subproblem over ${\varvec{\gamma }}_k$ can be simplified as

$$\begin{aligned} \begin{aligned}&\min \limits _{{\varvec{\gamma }}_k} \Vert \mathbf{{Y}}_k\Vert ^2_{{\varvec{\Gamma }}_k} + n_k\mathbf{z}^T{\varvec{\gamma }}^{-1}_k + n_k\log |{\varvec{\Gamma }}_k| + \sum \limits _{j} \varpi _{jk}\gamma _{ik},\\&\quad =\min \limits _{{\varvec{\gamma }}_k} \sum \limits _j \left( \frac{\overline{y}_j + n_kz_j}{\gamma _{ik}} + n_k\log \gamma _{jk} + \varpi _{jk}\gamma _{ik} \right) , \end{aligned} \end{aligned}$$

(31)

where $\overline{y}_i$ is the i-th entry of $\overline{\mathbf{y}} = \mathbf {diag}(\mathbf{{Y}}_k\mathbf{{Y}}^T_k) = [\overline{y}_1,\ldots ,\overline{y}_{n_d}]^T$. Since the variance ${\varvec{\gamma }}_k \ge 0$, this convex optimization over ${\varvec{\gamma }}_k$ gives a closed form solution over $\gamma _{jk}$ as

$$\begin{aligned} \begin{aligned} \gamma _{jk} = \left( \sqrt{4\varpi _{jk}(\overline{y}_j + n_kz_j) + n^2_k} - n_k\right) /\left( 2\varpi _{jk}\right) . \end{aligned} \end{aligned}$$

(32)

1.2 A.2 Optimization for ${\varvec{\varpi }}_k$

Given ${\varvec{\gamma }}_k$, ${\varvec{\varpi }}_k$ can be estimated by solving the problem

$$\begin{aligned} \begin{aligned} \min \limits _{{\varvec{\varpi }}_k} \sum \limits _{j}\left( \varpi _{jk}\gamma _{jk} - 2\log \varpi _{jk}\right) . \end{aligned} \end{aligned}$$

(33)

This problem produce a close-form solution of $\varpi _{jk}$ as

$$\begin{aligned} \begin{aligned} \varpi _{jk} = 2/{\gamma _{jk}}. \end{aligned} \end{aligned}$$

(34)

1.3 A.3 Optimization for ${\varvec{\eta }}_k$

Similar to optimization of ${\varvec{\gamma }}_k$, the subproblem over ${\varvec{\eta }}_k$ can be formulated as

$$\begin{aligned} \begin{aligned}&\min \limits _{{\varvec{\eta }}_k} \Vert \mathbf{{Y}}_k - \mathbf{{M}}_k\Vert ^2_{{{\varvec{\Sigma }}}_k} - n_k\log |{\varvec{\Lambda }}_k| + n_k\log |{{\varvec{\Sigma }}}_k|\\&\qquad + \sum \limits _{j}\left( \nu _{jk}\eta _{jk} - 2\log \nu _{jk}\right) \\&\quad = \min \limits _{{\varvec{\eta }}_k} \Vert \mathbf{{Y}}_k - \mathbf{{M}}_k\Vert ^2_{{{\varvec{\Sigma }}}_k} + n_k\log |{{\varvec{\Sigma }}}_{k}| \\&\qquad + \sum \limits _{j}\left( \nu _{jk}\eta _{jk} - 2\log \nu _{jk}\right) \\&\qquad + n_k\log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n\mathbf{{A}}{\varvec{\Phi }}|. \end{aligned} \end{aligned}$$

(35)

Let $\phi ({\varvec{\eta }}^{-1}_k) = \log | {\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A}{\varvec{\Phi }}}|$ with ${\varvec{\eta }}^{-1}_k = [\eta ^{-1}_{1k},\ldots ,\eta ^{-1}_{n_dk}]^T$. We have the upper bound of $\phi ({\varvec{\eta }}^{-1}_k)$

$$\begin{aligned} \begin{aligned} \phi ({\varvec{\eta }}^{-1}_k) \le {\varvec{\alpha }}^T{\varvec{\eta }}^{-1}_k - \phi ^*({\varvec{\alpha }}), \end{aligned} \end{aligned}$$

(36)

where $\phi ^*({\varvec{\alpha }})$ is the concave conjugate function with ${\varvec{\alpha }} = [\alpha _1,\ldots ,\alpha _{n_d}]^T$. The equality of this upper bound holds iff

$$\begin{aligned} \begin{aligned} {\varvec{\alpha }}&= \nabla _{{\varvec{\eta }}^{-1}_k}\log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A}{\varvec{\Phi }}}|\\&= \mathbf {diag}\left[ ({\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A}{\varvec{\Phi }}})^{-1}\right] . \end{aligned} \end{aligned}$$

(37)

Substituting this upper bound into Eq. (35), we have a simpler subproblem over ${\varvec{\eta }}_k$ as

$$\begin{aligned} \begin{aligned}&\min \limits _{{\varvec{\eta }}_k} \Vert \mathbf{{Y}}_k - \mathbf{{M}}_k\Vert ^2_{{{\varvec{\Sigma }}}_k} + n_k{\varvec{\alpha }}^T{\varvec{\eta }}^{-1}_k + n_k\log |{{\varvec{\Sigma }}}_k|\\&\qquad + \sum \limits _{j}\left( \nu _{jk}\eta _{jk} - 2\log \nu _{jk}\right) \\&\quad = \min \limits _{{\varvec{\eta }}_k} \sum \limits _j \left( \frac{\widehat{y}_j + n_k\alpha _j}{\eta _{jk}} + n_k\log \eta _{jk} + \nu _{jk}\eta _{jk} \right) , \end{aligned} \end{aligned}$$

(38)

where $\widehat{y}_j$ is the j-th entry of $\widehat{\mathbf{y}} =\mathbf {diag}[(\mathbf{{Y}}_k - \mathbf{{M}}_k)(\mathbf{{Y}}_k - \mathbf{{M}}_k)^T] = [\widehat{y}_1,\ldots ,\widehat{y}_{n_d}]^T$. This convex optimization problem gives a closed form solution over $\eta _{jk}$ as

$$\begin{aligned} \begin{aligned} \eta _{jk} = \frac{\sqrt{4\nu _{jk}(\widehat{y}_j + n_k\alpha _j) + n^2_k} - n_k}{2\nu _{jk}}. \end{aligned} \end{aligned}$$

(39)

1.4 A.4 Optimization for ${\varvec{\nu }}_k$

Given ${\varvec{\eta }}_k$, we can estimate ${\varvec{\nu }}_k$ by solving the following formula as

$$\begin{aligned} \begin{aligned} \min \limits _{{\varvec{\nu }}_k} \sum \limits _{j}\left( \nu _{jk}\eta _{jk} - 2\log \nu _{jk}\right) . \end{aligned} \end{aligned}$$

(40)

This convex optimization problem yields a closed form solution as

$$\begin{aligned} \begin{aligned} \nu _{jk} = \frac{2}{\eta _{jk}}. \end{aligned} \end{aligned}$$

(41)

Appendix B ${\varvec{\lambda }}$-subproblem

Finally, ${\varvec{\lambda }}$-subproblem can be formulated as

$$\begin{aligned} \begin{aligned}&\min \limits _{{\varvec{\lambda }}} \sum \limits _k (\Vert {\mathbf{A}{\varvec{\Phi }}}{} \mathbf{{Y}}_k - \mathbf{{F}}_k\Vert ^2_{{{\varvec{\Sigma }}}_n} - n_k\log |{\varvec{\Lambda }}_k| + n_k\log |{{\varvec{\Sigma }}}_n|)\\&\quad = \min \limits _{{\varvec{\lambda }}} \sum \limits _k (\Vert {\mathbf{A}{\varvec{\Phi }}}{} \mathbf{{Y}}_k - \mathbf{{F}}_k\Vert ^2_{{{\varvec{\Sigma }}}_n} + n_k\log |{{\varvec{\Sigma }}}_n|) \\&\qquad + \sum \limits _k n_k\log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n{\mathbf{A}{\varvec{\Phi }}}| \end{aligned} \end{aligned}$$

(42)

According to the following algebra equation

$$\begin{aligned} \begin{aligned}&\log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k + {\varvec{\Phi }}^T\mathbf{{A}}^T{{\varvec{\Sigma }}}^{-1}_n\mathbf{{A}}{\varvec{\Phi }}| + n_k\log |{{\varvec{\Sigma }}}_n|\\&\quad =\log |{{\varvec{\Sigma }}}_n + {\mathbf{A}{\varvec{\Phi }}}({\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k)^{-1}{\varvec{\Phi }}^T\mathbf{{A}}^T| \\&\qquad +\log |{\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k|, \end{aligned} \end{aligned}$$

(43)

the subproblem in Eq. (42) simplifies to

$$\begin{aligned} \begin{aligned} \min \limits _{{\varvec{\lambda }}}&\sum \limits _k n_k\log |{{\varvec{\Sigma }}}_n + {\mathbf{A}{\varvec{\Phi }}}({\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k)^{-1}{\varvec{\Phi }}^T\mathbf{{A}}^T| \\&+ \sum \limits _k (\Vert {\mathbf{A}{\varvec{\Phi }}}{} \mathbf{{Y}}_k - \mathbf{{F}}_k\Vert ^2_{{{\varvec{\Sigma }}}_n} \end{aligned} \end{aligned}$$

(44)

Let $g({\varvec{\lambda }}) = \log |{{\varvec{\Sigma }}}_n + {\mathbf{A}{\varvec{\Phi }}}({\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k)^{-1}{\varvec{\Phi }}^T\mathbf{{A}}^T|$, we can obtain its upper bound as

$$\begin{aligned} \begin{aligned} g({\varvec{\lambda }}) \le {\varvec{\beta }}_k^T{\varvec{\lambda }} - g^*({\varvec{\beta }}_k), \end{aligned} \end{aligned}$$

(45)

where $g^*({\varvec{\beta }}_k)$ is the corresponding concave conjugate function with ${\varvec{\beta }}_k = [\beta _{1k},...,\beta _{n_bk}]^T$. It can be found that the equality in Eq. (45) only holds when

$$\begin{aligned} \begin{aligned} {\varvec{\beta }}_k&= \nabla _{{\varvec{\lambda }}}\log |{{\varvec{\Sigma }}}_n + {\mathbf{A}{\varvec{\Phi }}}({\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k)^{-1}{\varvec{\Phi }}^T\mathbf{{A}}^T|\\&= \mathbf {diag}[({{\varvec{\Sigma }}}_n + {\mathbf{A}{\varvec{\Phi }}}({\varvec{\Gamma }}^{-1}_k + {{\varvec{\Sigma }}}^{-1}_k)^{-1}{\varvec{\Phi }}^T\mathbf{{A}}^T)^{-1}]. \end{aligned} \end{aligned}$$

(46)

Substituting this upper bound into Eq. (44), the subproblem over ${\varvec{\lambda }}$ further simplifies to

$$\begin{aligned} \begin{aligned}&\min \limits _{{\varvec{\lambda }}} \sum \limits _k (\Vert {\mathbf{A}{\varvec{\Phi }}}{} \mathbf{{Y}}_k - \mathbf{{F}}_k\Vert ^2_{{{\varvec{\Sigma }}}_n} + n_k{\varvec{\beta }}_k^T{\varvec{\lambda }}). \end{aligned} \end{aligned}$$

(47)

This amounts to the following optimization over each component $\lambda _j$ of ${\varvec{\lambda }}$ as

$$\begin{aligned} \begin{aligned} \min \limits _{\lambda _j} \frac{\overline{q}_j}{\lambda _j} + \lambda _j\sum \limits _k n_k\beta _{jk}, \end{aligned} \end{aligned}$$

(48)

where $\overline{q}_j$ is the jth component of ${\overline{\mathbf{q}}} = \sum \nolimits _k \mathbf {diag}[({\mathbf{A}{\varvec{\Phi }}}{} \mathbf{{Y}}_k - \mathbf{{F}}_k)({\mathbf{A}{\varvec{\Phi }}}{} \mathbf{{Y}}_k - \mathbf{{F}}_k)^T] = [\overline{q}_1,\ldots ,\overline{q}_{n_b}]^T$. This convex optimization gives a closed form solution as

$$\begin{aligned} \begin{aligned} \lambda _j = \sqrt{\frac{\overline{q}_j}{\sum \limits _k n_k\beta _{jk}}}. \end{aligned} \end{aligned}$$

(49)