Satellite imagery retrieval based on adaptive Gaussian–Markov random field model with Bayes deep convolutional neural network

Abstract

This paper introduces a novel method for satellite colour imagery retrieval, based on an adaptive Gaussian–Markov random field (AGMRF) model with the Bayes-driven deep convolutional neural network (AGMRF–BDCNN). The given input imagery is segregated into the structure, microstructure, and texture components, and the AGMRF-driven features and statistical features are extracted from the segregated components and are formulated as a feature vector of the query imagery. Cosine direction and Bhattacharyya distance measures are deployed to match the feature vector with the feature vector of the feature vector database. If the query imagery features match the feature-vector database's features, then the reference imagery in the database is marked and indexed. The indexed imageries are retrieved. Three different benchmark data sets, SceneSat, PatternNet, and UC Merced, have been used to validate the proposed AGMRF–BDCNN method. For the SceneSat data set, the AGMRF–BDCNN method results in 0.2319 scores for ANMRR and 0.7156 scores for mAP; for the UC Merced data set, it yields 0.2316 scores for ANMRR and 0.7816 scores for mAP; for PatternNet data set, it achieves 0.2405 scores for ANMRR and 0.6979 scores for mAP. The obtained results are comparable to state-of-the-art methods.

Appendices

Appendix A

The joint probability density function of the pixels of the imagery is defined as in Eq. (A1):

$$ Pr\left( {X\left| Q \right.} \right) \propto \left( {\sigma^{2} } \right)^{ - N/2} exp\left[ { - \frac{1}{{2\sigma^{2} }}\left\{ {T_{00} + K^{2} \sum\limits_{r = 1}^{N} {S_{r}^{2} T_{rr} + } 2K^{2} \sum\limits_{{\mathop {r,s = 1}\limits_{r < s} }}^{N} {S_{r} S_{s} T_{rs} - } 2K\sum\limits_{r = 1}^{N} {S_{r} T_{0r} } } \right\}} \right] $$

(A1)

where

$Q = T_{00} + K^{2} \sum\limits_{r = 1}^{N} {S_{r}^{2} T_{rr} + } 2K^{2} \sum\limits_{{\mathop {r,s = 1}\limits_{r < s} }}^{N} {S_{r} S_{s} T_{rs} - } 2K\sum\limits_{r = 1}^{N} {S_{r} T_{0r} }$, s. t. K ∈ R, α > 1, 0 < θ < π, 0 < ϕ < π/2 and $\sigma^{2} > 0$.

Each parameter of the parameter set, Θ, follows its own distribution, such as α follows displaced exponential distribution, θ, ϕ, and K follow the Uniform distribution. The prior-distribution of the parameters is assigned as follows.

1. α is distributed to displaced exponential distribution with parameter β, that is

$$ \Pr \left( \alpha \right) = \beta \exp \, \left( { - \beta \, \left( {\alpha - 1} \right)} \right); \, \alpha \, > \, 1; \, \beta \, > \, 0 $$

(A2)

2. $\sigma^{2} \,$ has the inverted Gamma distribution with parameter ν and δ, that is

$$ \Pr \, \left( {\sigma^{2} } \right) \propto \beta \exp \, \left( { - \nu /\sigma^{2} } \right) \, \left( {\sigma^{2} } \right)^{{ \, - \, \left( {\delta + 1} \right)}} ;\sigma^{2} > 0, \, \nu > 0 $$

(A3)

3. K, θ, and ϕ are uniformly distributed over their domain, that is

$$ Pr(K,q,f) \, = {\text{ C}},{\text{ a constant}}. K \in {\mathbb{R}}, \, 0 < \theta < \pi, 0 < \phi < \pi/{2} $$

(A4)

The joint priori density function of the parameter set, Θ, is given in Eq. (A5):

$$ Pr \left( \Theta \right) \propto \beta exp \left( { - \beta \left( {\alpha - 1} \right) - \nu /\sigma^{2} } \right) \left( {\sigma^{2} } \right)^{{ - \left( {\delta + 1} \right)}} ; \sigma^{2} > 0, \alpha > 0; 0 < \theta < \pi ; 0 < \theta < \varphi $$

(A5)

Using Eqs. (A1), (A2), and Bayes rule, the joint posterior density of α, θ, and ϕ is obtained as

$$ Pr \left( \Theta \right) \propto \left( {\sigma^{ \, 2} } \right)^{{ - \left( {N/2} \right)}} e^{{ - \left( {Q/2\sigma^{ \, 2} } \right)}} e^{{ - \beta \left( {\alpha - 1} \right) - \nu /\sigma^{ \, 2} }} \left( {\sigma^{ \, 2} } \right)^{{ - \left( {\delta + 1} \right)}} ; \, \sigma^{2} > 0; \, \nu ,\delta > 0; \, \alpha > 1; \, 0 < \theta < \pi ; \, 0 < \theta < \varphi $$

(A6)

Integrating Eq. (A6) with respect to $\sigma^{2}$, the joint posterior density of K, α, θ, and ϕ is obtained as

$$ \begin{gathered} Pr\left( {\alpha ,\theta ,\varphi ,K\left| X \right.} \right) = exp\left( { - \beta \left( {\alpha - 1} \right)} \right) \left( {Q + 2\nu } \right)^{{ - \left( {\frac{N}{2} + \delta } \right)}} \hfill \\ K \in R, \alpha > 1, 0 < \theta < \pi , 0 < \varphi < \pi /2 \hfill \\ \end{gathered} $$

(A7)

where

$$ \left( {Q + 2\nu } \right) = \left[ {K^{2} \sum\limits_{r = 1}^{N} {S_{r}^{2} T_{rr} + 2K^{2} \sum\limits_{\begin{subarray}{l} r,s = 1 \\ r < s \end{subarray} }^{N} {S_{r} S_{s} } T_{rs} - 2K\sum\limits_{\begin{subarray}{l} r,s = 1 \\ r < s \end{subarray} }^{N} {S_{r} T_{0r} + T_{00} + 2\nu } } } \right] $$

That is

$$ \begin{gathered} \left( {Q + 2\nu } \right) = aK^{2} - 2Kb + T_{00} + 2\nu \hfill \\ = C\left[ {1 + a_{1} \left( {K - b_{1} } \right)^{2} } \right] \hfill \\ \end{gathered} $$

(A8)

where

$a_{1} = \frac{a}{C}$, $b_{1} = \frac{b}{a}$, $a = \sum\limits_{r = 1}^{N} {S_{r}^{2} T_{rr} } + 2\sum\limits_{\begin{subarray}{l} r,s = 1 \\ r < s \end{subarray} }^{N} {S_{r} S_{s} T_{rs} }$, $b = \sum\limits_{r = 1}^{N} {S_{r}^{2} T_{rr} }$, $T_{rs} = \sum\limits_{t = 1}^{N} {X_{t - r} X_{t - s} ;} r,s = 0,1,2, \ldots ,N$ and $S_{r} = \frac{\sin (r\theta )\cos (r\varphi )}{{\alpha^{r} }}$.

The proper Bayesian inference on the parameters, α, θ, and ϕ, can be obtained from their respective posterior density function given in (A7):

$$ \left. \begin{gathered} Pr\left( \alpha \right) \propto \iint {Pr\left( {\alpha ,\theta ,\varphi /X} \right)d\theta d\varphi } \hfill \\ Pr\left( \theta \right) \propto \iint {Pr\left( {\alpha ,\theta ,\varphi /X} \right)d\alpha d\varphi } \hfill \\ Pr\left( \varphi \right) \propto \iint {Pr\left( {\alpha ,\theta ,\varphi /X} \right)d\alpha d\theta } \hfill \\ \end{gathered} \right\} $$

(A9)

The point estimates of the parameters, α, θ, and ϕ, are considered the respective marginal posterior distribution, i.e., posterior means. In a view to minimising the computational effort, first, the posterior mean of α is computed. Then, α is fixed at its posterior mean, and the conditional means of θ and ϕ are evaluated. The α, θ, and ϕ are fixed at their posterior means, then the conditional mean of K is evaluated. Thus, the estimates are

$$ \left. \begin{gathered} \mathop \alpha \limits^{ \wedge } = E\left( \alpha \right) \hfill \\ \left( {\mathop \theta \limits^{ \wedge } ,\mathop \varphi \limits^{ \wedge } } \right) = E\left( {\left. {\theta ,\varphi } \right|\mathop \alpha \limits^{ \wedge } } \right) \hfill \\ \mathop K\limits^{ \wedge } = E\left( {\left. K \right|\mathop \alpha \limits^{ \wedge } ,\mathop \theta \limits^{ \wedge } ,\mathop \varphi \limits^{ \wedge } } \right) \hfill \\ \end{gathered} \right\} $$

(A10)

The reason behind the proposed method is said to be a Bayes-driven Deep CNN.

Appendix B

$$ w\left( {L,L} \right) = \left[ {\begin{array}{*{20}c} {X\left( { - 2, - 2} \right)} & {X\left( { - 2, - 1} \right)} & {X\left( { - 2,0} \right)} & {X\left( { - 2,1} \right)} & {X\left( { - 2,2} \right)} \\ {X\left( { - 1, - 2} \right)} & {X\left( { - 1, - 1} \right)} & {X\left( { - 1,0} \right)} & {X\left( { - 1,1} \right)} & {X\left( { - 1,2} \right)} \\ {X\left( {0, - 2} \right)} & {X\left( {0, - 1} \right)} & {X\left( {0,0} \right)} & {X\left( {0,1} \right)} & {X\left( {0,2} \right)} \\ {X\left( {1, - 2} \right)} & {X\left( {1, - 1} \right)} & {X\left( {1,0} \right)} & {X\left( {1,1} \right)} & {X\left( {1,2} \right)} \\ {X\left( {2, - 2} \right)} & {X\left( {2, - 1} \right)} & {X\left( {2,0} \right)} & {X\left( {2,1} \right)} & {X\left( {2,2} \right)} \\ \end{array} } \right] $$

(B1)

$$ \Gamma \left( {L,L} \right) = \left[ {\begin{array}{*{20}c} {\Gamma_{5} } & {\Gamma_{4} } & {\Gamma_{3} } & {\Gamma_{4} } & {\Gamma_{5} } \\ {\Gamma_{4} } & {\Gamma_{2} } & {\Gamma_{1} } & {\Gamma_{2} } & {\Gamma_{4} } \\ {\Gamma_{3} } & {\Gamma_{1} } & 1 & {\Gamma_{1} } & {\Gamma_{3} } \\ {\Gamma_{4} } & {\Gamma_{2} } & {\Gamma_{1} } & {\Gamma_{2} } & {\Gamma_{4} } \\ {\Gamma_{5} } & {\Gamma_{4} } & {\Gamma_{3} } & {\Gamma_{4} } & {\Gamma_{5} } \\ \end{array} } \right] $$

(B2)

$$ g = \sum\limits_{p = - 2}^{2} {\sum\limits_{q = - 2}^{2} {\Gamma \left( {k + p,l + q} \right) * w \left( {k + p,l + q} \right);} } s.t. p = q \ne 0 \;{\text{ and }} k = l = 0 $$

(B3)

$$ R = \frac{{w\left( {k,l} \right)}}{g} \in \left( {0.95, 1.05} \right) $$

(B4)

$$ \Gamma_{1} = \frac{{K\left( {sin\left( {1\theta } \right)cos\left( {1\varphi } \right)} \right)}}{{\alpha^{1} }}and\;\;\Gamma_{2} = \frac{{K\left( {sin\left( {2\theta } \right)cos\left( {2\varphi } \right)} \right)}}{{\alpha^{2} }} $$

(B5)

Similarly, $\Gamma_{3}$, $\Gamma_{4}$, and $\Gamma_{5}$ are computed.