Combining categorical and continuous spatial information within the Bayesian maximum entropy paradigm

Abstract

Due to the fast pace increasing availability and diversity of information sources in environmental sciences, there is a real need of sound statistical mapping techniques for using them jointly inside a unique theoretical framework. As these information sources may vary both with respect to their nature (continuous vs. categorical or qualitative), their spatial density as well as their intrinsic quality (soft vs. hard data), the design of such techniques is a challenging issue. In this paper, an efficient method for combining spatially non-exhaustive categorical and continuous data in a mapping context is proposed, based on the Bayesian maximum entropy paradigm. This approach relies first on the definition of a mixed random field, that can account for a stochastic link between categorical and continuous random fields through the use of a cross-covariance function. When incorporating general knowledge about the first- and second-order moments of these fields, it is shown that, under mild hypotheses, their joint distribution can be expressed as a mixture of conditional Gaussian prior distributions, with parameters estimation that can be obtained from entropy maximization. A posterior distribution that incorporates the various (soft or hard) continuous and categorical data at hand can then be obtained by a straightforward conditionalization step. The use and potential of the method is illustrated by the way of a simulated case study. A comparison with few common geostatistical methods in some limit cases also emphasizes their similarities and differences, both from the theoretical and practical viewpoints. As expected, adding categorical information may significantly improve the spatial prediction of a continuous variable, making this approach powerful and very promising.

Appendices

Appendix 1

As we assumed that each Y(x) is Bernouilli distributed, its discrete pdf is

$$p_{Y_{k} ({\mathbf{x}})} (y) = \left\{ \begin{array}{*{20}l} P(C({\mathbf{x}}) = c_{k}) & \hbox{if}\; y=1\\ P(C({\mathbf{x}})\neq c_{k})& \hbox{if}\; y=0\\ 0 & \hbox{otherwise}\\ \end{array} \right. $$

(58)

and the equivalent continuous pdf is thus given by

$$ f_{Y_{k} ({\mathbf{x}})} (y) = P(C({\mathbf{x}}) = c_{k}) \delta(y-1) + P(C({\mathbf{x}}) \neq c_{k}) \delta(y).$$

(59)

In a similar way, as the joint discrete pdf for $\left(Y_{k}({\bf x}_{i}), Y_{k \prime}({\bf x}_{j})\right)$ is given by

$$p_{Y_{k} ({\mathbf{x}}_{i}), Y_{k \prime} ({\mathbf{x}}_{j})} (y,y^{\prime}) = \left\{ \begin{array}{*{20}l} P(C({\mathbf{x}}_i)\neq c_{k}\cap C({\mathbf{x}}_j)\neq c_{k{\prime}}) & \hbox{if}\;y=0, \, y^{\prime}=0 \\ P(C({\mathbf{x}}_i)\neq c_{k}\cap C({\mathbf{x}}_j)=c_{k{\prime}}) & \hbox{if} \; y=0, \, y^{\prime}=1 \\ P(C({\mathbf{x}}_i)=c_{k} \cap C({\mathbf{x}}_j)\neq c_{k{\prime}}) & \hbox{if} \; y=1, \, y^{\prime}=0 \\ P(C({\mathbf{x}}_i)=c_{k} \cap C({\mathbf{x}}_j)=c_{k{\prime}}) & \hbox{if} \; y=1, \, y^{\prime}=1 \\ 0 & \hbox{otherwise},\\ \end{array}\right.$$

(60)

the equivalent continuous pdf is

$$\begin{aligned} f_{Y_{k} ({\mathbf{x}}_{i}), Y_{k \prime}({\mathbf{x}}_j)} (y,y^{\prime}) &= P(C({\mathbf{x}}_i)\neq c_{k}\cap C({\mathbf{x}}_j)\neq c_{k{\prime}})\delta(y)\delta(y^{\prime})\\ &\quad + P(C({\mathbf{x}}_i)\neq c_{k} \cap C({\mathbf{x}}_j)=c_{k{\prime}})\delta(y)\delta(y^{\prime} - 1)\\ &\quad + P(C({\mathbf{x}}_i)=c_{k} \cap C({\mathbf{x}}_j)\neq c_{k{\prime}})\delta(y-1)\delta(y^{\prime})\\ &\quad + P(C({\mathbf{x}}_i)=c_{k} \cap C({\mathbf{x}}_j)=c_{k{\prime}})\delta(y-1)\delta(y^{\prime}-1),\\ \end{aligned}$$

(61)

whereas for the joint pdf of $(Y_{k} ({\bf x}_{i}),Z({\bf x}_{j}))$, we will express it as

$$f_{Y_{k} ({\mathbf{x}}_i), Z({\mathbf{x}}_j)} (y,z)= f_{Z({\mathbf{x}}_j) | y_{k}({\mathbf{x}}_i)} (z) f_{Y_{k} ({\mathbf{x}}_{i})}(y).$$

(62)

Similar notations can be, of course, obtained for the joint pdf of any combinations of several continuous and/or binary variables.

Appendix 2

The general expression for a multivariate Gaussian distribution $N(\varvec{\mu}_{\ell},\Sigma)$ over n+1 variables is given by

$$f({\mathbf{z}} | {\mathbf{y}}_{\ell}) = \frac{\exp \left((1/2) ({\mathbf{z}} - \varvec{\mu}_{\ell})^{\prime} \Sigma^{-1}({\mathbf{z}} - \varvec{\mu}_{\ell}) \right) }{\sqrt{(2 \pi)^{n+1} \det(\Sigma^{-1})}}$$

(63)

Expanding the quadratic term inside this exponential, we can write that

$$f({\mathbf{z}} | {\mathbf{y}}_{\ell}) \propto \exp \left(\frac{1}{2} {\mathbf{z}}^{\prime} \Sigma^{-1}{\mathbf{z}} + \varvec{\mu}_{\ell}^{\prime} \Sigma^{-1}{\mathbf{z}}\right),$$

(64)

where the other factors are constant with respect to z. By identification with Eq. 33, we thus have the results

$$\frac{1}{2} {\mathbf{z}}^{\prime} \Sigma^{-1} {\mathbf{z}} = {\mathbf{z}}^{\prime} {\mathbf{B}} {\mathbf{z}}, \quad \varvec{\mu}_{\ell}^{\prime} \Sigma^{-1} {\mathbf{z}} = - \varvec{\eta}_{\ell}^{\prime} {\mathbf{z}},$$

(65)

so that, as stated,

$$\Sigma = \frac{1}{2} {\mathbf{B}}^{-1}, \quad \varvec{\mu}_{\ell} = -\Sigma \varvec{\eta}_{\ell}.$$

(66)