The Bayesian maximum entropy method for lognormal variables

Abstract

The Bayesian maximum entropy (BME) method can be used to predict the value of a spatial random field at an unsampled location given precise (hard) and imprecise (soft) data. It has mainly been used when the data are non-skewed. When the data are skewed, the method has been used by transforming the data (usually through the logarithmic transform) in order to remove the skew. The BME method is applied for the transformed variable, and the resulting posterior distribution transformed back to give a prediction of the primary variable. In this paper, we show how the implementation of the BME method that avoids the use of a transform, by including the logarithmic statistical moments in the general knowledge base, gives more appropriate results, as expected from the maximum entropy principle. We use a simple illustration to show this approach giving more intuitive results, and use simulations to compare the approaches in terms of the prediction errors. The simulations show that the BME method with the logarithmic moments in the general knowledge base reduces the errors, and we conclude that this approach is more suitable to incorporate soft data in a spatial analysis for lognormal data.

Appendix A

In this appendix, we show that the multivariate lognormal distribution, Eq. (7), is the maximum entropy pdf for a vector of variables, ${\mathbf{Z}} = {\left( {Z_{1} ,Z_{2} ,...,Z_{N} } \right)}^{{\text{T}}} ,$ given the general knowledge base stated in Eq. 6.

For a transformation, Y = φ(Z), the pdfs for Z and for Y are linked by:

$$ f{\left( {\mathbf{z}} \right)} = {\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}g{\left( {\mathbf{y}} \right)}. $$

(A1)

So the entropy for f(z) is given by:

$$ \begin{aligned}{} H_{{\mathbf{Z}}} {\left[ {f{\left( {\mathbf{z}} \right)}} \right]} & = - {\int {f{\left( {\mathbf{z}} \right)}\ln f{\left( {\mathbf{z}} \right)}} }{\text{d}}{\mathbf{z}} \\ & = - {\int {{\left\{ {{{\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}g{\left( {\mathbf{y}} \right)}\ln {\left[ {{\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}g{\left( {\mathbf{y}} \right)}} \right]}} \mathord{\left/ {\vphantom {{{\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}g{\left( {\mathbf{y}} \right)}\ln {\left[ {{\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}g{\left( {\mathbf{y}} \right)}} \right]}} {{\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}}}} \right. \kern-\nulldelimiterspace} {{\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}}} \right\}}} }{\text{d}}{\mathbf{y}} \\ & = - {\int {g{\left( {\mathbf{y}} \right)}\ln g{\left( {\mathbf{y}} \right)}} }{\text{ d}}{\mathbf{y}} - {\int {g{\left( {\mathbf{y}} \right)}\ln {\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}} }{\text{d}}{\mathbf{y}} \\ & = H_{{\mathbf{Y}}} {\left[ {g{\left( {\mathbf{y}} \right)}} \right]} - {\text{E}}{\left[ {\ln {\left| {\frac{{{\text{d}}{\mathbf{y}}}} {{{\text{d}}{\mathbf{z}}}}} \right|}} \right]} \\ & = H_{{\mathbf{Y}}} {\left[ {g{\left( {\mathbf{y}} \right)}} \right]} - {\text{E}}{\left[ {\ln J} \right]}, \\ \end{aligned} $$

(A2)

where J is the Jacobian of the transformation. In the case of φ(Z) being the logarithmic transform we get $J = {\prod {1 \mathord{\left/ {\vphantom {1 {z_{i} }}} \right. \kern-\nulldelimiterspace} {z_{i} }} };$ the expectation on the right-hand side of Equation (A2) now gives the sum of the expectations of the y _i s, which are known because of our constraints on the logarithmic means, ${\int\limits_{ - \infty }^\infty {y_{i} g{\left( {\mathbf{y}} \right)}} }{\text{ d}}{\mathbf{y}} = \mu _{i} .$ Therefore we have:

$$ H_{{\mathbf{Z}}} {\left[ {f{\left( {\mathbf{z}} \right)}} \right]} = H_{{\mathbf{Y}}} {\left[ {g{\left( {\mathbf{y}} \right)}} \right]} + {\sum\limits_{i = 1}^N {\mu _{i} } }. $$

(A3)

Thus, if we choose g(y) such that it maximizes H _Y[g(y)] over all distributions for Y, then we know that this distribution gives the maximum value of the entropy, H _z[f(z)], over all distributions for Z. Since the Gaussian distribution for g(y) maximizes H _Y[g(y)], the back-transform of this distribution through Equation (A1) (i.e. the multivariate lognormal distribution) is the maximum entropy pdf for the SRF, Z, given the mean and covariance for Y.