Information Gathering in Bayesian Networks Applied to Petroleum Prospecting

Abstract

The optimal design of data acquisition is not obvious in Bayesian network models. The dependency structure may vary dramatically, which makes learning and information evaluation complicated and sometimes non-intuitive. The motivation for working on this topic is petroleum exploration, and the application of this paper is prospect selection in the North Sea. Here, the data gathering is often carried out during seasonal campaigns, and it is useful to plan the experimentation and to understand which data are likely to be most informative. Information measures are used to compare possible future observation sets. Four information measures are studied: Shannon Entropy, sum of variances, Node-wise Entropy and overall prediction error. The Shannon Entropy is commonly considered the standard measure of information, and the Node-wise Entropy measure can be interpreted as an approximation to the former. The variance measure links uncertainty and variance. The prediction error measure is tied to decision-making rules. The results lead to new insight about prospect selection. For example, the Node-wise Entropy and the variance measure behave similarly, and the optimal observation set of Shannon Entropy does not correspond to what one intuitively would consider as minimizing unknown information in this case.

Appendix: Proof of Theorem 1

Proof

Let $\emptyset \subseteq A \subset B \subseteq L$.

The Shannon Entropy measure has

$$\begin{aligned} \mu _\mathrm{ShE}(A)&= - \mathbb {E}_{[X_{L}]} [ \log \mathbb {P}(X_{L {\setminus } A}|X_{A}) ] \\&= - \mathbb {E}_{[X_{L}]} [ \log ( \mathbb {P}(X_{L {\setminus } B}|X_{B})\mathbb {P}(X_{B {\setminus } A}|X_{A}) ) ] \\&= - \mathbb {E}_{[X_{L}]} [ \log \mathbb {P}(X_{L {\setminus } B}|X_{B}) ] - \mathbb {E}_{[X_{L}]} [ \log \mathbb {P}(X_{B {\setminus } A}|X_{A}) ] \\&\ge - \mathbb {E}_{[X_{L}]} [ \log \mathbb {P}(X_{L {\setminus } B}|X_{B}) ] \\&=\mu _\mathrm{ShE}(B), \end{aligned}$$

with equality if and only if the distribution $\mathbb {P}(X_{B {\setminus } A}|X_{A})$ is trivial for each assignment to $X_{A}$.

Observe that $f_\mathrm{NwE}$ and $f_\mathrm{Var}$ are strictly concave, since

$$\begin{aligned} f''_\mathrm{NwE}=\frac{-1}{p(1-p)} \quad \text { and }\quad f''_\mathrm{Var}=-2 \quad \forall p \in \left<0,1\right>, \end{aligned}$$

while $f_\mathrm{PrE}$ is concave on $\left[ 0,1 \right]$ and linear on $\left[ 0,1/2 \right]$ and on $\left[ 1/2,1\right]$. Fix an $i \in L$, and assume a measure term of the form

$$\begin{aligned} \mu ^{i}(B) = \mathbb {E}_{[X_{B}]} f\left( \mathbb {P}(X_{i}=1|X_{B}) \right) , \end{aligned}$$

for a concave function $f:[0,1] \rightarrow \mathbb {R}$ with $f^{-1}(0)= \{0,1\}$. For a given assignment $X_{A}=x_{A}$ to the random variables in A, $\mathbb {P}(X_{i}=1|X_{B})$ is a function of $X_{B {\setminus } A}$ and thus a random variable. By Jensen’s inequality,

$$\begin{aligned} f(\mathbb {P}(X_{i}=1|X_{A})) \ge \mathbb {E}_{[X_{B{\setminus } A}|X_{A}]}f(\mathbb {P}(X_{i}=1|X_{B})), \end{aligned}$$

with equality if and only if f is linear on

$$\begin{aligned} \left[ \min _{x_{B {\setminus } A}} \{\mathbb {P}(X_{i}=1|X_{B})\}, \max _{x_{B {\setminus } A}} \{\mathbb {P}(X_{i}=1|X_{B})\}\right] . \end{aligned}$$

If $i \in B$, the right-hand side of the inequality is zero valued, and there is equality if and only if $\mathbb {P}(X_{i}|X_{A})$ is trivial as well. If $i \in L {\setminus } B$ and f is strictly concave, the inequality is strict unless

$$\begin{aligned} \mathbb {P}(X_{i}=1|X_{B}) \equiv \mathbb {P}(X_{i}=1|X_{A}). \end{aligned}$$

Since the assignment $X_{A}=x_{A}$ was arbitrary,

$$\begin{aligned} \mathbb {E}_{[X_{A}]} \left[ f(\mathbb {P}(X_{i}=1|X_{A})) \right] \ge \mathbb {E}_{[X_{B}]} \left[ f(\mathbb {P}(X_{i}=1|X_{B})) \right] , \end{aligned}$$

and the claims follow. $\square$