Optimal Thresholding of Predictors in Mineral Prospectivity Analysis

Abstract

Some methods for analysing mineral prospectivity, especially the weights of evidence technique, require the predictor variables to be binary values. When the original evidence data are numerical values, such as geochemical indices, they can be converted to binary values by thresholding. When the evidence layer is a spatial feature such as a geological fault system, it can be converted to a binary predictor by buffering at a suitable cut-off distance. This paper reviews methods for selecting the best threshold or cut-off value and compares their performance. The review covers techniques which are well known in prospectivity analysis as well as unfamiliar techniques borrowed from other literature. Methods include maximisation of the estimated contrast, Studentised contrast, $\chi ^2$ test statistic, Youden criterion, statistical likelihood, Akman–Raftery criterion, and curvature of the capture–efficiency curve. We identify connections between the different methods, and we highlight a common technical error in their application. Simulation experiments indicate that the Youden criterion has the best performance for selection of the threshold or cut-off value, assuming that a simple binary threshold relationship truly holds. If the relationship between predictor and prospectivity is more complicated, then the likelihood method is the most easily adaptable. The weights-of-evidence contrast performs poorly overall. These conclusions are supported by our analysis of data from the Murchison goldfields, Western Australia. We also propose a bootstrap method for calculating standard errors and confidence intervals for the location of the threshold.

Appendices

Appendices

Capture–Efficiency Curve as a CDF

In the “Threshold Selection Using the Capture–Efficiency Curve” section, “Principle” section, we mentioned that the capture–efficiency curve can be regarded as a cumulative distribution function (cdf) in its own right. Here we clarify that comment.

A very subtle interpretation of the capture–efficiency curve used by statisticians is the “transformation to uniformity” or “probability integral transformation” (Kendall and Stuart 1973, p. 459 ff.; Hogg and Craig 1970, pp. 349–350). Suppose that the original predictor Z is replaced by a new predictor V, defined at each spatial location u by $V(u) = G(Z(u)) = a(Z(u))/a$, which is the area fraction of spatial locations where the predictor value does not exceed the value z. In words, at a given spatial location u, the value of V(u) is the fraction of area of the survey region where the original predictor Z does not exceed the value Z(u) which it takes at u. If Z is distance-to-nearest-fault, then V(u) is the area fraction occupied by the buffer at distance equal to the distance from u to the faults, that is, the buffer whose boundary passes through the point u. Then, the capture–efficiency curve is the cumulative distribution function of the transformed predictor V at the deposit points, while the diagonal line is the cumulative distribution function of V over all spatial locations in the survey region.

Likelihood Function for Threshold Model

This appendix provides elementary explanations for the appearance of the binomial probability distribution discussed in the “Significance Tests” section, “Significance Test for a Binary Predictor” section, and for the form of the likelihood function based on this distribution, and for the form of the likelihood discussed in the section on “Threshold Selection Using Change-Point Analysis”, “Profile Likelihood for the Threshold Model” section, which results from the assumption that the grid cells are very small in area relative to the survey region.

To simplify discussion, we assume that the geometry of the survey region S and of the prospective map feature B are fixed and known in advance, while the mineral deposit locations are discovered during the survey. In the notation of the section on Binary Predictors, the survey region has area a and the feature B has area $a_B$.

The survey process begins by dividing the survey region into N grid cells of equal area, then determining whether each grid cell contains or does not contain a deposit. For our purposes the results of the survey are the count $n_B$ of grid cells inside B which contain a deposit, and the count $n_{\overline{B}}$ of grid cells outside B which contain a deposit. The total number of grid cells containing deposits is $n = n_B + n_{\overline{B}}$.

Uniform Prospectivity Model Using Binomial Probabilities

We first consider the simple model in which prospectivity is uniform over the entire survey region. Each grid cell has the same probability p of containing a deposit. The outcomes in different grid cells are assumed to be statistically independent. There are N grid cells altogether. Therefore, the number n of grid cells containing a deposit follows a binomial distribution on N trials with success probability p; the probability that exactly n grid cells contain deposits is

$$\begin{aligned} {N \atopwithdelims ()n} p^n (1-p)^{N-n} . \end{aligned}$$

(26)

The expected total number of grid cells which contain deposits is Np.

The same principle applies to the grid cells inside the feature B; the probability that exactly $n_B$ grid cells inside B contain deposits is

$$\begin{aligned} {N_B \atopwithdelims ()n_B} p^{n_B} (1-p)^{N_B-n_B} , \end{aligned}$$

(27)

where $N_B = (a_B/a) N$ is the number of grid cells that constitute the feature B. The expected number of grid cells inside B that contain deposits is $p N_B = p (a_B/a) N$.

Again this applies to the grid cells outside B; the probability of obtaining $n_{\overline{B}}$ grid cells outside B which contain deposits is

$$\begin{aligned} {N_{\overline{B}}\atopwithdelims ()n_{\overline{B}}} p^{n_{\overline{B}}} (1-p)^{N_{\overline{B}}-n_{\overline{B}}} , \end{aligned}$$

(28)

where $N_{\overline{B}}= (a_{\overline{B}}/a) N$ is the number of grid cells in ${\overline{B}}$. The expected number of grid cells outside B that contain deposits is $p N_{\overline{B}}= (a_{\overline{B}}/a) p N$.

Combining (27) and (28), the probability of obtaining $n_B$ cells with deposits inside B and $n_{\overline{B}}$ cells with deposits outside B is

$$\begin{aligned} {N_B \atopwithdelims ()n_B} p^{n_B} (1-p)^{N_B-n_B} {N_{\overline{B}}\atopwithdelims ()n_{\overline{B}}} p^{n_{\overline{B}}} (1-p)^{N_{\overline{B}}-n_{\overline{B}}} \end{aligned}$$

which simplifies to

$$\begin{aligned} {N_B \atopwithdelims ()n_B} {N_{\overline{B}}\atopwithdelims ()n_{\overline{B}}} p^{n} (1-p)^{N-n} . \end{aligned}$$

(29)

This combining is justified because individual grid cell outcomes, and hence outcomes in disjoint parts of the survey region, are independent.

The likelihood function of the model is the probability (29) of observing the data $(n_B,n_{\overline{B}})$, treated as a function of the parameter p:

$$\begin{aligned} L(p) = {N_B \atopwithdelims ()n_B} {N_{\overline{B}}\atopwithdelims ()n_{\overline{B}}} p^{n} (1-p)^{N-n} . \end{aligned}$$

Constant factors that do not depend on p can be omitted, so the likelihood would often be reported as

$$\begin{aligned} L(p) = p^{n} (1-p)^{N-n} . \end{aligned}$$

(30)

Simple Threshold Model Using Binomial Probabilities

In the simple threshold model we assume that prospectivity is higher inside the feature B. Each grid cell inside B has probability $p_B$ of containing a deposit, while each grid cell outside B has a different probability $p_{\overline{B}}$ of containing a deposit, where $p_{\overline{B}}< p_B$. We simply replace p by $p_B$ in equation (27) to find that the probability of obtaining exactly $n_B$ cells with deposits inside B is

$$\begin{aligned} {N_B \atopwithdelims ()n_B} p_B^{n_B} (1-p_B)^{N_B-n_B} . \end{aligned}$$

(31)

Replacing p by $p_{\overline{B}}$ in Eq. (28), the probability of obtaining $n_{\overline{B}}$ grid cells outside B which contain deposits is

$$\begin{aligned} {N_{\overline{B}}\atopwithdelims ()n_{\overline{B}}} p_{\overline{B}}^{n_{\overline{B}}} (1-p_{\overline{B}})^{N_{\overline{B}}-n_{\overline{B}}} . \end{aligned}$$

(32)

Combining (31) and (32), the probability of obtaining $n_B$ cells containing deposits inside B and $n_{\overline{B}}$ cells containing deposits outside B is

$$\begin{aligned} {N_B \atopwithdelims ()n_B} p_B^{n_B} (1-p_B)^{N_B-n_B} {N_{\overline{B}}\atopwithdelims ()n_{\overline{B}}} p_{\overline{B}}^{n_{\overline{B}}} (1-p_{\overline{B}})^{N_{\overline{B}}-n_{\overline{B}}} . \end{aligned}$$

The likelihood function for the simple threshold model is (again omitting constant factors)

$$\begin{aligned} L(p_B, p_{\overline{B}}) = p_B^{n_B} (1-p_B)^{N_B-n_B} p_{\overline{B}}^{n_{\overline{B}}} (1-p_{\overline{B}})^{N_{\overline{B}}-n_{\overline{B}}} . \end{aligned}$$

(33)

The likelihood now has two arguments $p_B$ and $p_{\overline{B}}$ representing the probabilities of a deposit for grid cells inside and outside B, respectively. Notice that if the two probabilities were equal, $p_B = p_{\overline{B}}= p$, then (33) would collapse to (30).

Rescaling in Terms of Density

Since the grid cells are artificial (and in particular their size is an arbitrary choice), it is useful to rescale the equations so that they depend as little as possible on the grid geometry. This can be done by using the average density of deposits, $\mu$, defined as the expected number per unit area.

In the uniform prospectivity model, the expected total number of grid cells that contain deposits is Np. The average density is therefore $\mu = Np/a$. Noting that the area of one grid cell is $\epsilon = a/N$, we see that the average density is equal to $\mu = p/\epsilon$. Equivalently, $p = \mu \epsilon$, that is, the probability of a deposit in any given cell is equal to the average density of deposits times the area of the grid cell. Replacing p by $\mu \epsilon$ in Eq. 30, and removing constant factors, we get the likelihood

$$\begin{aligned} L(\mu ) = \mu ^{n} (1-\mu \epsilon )^{N-n} . \end{aligned}$$

(34)

In the simple threshold model, we have two different densities inside and outside the feature B. Inside B, the expected number of grid cells containing deposits is $N_B p_B$ and the area is $a_B= N_B \epsilon$ so the density is $\mu _B = N_B p_B/a_B = p_B/\epsilon$. Outside B, the density is $\mu _{\overline{B}}= p_{\overline{B}}N_{\overline{B}}/a_{\overline{B}}= p_{\overline{B}}/\epsilon$. Substituting into (33) we get the likelihood

$$\begin{aligned} L(\mu _B, \mu _{\overline{B}}) = \mu _B^{n_B} (1-\epsilon \mu _B)^{N_B-n_B} \mu _{\overline{B}}^{n_{\overline{B}}} (1- \epsilon \mu _{\overline{B}})^{N_{\overline{B}}-n_{\overline{B}}} . \end{aligned}$$

(35)

Small Grid Cells

Finally, we suppose that the grid cells are very small. Then, N is very large, $\epsilon = a/N$ is very small. For the uniform prospectivity model, in the likelihood (30) the term $(1-\mu \epsilon )^{N-n}$ converges to the exponential $\exp (-\mu N \epsilon ) = \exp (-\mu a)$ as N becomes large, giving the likelihood

$$\begin{aligned} L(\mu ) = \mu ^{n} \exp (-\mu a). \end{aligned}$$

For the simple threshold model, the likelihood (33) similarly converges to

$$\begin{aligned} L(\mu _B, \mu _{\overline{B}}) = \mu _B^{n_B} \mu _{\overline{B}}^{n_{\overline{B}}} \exp (-\mu _B a_B - \mu _{\overline{B}}a_{\overline{B}}). \end{aligned}$$

Taking logarithms, the log-likelihood for the null model of uniform prospectivity is

$$\begin{aligned} \ln L(\mu ) = n \ln \mu -\mu a , \end{aligned}$$

(36)

and for the simple threshold model

$$\begin{aligned} \ln L(\mu _B, \mu _{\overline{B}}) = n_B \ln \mu _B + n_{\overline{B}}\ln \mu _{\overline{B}}-\mu _B a_B - \mu _{\overline{B}}a_{\overline{B}}. \end{aligned}$$

(37)

If the feature B is the region determined by thresholding a spatial predictor function Z(u) to have values less than or equal to a threshold z, then $a_B = a(z)$ is the area of this region, $n_B = n(z)$ is the number of deposits with predictor values less than or equal to z, and we have $a_{\overline{B}}= a - a_B$ and $n_{\overline{B}}= n - n_B = n - n(z)$, so that Eq. 37 is translated into Eq. 10 of the paper.

Connections Between the Threshold Selection Criteria

In this appendix, we provide proofs of claims made in the “Significance Tests” section and “Mathematical Connections between the Criteria” section regarding relationships between various threshold selection criteria.

Forms of the $\chi ^2$ Statistic

First, we prove the claim in the “Significance Tests” section that the general form of the $\chi ^2$ statistic (18) reduces to the special form (19) in this case. For any feature B, define $e_B = (n/ a) a_B = ( a_B/ a) n$ and $e_{\overline{B}}= (n/ a) a_{\overline{B}}= ( a_{\overline{B}}/ a) n$, the expected counts in B and ${\overline{B}}$, respectively, when the deposits are randomly distributed with constant density. Then

$$\begin{aligned} n_{\overline{B}}- e_{\overline{B}}= (n - n_B) - (n - e_B) = -(n_B - e_B) \end{aligned}$$

so that

$$\begin{aligned} X^2&= \frac{(n_B - e_B)^2}{e_B} + \frac{(n_{\overline{B}}- e_{\overline{B}})^2}{e_{\overline{B}}} = \frac{(n_B - e_B)^2}{e_B} + \frac{(n_B - e_B)^2}{e_{\overline{B}}} = (n_B - e_B)^2 \left( \frac{1}{e_B} + \frac{1}{e_{\overline{B}}} \right) \\&= (n_B - e_B)^2 \frac{e_{\overline{B}}+ e_B}{ e_B e_{\overline{B}}} = \frac{n}{e_B e_{\overline{B}}} (n_B - e_B)^2, \end{aligned}$$

which is equivalent to (19).

Connection Between Akman–Raftery and $\chi ^2$ Criteria

Next we prove the connection between $\hbox {AR}(z)$ and X(z) stated in Eq. 14, in the “Mathematical Connections between the Criteria” section, and again in the “Significance Tests” section, “Akman–Raftery (Constrained $\chi ^2$ Test)” section.

For candidate threshold value z, let n(z) be the number of deposits below the threshold, a(z) the area of study region below the threshold, n the total number of deposits, a the total area of study region. Define $s(z) = a(z)/a$, the area fraction.

Then, from the definition (13) of the Akman–Raftery criterion,

$$\begin{aligned} \hbox {AR}(z)&= \sqrt{s(z) (1-s(z))} \left( \frac{n(z)}{s(z)} - \frac{n - n(z)}{1- s(z)} \right) \\&= \sqrt{s(z) (1-s(z))} \frac{ n(z) (1 - s(z)) - s(z) (n - n(z)) }{ s(z) (1-s(z)) } \\&= \sqrt{s(z) (1-s(z))} \frac{ n(z) - n s(z) }{ s(z) (1-s(z)) } \\&= n \frac{ \frac{n(z)}{n} - s(z) }{ \sqrt{s(z) (1-s(z))} } \\&= \sqrt{n} X(z). \end{aligned}$$

This proves (14).

Relation Between $\chi ^2$ Statistic and Youden Criterion

Here, we prove the claim, made in the “Mathematical Connections between the Criteria” section, that the $\chi ^2$ statistic is the standardised version of the Youden criterion.

Under the null hypothesis that the density of deposits is uniform, if we treat the total number of deposits as fixed, then for any threshold z the count n(z) follows a binomial distribution with n trials and success probability $p = a(z)/ a = s(z)$. This distribution has variance $n p(1-p) = n s(z) (1-s(z))$. Accordingly $Y(z) = (n(z)/n) - s(z)$ has variance $s(z)(1-s(z))/n$ so that the standard error of Y(z), under the null hypothesis of a uniform density of deposits, is $\hbox {se}_0(Y(z)) = \sqrt{s(z) (1-s(z))/n}$. Finally, inspecting (7) yields (15).

Derivation of Optimal Threshold for Gradual Decline in Prospectivity

This appendix provides the proof of the claim in the “Gradual Decline in Prospectivity” section of the “Discussion” section that, if the prospectivity (i.e. the density of deposit points) is a decreasing function of the predictor value, then the optimal threshold for the Youden criterion is the threshold at which that function equals the average density of deposits over the survey region.

We define the prospectivity as the spatially varying density (intensity) of deposit points considered as a function $\lambda (u)$ of spatial location u. In any given grid cell, the expected number of deposits is equal to $\lambda (u)\epsilon$ where u is the location of the cell centre and $\epsilon > 0$ is the cell area. See Baddeley (2018) for an explanation.

Assume first that the density of deposits depends only on the predictor Z. That is, we assume that $\lambda (u)$ depends on Z(u) through the relation $\lambda (u) = \rho (Z(u))$, where $\rho$ is a nonnegative function. Several examples of the relationship between predictor and prospectivity are shown in the top row of Figure 23; these are graphs of $\rho (z)$ against z. Assume further, as we did in “Gradual Decline in Prospectivity” section, that $\rho (z)$ is a decreasing function of z, such as those in the left and right columns of Figure 23. Then, G(z) is the spatial cumulative distribution function of the predictor over the study region S, that is,

$$\begin{aligned} G(z) = \frac{1}{ a} \int _S {{\mathbf {1}}} \{ Z(u) \le z \} \, \mathrm{d}u, \end{aligned}$$

where ${{\mathbf {1}}} \{ Z(u) \le z \}$ is the indicator function, equal to 1 if $Z(u) \le z$ and equal to 0 otherwise. The expected total number of deposits ${{\mathbb {E}}}[n]$ is equal to the integral of the intensity function,

$$\begin{aligned} {{\mathbb {E}}}[n] = \int _S \lambda (u) \, \mathrm{d}u = \int _S \rho (Z(u)) \, \mathrm{d} u \end{aligned}$$

This integral over the spatial domain S can be transformed into a one-dimensional integral

$$\begin{aligned} {{\mathbb {E}}}[n] = a \int _{-\infty }^\infty \rho (z) \, \mathrm{d} G(z). \end{aligned}$$

The cumulative distribution function of the values of Z at the deposit points, averaged over all random outcomes, is

$$\begin{aligned} {\widetilde{F}}(z) = \frac{ a \, \int _{-\infty }^z \rho (v) \, \mathrm{d} G(v) }{ a \, \int _{-\infty }^\infty \rho (v) \, \mathrm{d} G(v) } = \frac{1}{\mu } \int _{-\infty }^z \rho (v) \, \mathrm{d} G(v), \end{aligned}$$

(38)

where $\mu = {{\mathbb {E}}}[n]/ a$ is the average density of deposits over the whole domain, and v is the dummy variable of integration.

The expected capture–efficiency curve is the graph of ${\widetilde{F}}(z)$ against G(z) for all z; equivalently it is the graph of the function $s \mapsto {\widetilde{F}}(G^{-1}(s))$ where $G^{-1}$ is the inverse function of G. Assuming differentiability, the slope of the expected capture–efficiency curve is

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d} s} {\widetilde{F}}(G^{-1}(s)) = \frac{{\widetilde{F}}^\prime (G^{-1}(s))}{G^\prime (G^{-1}(s))}. \end{aligned}$$

But from (38) we have ${\widetilde{F}}^\prime (z) = \rho (z) G^\prime (z)/\mu$, so that the slope of the expected capture–efficiency curve at a given area fraction s is equal to $\rho (z)/\mu$, where $z = G^{-1}(s)$.

The Youden method selects the point on the capture–efficiency curve with slope equal to 1. Ignoring sampling variability, that is, if we replace the observed capture–efficiency curve by the expected capture–efficiency curve, the Youden method selects the point with slope $\rho (z)/\mu = 1$, i.e., it selects the threshold value z for which $\rho (z) = \mu$.

Non-negligible Grid Cell Size

Equations in the paper assume, for simplicity and clarity, that the area of a grid cell is negligible. To be precise, if $\epsilon$ denotes the area of one grid cell, then we assume that $n \epsilon$, the total area of all cells containing deposits, can be treated as zero. This appendix lists the modifications to these equations that are necessary when $n\epsilon$ is not negligible.

In Eq.  1, the revised formulae are

$$\begin{aligned} W_{+}= \ln \frac{n_B}{ a_B - \epsilon n_B} - \ln \frac{n}{ a} \end{aligned}$$

(39)

$$\begin{aligned} W_{-}= \ln \frac{n_{\overline{B}}}{ a_{\overline{B}}- \epsilon n_{\overline{B}}} - \ln \frac{n}{ a} \end{aligned}$$

(40)

$$\begin{aligned} {\widehat{C}}= \ln \frac{n_B}{ a_B - \epsilon n_B} - \ln \frac{n_{\overline{B}}}{ a_{\overline{B}}- \epsilon n_{\overline{B}}}. \end{aligned}$$

(41)

Equation 2 becomes

$$\begin{aligned} {\widehat{C}}(z) = \ln \left( \frac{n(z)}{ a(z) - \epsilon n(z)} \bigg / \frac{n-n(z)}{ a- a(z) - \epsilon (n-n(z))} \right). \end{aligned}$$

(42)

Equation 3 becomes

$$\begin{aligned} \hbox {se} ({\widehat{C}}) = \sqrt{ \frac{1}{n_B} + \frac{1}{n_{\overline{B}}} + \frac{1}{ a_B/\epsilon - n_B} + \frac{1}{ a_{\overline{B}}/\epsilon - n_{\overline{B}}} } \end{aligned}$$

(43)

and Eq. 5 becomes

$$\begin{aligned} \hbox {se}({\widehat{C}}(z)) = \sqrt{ \frac{1}{n(z)} + \frac{1}{n-n(z)} + \frac{1}{ a(z)/\epsilon - n(z)} + \frac{1}{( a - a(z))/\epsilon - (n - n(z))} }. \end{aligned}$$

(44)

Equation (8) for the Youden criterion becomes

$$\begin{aligned} Y(z) = \frac{n(z)}{n} - \frac{ a(z)-\epsilon n(z)}{ a - \epsilon n}. \end{aligned}$$

(45)

There are no changes in other equations, except that equation (12) does not hold. In the right panel of Figure 10, the ROC curve should be used instead of the capture–efficiency curve.