Model-Based Geostatistics from a Bayesian Perspective: Investigating Area-to-Point Kriging with Small Data Sets

2.1 Geostatistics Basics: Gaussian Random Field

2.2 Area-to-Point Kriging

For a mathematical elaboration of the above equations, refer to Wackernagel (2014). The term regression kriging (RK) is used later to refer to simple kriging of the trend residuals, an approach which disregards any uncertainty in the estimated plug-in values of the trend parameters, which results in a different kriging variance. In the following sections, the framework presented above will be extended to a Bayesian one.

2.3 Bayesian Statistics

In the Bayesian framework, the prior \(p({\varTheta })\) needs to be defined, stating the current state of knowledge—or, inversely formulated, the current state of ignorance—about the parameters. In many cases, a low-informative prior is desired; however, formulating a prior that ‘lets the data speak for itself’ is not always straightforward (Seaman et al. 2012; Lindley 2004). Also, a ‘conjugate’ prior is often chosen so that its distribution function matches the likelihood, resulting in a closed-form description of the posterior (Albert 2009).

For various reasons, prior distributions that do not integrate to a finite value (i.e., cannot be normalised to integrate to one) might be considered, termed in the Bayesian framework as ‘improper’ priors. Improper priors can result in improper (poorly defined) posterior probability densities.

In the remainder of this paper, a posterior probability distribution—proper or not—is indicated by \(f_p(\ldots |\ldots )\), the likelihood by \(f_l(\ldots |\ldots )\), and a prior distribution—again having propriety or not—by \(f_0(\ldots )\). When the function type is ambiguous, its interpretation depends on the context and its arguments.

3.1 Partly Analytical Bayesian Area-to-Point Algorithm

Given the above prior and likelihood function, the joint posterior distribution for the parameters is (up to a constant of proportionality)

$$\begin{aligned} \begin{aligned}&f_p(\varvec{\beta }, \sigma ^2, \phi | \bar{\varvec{z}}) \\&\quad \propto \frac{1}{(2\pi )^\frac{m}{2} ({\sigma ^2})^{\frac{m}{2}} \left| \varvec{ \bar{C}}\right| ^\frac{1}{2} } \hbox {exp} \left\{ - \frac{1}{2{\sigma ^2}}(\bar{\varvec{z}} - \bar{\varvec{X}}\varvec{\beta })^\mathrm{T} \varvec{ \bar{C}}^{-1} (\bar{\varvec{z}} - \bar{\varvec{X}}\varvec{\beta }) \right\} \frac{1}{\sigma ^2} f_0(\phi ). \end{aligned} \end{aligned}$$

(14)

Based on the above assumptions, Fig. 1 illustrates our partly analytical Bayesian algorithm, Bayesian areal kriging (BAK), to infer the marginal posterior distributions of all parameters and to calculate and summarise predictive distributions. The relevant equations and their derivation are given in Online Resource A; the summary stating the main equations follows in the coming sections.

Open image in new window

3.2 Methodological Details

In this work, a number of increasingly Bayesian approaches to ATPK are applied and compared (Table 1). The first three rows of the table represent plug-in approaches for some of the parameters (i.e., the stated parameters are first estimated, by maximising a likelihood or marginal likelihood function, before being plugged into the relevant predictive distribution equation for prediction), while the final row represents the fully Bayesian approach. In the case of maximum likelihood estimation (ML, and not implemented in this work), all parameters (in the geostatistical context: regression coefficients and spatial covariance parameters) are estimated by analytically or numerically maximising the likelihood. This general approach was consolidated by Fisher almost a century ago (Stigler 2007) and applied in geostatistics for example by Kitanidis (1983) and Lark (2000). REML, which has been advocated for several decades in geostatistics, is based on a likelihood function for a set of projected data rather than the original data, and gives conditionally unbiased estimates for the spatial parameters (Webster and Oliver 2007; Lark and Cullis 2004); see also Sect. A2 in Online Resource A. REML represents a form of marginal likelihood (a likelihood function in which some parameters have been marginalised), and has been presented in a Bayesian framework as such (the integral of the likelihood function with respect to the trend parameters, assuming a flat improper prior for these parameters) (Harville 1974). Note that \(f_0(\varvec{\beta })\) can be considered an uninformative prior when neglected—this is often valid for centrality parameters but not for other parameters. Underpinning the same approach, UK takes the uncertainty in the trend coefficients into account, making it a logical combination with REML. Within this research, the combined application of REML and UK is indicated by ‘REML approach’. The next gradation towards the fully Bayesian approach is maximum likelihood with both trend and variance integrated out, in the context of this paper indicated by the generic term ‘maximum marginal likelihood’ (MML). Finally, the full Bayesian approach (also referred to as ‘Bayesian approach’) provides a posterior distribution of all parameters, while in the prediction all parameters are integrated out and the uncertainty of all parameters is taken into account.

In the following sections, REML, MML and the Bayesian approach are compared, and for the Bayesian approach different priors for \(\phi \) (as defined in the following section) are applied. All algorithms (including the central BAK algorithm as presented in Fig. 1) are written in the statistical programming language R, and are available at Steinbuch et al. (2019).

4.1 Single Simulation

4.1.2 Results

The original signal (assumed to be unobservable, and represented as point values), the areal means (the ‘observations’) and the predictions are shown in Fig. 2. The difference between the REML and the full Bayesian approach is mainly in the prediction uncertainty: the Bayesian approach gives a slightly larger prediction interval.

Open image in new window

The marginal densities in Fig. 3 show that, based on this—small—simulated data set, it is rather difficult to identify the distance parameter \(\phi \), which has a very flat mode. The REML estimates for \(\phi \) and \(\sigma ^2\) (point values) are close to the modes of the respective marginal posteriors from the Bayesian approach. For the trend parameters, the marginal Bayesian posterior distributions are slightly skewed and wider than the corresponding distributions based on REML (Gaussian distributions parameterised by the GLS estimate and estimation variance), indicating that more uncertainty is included.

Open image in new window

Open image in new window

Open image in new window

The pairwise joint posteriors (\(\phi \) with each other parameter) are shown in Fig. 4: \(\sigma ^2\) and \(\phi \) seem to have a positive correlation (subfigure a), while \(\beta _1\) has a slightly negative correlation with \(\phi \) (b) and \(\beta _2\) a slightly positive correlation (c). Figure 5 shows the variogram models obtained with REML and the Bayesian-uniform approach. Table 2 shows the validation results of this single simulation. The quality of the prediction (RMSE) is almost equal for the REML and Bayesian approaches, and better than for the baseline approach. The ME and max(MPP) are close to zero, indicating the absence of bias, and a discretisation grid (which is different from the prediction grid) of sufficient density to provide good approximations of areal-average covariances and areal-average values of covariates. The uncertainty validation value mean(StSE) shows that REML is on average optimistic, while the Bayesian approach is conservative. Note that the baseline approach does not provide a quantification of prediction uncertainty.

Table 2

Results of single simulation run with validation on original data points

	mean(StSE)	RMSE	ME	max(MMP)
REML	1.169	0.585	0.000	0.009
Bayesian-uniform	0.827	0.588	0.000	0.009
Baseline	–	0.666	0.000	0.000

REML restricted maximum likelihood, StSE standardised squared error, ME mean error, RMSE root mean squared error, max(MPP) maximal found deviation from the mass preserving property

4.2 Simulation Ensemble

In this section, results are generalised by generating many simulations, varying only the random number seed, while comparing validation statistics on the outcomes of several approaches: REML, MML and full Bayesian with the three different priors for \(\phi \) indicated earlier. The applied inverse-gamma prior for \(\phi \) is set as somewhat informative with shape \(= 11\) and rate \(= 600\). This results in a mean of 60 au, emulating a situation where decent prior knowledge about the range is available. Also, the number of observations m is varied by dividing the line into m sections of equal length; this together is one ‘session’. Furthermore, to investigate how the approaches behave for differently simulated data sets and different inference settings, both are varied into an ‘ensemble’ of many sessions. The settings as used in standard session 1 are given in Sect. 4.1.1. Sessions 2 and 3 vary the upper bound for the uniform prior and for the REML and MML searches for estimation of \(\phi \) in comparison with standard session 1 (where \(\phi = 300\) au). In session 4, the trend is removed (so that the inferential model has to infer the mean only); in session 5, the trend is based on a separate Gaussian random field (GRF) rather than on the coordinates. Sessions 6, 7 and 8 introduce a misfit between the simulation model and the inferential model, where the correlation function in the simulation model is changed or a nugget component is added—the inferential model stays unchanged. Finally, sessions 9 and 10 show the effects of a misfit between the actual signal and the support of the available data (i.e., short or very long distance parameter used in simulation compared with the area sizes and total extent), which might make it difficult to identify parameters.

Table 3 presents the results expressed as the average (and, in small font, the corresponding standard deviation) over 250 means of the standardised squared error (mean(StSE)). Table D1 in Online Resource D shows the results of the analogous two-dimensional simulations. Additional validation statistics and assessments regarding \(\phi \) and \(\sigma ^2\) for the simulation ensemble can be found in Online Resource C, which also includes \(m = 15\) and \(m = 30\) for the one-dimensional simulations, and in Online Resource E for the corresponding figures of the two-dimensional simulations.

Table 3

Mean standardised squared error (mean(StSE)) for the one-dimensional simulation ensemble, comparing restricted maximum likelihood (REML), maximum marginal likelihood (MML) and three full Bayesian approaches with different priors for \(\phi \)

Open image in new window

In the online version of this paper, colours are used to highlight larger and smaller values for the readers’ convenience

5.1 Synthetic Case Study: Vegetation Index Data, with Validation on Point Support

To briefly investigate how REML, MML and full Bayesian would perform for a real-world data set, a remote-sensing vegetation index, CFAPAR-27, was used as the variable of interest. These data are used as a covariate in the real case study (spatial prediction of crop yield in Burkina Faso, Sect. 5.2) and therefore concisely described in Online Resource F. This spatial variable is, obviously, available on pixel support. The CFAPAR-27 data were masked using the crop yield mask (see also Sect. 5.2), and subsequently aggregated over the 45 provinces of Burkina Faso. As covariates for inference, two climate variables broadly representing rainfall and temperature (CRAIN-EC-27 and TMIN-EC-21) and one variable representing soil pH (PHAQ) were used. Gaussianity for all real-world variables of interest was assumed.

ATPK was applied using four approaches: (1) REML, (2) MML, (3) the full Bayesian approach using the uniform prior for \(\phi \), and (4) the full Bayesian approach using the reference prior for \(\phi \). For REML and MML, the parameter search for \(\phi \) was bounded between 37 and 300 km, being roughly the smallest distance between the centres of any two areas, and one third of the largest extent of the region of interest, respectively. The same bounds defined the uniform prior for the full Bayesian approach.

The resulting mean(StSE) was 2.87, 2.73, 2.92 and 2.59 for the REML, MML, Bayesian-uniform and Bayesian-reference approaches, respectively, showing that prediction uncertainty was seriously underestimated by all approaches. The mean(StSE) of the Bayesian-uniform approach could be changed by several tenths by adjusting the bounds of the uniform prior. All RMSE values for the four approaches were around 6.19 (compared with the baseline approach RMSE of 16.54), indicating that they offered the same prediction quality and probably quite similar predictions.

5.2 Real Case Study: Crop Yield Data

As a real-world case study, this paper predicts yields of sorghum and millet, both cereal staple foods, in Burkina Faso, West Africa. The observation areas are the 45 provinces, for which only the average yields are known (averaged over the years 2000–2013, and provided by AGRHYMET), as shown in Fig. 6 for millet. Covariates for the trends as suggested by Brus et al. (2018) are used: for millet no covariates, and for sorghum four covariates are shown and briefly explained in Online Resource F.

REML, MML, Bayesian-uniform and Bayesian-reference approaches were applied, with similar settings to those of Sect. 5.1 in the previous analysis of the vegetation index data. The observed millet yields, and the resulting predictions and prediction uncertainties (standard deviations of the predictive distributions) when applying MML, are presented in Fig. 6; maps are presented first over the entire study region, then focused on a subregion to reveal more detail. Similar maps for sorghum are presented in Online Resource F.

Open image in new window

Figure 7 shows the densities of the millet yield predictions and prediction standard deviations based on all four approaches, indicating that the Bayesian and, to a lesser extent, MML approaches generated larger prediction uncertainties than REML. For sorghum (see Online Resource F), the Bayesian-reference-calculated prediction diverted from the other approaches, due to the tendency of the distance parameter to move as close as possible to zero; the applied lower bound for the uniform prior for \(\phi \) and for the REML and MML parameter searches imposed a limit on this effect. This shows again that a seemingly arbitrary choice of a uniform prior or of a parameter range for REML or MML might influence the resulting prediction uncertainty.

Open image in new window

6.1 Setting Uniform Prior

Both in the simulations (Table 3 and Online Resource C) and in the case studies, the choice of the upper and lower bounds of a uniform prior for \(\phi \) can influence the prediction uncertainty, especially (but not exclusively) with smaller data sets and if the posterior mode of \(\phi \) coincides with one of the bounds of the prior. This effect can also occur with REML and MML approaches, where the search for the optimum value of \(\phi \) is bounded by the same limits. It should be stressed that, in this context, this ‘flat’ uniform prior cannot be considered uninformative. The fact that the posterior modes of \(\phi \) (resulting from Bayesian approaches), or \(\hat{\phi }\) (from the REML approach), often coincided with one of the bounds (for example see the ‘\(\hat{\phi }\), \(\hbox {mode}(\phi ) \approx ~ \min , ~ \max \)’ columns in Online Resources C and E, but also sorghum in the case study) highlights the importance of carefully considering such prior or parameter search settings in geostatistical practice.

6.2 Reference Prior

According to the simulations, the reference prior did not perform well, being in many cases too conservative about prediction uncertainty, and pushing posterior distributions of the distance parameter too strongly towards zero (see also Online Resources C and D). In the case of small \(\nu _\mathrm{sim}\) or \(\tau _\mathrm{sim}^2 > 0\), this conservatism compensated to some extent for model misspecification. Berger et al. (2001) derived the form of the reference prior for analysis of spatial point-support data, and the same logic was applied—with area-to-area average correlations replacing the point-to-point correlations of Berger et al. (2001)—to justify a similar prior for analysis of areal-support data. However, although the logic to derive the form of the prior follows analogously, the authors are unsure of the analogous logic to ensure propriety of the resulting prior and posterior distributions. As such, even if the simulations would have demonstrated a strong advantage, it would require further work to derive the required proofs of propriety. With the simulation results not demonstrating strong advantages, other priors for \(\phi \) are currently recommended.

6.3 Misspecified Model

Validation statistics about prediction uncertainties are very sensitive to the misspecification of variogram parameters that determine the smoothness of the spatial signal. Examples are the nugget parameter and the smoothness parameter of the Matérn covariance function, as demonstrated in the simulation sessions 6, 7 and 8 and, in the authors’ opinion, in the vegetation index synthetic case study. Short-range spatial relationships are however difficult, if not impossible, to assess if only areal means are available. Situations with areal data combined with some high-density point data could improve the results, see for example Moraga et al. (2017); another approach would be to use prior information, such as expert opinions, for the nugget (Truong et al. 2014). In cases in which more summary data per area are available than only the mean, Orton et al. (2012) proposed a method for incorporating this information. In this situation, the exponential covariance model without a nugget was applied for convenience, as is often the case in comparable research; this is however a quite arbitrary choice, and given the results, a careful consideration of all model parameters that determine the smoothness of realisations is suggested for future research.

6.4 Number of Observations

In simulation sessions 6, 7 and 8 (very smooth or very rough simulated signals), mean(StSE) did not converge to one with an increasing number of observations m as might be expected, and actually diverged from one in most cases. Therefore, more data do not alleviate a poor choice of model. Furthermore, in the simulation set-up, increasing m had the side effect of decreasing the size of the individual areas, which might also have influenced this behaviour due to short-range variations becoming better observable.

Very small data sets (nine or ten observations) were analysed in the simulations. The main point of interest was to see how the different approaches behave in such extreme situations, assessed by taking averages over many simulations. The authors stress that, even when using a Bayesian approach, any geostatistical conclusion based on nine or ten observations should be interpreted with caution, except perhaps if strong and honest prior information is available and can be incorporated.

6.5 One-Dimensional Versus Two-Dimensional Simulations

The effect of simulation choices and statistical inference approaches was quite similar in the one-dimensional and two-dimensional simulations. Differences might be due to different mutual spatial relationships. For example, for the two-dimensional simulations with nine observations, the closest pairs of units have centroids separated by 100 au. For the one-dimensional simulations with ten observations, neighbouring units’ centroids are separated by 30 au. Therefore, despite there being an almost equal amount of data, there is much more short-range information in the one-dimensional data in the set-up used. This might explain the extremely large mean(StSE) values in session 9 of the two-dimensional study (up to 49.2) compared with the less extreme values in the one-dimensional study (up to 4.9).

The approximation of the average covariance matrices might have been less successful for the two-dimensional simulations. This would explain the relatively high max(MPP) and the unexpected irregular spatial pattern of the prediction error sd (see Online Resource D, Fig. D1 d).

6.6 The Algorithm

Although the authors did not compare the approach used here with more conventional MCMC methods, it proved an effective and efficient means of performing Bayesian (and also MML) spatial data analysis in the area-to-point context presented. As indicated in Sect. 3.2.3, several different methods can be used to approximate the average covariance matrices. For example, the Legendre–Gauss quadrature—as described by Orton et al. (2017)—is computationally and memory-wise much cheaper, but perhaps less accurate than the applied discretisation points method. Both methods, including some variations, are included in the code, as is area-to-area kriging. Future extensions might include directionality and point-to-point kriging.

The Integrated Nested Laplace Approximations (INLA) alternative to MCMC (Blangiardo et al. 2013) has some similarities to our partly analytical Bayesian approach, such as a gridded search in parameter space and numerical integration. However, it uses Laplace approximations of some integrals and is applicable to a much wider range of models, including hierarchical ones. Furthermore, its spatial implementation assumes the Markovian property on the spatial Gaussian random field (meaning that any point or area in the region is influenced only by its immediate neighbours), leading to sparse covariance matrices and thus reductions in computational costs. In our opinion, our approach offers specific and transparent insights into the Bayesian approach of model-based geostatistics. For future research, however, it would be interesting to redo the calculations with INLA, or to integrate some of the sophisticated and cost-reducing details of INLA into the code.

6.7 General Remarks

The general impression obtained from the ensemble of simulations is that REML tended to underestimate prediction uncertainty the most, followed by Bayesian-uniform. The Bayesian-reference approach tended to be more conservative, while MML was slightly conservative but seemed relatively stable. The differences between the approaches decreased with increasing m.

Given a covariance model that is more or less accurate in terms of short-range behaviour, the conclusion is that, for data sets of sufficient size, or if a slight underestimation of prediction uncertainty is allowed, the REML approach as demonstrated by Brus et al. (2018) should be sufficient. For smaller data sets with no prior information available, the most robust and in many cases best approach, albeit somewhat conservative, appeared to be MML. An additional advantage of MML is its relative insensitivity to arbitrary choices such as bounds on the correlation distance parameter. In several sessions, MML even outperformed Bayesian-inverse-gamma when the supplied prior information about \(\phi \) was correct. Finally, MML has additional computational benefits for prediction over the fully Bayesian approach.

The authors suggest focusing future research on modelling short-range variation and including a smoothness parameter in the inferential models. Using honest and informative priors—depending on the research question at hand—might also yield interesting results. The Matérn smoothness parameter \(\nu _\mathrm{M}\) could be made an integral part of the Bayesian model, or alternatively incorporated as an extra model parameter to be optimised in an MML approach, which could then be used as a plug-in value for prediction.