Statistical Interpolation of Spatially Varying but Sparsely Measured 3D Geo-Data Using Compressive Sensing and Variational Bayesian Inference

Abstract

Real geo-data are three-dimensional (3D) and spatially varied, but measurements are often sparse due to time, resource, and/or technical constraints. In these cases, the quantities of interest at locations where measurements are missing must be interpolated from the available data. Several powerful methods have been developed to address this problem in real-world applications over the past several decades, such as two-point geo-statistical methods (e.g., kriging or Gaussian process regression, GPR) and multiple-point statistics (MPS). However, spatial interpolation remains challenging when the number of measurements is small because a suitable covariance function is difficult to select and the parameters are challenging to estimate from a small number of measurements. Note that a covariance function form and its parameters are key inputs for some methods (e.g., kriging or GPR). MPS is a non-parametric simulation method that combines training images as prior knowledge for sparse measurements. However, the selection of a suitable training image for continuous geo-quantities (e.g., soil or rock properties) faces certain difficulties and may become increasingly complicated when the geo-data to be interpolated are high-dimensional (e.g., 3D) and exhibit non-stationary (e.g., with unknown trends or non-stationary covariance structure) and/or anisotropic characteristics. This paper proposes a non-parametric approach that systematically combines compressive sensing and variational Bayesian inference for statistical interpolation of 3D geo-data. The method uses sparse measurements and their locations as the input and provides interpolated values at unsampled locations with quantified interpolation uncertainty as the output. The proposed method is illustrated using a series of numerical 3D examples, and the results indicate a reasonably good performance.

Appendices

Appendix A: Construction of B ^−3D_t

In this appendix, the 3D basis function $\underline{{\mathbf{B}}}_{t}^{3D}$ is reconstructed from columns of three independent 1D basis function matrices, i.e., B¹, B², and B³, which have dimensions of N₁ × N₁, N₂ × N₂, and N₃ × N₃, respectively. The columns of B¹, B², and B³ represent orthonormal basis functions, e.g., discrete cosine functions. B¹, B², and B³ can be obtained using the formula for discrete cosine functions (see Salomon 2007) or constructed readily using a built-in function in commercial software, such as “dctmtx” in MATLAB (Mathworks 2020), which requires only N₁, N₂, and N₃ as the input. A 3D basis function is then constructed as $\underline{{\mathbf{B}}}_{i,j,k}^{3D} = {\varvec{b}}_{i}^{1} \otimes {\varvec{b}}_{j}^{2} \otimes {\varvec{b}}_{k}^{3}$ (i = 1, 2, …, N₁; j = 1, 2, …, N₂; k = 1, 2, …, N₃). The subscript of $\underline{{\mathbf{B}}}_{i,j,k}^{3D}$, i.e., “i, j, k” changes to “t” later in this paragraph. ${\varvec{b}}_{i}^{1}$, ${\varvec{b}}_{j}^{2}$, and ${\varvec{b}}_{k}^{3}$ represent the ith, jth, and kth columns of B¹, B², and B³, respectively; “$\otimes$” represents an outer product and an element of $\underline{{\mathbf{B}}}_{i,j,k}^{3D}$, such as the element indexed by (m, n, l), i.e., $\underline{{\text{B}}}_{i,j,k}^{3D} (m,n,l)$, is expressed as $\underline{{\text{B}}}_{i,j,k}^{3D} (m,n,l) = b_{i}^{1} (m)b_{j}^{2} (n)b_{k}^{3} (l)$ (Kroonenberg 2008). $b_{i}^{1} (m)$, $b_{j}^{2} (n)$, and $b_{k}^{3} (l)$ are the mth, nth, and lth elements of ${\varvec{b}}_{i}^{1}$, ${\varvec{b}}_{j}^{2}$, and ${\varvec{b}}_{k}^{3}$, respectively. After the construction of $\underline{{\mathbf{B}}}_{i,j,k}^{3D}$, the subscript “(i, j, k)” of $\underline{{\mathbf{B}}}_{i,j,k}^{3D}$ changes to t for derivation convenience. “t” is numbered in increasing order of i, followed by j and k, respectively. It is worth noting that although the discrete cosine function is adopted in this study to construct the 3D basis function, other basis functions (e.g., wavelets functions) can also be used in the proposed method. The discrete cosine function is adopted here because it has analytical function forms and the basis function can be readily obtained.

Appendix B: Derivation of Eq. (15)

The expression of KL divergence defined in Eq. (14) is expanded to

$$ \begin{aligned} KL(q||p) & = - \int {q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )} \ln \frac{{p(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau |\varvec{y}^{3D} )}}{{q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )}}{\text{d}}\hat{\varvec{\omega }}^{3D} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau \\ & = - \int {q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )} [\ln p(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau |\varvec{y}^{3D} )\\ &\quad - \ln q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )]{\text{d}}\hat{\varvec{\omega }}^{3D} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau . \\ \end{aligned} $$

(27)

In accordance with the rules of conditional probability, $p(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau |\varvec{y}^{3D} ) = p(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau ,\varvec{y}^{3D} )/p(\varvec{y}^{3D} )$. Substituting this expression into Eq. (27) and rearranging the terms lead to

$$\begin{aligned} KL(q||p)& = - \int {q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )} [\ln p(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau ,y^{3D} )\\ &\quad - \ln q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau ) - \ln p(\varvec{y}^{3D} )]{\text{d}}\hat{\varvec{\omega }}^{3D} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau . \end{aligned}$$

(28)

Note that ln[p(y^3D)] is independent of the distribution $q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )$. Therefore, $\int {q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )} \ln p(\varvec{y}^{3D} ){\text{d}}\hat{\varvec{\omega }}^{3D} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau = \ln p(\varvec{y}^{3D} )$. As a result, Eq. (28) is simplified as

$$\begin{aligned} KL(q||p) = - \int {q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )} \left[ {\ln \frac{{p(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau ,\varvec{y}^{3D} )}}{{q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )}}} \right]{\text{d}}\hat{\varvec{\omega }}^{3D} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau + \ln p(\varvec{y}^{3D} ).\end{aligned} $$

(29)

Let $L(q) = \int {q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )} \ln \left( {\frac{{q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau ,\varvec{y}^{3D} )}}{{q(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau )}}} \right){\text{d}}\hat{\varvec{\omega }}^{3D} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau$. Equation (29) can then be rewritten as KL(q||p) = − L(q) + ln[p(y^3D)]. Subsequently, ln[p(y^3D)] = L(q) + KL(q||p), i.e., Equation (15) can be obtained.

Appendix C: Derivation of q($\hat{\varvec{\omega }}^{3D}$), q(α), q(γ), and q(τ)

In this appendix, the framework for using VBI to derive the tractable distribution is introduced. Let Θ = [θ₁, θ₂, …, θ_n]^T represent a set of random variables and “p(Θ|Data)” represent the true posterior PDF of Θ updated by “Data.” Suppose that “p(Θ|Data)” has no analytical solution and VBI is adopted to seek an approximate distribution $q(\varvec{\varTheta})$ that can properly represent p(Θ|Data). As mentioned in the main text, $q(\varvec{\varTheta})$ is usually factorized as $q(\varvec{\varTheta}) = \prod\nolimits_{i = 1}^{n} {q(\theta_{i} )}$. In accordance with the derivation by Bishop (2006) (pp. 461–517), the q(θ_i) that minimizes the KL divergence between q(Θ) and p(Θ|Data) is then expressed as

$$ \begin{aligned} \ln [q(\theta_{i} )] & = \int {q(\varvec{\varTheta}_{ - i} )\ln p(\varvec{\varTheta},Data)} {\text{d}}\varvec{\varTheta}_{ - i} + const \\ & = \int {q(\varvec{\varTheta}_{ - i} )\ln [p(Data|\varvec{\varTheta})} p(\varvec{\varTheta})]{\text{d}}\varvec{\varTheta}_{ - i} + const, \\ \end{aligned} $$

(30)

where Θ_−i represents Θ with θ_i removed and “const” represents a term that ensures that the integration of q(θ_i) = 1.

In this paper, the random variables of interest are $\hat{\varvec{\omega }}^{3D}$, α, γ, and τ, and their approximate distribution can be individually derived using Eq. (30). Consider, for example, q($\hat{\varvec{\omega }}^{3D}$). In accordance with Eq. (30), $q(\hat{\varvec{\omega }}^{3D} )$ is expressed as Eq. (17). Substituting Eqs. (6), (8–11), and (18) into Eq. (17) leads to

$$ \begin{aligned} \ln [q(\hat{\varvec{\omega }}^{3D} )] & = \int {q(\varvec{\alpha})q(\gamma )q(\tau )\ln p(\hat{\varvec{\omega }}^{3D} ,\varvec{\alpha},\gamma ,\tau ,y^{3D} )} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau + const_{1} \\ & = \int {q(\tau )\ln p(\varvec{y}^{3D} |\hat{\varvec{\omega }}^{3D} ,\tau )} {\text{d}}\tau + \int {q(\varvec{\alpha})\ln p(\hat{\varvec{\omega }}^{3D} |\varvec{\alpha})} {\text{d}}\varvec{\alpha}+ const_{2} , \\ \end{aligned} $$

(31)

where “const₂” represents the term that does not involve $\hat{\varvec{\omega }}^{3D}$. Note that q(α) and q(γ) are independent of $\ln p(\varvec{y}^{3D} |\hat{\varvec{\omega }}^{3D} ,\tau )$. Therefore, $\int {q(\varvec{\alpha})q(\gamma )q(\tau )\ln p(\varvec{y}^{3D} |\hat{\varvec{\omega }}^{3D} ,\tau )} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau = \int {q(\tau )\ln p(\varvec{y}^{3D} |\hat{\varvec{\omega }}^{3D} ,\tau )} {\text{d}}\tau$. Similarly, $\int {q(\varvec{\alpha})q(\gamma )q(\tau )\ln p(\hat{\varvec{\omega }}^{3D} |\alpha )} {\text{d}}\varvec{\alpha}{\text{d}}\gamma {\text{d}}\tau = \int {q(\varvec{\alpha})\ln p(\hat{\varvec{\omega }}^{3D} |\varvec{\alpha})} {\text{d}}\varvec{\alpha}$. Subsequently, substituting Eqs. (6) and (8) into Eq. (31) and rearranging the terms lead to

$$ \begin{aligned} \ln [q(\hat{\varvec{\omega }}^{3D} )] & = \int {q(\tau )\ln p(\varvec{y}^{3D} |\hat{\varvec{\omega }}^{3D} ,\tau )} {\text{d}}\tau + \int {q(\varvec{\alpha})\ln p(\hat{\varvec{\omega }}^{3D} |\varvec{\alpha})} {\text{d}}\varvec{\alpha}+ const_{2} \\ & = - \frac{{E(\tau )(\varvec{y}^{3D} - {\mathbf{A}}\hat{\varvec{\omega }}^{3D} )^{T} (\varvec{y}^{3D} - {\mathbf{A}}\hat{\varvec{\omega }}^{3D} )}}{2} - \frac{{(\hat{\varvec{\omega }}^{3D} )^{T} E(\varvec{D}^{\alpha } )^{T} \hat{\varvec{\omega }}^{3D} }}{2} + const_{3} , \\ \end{aligned} $$

(32)

where “const₃” represents the terms that do not involve $\hat{\varvec{\omega }}^{3D}$. Completing the square for $\hat{\varvec{\omega }}^{3D}$ in Eq. (32) and rearranging the terms lead to

$$ \ln [q(\hat{\varvec{\omega }}^{3D} )] = - \frac{{(\hat{\varvec{\omega }}^{3D} )^{T} [{\mathbf{A}}^{T} {\mathbf{A}}E(\tau ) + E({\mathbf{D}}^{\alpha } )]\hat{\varvec{\omega }}^{3D} - 2(\hat{\varvec{\omega }}^{3D} )^{T} {\mathbf{A}}^{T} \varvec{y}^{3D} E(\tau ) + (\varvec{y}^{3D} )^{T} \varvec{y}^{3D} E(\tau )}}{2} + const_{3} . $$

(33)

Let ${\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} = [{\mathbf{A}}^{T} {\mathbf{A}}E(\tau ) + E({\mathbf{D}}^{\alpha } )]^{ - 1}$. Equation (33) is rewritten as

$$ \ln [q(\hat{\varvec{\omega }}^{3D} )] = - \frac{{(\hat{\varvec{\omega }}^{3D} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1} \hat{\varvec{\omega }}^{3D} - 2(\hat{\varvec{\omega }}^{3D} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1} {\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} {\mathbf{A}}^{T} \varvec{y}^{3D} E(\tau )}}{2} + const_{4} , $$

(34)

where “const₄” is a term that incorporates “const₃” and new terms that do not involve $\hat{\varvec{\omega }}^{3D}$. Let $\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} = {\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} {\mathbf{A}}\varvec{y}^{3D} E(\tau )$. Equation (34) is rewritten as

$$ \begin{aligned} \ln [q(\hat{\varvec{\omega }}^{3D} )] & = - \frac{{(\hat{\varvec{\omega }}^{3D} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1} \hat{\varvec{\omega }}^{3D} - 2(\hat{\varvec{\omega }}^{3D} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1}\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} + (\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1}\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} }}{2} \\ & \quad + \frac{{(\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1}\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} }}{2} + const_{4} \\ & = \frac{{(\hat{\varvec{\omega }}^{3D} -\varvec{\mu}_{{\hat{\omega }^{3D} }} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1} (\hat{\varvec{\omega }}^{3D} -\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )}}{2} + const_{5} , \\ \end{aligned} $$

(35)

where $\frac{{(\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1}\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} }}{2}$ is incorporated into the term “const₅”. Therefore, $q(\hat{\varvec{\omega }}^{3D} )$ can be derived as

$$ \begin{aligned} q(\hat{\varvec{\omega }}^{3D} ) & = \frac{1}{{\sqrt {(2\pi )^{N} \det ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )} }}\exp \left[ { - \frac{{(\hat{\varvec{\omega }}^{3D} -\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1} (\hat{\varvec{\omega }}^{3D} -\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )}}{2}} \right] \\ & \quad \times \sqrt {(2\pi )^{N} \det ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )} \exp (const_{5} ), \\ \end{aligned} $$

(36)

A close examination of Eq. (36) shows that $\frac{1}{{\sqrt {(2\pi )^{N} \det ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )} }}$ $\exp \Big[ { - \frac{{(\hat{\varvec{\omega }}^{3D} -\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )^{T} ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )^{ - 1} (\hat{\varvec{\omega }}^{3D} -\varvec{\mu}_{{\hat{\varvec{\omega }}^{3D} }} )}}{2}} \Big]$ is the multivariate normal distribution, and its integration with respect to $\hat{\varvec{\omega }}^{3D}$ is therefore 1. In addition, note that $q(\hat{\varvec{\omega }}^{3D} )$ is a PDF, which leads to $\int {q(\hat{\varvec{\omega }}^{3D} )} {\text{d}}\hat{\varvec{\omega }}^{3D} = 1$. As a result, the term “$\sqrt {(2\pi )^{N} \det ({\varvec{\Sigma}}_{{\hat{\varvec{\omega }}^{3D} }} )} \exp (const_{5} )$” in Eq. (36) is equal to 1. In such a case, Eq. (36) is simplified as Eq. (19) in the main text.

Similarly, q(α) is derived as

$$ q({{\alpha }}) = \prod\limits_{t = 1}^{N} {\exp \left[ { - \frac{{a_{t} \alpha_{t} + b_{t} \alpha_{t}^{ - 1} }}{2}} \right]} (\alpha_{t} )^{p - 1} \times \frac{{(a_{t} /b_{t} )^{p/2} }}{{2K_{p} (\sqrt {a_{t} b_{t} } )}} = \prod\limits_{t = 1}^{N} {q(\alpha_{t} )} , $$

(37)

where $a_{t} = E[(\hat{\omega }_{t}^{3D} )^{2} ]$, $b_{t} = E(\gamma )$, p = − 1/2, and $q(\alpha_{t} )$ = $\exp \left[ { - \frac{{a_{t} \alpha_{t} + b_{t} \alpha_{t}^{ - 1} }}{2}} \right](\alpha_{t} )^{p - 1} \times \frac{{(a_{t} /b_{t} )^{p/2} }}{{2K_{p} (\sqrt {a_{t} b_{t} } )}}$, which is a generalized inverse Gaussian (GIG) PDF (Zhao et al. 2015; Dumitru 2017). The mean or expectation of α_t is shown in Eq. (21a). In addition, note that E(α ⁻¹_t ) is needed in the proposed method [see Eq. (24)], which cannot be directly evaluated even if q(α_t) is available. This is because 1/α_t is a non-linear function of α_t. To address this problem, the PDF of 1/α_t, i.e., q(α ⁻¹_t ) is derived as (Ang and Tang 2007)

$$ \begin{aligned} q(\alpha_{t}^{ - 1} ) & = \frac{{(a_{t} /b_{t} )^{p/2} }}{{2K_{p} (\sqrt {a_{t} b_{t} } )}}\exp \left[ { - \frac{{a_{t} \alpha_{t}^{ - 1} + b_{t} \alpha_{t} }}{2}} \right](\alpha_{t}^{ - 1} )^{p - 1} \times \left| {\frac{{d(\alpha_{t}^{ - 1} )}}{{\alpha_{t} }}} \right| \\ & = \frac{{(a_{t} /b_{t} )^{p/2} }}{{2K_{p} (\sqrt {a_{t} b_{t} } )}}\exp \left[ { - \frac{{a_{t} \alpha_{t}^{ - 1} + b_{t} \alpha_{t} }}{2}} \right](\alpha_{t}^{ - 1} )^{ - p - 1} . \\ \end{aligned} $$

(38)

Equation (38) shows that (1/α_t) follows a GIG with parameters b_t, a_t, and −p. The mean of (1/α_t), i.e., E(α ⁻¹_t ), is obtained as Eq. (24).

In a manner similar to the derivation of q($\hat{\varvec{\omega }}^{3D}$) and q(α), both q(τ) and q(γ) are derived to follow a Gamma distribution, which is shown in Eqs. (39) and (40), respectively

$$ q(\tau ) = \frac{{(d_{n} )^{{c_{n} }} }}{{\varGamma (c_{n} )}}\tau^{{c_{n} - 1}} \exp ( - \tau d_{n} ), $$

(39)

$$ q(\gamma ) = \frac{{(\gamma_{b} )^{{\gamma_{a} }} }}{{\varGamma (\gamma_{a} )}}\gamma^{{(\gamma_{a} - 1)}} \exp ( - \gamma \gamma_{b} ). $$

(40)