We now assess the performance of the proposed method on two example partial differential equation problems: a Darcy flow problem with a contaminant mass balance, modelling steady-state groundwater flow in a 2-d porous medium; and Richards equation, modelling single-phase flow through a 3-d porous medium. As explained in Sect. 5, the analysis includes: (i) predictions that are conditioned on an input; and (ii) predictions that are marginalized over the stochastic input.
When making conditioned predictions, we use the conditional predictive distribution (30) for \(\mathbf {y}\), or the distribution (27) for \(\mathbf {z}\) in conjunction with the pre-image map (23). As explained in 4.1, we place a prior over the hyperparameters \(\varvec{\Theta }\) and signal variances \(\pmb {\beta }\) and use a HMC scheme to sample from the posterior distributions over these parameters. Each sample can be used to obtain a different normal predictive distribution, conditioned on an input. We are therefore able to see how the predictive mean and variance change with respect to the uncertainty in the GP parameters. In the results, we plot the expectation and standard deviation of first two predictive distribution moments.
For the forward UQ problem we marginalize the conditional predictive distributions over a stochastic input (Eq. 32) to obtain the pushforward measure (non analytically). We are able to analytically find the mean using (A2) and (A3) together with the pre-image map, or, using Algorithm 1, sample from the marginalized distribution via MC (Eq. 36).
The accuracies of both the point predictions and the predictions of the pushforward measure are assessed by comparison with the true values obtained with the simulator (on the test inputs \(\{\varvec{\xi }_q^*\}_{q=1}^Q\)). We run the solver for each test input to generate the true response, denoted \(\tilde{\mathbf {y}}^{*}_q\). For the UQ comparison we again approximate the pdf using KDE (or simply extract the moments) based on \(\{\tilde{\mathbf {y}}^{*}_q\}_{q=1}^Q\). The latter approximation is guaranteed to converge to the truth as the number of test inputs increases.
Darcy Flow: Non-point Source Pollution
The first example is a linear model of steady-state groundwater flow in 2-d. The approach was developed by Kourakos et al. (2012) and implemented in the mSim package.Footnote 3 The model comprises Darcy’s law and a contaminant mass balance in a 2-d polygonal domain \(\Omega \) of total area 18.652 km\(^2\) containing wells and a stream, and subdivided into polygonal regions of different land use (Fig. 5). Full details of the model and the numerical method can be found in Kourakos et al. (2012). Below we provide a brief description. The model equations are given by:
$$\begin{aligned} \begin{aligned} \nabla \cdot \left( K\nabla h\right)&=Q\\ \displaystyle R\frac{\partial C}{\partial t}&=\nabla \cdot \left( \mathbf{D}\nabla C\right) -\nabla \cdot \left( \mathbf{v}C\right) =G \end{aligned} \end{aligned}$$
(37)
where \(K(\mathbf {x})\) is the hydraulic conductivity, \(h(\mathbf {x})\) is the pressure head, \(C(\mathbf {x},t)\) is the contaminant concentration, R is the retardation factor, \(\mathbf{D}\) is the dispersion tensor, \(\mathbf{v}\) is the fluid velocity, and Q and G represent sources/sinks. The contaminant transport equation is replaced by a 1-d approximation and is solved through an ensemble of one-dimensional streamline-based solutions (Kourakos et al. 2012).
The contaminant balance and flow (Darcy) equations are decoupled. The latter is solved using the finite element method based on triangular elements and first-order (linear) shape functions. The boundary conditions are given by: (i) a constant head equal to 30 m on the left boundary; (ii) a general head boundary equal to 40 m with conductance equal 160 m\(^3\) day\(^{-1}\) on the right boundary; and (iii) no flow on the top and bottom boundaries. Each land use polygon is assigned its own recharge rate. Stream rates are assigned directly to nodes. (Any node closer than 10 m to the stream is considered to be part of the stream.)
We assume that \(K(\mathbf {x})\) is log-normally distributed and treat it as an input. The output field upon which we focus is the pressure head, that is, \(u(\mathbf {x};K)=h(\mathbf {x})\) in the notation of Sect. 2. We use the input model described in Sect. 2, defining a discretized random field corresponding to realizations of \(K(\mathbf {x})=\exp (Z(\mathbf {x}))\) at the nodes \(\{\mathbf {x}_k\}_{k=1}^{{k_{y}} }\subset \mathcal{R}\) on the finite element mesh. The covariance function for the random field \(Z(\mathbf {x})\) is given by:
$$\begin{aligned} c_Z(\mathbf {x},\mathbf {x}') = \sigma _Z^2\exp \left\{ -\frac{(x_1-x_1')^2}{l_1^2}-\frac{(x_2-x_2')^2}{l_2^2}\right\} ,\quad \mathbf {x}=(x_1,x_2)^T\in \mathcal{R}, \end{aligned}$$
(38)
where \(l_1\) and \(l_2\) are correlation lengths. This separable form was suggested in Zhang and Lu (2004) and is used extensively in the literature to model hydraulic permeability fields (often by setting the correlation lengths equal). The generalized variance (value of \({k_{\xi }} \)) was chosen to satisfy \(\sum _{j=1}^{{k_{\xi }} } \sqrt{\lambda _j}/ \sum _{j=1}^{{k_{y}} } \sqrt{\lambda _j} > 0.98\).
Both the training and test input samples were drawn independently: \(\varvec{\xi }_n \sim \mathcal {N}\left( \mathbf{0},\mathbf{I}\right) \), \(n=1,\ldots , N\) to yield \(\{\mathbf {y}_n\}_{n=1}^N\) for training; and \(\varvec{\xi }_q \sim \mathcal {N}\left( \mathbf{0},\mathbf{I}\right) \), \(q=1,\ldots , Q\) to yield \(\{\widetilde{\mathbf {y}}^{*}_q\}_{q=1}^Q\) for testing and the forward problem (UQ). We set \(Q=5000\) and \(N\in \{25,50,75,100\}\). Running the solver with an input generated using the KL truncation necessarily leads to a response surface with intrinsic dimension at most \({k_{\xi }} \), which was therefore the value chosen for the approximating manifold dimension \({k_{z}} \). In all of the results presented below, \({k_{y}} = 1933\) nodes (elements) were used in the simulation. The number of neighbours P in the LTSA algorithm was chosen according to the error between the solver response and the predictive mean at the test points. We define a scaled measure of error on each test point as follows:
$$\begin{aligned} e_q=||\widetilde{\mathbf {y}}^{*}_q-\overline{\mathbf {y}}_q^*||/||\widetilde{\mathbf {y}}^{*}_q||, \quad q=1,\ldots ,Q, \end{aligned}$$
(39)
where \(\widetilde{\mathbf {y}}^{*}_q\) is the response predicted by the solver, and \(\overline{\mathbf {y}}_q^*\) is the point recovered by application of the pre-image map (23) on the GP predictive mean (26). The scaling ensures that the errors are comparable and can be interpreted as percentage errors.
We present results for three stochastic input models:
-
M1
:
-
We set \(m_Z=\ln (40)\) and \(\sigma _Z^2 = 0.2\), yieldingFootnote 4 a mean for \(k(\mathbf {x})\) of 44.2 m day\(^{-1}\), which is close to the default value in the mSim package, and a standard deviation of 13.63 m day\(^{-1}\). The correlation lengths were chosen as \(l_1=2000\) m and \(l_2=1000\) m, which correspond to dimensionless values of 1 / 3 and 2 / 7, respectively. These choices require \({k_{\xi }} =5\) input dimensions to capture \(98\%\) of the generalized variance.
-
M2
:
-
We set \(m_Z=\ln (36.18)\) and \(\sigma _Z^2 = 0.4\), again yielding a mean 44.2 m day\(^{-1}\) and a standard deviation of 18.80 m day\(^{-1}\). We set \(l_1=2000\) m and \(l_2=1000\) m. \({k_{\xi }} =5\) captures \(98\%\) of the generalized variance.
-
M3
:
-
We set \(m_Z=\ln (40)\), and \(\sigma _Z^2 = 0.4\) and reduce the correlation lengths to \(l_1=1000\) m and \(l_2=500\) m (dimensionless values of 1 / 6 and 1 / 7, respectively). We now require \({k_{\xi }} =15\) to capture \(98\%\) of the generalized variance.
For model M1, the distributions of \(\{e_q\}_{n=1}^{Q}\) for different training set sizes N are shown as boxplots for increasing values of P in Fig. 1. The performance of the emulator is good even for \(N=25\) training points (maximum \(e_q\) of approximately \(e^{-3}\)), although there is a clear decrease in the error when N is increased to 100. The relationship between the errors and P is more complicated. The errors are high for \(P<8\) (not shown in the boxplots) at all values of N and decrease as P increases. This is due to the linear approximation of points in local tangent spaces via PCA in the LTSA algorithm. As more points are added, the approximation improves. As P is increased beyond a certain value, however, the errors increase (this is most clearly visible for \(N=100\)). The reason for this behaviour is that for large enough neighbourhood sizes the linear approximation breaks down. Thus, there is an optimal choice of P for each value of N and the higher the value of N the more sensitive are the errors to the value of P. In the subsequent results we use \(P=15\) unless otherwise specified.
In Fig. 2 we plot the normalized pressure head prediction (for each coordinate of the predicted pressure head we subtract the mean and divide by the standard deviation) corresponding to the highest \(e_q\) for both \(N=25 \) and \(N=50\) (using \(P=15\)). The normalization highlights the differences between the true values and the predictions (the errors) more clearly. The predicted means of the means (middle row) are the mean predictions averaged over all hyperparameter and precision samples. Also shown (bottom row) are the standard deviations of the predictions averaged over all hyperparameter and precision samples. We observe that the prediction at \(N=75\) is highly accurate, while the prediction at \(N=25\) is still reasonably accurate even in this worst case (an outlier in Fig. 1). For both values of N, the true values lie within the credible regions. In Fig. 3 we show the corresponding predictions for cases where the errors are close to the medians. Both predictions are highly accurate and again the true values lie inside the credible regions.
We now focus on the forward problem, in which we estimate the marginalized predictive distribution (32) using Algorithm 1. KDE is used to obtain estimates of the pdf of a feature for different predictive posterior, hyperparameter and precision samples, as previously described. The feature we choose is the pressure head at the spatial location \(\mathbf {x}= \left( 2511,486\right) \in \mathcal{R}\). We plot a heat map of the pdfs in Fig. 4 for different N.
The distributions are accurately estimated for all values of N. While the predictions improve as the number of training samples N increases, the true value does not always lie within the contours. This is because: (i) as stated earlier, an increased GP predictive variance acts to smooth the density, rather than increase the width of the contours; (ii) by choosing a priori the number of neighbours we also a priori assume a global smoothness of the emulator; and (iii) we have a pre-image map \(\widehat{\mathbf {f}} : \mathcal {F}\rightarrow \mathcal Y\) for which the error is unknown (as with all methods), but not estimated (as with probabilistic methods).
We can find the means and standard deviations across the samples obtained for different predictive posterior, hyperparameter and precision samples using Algorithm 1. We obtain distributions over the moments of the marginalized predictive distribution (32). In Fig. 5 we plot the mean and standard deviation of the marginalized predictive mean and standard deviation for \(N=25\), with comparisons to the true values obtained by finding the mean and standard deviation across the test responses \(\{\widetilde{\mathbf {y}}^{*}_q\}_{q=1}^Q\). Even for this low number of training points the results are highly accurate.
We now consider Model M2, in which we increase the variance of the stochastic input, while keeping the mean fixed. For this example we again set \(l_1 = 2000\) m and \(l_2 = 1000\) m, requiring \({k_{\xi }} =5\). The distributions of \(\{e_q\}_{q=1}^{Q}\) for different training set sizes N and increasing P are shown in Fig. 6. We observe trends similar to those observed using Model M1, although the increased variance leads to larger errors at fixed N and P (higher maxima and minima). With the exception of an isolated outlier (shown later), the predictions are nevertheless accurate for \(N=75\).
The worst case (highest \(e_q\)) for \(P=15\) is shown in Fig. 7 for \(N=25\) and 75 points (see Fig. 6). As before the top row is the test (solver prediction), while the middle and bottom rows are the mean prediction and standard deviation of the prediction averaged over all hyperparameter and precision samples. The true values lie within the credible regions, although for this model a higher number of training points are required to ensure that even the worst-case predictions are accurate. Figure 8 demonstrates the quality of the predicted responses when the errors are at the median in the \(P=15\) boxplots in Fig. 6. Here, even \(N=25\) provides accurate results.
Figure 9 shows heat maps of the pdfs of the pressure head at the spatial location \(\mathbf {x}= \left( 2511,486\right) \) for different N (generated using KDE) in the case of Model M2. Using \(N=75\) we achieve very good agreement with the MC prediction based on the simulator results (test points), although again the true value does not lie within the contours. For \(N=25\), we plot the mean and standard deviation of the marginalized predictive mean and standard deviation in Fig. 10, with a comparison to the true values obtained from \(\{\widetilde{\mathbf {y}}^{*}_q\}_{q=1}^Q\). The predictions are highly accurate. In fact, even for \(N=25\) (not shown to conserve space) the mean was very accurate and the standard deviation exhibited only slight differences from the true value.
For Model M3 (decreased correlation lengths, high standard deviation and \({k_{\xi }} =15\)), the distributions of \(\{e_q\}_{q=1}^{Q}\) for increasing N and P in are shown in Fig. 11. In this case it is clear that a much higher value of P (\(P>60\), giving a similar neighbourhood radius in-line with the increased sample density) is required to obtain a reasonable accuracy. For \(N=500\) and \(P=80\), there are a small (9 out of 5000) number of outliers with low accuracy, while the errors for the remaining points satisfy \(\ln (e_q)<-3.25\). The worst cases (highest \(e_q\)) for \(P=70\), \(N=300\) and \(P=80\), \(N=500\) are shown in Fig. 12, and in Fig.13 we show predicted responses with errors at the medians for the same values of P and N. There are noticeable differences in the worst cases, although the qualitative agreement is very good at both values of N. For the median error cases both emulators perform extremely well.
In Fig. 14 we show the heat maps of the pdfs of the pressure head at \(\mathbf {x}= \left( 2511,486\right) \) for different N. For both values of N there is very good agreement with the simulator result and the true value this time lies within the contours. For \(N=500\), we show the mean and standard deviation of the marginalized predictive mean and standard deviation in Fig. 15, with a comparison to the true values obtained from \(\{\widetilde{\mathbf {y}}^{*}_q\}_{q=1}^Q\). The predictions are again highly accurate (which was also the case for \(N=300\)).
Richards Equation: Unsaturated Flow in Porous Media
Consider a single-phase flow through a 3-d porous region \(\mathcal{R}\subset \mathbb {R}^3\) containing unsaturated soil with a random permeability field. The vertical flow problem can be solved using Richards equation (Darcy’s law combined with a mass balance). There are three standard forms of Richards equation: the pressure head based (h-based) form; the water content-based (\(\theta \)-based) form; and the mixed-based form. For flow in saturated or layered soils, the h-based form is particularly appropriate (Huang et al. 1996; Shahraiyni and Ataie-Ashtiani 2011).
The h-based form with an implicit or explicit finite difference (FD) scheme has been shown to provide good accuracy, although this approach may result in high mass balance errors (Zarba et al. 1990; Huang et al. 1996). The mixed-based form, on the other hand, exhibits low mass balance errors with highly accurate predictions using a fully implicit FD scheme (Ray and Mohanty 1992; Zarba et al. 1990; Celia et al. 1987). The latest work of Shahraiyni and Ataie-Ashtiani (2011) showed that a fully implicit FD scheme with a standard chord slope (CSC) approximation (Rathfelder and Abriola 1994) not only solved the mass balance problem of the h-based form but also improved convergence. Thus, in the paper we adopt this approach, although other numerical formulations are by no means precluded. The h-based form of Richards equation can be written as follows:
$$\begin{aligned} u(h) \frac{\partial h}{\partial t} - \nabla \cdot \textit{K}(h) \nabla (h+x_3)=0, \qquad (\mathbf x ,t) \in \mathcal{R} \times (0, T], \end{aligned}$$
(40)
where h is the pressure head, \(u(h)=\partial \theta / \partial h\) is the specific moisture capacity, in which \(\theta \) is the moisture content, K(h) is the unsaturated hydraulic conductivity, and \({{\varvec{x}}}=(x_1,x_2,x_3)^T\) is the spatial coordinate, in which \(x_3\) is the vertical coordinate. The nonlinear functions \(\theta (h)\) and k(h) can take on different forms. For example, in Haverkamp et al. (1977), a least square fit to experimental data was used to derive:
$$\begin{aligned} \begin{aligned} \displaystyle \theta (h)&= \frac{\alpha _1 (\theta _s-\theta _r)}{\alpha _1 + |h|^{\alpha _2}} +\theta _r, \\ \displaystyle \textit{K}(h)&=K_s (\mathbf {x}) \frac{\alpha _3}{\alpha _3 +|h|^{\alpha _4}}, \end{aligned} \end{aligned}$$
(41)
where \(\theta _r\) and \(\theta _s\) are the residual the saturated water contents, \(K_s (\mathbf{x})\) is the saturated hydraulic conductivity, and \(\alpha _1\), \(\alpha _2\), \(\alpha _3\) and \(\alpha _4\) are fitting parameters. We adopt the relationships (41) and use the parameter values in Haverkamp et al. (1977): \(\alpha _1=1.611\times 10^6\), \(\alpha _2=3.96\), \(\alpha _3=1.175\times 10^6\), \(\alpha _4=4.74\), \(\theta _s=0.287\) and \(\theta _r=0.075\). The domain \(\mathcal{R}\) is taken to be \(20 \text{ cm } \times 20 \text{ cm }\times 20 \text{ cm }\). \(K_s({\mathbf {x}})\) is treated as a random field input with a log-normal distribution (\(K_s({\mathbf {x}})=\exp (Z(\mathbf {x})\)), again discretized using the Karhunen–Loève theorem. We generate realizations of a corresponding discrete random field on an \(n_1\times n_2 \times n_3\) finite difference grid (\({k_{y}} =n_1n_2n_3\)), with grid spacings \(\Delta x_1\), \(\Delta x_2\) and \(\Delta x_3\) in the directions \(x_1\), \(x_2\) and \(x_3\), respectively. The output field of interest is again the pressure head, at a fixed time T. Thus, we set \(u(\mathbf {x};K)=h(\mathbf {x},T)\).
The boundary conditions are those used in Haverkamp et al. (1977), corresponding to laboratory experiments of infiltration in a plexiglass column packed with sand. Along the top boundary (surface) \(x_3=20\) cm, the pressure head is maintained at \(h=-20.7\) cm (\(\theta =0.267\) cm\(^3\) cm\(^{-3}\)), and along the bottom boundary \(x_3=0\) cm, it is maintained at \(h=-\,61.5\) cm. At all other boundaries a no-flow condition is imposed: \(\nabla h \cdot \mathbf{n}=0\), where \(\mathbf{n}\) is the unit, outwardly pointing normal to the surface. The initial condition is \(h(\mathbf {x},0)=-\,61.5\) cm.
The covariance function for the random field \(Z(\mathbf {x})\) is again of the form:
$$\begin{aligned} c_Z(\mathbf {x},\mathbf {x}') = \sigma _Z^2\exp \left\{ -\frac{(x_1-x_1')^2}{l_1^2}-\frac{(x_2-x_2')^2}{l_2^2}-\frac{(x_3-x_3')^2}{l_3^2}\right\} , \end{aligned}$$
(42)
where the \(l_i\) are correlation lengths, chosen as \(l_1=l_2=l_3=7.5\) cm. The mean \(m_Z\) and variance \(\sigma _Z^2\) are chosen such that the mean and standard deviation of \(K(\mathbf {x})\) are 0.0094 cm s\(^{-1}\) (Shahraiyni and Ataie-Ashtiani 2011; Haverkamp et al. 1977) and 0.00235 cm s\(^{-1}\) (25 % of the mean), respectively. The generalized variance satisfies \(\sum _{i=1}^{{k_{\xi }} } \sqrt{\lambda _i}/ \sum _{i=1}^n \sqrt{\lambda _i} = 0.75\) for \({k_{\xi }} =15\).
The training and test input samples were drawn independently: \(\varvec{\xi }_n \sim \mathcal {N}\left( \mathbf{0},\mathbf{I}\right) \) and \(\varvec{\xi }_q \sim \mathcal {N}\left( \mathbf{0},\mathbf{I}\right) \) to yield \(\{\mathbf {y}_n\}_{n=1}^N\) for training and \(\{\widetilde{\mathbf {y}}^{*}_q\}_{q=1}^Q\) for testing and UQ. We set \(Q=5000\) and \(N\le 800\). As before, the manifold dimension was set to \({k_{z}} ={k_{\xi }} \). The number of neighbours P and the number of training points N were chosen as in the first example by examining the errors \(e_q=||\widetilde{\mathbf {y}}^{*}_q-\overline{\mathbf {y}}_q^*||/||\widetilde{\mathbf {y}}^{*}_q||\) on the test set, where again \(\widetilde{\mathbf {y}}^{*}_q\) is the solver output (truth) and \(\overline{\mathbf {y}}_q^*\) is emulator prediction based on the GP predictive mean (26).
Equation (40) was solved using a finite difference scheme with first-order differencing for the first-order derivatives, central differencing for the second-order derivatives and a fully implicit backward Euler time stepping scheme. A picard iteration scheme is used (Celia et al. 1990) at each time step. Details are provided in “Appendix C”.
We followed the procedure of the first example. Training point numbers below 600 led to inaccurate results. For \(N=600\), the results were reasonably accurate but to achieve good accuracy we required \(N>700\). We present the results for \(N=800\). The pressure head is normalized as in the first example in order to highlight the errors in the predictions more clearly. In Fig. 16a we plot the log normalized error \(\ln (e_q)\) for an emulator trained on \(N=800\) points \(\mathbf {y}_n\) and tested with \(Q=5000\) points \(\widetilde{\mathbf {y}}^{*}_q\) for different nearest neighbour numbers \(P> 20\) (averaging over hyperparameter and precision posterior samples). For \(P\le 20\) the errors were high, with the same trend as seen in the first example.
We use Algorithm 1 and KDE to obtain predictions of the pdf of a feature of the response. We choose as a feature the pressure head at the location \(\mathbf {x}=(10.4,10.4,10.4)^T\) (grid point number 4411). The distributions are shown in Fig. 16b for \(N=800\). We can again find the means and standard deviations across predictive posterior, hyperparameter and precision samples to obtain distributions over the moments of the marginalized distribution (32). These are plotted in Fig. 17, alongside comparisons to the true values obtained from \(\{\widetilde{\mathbf {y}}^{*}_q\}_{q=1}^Q\). These results show that the emulator performs extremely well, accurately capturing both the mean and standard deviation with high precision.