Gaussian process for estimating parameters of partial differential equations and its application to the Richards equation

Abstract

This paper proposes a new collocation method for estimating parameters of a partial differential equation (PDE), which uses Gaussian process (GP) as a basis function and is termed as Gaussian process for partial differential equation (GPPDE). The conventional method of estimating parameters of a differential equation is to minimize the error between observations and their estimates. The estimates are produced from the forward solution (numerical or analytical) of the differential equation. The conventional approach requires initial and boundary conditions, and discretization of differential equations if the forward solution is obtained numerically. The proposed method requires fitting a GP regression model to the observations of the state variable, then obtaining derivatives of the state variable using the property that derivative of a GP is also a GP, and finally adjusting the PDE parameters so that the GP derived partial derivatives satisfy the PDE. The method does not require initial and boundary conditions, however if these conditions are available (exactly or with measurement errors), they can be easily incorporated. The GPPDE method is evaluated by applying it on the diffusion and the Richards equations. The results suggest that GPPDE can correctly estimate parameters of the two equations. For the Richards equation, GPPDE performs well in the presence of noise. A comparison of GPPDE with HYDRUS-1D software showed that their performances are comparable, though GPPDE has significant advantages in terms of computational time. GPPDE could be an effective alternative to conventional approaches for finding parameters of high-dimensional PDEs where large datasets are available.

Appendices

Appendix 1: Derivative of Gaussian covariance function $k({{\mathbf {x}}},{\mathbf {x'}})$

Gaussian covariance function is given by relation

$$\begin{aligned} k({{\mathbf {x}}},{\mathbf {x'}}) = {\sigma _s}^2\exp \left\{ {-\dfrac{1}{2}\sum _{i=1}^{m} \dfrac{{\left( {{x_i} - {x'_i}} \right) }^{2}}{l_i^2}} \right\} \end{aligned}$$

(25)

where $l_i$’s and ${\sigma _s}$ are the length and the scale parameters, respectively.

1.1 Derivative of covariance function with respect to hyperparameters

The derivatives of ${k_y}\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) ={k}\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) +\sigma _y^2$, where $\sigma _y$ is the noise term, are given by.

$$\begin{aligned} {k_y}\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right)&= {\sigma _s}^2\exp \left\{ {-\dfrac{1}{2}\sum _{i=1}^{m} \dfrac{{\left( {{x_i} - {x'_i}} \right) }^{2}}{l_i^2}} \right\} + \sigma _y^2 \end{aligned}$$

(26)

$$\begin{aligned} \dfrac{{\partial {k_y}}}{{\partial l_j}}= {\sigma _s}^2\exp \left\{ {-\dfrac{1}{2}\sum _{i=1}^{m} \dfrac{{\left( {{x_i} - {x'_i}} \right) }^{2}}{l_i^2}} \right\} {\dfrac{{\left( {{x_i} - {x'_i}} \right) }^{2}}{l_j^3}}; & \\ 1\leqslant j \leqslant m &\\\end{aligned}$$

(27)

$$\begin{aligned} \dfrac{{\partial {k_y}}}{{\partial {\sigma _s}}}&= 2{\sigma _s}\exp \left\{ {-\dfrac{1}{2}\sum _{i=1}^{m} \dfrac{{\left( {{x_i} - {x'_i}} \right) }^{2}}{l_i^2}} \right\} \end{aligned}$$

(28)

$$\begin{aligned} \dfrac{{\partial {k_y}}}{{\partial {\sigma _y}}}&= 2{\sigma _y} \end{aligned}$$

(29)

1.2 Derivatives of covariance function with respect to arguments

First derivative

$$\begin{aligned} \frac{{\partial k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) }}{{\partial x_j^\prime }}&= \frac{\partial }{{\partial x_j^\prime }}\left[ {{\sigma _s}^2\exp \left\{ { - \frac{1}{2}\sum \limits _{i = 1}^m {\frac{{{{\left( {{x_i} - {{x'}_i}} \right) }^2}}}{l_i^2}} } \right\} } \right] ; 1\leqslant j \leqslant m\\ &={k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) }\frac{\partial }{{\partial x_j^\prime }} \left[ { - \frac{1}{2}\sum \limits _{i = 1}^m {\frac{{{{\left( {{x_i} - {{x'}_i}} \right) }^2}}}{{l_i^2}}} } \right] \\&=k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) \frac{{\left( {{x_j} - {{x'}_j}} \right) }}{{l_j^2}}\\ \end{aligned}$$

(30)

Second derivative

$$\begin{aligned} \begin{aligned} \frac{{{\partial ^2}k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) }}{{\partial x_j^{\prime 2}}}&= \frac{\partial }{{\partial x_j^\prime }}\left[ {k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) \frac{{\left( {{x_j} - {{x'}_j}} \right) }}{{l_j^2}}} \right] ; 1\leqslant j \leqslant m \\ \dfrac{{{\partial ^2}k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) }}{{\partial {{{x}_j^{\prime }}}^2}}&= k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) \frac{{\left( {{x_j} - {{x'}_j}} \right) }}{{l_j^2}}\frac{{\left( {{x_j} - {{x'}_j}} \right) }}{{l_j^2}} \\&\quad + k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) \left( {\frac{{ - 1}}{{l_j^2}}} \right) \\&=\frac{{k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) }}{{l_j^2}}\left[ {\frac{{{{\left( {{x_j} - {{x'}_j}} \right) }^2}}}{{l_j^2}} - 1} \right] \\ \end{aligned} \end{aligned}$$

(31)

Auto covariance

$$\begin{aligned} \begin{aligned} \frac{{{\partial ^2}k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) }}{{\partial {x_j}\partial x_j^\prime }}&= \frac{\partial }{{\partial {x_j}}}\left[ {k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) \frac{{\left( {{x_j} - {{x'}_j}} \right) }}{{l_j^2}}} \right] \\&= \frac{{k\left( {{{\mathbf {x}}},{\mathbf {x'}}} \right) }}{{l_j^2}}\left[ {1 - \frac{{{{\left( {{x_j} - {{x'}_j}} \right) }^2}}}{{l_j^2}}} \right] \end{aligned} \end{aligned}$$

(32)

Appendix 2: Factorization of the Richards equation

This appendix provides the representation of the Richards equation in terms of state variable $(\theta )$ and its partial derivatives $\left( {\dfrac{{\partial \theta }}{{\partial t}},\dfrac{{\partial \theta }}{{\partial x}}\ {\mathrm {and}} \ \dfrac{{{\partial ^2}\theta }}{{\partial {x^2}}}} \right)$.

The van Genuchten (1980) model is used to relate hydraulic conductivity (k), pressure head (h) and relative saturation $(\Theta )$ as

$$\begin{aligned} \Theta &= \dfrac{1}{{{{\left[ {1 + {{\left( {\alpha \left| h \right| } \right) }^n}} \right] }^m}}} \end{aligned}$$

(33)

$$\begin{aligned} k &= {k_s}{\Theta ^{1/2}}{\left[ {1 - {{\left( {1 - {\Theta ^{1/m}}} \right) }^m}} \right] ^2} \end{aligned}$$

(34)

where ${k_s}$ is saturated hydraulic conductivity, and ${\theta _r}$ and ${\theta _s}$ are residual and saturated soil moisture content, respectively. The relative saturation terms $\alpha \,\,({\hbox {L}}^{-1})$, m and n represent model parameters where, m and n are related as $n\left( {1 - m} \right) = 1$. Term $(\Theta )$ is defined as

$$\begin{aligned} \Theta = \dfrac{{\theta - {\theta _r}}}{{{\theta _s} - {\theta _r}}} \end{aligned}$$

(35)

To write the Richards equation in terms of soil moisture and its partial derivatives, we need to estimate $\dfrac{\partial {\Theta }}{\partial {\theta }}$, $\dfrac{\partial {k}}{\partial {\Theta }}$, $\dfrac{\partial {\Theta }}{\partial {h}}$ and $\dfrac{\partial {h}}{\partial {\theta }}$. These partial derivatives were estimated as follows. Differentiating $\Theta$ with respect to $\theta$

$$\begin{aligned} \dfrac{{\partial \Theta }}{{\partial \theta }} = \dfrac{1}{{{\theta _s} - {\theta _r}}} \end{aligned}$$

(36)

Differentiating (34) with respect to $\Theta$

$$\begin{aligned}\dfrac{{\partial k}}{{\partial \Theta }} &= 2{k_s}{\Theta ^{1/2}}\left[ {1 - {{\left( {1 - {\Theta ^{1/m}}} \right)}^m}} \right]\left[ {{{\left( {1 - {\Theta ^{1/m}}} \right)}^{m - 1}}\left({{\Theta ^{1/m - 1}}} \right)} \right] \\&+\dfrac{k_s}{2}{\Theta ^{-1/2}}{\left[ {1 - {{\left( {1 -{\Theta ^{1/m}}} \right)}^m}} \right]^2}\end{aligned}$$

(37)

Rearranging Eq. (34)

$$\begin{aligned} \begin{aligned} 1 + {\left( {\alpha \left| h \right| } \right) ^n}&= {\left( {\dfrac{1}{\Theta }} \right) ^{{1}/{m}}}\\ \left( {\alpha \left| h \right| } \right)&= {\left[ {{{\left( {\dfrac{1}{\Theta }} \right) }^{{1}/{m}}} - 1} \right] ^{{1}/{n}}} \end{aligned} \end{aligned}$$

(38)

Again rearranging (33) and differentiating it with respect to h

$$\begin{aligned} \dfrac{1}{\Theta }&= {\left[ {1 + {{\left( {\alpha \left| h \right| } \right) }^n}} \right] ^m}\end{aligned}$$

(39)

$$\begin{aligned} \dfrac{\partial }{{\partial h}}\left( {\dfrac{1}{\Theta }} \right)&= m{\left\{ {1 + {{\left( {\alpha \left| h \right| } \right) }^n}} \right\} ^{m - 1}}\left\{ {n{{\left( {\alpha \left| h \right| } \right) }^{n - 1}}} \right\} \alpha {\mathop {\mathrm{sign}}\nolimits } \left( h \right) \end{aligned}$$

(40)

$$\begin{aligned} \dfrac{{ - 1}}{{{\Theta ^2}}}\dfrac{{\partial \Theta }}{{\partial h}}&= \alpha mn{\mathop {\mathrm{sign}}\nolimits } \left( h \right) {\left\{ {1 + {{\left( {\alpha \left| h \right| } \right) }^n}} \right\} ^{m - 1}}\left\{ {{{\left( {\alpha \left| h \right| } \right) }^{n - 1}}} \right\} \end{aligned}$$

(41)

Substituting (38) into (41) (to represent $\dfrac{\partial \Theta }{\partial h}$ as a function of $\Theta )$

$$\begin{aligned} \begin{aligned} \dfrac{{\partial \Theta }}{{\partial h}}&= - \dfrac{{\alpha m{\mathop {\mathrm{sign}}\nolimits } \left( h \right) }}{{1 - m}}{\Theta ^2}{\left\{ {{{\left( {\dfrac{1}{\Theta }} \right) }^{{1}/{m}}}} \right\} ^{m - 1}}{\left\{ {{{\left( {\dfrac{1}{\Theta }} \right) }^{{1}/{m}}} - 1} \right\} ^m}\\&= - \dfrac{{\alpha m{\mathop {\mathrm{sign}}\nolimits } \left( h \right) }}{{1 - m}}{\Theta ^{\left(- m + 1\right)/{m} + 2}}{\left\{ {{{\left( {\dfrac{1}{\Theta }} \right) }^{{1}/{m}}} - 1} \right\} ^m}\\&= - \dfrac{{\alpha m{\mathop {\mathrm{sign}}\nolimits } \left( h \right) }}{{1 - m}}{\Theta ^{\left(m + 1\right)/{m}}}{\left\{ {{{\left( {\dfrac{1}{\Theta }} \right) }^{{1}/{m}}} - 1} \right\} ^m}\\&= {B_0}{\Theta ^{\left(1 + m\right)/{m}}}{\left\{ {{{\Theta }^{{{ - 1}}/{m}}} - 1} \right\} ^m} \end{aligned} \end{aligned}$$

(42)

where ${B_0} = - \dfrac{{\alpha m{\mathop {\mathrm{sign}}\nolimits } \left( h \right) }}{{1 - m}}$

Using Eq. (42), $\dfrac{{\partial h}}{{\partial \theta }}$ could be obtained as

$$\begin{aligned}\dfrac{{\partial h}}{{\partial \theta }} = {\left( {\dfrac{{\partial \Theta }}{\partial h}} \right) ^{ - 1}}\dfrac{1}{{{\theta _s} - {\theta _r}}}=\dfrac{1}{B_0}{\Theta ^{ - \left( {1 + m}\right) /{m} }}{\left\{ {{\Theta ^{{-1}/{m}}} - 1} \right\} ^{ - m}}\end{aligned}$$

(43)

The Richards equation can be factorized into three terms as

$$\begin{aligned} \dfrac{{\partial \theta }}{{\partial t}}&= \dfrac{\partial }{{\partial z}}\left[ {k\dfrac{{\partial h}}{{\partial z}}} \right] + \dfrac{{\partial k}}{{\partial z}}\end{aligned}$$

(44)

$$\begin{aligned}&= \underbrace{\dfrac{{\partial k}}{{\partial z}}\dfrac{{\partial h}}{{\partial z}}}_{\mathrm{{Term 1}}} + \underbrace{k\dfrac{{{\partial ^2}h}}{{\partial {z^2}}}}_{\mathrm{{Term 2}}} + \underbrace{\dfrac{{\partial k}}{{\partial z}}}_{\mathrm{{Term 3}}} \end{aligned}$$

(45)

Term1:

$$\begin{aligned} \dfrac{{\partial k}}{{\partial z}}\dfrac{{\partial h}}{{\partial z}}&= {\mathrm {Term3}}\left[ {\dfrac{{\partial h}}{{\partial \theta }}\dfrac{{\partial \theta }}{{\partial z}}} \right] \end{aligned}$$

(46)

$$\begin{aligned}&= {\mathrm {Term3}}\left[ {\left\{ {{{\left( {\dfrac{{\partial \Theta }}{{\partial h}}} \right) }^{ - 1}}\dfrac{1}{{{\theta _s} - {\theta _r}}}} \right\} \dfrac{{\partial \theta }}{{\partial z}}} \right] \end{aligned}$$

(47)

Term1 is obtained by substituting (42) in (47).

Term2:

$$\begin{aligned} \begin{aligned} \dfrac{{{\partial ^2}h}}{{\partial {z^2}}}&= \dfrac{\partial }{{\partial z}}\left( {\dfrac{{\partial h}}{{\partial z}}} \right) \\&= \dfrac{\partial }{{\partial z}}\left( {\dfrac{{\partial h}}{{\partial \theta }}\dfrac{{\partial \theta }}{{\partial z}}} \right) \\&= \dfrac{{{\partial ^2}h}}{{\partial z\partial \theta }}\dfrac{{\partial \theta }}{{\partial z}} + \left( {\dfrac{{\partial h}}{{\partial \theta }}} \right) \dfrac{{{\partial ^2}\theta }}{{\partial {z^2}}} \end{aligned} \end{aligned}$$

(48)

Deriving $\dfrac{{{\partial ^2}h}}{{\partial z\partial \theta }}$ by differentiating (43) with respect to z.

$$\begin{aligned}\dfrac{\partial }{{\partial z}}\left( {\dfrac{{\partial h}} {{\partial \theta }}} \right) &= -\dfrac{1}{{{B_0}}}\left({{\dfrac{{1 + m}}{m}} } \right){\Theta^{-\left(1+2m \right)/m}}{\left\{ {{\Theta ^{{-1}/{m}}} - 1} \right\}^{ - m}}\dfrac{{\partial \Theta }}{{\partial z}} \\& + \dfrac{1}{{{B_0}}}{\Theta ^{ - \left( {1 + m} \right)/{m}}}{\left\{ { {{\Theta ^{- 1/{m}}} - 1}} \right\}^{-m- 1}}\left({{\Theta ^{{-\left( 1+m\right)}/{m}}}} \right)\dfrac{{\partial\Theta }}{{\partial z}}\end{aligned}$$

(49)

$$\begin{aligned} \dfrac{\partial }{{\partial z}}\left( {\dfrac{{\partial h}}{{\partial \theta }}} \right)&= \dfrac{1}{{{B_0}}}\dfrac{{\partial \Theta }}{{\partial z}} \nonumber \times \\&\quad \quad \left[ \begin{array}{l} - \left( {\dfrac{{1 + m}}{m}} \right) {\Theta ^{{{ - \left( {1 + 2m} \right) }}/{m}}}{\left( {{\Theta ^{{{ - 1}}/{m}}} - 1} \right) ^{ - m}}\\ + {\Theta ^{ - \left(2 + 2m\right)/m }}{\left( {{\Theta ^{{ - 1}/{m}}} - 1} \right) ^{ - m - 1}} \end{array} \right] \end{aligned}$$

(50)

$$\begin{aligned} \dfrac{{{\partial ^2}h}}{{\partial z\partial \theta }}&= \dfrac{1}{{{B_0}}}\dfrac{1}{{{\theta _s} - {\theta _r}}}\dfrac{{\partial \theta }}{{\partial z}}\nonumber\times \\&\quad \quad \left[ \begin{array}{l} - \left( {\dfrac{{1 + m}}{m}} \right) {\Theta ^{{ - \left( {1 + 2m} \right) }/{m}}}{\left\{ {{\Theta ^{{ - 1}/{m}}} - 1} \right\} ^{ - m}}...\\ + {\Theta ^{ - \left( {2 + 2m}\right)/{m} }}{\left( {{\Theta ^{{ - 1}/{m}}} - 1} \right) ^{ - m - 1}} \end{array} \right] \end{aligned}$$

(51)

Term2 is obtained by substituting (43) and (51) in (48)

Term3:

$$\begin{aligned} \dfrac{{\partial k}}{{\partial z}} &= \dfrac{{\partial k}}{{\partial \theta }}\dfrac{{\partial \theta }}{{\partial z}} \end{aligned}$$

(52)

$$\begin{aligned} \dfrac{{\partial k}}{{\partial \theta }} &= \dfrac{{\partial k}}{{\partial \Theta }}\dfrac{{\partial \Theta }}{{\partial \theta }} \end{aligned}$$

(53)

Using (36), (53) and (37), Eq. (52) is expressed as

$$\begin{aligned} \begin{aligned} \dfrac{{\partial k}}{{\partial z}}&= \dfrac{{\partial k}}{{\partial \theta }}\dfrac{{\partial \theta }}{{\partial z}} = \dfrac{1}{{{\theta _s} - {\theta _r}}}{k_s}\dfrac{{\partial \theta }}{{\partial z}} \times\\&\quad \quad \left[ \begin{array}{l} 2{\Theta ^{1/2}}\left\{ {1 - {{\left( {1 - {\Theta ^{1/m}}} \right) }^m}} \right\} \left\{ {{{\left( {1 - {\Theta ^{1/m}}} \right) }^{m - 1}}\left( {{\Theta ^{1/m - 1}}} \right) } \right\} \\ + \dfrac{1}{2}{\Theta ^{ - 1/2}}{\left\{ {1 - {{\left( {1 - {\Theta ^{1/m}}} \right) }^m}} \right\} ^2} \end{array} \right] \end{aligned} \end{aligned}$$

(54)