Parameter estimation of complex mixed models based on meta-model approach

Abstract

Complex biological processes are usually experimented along time among a collection of individuals, longitudinal data are then available. The statistical challenge is to better understand the underlying biological mechanisms. A standard statistical approach is mixed-effects model where the regression function is highly-developed to describe precisely the biological processes (solutions of multi-dimensional ordinary differential equations or of partial differential equation). A classical estimation method relies on coupling a stochastic version of the EM algorithm with a Monte Carlo Markov Chain algorithm. This algorithm requires many evaluations of the regression function. This is clearly prohibitive when the solution is numerically approximated with a time-consuming solver. In this paper a meta-model relying on a Gaussian process emulator is proposed to approximate the regression function, that leads to what is called a mixed meta-model. The uncertainty of the meta-model approximation can be incorporated in the model. A control on the distance between the maximum likelihood estimates of the mixed meta-model and the maximum likelihood estimates of the exact mixed model is guaranteed. Eventually, numerical simulations are performed to illustrate the efficiency of this approach.

Appendix

1.1 Proof of Proposition 3

We have

$$\begin{aligned}&|p(\mathbf {y};\theta )-\tilde{p}_D(\mathbf {y};\theta )|\\&\quad \le \int |p(\mathbf {y}|\varvec{\psi };\varvec{\theta })-\tilde{p}_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })|p(\varvec{\psi })d\varvec{\psi }\,. \end{aligned}$$

Therefore, we start by studying $|p(\mathbf {y}|\varvec{\psi };\varvec{\theta })-\tilde{p}_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })|$:

$$\begin{aligned}&(2\pi \sigma _\varepsilon ^2)^{n_{tot}/2} |p(\mathbf {y}|\varvec{\psi };\varvec{\theta })-\tilde{p}_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })|\\&\quad = \Bigg | \exp \bigg (-\frac{1}{2\sigma _\varepsilon ^2}\sum _{ij}(y_{ij}-f(t_{ij},\psi _i))^2\bigg )\\&\qquad -\exp \bigg (-\frac{1}{2\sigma _\varepsilon ^2}\sum _{ij}(y_{ij}-m_D(t_{ij},\psi _i))^2\bigg )\Bigg |\\&\quad = \ \exp \left( -\frac{1}{2\sigma _\varepsilon ^2}\sum _{ij}(y_{ij}-f(t_{ij},\psi _i))^2\right) \times \\&\qquad \Bigg |1-\exp \bigg (-\frac{1}{2\sigma _\varepsilon ^2}\sum _{ij}\Big ((y_{ij}-m_D(t_{ij},\psi _i))^2\\&\qquad -(y_{ij}-f(t_{ij},\psi _i))^2\Big )\bigg ) \Bigg |\\&\quad \le \Bigg |1-\exp \bigg (-\frac{1}{2\sigma _\varepsilon ^2}\sum _{ij}\Big ( (f(t_{ij},\psi _i)-m_D(t_{ij},\psi _i))\\&\quad (2y_{ij}-f(t_{ij},\psi _i) -m_D(t_{ij},\psi _i))\Big )\bigg )\Bigg |\,. \end{aligned}$$

Under the assumption that the functions f and $m_D$ are uniformly bounded on the support of $\psi$, there exists a constant $C_y$ which is uniform according to $\psi$ such that $|2y_{ij}-f(t,\psi )-m_D(t,\psi )|\le C_y$. Proposition 1 implies that the approximation error due to the metamodel $|f(t_{ij},\psi _i)-m_D(t_{ij},\psi _i)|$ is controlled by inequality (8):

$$\begin{aligned} |f(t_{ij},\psi _i)-m_D(t_{ij},\psi _i)|\le \Vert f\Vert _{\mathcal {H}_K}G_K(a_D)\,. \end{aligned}$$

Then there exists a constant $C_y$ depending only on $\mathbf {y}$ such that

$$\begin{aligned}&(2\pi \sigma _\varepsilon ^2)^{n_{tot}/2} |p(\mathbf {y}|\varvec{\psi };\varvec{\theta })-\tilde{p}_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })|\\&\quad \le C_y \frac{n_{tot}}{2\sigma _\varepsilon ^2} \Vert f\Vert _{\mathcal {H}_K}G_K(a_D). \end{aligned}$$

Finally

$$\begin{aligned} |p(\mathbf {y};\theta )-\tilde{p}_D(\mathbf {y};\theta )|\le \frac{C_y}{(2\pi \sigma _\varepsilon ^2)^{n_{tot}/2}}\frac{n_{tot}}{2\sigma _\varepsilon ^2} \Vert f\Vert _{\mathcal {H}_K}G_K(a_D)\,. \end{aligned}$$

$\square$

1.2 Proof of Proposition 4

We study the distance between the two likelihoods $p_D$ and $\tilde{p}_D$. As in Proposition 3, we start by studying $|p_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })-\tilde{p}_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })|$. We consider two Gaussian distributions with same expectations and different covariance matrix. Thus this distance is maximum when $\sum (y_{ij}-m_D(t_{ij},\psi _i))^2=0$. This yields

$$\begin{aligned}&(2\pi )^{n_{tot}/2} |p_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })-\tilde{p}_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })|\\&\quad \le \left| \frac{1}{\sigma _\varepsilon ^{n_{tot}}}-\frac{1}{\sqrt{|\sigma _\varepsilon ^2I_{n_{tot}}+\mathbf {C}_D|}}\right| \\&\quad =\frac{1}{ \sigma _\varepsilon ^{n_{tot}}}\left| 1-\frac{\sigma _\varepsilon ^{n_{tot}}}{|\sigma _\varepsilon ^2I_{n_{tot}}+\mathbf {C}_D|^{1/2}}\right| \\&\quad \le \frac{1}{ \sigma _\varepsilon ^{n_{tot}}}\left| 1-\frac{\sigma _\varepsilon ^{n_{tot}}}{(\sigma _\varepsilon ^2+\frac{1}{n_{tot}}\sum _{ij} C_D(t_{ij},\psi _i;t_{ij},\psi _i))^{n_{tot}/2}}\right| \\&\quad \le \frac{1}{ \sigma _\varepsilon ^{n_{tot}}}\left| 1-\frac{1}{(1+\frac{1}{\sigma _\varepsilon ^2n_{tot}}\sum _{ij} C_D(t_{ij},\psi _i; t_{ij},\psi _i))^{n_{tot}/2}}\right| \\ \end{aligned}$$

where we use that the determinant, as a product of eigen values, is smaller than a function of the trace of the matrix. Thus, the sum is over the diagonal of the matrix $\mathbf {C}_D$ i.e. the sum of the variances. Then, we obtain that there exists a constant C such that

$$\begin{aligned}&|p_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })-\tilde{p}_D(\mathbf {y}|\varvec{\psi };\varvec{\theta })|\\&\quad \le C \frac{1}{ \sigma _\varepsilon ^{n_{tot}}}\left| \frac{1}{2\sigma _\varepsilon ^2}\sum _{ij} C_D(t_{ij},\psi _i; t_{ij},\psi _i)\right| \\&\quad \le C (2\pi )^{n_{tot}/2}\frac{n_{tot}}{ \sigma _\varepsilon ^{n_{tot}+2}}G_K(a_D) \end{aligned}$$

where the last inequality holds using Proposition 1. Finally, we obtain

$$\begin{aligned} |p_D(\mathbf {y};\theta )-\tilde{p}_D(\mathbf {y};\theta )|\le C_y \frac{n_{tot}}{ \sigma _\varepsilon ^{n_{tot}+2}}G_K(a_D). \end{aligned}$$

The proof is similar for the distance between the two likelihoods $\bar{p}_D$ and $\tilde{p}_D$. $\square$

1.3 Details of assumptions for Proposition 2

(M1) :

function f and the individual parameters distribution $p(\psi ; \beta )$ are such that there exist functions $g, \nu$ of $\theta$ verifying

$$\begin{aligned} \log p(y, \psi ;\theta )= - g(\theta ) + \left\langle S(y,\psi ), \nu (\theta ) \right\rangle \end{aligned}$$

where $S(y, \psi )$ is a minimal sufficient statistic of the complete model, taking its value in a subset ${\mathcal {S}}$ and $\left\langle \cdot , \cdot \right\rangle$ is the scalar product.

(M2) :

The functions $g(\theta )$ and $\nu (\theta )$ are twice continuously differentiable on $\varTheta$.

(M3) :

The function $\bar{s}:\varTheta \longrightarrow \mathcal {S}$ defined as $\bar{s} (\theta ) = \int S(y,\psi ) p(\psi |y;\theta ) d\psi$ is continuously differentiable on $\varTheta$.

(M4) :

The function $\ell (\theta ) = \log p(y,\theta )$ is continuously differentiable on $\varTheta$ and

$$\begin{aligned} \partial _\theta \int p(y,\psi ;\theta ) d\psi = \int \partial _\theta p(y,\psi ;\theta ) d\psi . \end{aligned}$$

(M5) :

Define $L:\mathcal {S}\times \varTheta \longrightarrow \mathbb {R}$ as $L(s,\theta ) = -g(\theta )+\langle s,\nu (\theta )\rangle$. There exists a function $\hat{\theta }:\mathcal {S}\longrightarrow \varTheta$ such that

$$\begin{aligned} \forall \theta \in \varTheta , \quad \forall s \in \mathcal {S}, \quad L(s,\hat{\theta }(s))\ge L(s,\theta ). \end{aligned}$$

(SAEM1) :

The positive decreasing sequence of the stochastic approximation $(\alpha _m)_{m \ge 0}$ is such that $\sum _{m} \alpha _m = \infty$ and $\sum _{m}\alpha ^2_m < \infty$.

(SAEM2) :

$\ell :\varTheta \rightarrow \mathbb {R}$ and $\hat{\theta }:\mathcal {S}\rightarrow \varTheta$ are d times differentiable, where d is the dimension of $S(y,\psi )$.

(SAEM3) :

For all $\theta \in \varTheta$, $\int || S(y,\psi )||^2\, p(\psi |y;\theta ) d\psi < \infty$ and the function $\varGamma (\theta )=Cov_\theta (S(\psi ))$ is continuous.

(SAEM4) :

S is a bounded function.

(SAEM5’) :

Let us denote $\Pi _\theta$ the transition kernel of the PMCMC algorithm and $\pi (\psi )=p(\psi |y;\theta )$ its stationary distribution. We assume that $\Pi _\theta$ is Lipschitz in $\theta$ and generates a ergodic chain such that for any starting point $(\varphi )$

$$\begin{aligned} \sum _{m\ge 0} \Vert 1_{\varphi }\Pi _\theta ^m- \pi \Vert _{TV} <\infty \end{aligned}$$

where $\Vert \cdot \Vert _{TV}$ is the total variation of probability measures. This property is also called ergodicity of degree 2.

1.4 Additional results for Section 7

In Tables 1, 3 and 2, relative bias and relative root mean square error (RMSE) are displayed for each population parameter. The $95\,\%$ coverage rates correspond to the coverage rate of the confidence interval on parameters based on the stochastic approximation of the Fisher Information matrix are also displayed in these tables. These results are obtained from 100 replications in Table 1 and from 1000 replications in Tables 3 and 2.

Table 1 One compartment simulations: relative bias ($\,\%$), relative MSE ($\,\%$) and coverage rate ($\,\%$) computed over 100 simulations, with the complete meta-, intermediate meta-, the simple meta- and the exact mixed models

Table 2 One compartment simulations: relative bias ($\,\%$), relative MSE ($\,\%$) and coverage rate ($\,\%$) computed over 1000 simulations, with the intermediate meta-, the simple meta- and the exact mixed models

Table 3 Michaelis–Menten pharmacokinetic simulations: relative bias ($\,\%$), relative MSE ($\,\%$) and coverage rate ($\,\%$) computed over 1000 simulations, with the intermediate meta-, the simple meta- and the exact mixed models