Continuous-time Markov modelling of flexible-dose depression trials

Abstract

The aim of this paper is to provide a systematic methodology for modelling longitudinal data to be used in contexts of limited or even absent knowledge of the physiological mechanism underlying the disease time course. Adopting a system-theoretic paradigm, a population response model is developed where the clinical endpoint is described as a function of the patient’s health state. In particular, a continuous-time stochastic approach is proposed where the clinical score and its time-derivative summarize the patient’s health state affected by a random term accounting for exogenous unpredictable factors. The proposed approach is validated on experimental data from the placebo and drug arms of a Phase II depression trial. Since some subjects in the trial may undergo changes in their treatment dose due to the flexible dosing scheme, dose escalations are modelled as instantaneous perturbations on the state. In its simplest form—an integrated Wiener process—was able to correctly capture the individual responses in both treatment arms. However, a better description of inter-individual variability was obtained by means of a stable Markovian model. Parameter estimation has been carried out according to the empirical Bayes method.

Appendices

Appendix 1: Auto-covariance function of Markov processes

In this appendix, the formulas to derive the auto-covariance functions $\overline{R}(t, \tau )$ and $\widetilde{R}(t, \tau )$ for the average curve and individual shift, respectively, are detailed. First, some basic properties of the auto-covariance of Markov processes are recalled. Then, we consider their application to the model used for the typical response and the individual shift. Differently from [13, 14, 19] the present derivation accounts for non-zero initial variance of the individual shift and the possible presence of exogenous factors as dose changes.

Definition of auto-covariance

A random process can be seen as a sequence of random variables over time. For example, one common random process is the white Gaussian noise where each random variable $w(t)$ is a Gaussian random variable with zero-mean and variance $\lambda ^2$. Given two distinct time points t and $\tau$, the associated values of the random process, i.e. the random variables $w(t)$ and $w(\tau )$, are characterized by a covariance $R(t, \tau ) = cov[w(t), w(\tau )]$ for all possible choices of $t$ and $\tau$. The function $R(t, \tau )$ is commonly known as the auto-covariance function of the process $w(t)$.

Basic properties

Given the following system of equations:

$$\begin{aligned} \left\{ \begin{array}{lll} \dot{x}(t) = A x(t) + B w(t) \\ z(t) = C x(t) \end{array} \right. \end{aligned}$$

(12)

where $x(t_0) \sim N(0, P_0), w(t) \sim WGN(\lambda ^2)$, and $E[x(t_0)w(t)] = 0$, its solution $x(t)$ is given by the Lagrange formula

$$\begin{aligned} x(t) = e^{A(t - t_0)} x(t_0) + \int \limits _{t_0}^{t} e^{A(t - s) B w(s) ds} \end{aligned}$$

(13)

Let $P(t) = var[x(t)] = E[x(t)x(t)^T]$ be the variance of the state. The auto-covariance function of $x(t)$ is given by

$$\begin{aligned} cov[x(t), x(\tau )] = E[x(t)x(\tau )^T] = \left\{ \begin{array}{ll} P(t) e^{A^T(\tau - t)} &{} t \le \tau \\ e^{A(t - \tau )} P(t) &{} t > \tau \end{array} \right. \end{aligned}$$

(14)

Accordingly, the auto-covariance function of the output $z(t)$ is

$$\begin{aligned} cov[z(t), z(\tau )] = cov[Cx(t), Cx(\tau )] = C E[x(t)x(\tau )^T] C^T \end{aligned}$$

(15)

The state variance $P(t)$ satisfies the following Lyapunov differential equation [23]

$$\begin{aligned} \dot{P}(t) = A P(t) + P(t) A^T + B \lambda ^2 B^T , \qquad P(0) = P_0 \end{aligned}$$

(16)

from which

$$\begin{aligned} P(t) = e^{At} P(0) e^{A^Tt} + \int \limits _{0}^{t} e^{A(t - s)} B \lambda ^2 B^T e^{A^T(t - s)} ds \end{aligned}$$

(17)

and

$$\begin{aligned} R(t, \tau ) = \left\{ \begin{array}{ll} C P(t) e^{A^T(\tau - t)} C^T &{} t \le \tau \\ C e^{A(t - \tau )} P(\tau ) C^T &{} t > \tau \end{array} \right. \end{aligned}$$

(18)

Auto-covariance function of the population curve

Based on the matrices $A, B$, and $C$ defined in (4) and recalling that $P_0 = 0$, it can be found that

$$\begin{aligned} e^{At} = \left[ \begin{array}{cc} 1 &{} t \\ 0 &{} 1 \end{array} \right] \end{aligned}$$

(19)

and, according to (17)

$$\begin{aligned} P(t) = \left[ \begin{array}{cc} p_{11}(t) &{} p_{12}(t) \\ p_{21}(t) &{} p_{22}(t) \end{array} \right] \end{aligned}$$

(20)

where

$$\begin{aligned} p_{11}(t)&= \dfrac{\overline{\lambda }^2 t^3}{3} \nonumber \\ p_{12}(t)&= p_{21}(t) = \dfrac{\overline{\lambda }^2 t^2}{2} \nonumber \\ p_{22}(t)&= \overline{\lambda }^2 t \end{aligned}$$

(21)

Moreover, based on (18), the auto-covariance function $\overline{R}(t, \tau )$ of the population curve is

$$\begin{aligned} \overline{R}(t, \tau ) = \overline{\lambda }^2 \left\{ \begin{array}{ll} \dfrac{t^2}{2} \left( \tau - \dfrac{t}{3}\right) &{} t \le \tau \\ \dfrac{\tau ^2}{2} \left( t - \dfrac{\tau }{3}\right) &{} t > \tau \end{array} \right. \end{aligned}$$

(22)

Auto-covariance function of the individual shifts: non-escalating subjects

In this case

$$\begin{aligned} A&= \widetilde{A} = \left[ \begin{array}{cc} a &{} 1 \\ 0 &{} a \end{array} \right] \end{aligned}$$

(23)

$$\begin{aligned} e^{\widetilde{A}t}&= \left[ \begin{array}{cc} e^{at} &{} t e^{at} \\ 0 &{} e^{at} \end{array} \right] \end{aligned}$$

(24)

and, based on (20),

$$\begin{aligned} P(t) = e^{\widetilde{A}t} \left[ \begin{array}{cc} \sigma ^2_0 &{} 0 \\ 0 &{} \sigma ^2_1 \end{array}\right] e^{\widetilde{A}^Tt} + \int \limits _{0}^{t} e^{\widetilde{A}(t - s)} B \widetilde{\lambda }^2 B^T e^{\widetilde{A}^T(t - s)} ds \end{aligned}$$

(25)

Then, the auto-covariance function $\widetilde{R}(t, \tau )$ is

$$\begin{aligned} \widetilde{R}(t, \tau ) = \left\{ \begin{array}{ll} e^{a(\tau - t)} p_{11}(t) + (\tau - t) e^{a(\tau - t)} p_{12}(t) &{} t \le \tau \\ e^{a(t - \tau )} p_{11}(\tau ) + (t - \tau ) e^{a(t - \tau )} p_{12}(\tau ) &{} t > \tau \end{array} \right. \end{aligned}$$

(26)

where

$$\begin{aligned} p_{11}(t)&= \widetilde{\lambda }^2 \left( \dfrac{t^2 e^{2at}}{2a} - \dfrac{t e^{2at}}{2a^2} + \dfrac{e^{2at} - 1}{4a^3} \right) + f^2_{11} \sigma ^2_0 + f^2_{12} \sigma ^2_1 \nonumber \\ p_{12}(t)&= p_{21}(t) = \dfrac{\widetilde{\lambda }^2}{4a^2} \left( 2ate^{2at} - e^{2at} + 1 \right) + f_{12} \sigma ^2_1 f_{22} \nonumber \\ p_{22}(t)&= \dfrac{\widetilde{\lambda }^2}{2a} \left( e^{2at} - 1 \right) + f^2_{22} \sigma ^2_1 \end{aligned}$$

(27)

where $f_{11}(t) = f_{22}(t) = e^{at}$ and $f_{12}(t) = te^{at}$.

Auto-covariance function of the individual shifts: escalating subjects

In order to define the auto-covariance function $\widetilde{R}(t, \tau )$ for the escalating subjects, three scenarios must be taken into account with respect to the occurrence of the time of dose change $t_{esc}$:

• 1. $t < \tau < t_{esc}$

• 2. $t < t_{esc} < \tau$

• 3. $t_{esc} < t < \tau$

The auto-covariance function relative to the first two scenarios coincides with the one defined in (26) for the non-escalating subjects. For the third scenario, Eq. (25) has to be extended to account for the exogenous event:

$$\begin{aligned} P(t) = e^{\widetilde{A}t} \left[ \begin{array}{cc} \sigma ^2_0 &{} 0 \\ 0 &{} \sigma ^2_1 \end{array}\right] e^{\widetilde{A}^Tt} + \int \limits _{0}^{t} e^{\widetilde{A}(t - s)} B \widetilde{\lambda }^2 B^T e^{\widetilde{A}^T(t - s)} ds + e^{\widetilde{A}(t - t_{esc})} \left[ \begin{array}{cc} 0 &{} 0 \\ 0 &{} \sigma ^2_{\varDelta } \end{array}\right] e^{\widetilde{A}^T (t - t_{esc})} \end{aligned}$$

(28)

More specifically, the elements of the matrix (20) are defined as:

$$\begin{aligned} p_{11}(t)&= \widetilde{\lambda }^2 \left( \dfrac{t^2 e^{2at}}{2a} - \dfrac{t e^{2at}}{2a^2} + \dfrac{e^{2at} - 1}{4a^3} \right) + f^2_{11} \sigma ^2_0 + f^2_{12} \sigma ^2_1 + \left( (t - t_{esc}) e^{a(t - t_{esc})}\right) ^2 \sigma ^2_{\varDelta } \nonumber \\ p_{12}(t)&= p_{21}(t) = \dfrac{\widetilde{\lambda }^2}{4a^2} \left( 2ate^{2at} - e^{2at} + 1 \right) + f_{12} \sigma ^2_1 f_{22} + (t - t_{esc}) e^{a(t - t_{esc})} \sigma ^2_{\varDelta } e^{a(t - t_{esc})}\nonumber \\ p_{22}(t)&= \dfrac{\widetilde{\lambda }^2}{2a} \left( e^{2at} - 1 \right) + f^2_{22} \sigma ^2_1 + e^{2a(t - t_{esc})} \sigma ^2{\varDelta } \end{aligned}$$

(29)

with $f_{11}(t) = f_{22}(t) = e^{at}$ and $f_{12}(t) = te^{at}$. Therefore, the auto-covariance function $\widetilde{R}(t, \tau )$ becomes

$$\begin{aligned} \widetilde{R}(t, \tau ) = \left\{ \begin{array}{ll} C \left( P(t) e^{\widetilde{A}^T(\tau - t)} + e^{\widetilde{A}(t - t_{esc})} \left[ \begin{array}{cc} 0 &{} 0 \\ 0 &{} \sigma ^2_{\varDelta } \end{array}\right] e^{\widetilde{A}^T (t - t_{esc})}\right) C^T &{} t \le \tau \\ C \left( e^{\widetilde{A}^T(t - \tau )} P(\tau ) + e^{\widetilde{A}(t - t_{esc})} \left[ \begin{array}{cc} 0 &{} 0 \\ 0 &{} \sigma ^2_{\varDelta } \end{array}\right] e^{\widetilde{A}^T (t - t_{esc})}\right) C^T&t > \tau \end{array} \right. \end{aligned}$$

(30)

Appendix 2: Parameter estimation

The algorithms reported in this section are essentially the same as in [13, 14]. They are repeated here for the sake of self-consistency.

As explained in Section Methods, the initial conditions $\overline{x}(0)$ for the average curve are characterized by infinite variance. Therefore, Eq. (2) can be reformulated as

$$\begin{aligned} z_i(t)&= \overline{z}^*(t) + \widetilde{z}_i(t) \nonumber \\ \overline{z}^*(t)&= \varPhi ^T(t) \varPsi + \overline{z}(t) \end{aligned}$$

(31)

where $\varPhi ^T(t) = [1 \quad t]$ and $\varPsi \sim N(0, \infty I)$.

Moreover, let $M$ be the total number of subjects and $n = \sum \nolimits _{i = 1}^{M} n_i$ the total number of data, Eq. (31) can be defined in vector notation:

$$\begin{aligned} \begin{array}{c} \mathbf {\overline{z}} = [\overline{z}(t^1_1) \quad \ldots \quad \overline{z}^{n_1}_1 \quad \ldots \quad \overline{z}(t^1_M) \quad \ldots \quad \overline{z}(t^{n_M}_M)]^T \\ \mathbf {\widetilde{z}} = [\widetilde{z}(t^1_1) \quad \ldots \quad \widetilde{z}^{n_1}_1 \quad \ldots \quad \widetilde{z}(t^1_M) \quad \ldots \quad \widetilde{z}(t^{n_M}_M)]^T \\ \mathbf {v} = [v^1_1 \quad \ldots \quad v^{n_1}_1 \quad \ldots \quad v^1_M \quad \ldots \quad v^{n_M}_M]^T \\ \varvec{\varPhi } = [\varPhi (t^1_1) \quad \ldots \quad \varPhi ^{n_1}_1 \quad \ldots \quad \varPhi (t^1_M) \quad \ldots \quad \varPhi (t^{n_M}_M)]^T \\ \\ \mathbf {Y} = \varvec{\varPhi } \varPsi + \mathbf {\overline{z}} + \mathbf {\widetilde{z}} + \mathbf {v} \end{array} \end{aligned}$$

(32)

The variance of the vectors $\mathbf {\overline{z}}$ and $\mathbf {\widetilde{z}}$, which are ${\mathbf {\overline{R}}}(t, \tau )$ and ${\mathbf {\widetilde{R}}}(t, \tau )$, respectively, can be defined as

$$\begin{aligned} {\mathbf {\overline{R}}}&= var [{\mathbf {\overline{z}}}] = \left[ \begin{array}{ccc} \overline{R}(t^1_1, \tau ^1_1) &{} \ldots &{} \overline{R}(t^1_1, \tau ^{n_M}_M) \\ \vdots &{} \vdots &{} \vdots \\ \overline{R}(t^{n_M}_M, \tau ^1_1) &{} \ldots &{} \overline{R}(t^{n_M}_M, \tau ^{n_M}_M) \end{array} \right] \end{aligned}$$

(33)

$$\begin{aligned} {\mathbf {\widetilde{R}}}&= var [{\mathbf {\widetilde{z}}}] = blockdiag \left\{ {\mathbf {\widetilde{R}}}_1, \ldots , {\mathbf {\widetilde{R}}}_M \right\} \end{aligned}$$

(34)

where ${\mathbf {\widetilde{R}}}_i$ for the $i$-th subject is

$$\begin{aligned} {\mathbf {\widetilde{R}}}_i = \left[ \begin{array}{ccc} \widetilde{R}(t^1_i, \tau ^1_j) &{} \ldots &{} \widetilde{R}(t^1_i, \tau ^{n_i}_i) \\ \vdots &{} \vdots &{} \vdots \\ \widetilde{R}(t^{n_i}_i, \tau ^1_i) &{} \ldots &{} \widetilde{R}(t^{n_i}_i, \tau ^{n_i}_i) \end{array} \right] \end{aligned}$$

(35)

Based on the definition of the auto-covariances $\overline{R}$ and $\widetilde{R}_i$, model (31) can be formulated as

$$\begin{aligned} \hat{\overline{z}}^*(t)&= E[\overline{z}^*(t)|{\mathbf {Y}}] = \sum \limits _{i = 1}^{M} \sum \limits _{k = 1}^{n_i} c^k_i \overline{R}(t, \tau ^k_i) + \varPhi ^T(t) {\mathbf {b}} \nonumber \\ \hat{z}_i(t)&= E[z_i(t)|{\mathbf {Y}}] = \hat{\overline{z}}^* + \sum \limits _{k = 1}^{n_i} c^k_i \widetilde{R}(t, \tau ^k_i) \end{aligned}$$

(36)

where $c^k_i$ represent the weights of the regularization network [24, 25] and

$$\begin{aligned} \mathbf {b}&= \left( \varvec{\varPhi }^T \mathbf {M}^{-1} \varvec{\varPhi } \right) ^{-1} \varvec{\varPhi }^T \mathbf {M}^{-1} \mathbf {Y} \\ \mathbf {c}&= \mathbf {M}^{-1} \left( \mathbf {Y} - \varvec{\varPhi } \mathbf {b} \right) \\ \mathbf {M}&= \mathbf {\overline{R}} + \mathbf {\widetilde{R}} + \varvec{\varSigma }_v \\ \varvec{\varSigma }_v&= \sigma ^2 \mathbf {I}_{Mn} \end{aligned}$$

Let $\alpha = [\overline{\lambda }^2 \quad \widetilde{\lambda }^2 \quad \sigma ^2_0 \quad \sigma ^2_1 \quad \sigma ^2_{\varDelta } \quad \sigma ^2]$ be the vector of the unknown hyperparameters. According to the Empirical Bayes paradigm [14, 15], two steps are performed: initially $\alpha$ is estimated through the Maximum Likelihood (ML) approach, by maximizing the marginal likelihood $p(Y|\alpha )$; once the ML estimates of the hyperparameters $\alpha ^{ML}$ are available, they are plugged into the model and the Bayesian estimate is calculated through the standard conditional posterior formula for linear Gaussian models. Uninformative priors were assumed in this analysis.

In particular,

$$\begin{aligned} \alpha ^{ML}&= \arg \min \nolimits _{\varvec{\alpha }} \left\{ log(det(\mathbf {M})) + \mathbf {Y}^T \mathbf {M}^{-1} \mathbf {Y} \right\} \nonumber \\ \alpha ^{ML}&= [\overline{\lambda }^2 \quad \widetilde{\lambda }^2 \quad \sigma ^2_0 \quad \sigma ^2_1 \quad \sigma ^2_{\varDelta } \quad \sigma ^2] \end{aligned}$$

(37)

More technical details can be found in [13].