Evaluation of estimation, prediction and inference for autocorrelated latent variable modeling of binary data—a simulation study

Abstract

Longitudinal models of binary or ordered categorical data are often evaluated for adequacy by the ability of these to characterize the transition frequency and type between response states. Drug development decisions are often concerned with accurate prediction and inference of the probability of response by time and dose. A question arises on whether the transition probabilities need to be characterized adequately to ensure accurate response prediction probabilities unconditional on the previous response state. To address this, a simulation study was conducted to assess bias in estimation, prediction and inferences of autocorrelated latent variable models (ALVMs) when the transition probabilities are misspecified due to ill-posed random effects structures, inadequate likelihood approximation or omission of the autocorrelation component. The results may be surprising in that these suggest that characterizing autocorrelation in ALVMs is not as important as specifying a suitably rich random effects structure.

Appendices

Appendix 1: Example NONMEM code

The bivariate normal subroutine is available at https://nonmem.iconplc.com/nonmem/bivariate/ along with some sample control streams. Below is an example control stream that corresponds to the simulation study.

Appendix 2: Transition probability bias calculation

The calculation of bias for the population transition probabilities (eliminating the subscript i for simplicity and indexing two arbitrary times by 1 and 2) requires the quantity $E_{\eta } \Phi_{2} \left( { {\mathop\mu\limits^{\prime}}_{1} , {\mathop\mu\limits^{\prime}}_{2} ,\nu_{12} \left( \rho \right)} \right)$, where ${\mathop\mu\limits^{\prime}}_{1} = \mu_{1} - g_{1} \eta$. From the latent variable formulation

$$E_{\eta } \varPhi_{2} = \int \limits_{ - \infty }^{\infty } \int \limits_{ - \infty }^{{\mu_{1} }} \int \limits_{ - \infty }^{{\mu_{2} }} \phi_{2} \left( {z_{1} ,z_{2} ,\nu_{12} \left( \rho \right)} \right)p\left( \eta \right)dz_{2} dz_{1} d\eta = \int \limits_{ - \infty }^{\infty } \varPhi_{2} \left( {{\mathop\mu\limits^{\prime}}_{1} , {\mathop\mu\limits^{\prime}}_{2} ,\nu_{12} \left( \rho \right)} \right)p\left( \eta \right)d\eta$$

where $\phi_{2} \left( {z_{1} ,z_{2} ,\nu_{12} \left( \rho \right)} \right) = \frac{1}{{2\pi \sqrt {1 - \nu_{12} \left( \rho \right)^{2} } }}exp\left\{ { - \frac{{\left( {z_{1}^{2} - 2\nu_{12} \left( \rho \right)z_{1} z_{2} + z_{2}^{2} } \right)}}{{2\left( {1 - \nu_{12} \left( \rho \right)^{2} } \right)}}} \right\}$. Making a suitable change of variables based on the square root of the diagonal elements of $\Xi$ (based on times 1 and 2 only),

$$E_{\eta } \Phi_{2} = \Phi_{2} \left( {\mu_{1} ,\mu_{2} ,\kappa_{12} \left( {\rho ,\Omega } \right)} \right)$$

where $\kappa_{12} \left( \rho \right)$ is the off-diagonal from $\left[ {diag\left( \Xi \right)} \right]^{ - 1/2} \Xi \left[ {diag\left( \Xi \right)} \right]^{ - 1/2}$, which for the example yields $\kappa_{12} \left( \rho \right) = \left( {\nu_{12} \left( \rho \right) + \Omega_{11} + \Omega_{22} U_{1} U_{2} } \right)V_{1}^{ - 1/2} V_{2}^{ - 1/2}$. Monte Carlo methods could also be used: $p_{ij}^{{\left( {1|0} \right)}} \cong \left[ {p_{{ij^{'} }}^{\left( 0 \right)} } \right]^{ - 1} \left[ {\frac{1}{M}\mathop \sum \limits_{m = 1}^{M} p\left( {\eta^{*} } \right)_{{ijj^{'} }}^{{\left( {1,0} \right)}} } \right]$ and $p_{ij}^{{\left( {0|1} \right)}} \cong \left[ {p_{{ij^{'} }}^{\left( 1 \right)} } \right]^{ - 1} \left[ {\frac{1}{M}\mathop \sum \limits_{m = 1}^{M} p\left( {\eta^{*} } \right)_{{ijj^{'} }}^{{\left( {0,1} \right)*}} } \right]$, $j^{'} < j$, where the ‘*’ indicates the probability is a function of $\eta_{m}^{*}$ which is sampled from $N\left( {0,\Omega } \right)$, with $\Omega$ estimated, using a sufficiently large M.

Appendix 3: Delta method for general prediction

Let $\psi = \left( {\beta ,\Omega } \right)$ be the vector of parameters and $\hat{\psi } = \left( {\hat{\beta },\hat{\Omega }} \right)$ correspond to their estimates, which have an estimated covariance matrix (e.g., COV step) $\widehat{}\left( {\hat{\psi }} \right) = \hat{\Delta }$. Then the variance of the prediction can be calculated using

$$\widehat{Var}\left[ {\Phi^{ - 1} \left( {p_{ij}^{\left( 1 \right)} } \right)} \right] \approx \left. {\frac{{\partial \left[ {\Phi^{ - 1} \left( {p_{ij}^{\left( 1 \right)} } \right)} \right]}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}} \hat{\Delta }\left. {\frac{{\partial \left[ {\Phi^{ - 1} \left( {p_{ij}^{\left( 1 \right)} } \right)} \right]}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}}^{T}$$

such that the confidence limit (e.g., at 5th percentile) is

$$CL_{0.05} = \Phi \left[ {\Phi^{ - 1} \left( {p_{ij}^{\left( 1 \right)} } \right) + Z_{0.05} \cdot \sqrt {\widehat{Var}\left[ {\Phi^{ - 1} \left( {p_{ij}^{\left( 1 \right)} } \right)} \right]} } \right]$$

where $Z_{0.05}$ is the quantile corresponding to probability level 0.05. To apply this method when a closed form solution to the population mean is not available, suitable regulatory conditions are required. In general, let $E\left( {\left. Y \right|\eta } \right) = \mu \left( \eta \right)$ and $E\left( {Y;\psi } \right) = E_{\eta } \left( {\mu \left( \eta \right);\psi } \right)$ where inference on $E\left( Y \right)$ is desired. Applying the delta-method

$$\widehat{Var}\left[ {E\left( {Y;\psi } \right)} \right] \approx \left. {\frac{{\partial \left[ {E\left( {Y;\psi } \right)} \right]}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}} \hat{\Delta }\left. {\frac{{\partial \left[ {E\left( {Y;\psi } \right)} \right]}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}}^{T}$$

and passing the differentiation through the integration

$$\left. {\frac{{\partial \left[ {E\left( {Y;\psi } \right)} \right]}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}} = \left. {\frac{{\partial \left[ {E_{\eta } \left( {\mu \left( \eta \right);\psi } \right)} \right]}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}} = \left. {\frac{{E_{\eta } \left[ {\partial \left( {\mu \left( \eta \right);\psi } \right)} \right]}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}} \cong \mathop \sum \limits_{m = 1}^{M} \left. {\frac{{\partial \left( {\mu \left( {\eta_{m}^{*} } \right);\psi } \right)}}{\partial \psi }} \right|_{{\psi = \hat{\psi }}}$$

where the model needs to be parameterized such that $\eta_{{}}^{*} \sim MVN\left( {0,1} \right)$. Finite differences could be used, for example $\frac{{\partial \left( {\mu \left( {\eta_{m}^{*} } \right);\psi } \right)}}{\partial \psi } = \left[ {\left( {\mu \left( {\eta_{m}^{*} } \right);\psi + \delta } \right) - \left( {\mu \left( {\eta_{m}^{*} } \right);\psi } \right)} \right]/\delta$.