Quantile regression for nonlinear mixed effects models: a likelihood based perspective

Abstract

Longitudinal data are frequently analyzed using normal mixed effects models. Moreover, the traditional estimation methods are based on mean regression, which leads to non-robust parameter estimation under non-normal error distribution. However, at least in principle, quantile regression (QR) is more robust in the presence of outliers/influential observations and misspecification of the error distributions when compared to the conventional mean regression approach. In this context, this paper develops a likelihood-based approach for estimating QR models with correlated continuous longitudinal data using the asymmetric Laplace distribution. Our approach relies on the stochastic approximation of the EM algorithm (SAEM algorithm), obtaining maximum likelihood estimates of the fixed effects and variance components in the case of nonlinear mixed effects (NLME) models. We evaluate the finite sample performance of the SAEM algorithm and asymptotic properties of the ML estimates through simulation experiments. Moreover, two real life datasets are used to illustrate our proposed method using the $\texttt {qrNLMM}$ package from $\texttt {R}$.

Appendix

1.1 A1: Specification of initial values

It is well known that a smart choice of the initial values for the ML estimates can assure a fast convergence of the algorithm to the global maxima solution. Without considering the random effect term, i.e., $\mathbf {b}_{i}=\mathbf {0}$, let $\mathbf{y}_{i}\sim \text {AL}({\varvec{\eta }}({\varvec{\beta }}_{ p},\mathbf {0}),\sigma ,p)$. Next, considering the ML estimates for ${\varvec{\beta }}_{ p}$ and $\sigma$ as defined in Yu and Zhang (2005) for this model, we follow the steps below for the QR-LME model implementation:

• 1. Compute an initial value $\widehat{{\varvec{\beta }}}_{ p}^{{\scriptscriptstyle (0)}}$ as

  $$\begin{aligned} \widehat{{\varvec{\beta }}}_{ p}^{{\scriptscriptstyle (0)}}=\textit{argmin}{\scriptscriptstyle \beta _{ p}\in \mathbb {R}^{{k}}}\sum ^n_{i=1}\rho _{p}({\mathbf{y}_{i}-{\varvec{\eta }}({\varvec{\beta }}_{ p},\mathbf {0})}). \end{aligned}$$

• 2. Using the initial value of $\widehat{{\varvec{\beta }}}_{ p}^{{\scriptscriptstyle (0)}}$ obtained above, compute $\widehat{\sigma }^{(0)}$ as

  $$\begin{aligned} \widehat{\sigma }^{(0)}=\frac{1}{n}\sum ^n_{i=1}\rho _{p}({\mathbf{y}_{i}-{\varvec{\eta }}({\varvec{\beta }}_{ p},\mathbf {0})}). \end{aligned}$$

• 3. Use a $q\times q$ identity matrix $\mathbf I _{{\scriptscriptstyle q\times q}}$ for the the initial value ${\varvec{\varPsi }}^{(0)}$.

1.2 A2: Computing the conditional expectations

Due to the independence between $u_{ij}\!\mid \! y_{ij},\mathbf {b}_{i}$ and $u_{ik}\!\mid \! y_{ik},\mathbf {b}_{i}$, for all $j,k\!=\!1,2,\ldots ,n_{i}$ and $j\ne k$, we can write $\mathbf {u}_{i}\mid \mathbf {y}_{i},\mathbf {b}_{i}=[\begin{array}{cccc} u_{i1}\mid y_{i1},\mathbf {b}_{i}&u_{i2}\mid y_{i2},\mathbf {b}_{i}&\cdots&u_{in_{i}}\mid y_{in_{i}},\mathbf {b}_{i}\end{array}]^{{\scriptscriptstyle {\top }}}$. Using this fact, we are able to compute the conditional expectations $\mathcal {E}(\mathbf {u}_{i})$ and $\mathcal {E}({\varvec{D}}_{i}^{{\scriptscriptstyle -1}})$ in the following way:

$$\begin{aligned} \mathcal {E}(\mathbf {u}_{i})= & {} \left[ \mathcal {E}(u_{i1})\;\mathcal {E}(u_{i1})\;\cdots \;\mathcal {E}(u_{in_{i}})\right] ^{{\scriptscriptstyle \top }}, \end{aligned}$$

(12)

$$\begin{aligned} \mathcal {E}({\varvec{D}}_{i}^{{\scriptscriptstyle -1}})= & {} \left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c}\mathcal {E}(u_{i1}^{-1}) &{} 0 &{} \ldots &{} 0\\ 0 &{} \mathcal {E}(u_{i2}^{-1}) &{} \ldots &{} 0\\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ 0 &{} 0 &{} \ldots &{} \mathcal {E}(u_{in_{i}}^{-1}) \end{array}\right] . \end{aligned}$$

(13)

We have $u_{ij}\mid y_{ij},\mathbf {b}_{i}\sim \text {GIG}(\frac{1}{2},\chi _{ij},\psi )$, where $\chi _{ij}$ and $\psi$ are defined in Sect. 3.2. Then, using (2), we compute the moments involved in the equations above as $\mathcal {E}({u}_{ij})=\frac{\chi _{ij}}{\psi }(1+\frac{1}{\chi _{ij}\psi })$ and $\mathcal {E}({u}_{ij}^{-1})=\frac{\psi }{\chi _{ij}}$. Thus, for the k-th iteration of the algorithm and for the $\ell$-th Monte Carlo realization, we can compute $\mathcal {E}(\mathbf {u}_{i})^{{\scriptscriptstyle (\ell ,k)}}$ and $\mathcal {E}[{\varvec{D}}_{i}^{{\scriptscriptstyle -1}}]^{{\scriptscriptstyle (\ell ,k)}}$ using equations (12)-(13) where

$$\begin{aligned} \mathcal {E}({u}_{ij})^{{\scriptscriptstyle (\ell ,k)}}= & {} \frac{2|y_{ij}-\eta _{ij}({\varvec{\beta }}_{ p}^{{\scriptscriptstyle (k)}},\mathbf {b}_{i}^{{\scriptscriptstyle (\ell ,k)}})|+4\sigma ^{({\scriptscriptstyle k)}}}{{\displaystyle \tau _{p}^{2}}},\\ \mathcal {E}({u}_{ij}^{-1})^{{\scriptscriptstyle (\ell ,k)}}= & {} \frac{{\displaystyle \tau _{p}^{2}}}{2|y_{ij}-\eta _{ij}({\varvec{\beta }}_{ p}^{{\scriptscriptstyle (k)}},\mathbf {b}_{i}^{{\scriptscriptstyle (\ell ,k)}})|}. \end{aligned}$$

1.3 A3: The empirical information matrix

Using (5), the complete log-likelihood function can be rewritten as

$$\begin{aligned} \ell _{ci}({\varvec{\theta }})= & {} -\frac{3}{2}n_{i}\text{ log }\sigma -\frac{1}{2\sigma \tau _{p}^{2}}\zeta _{i}^{\top }{\varvec{D}}_{i}^{-1}\zeta _{i}-\frac{1}{2}\text{ log }\bigl |{\varvec{\varPsi }}\bigr |-\frac{1}{2}\mathbf {b}_{i}^{\top }{\varvec{\varPsi }}^{-1}\mathbf {b}_{i}-\frac{1}{\sigma }\mathbf {u}_{i}^{\top }\mathbf {1}{}_{n_{i}}, \end{aligned}$$

where $\zeta _{i}=\mathbf {y}_{i}-{\varvec{\eta }}({\varvec{\beta }}_{ p},\mathbf {b}_{i})-\vartheta _{p}\mathbf {u}_{i}$ and ${\varvec{\theta }}=({\varvec{\beta }}_{{\scriptscriptstyle p}}^{{\scriptscriptstyle \top }},\sigma ,\mathbf {{\varvec{\alpha }}^{{\scriptscriptstyle \top }}})^{{\scriptscriptstyle \top }}$. Differentiating with respect to ${\varvec{\theta }}$, we have the following score functions:

$$\begin{aligned} \frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial {\varvec{\beta }}_{{\scriptscriptstyle p}}}= & {} \frac{\partial {\varvec{\eta }}}{\partial {\varvec{\beta }}_{{\scriptscriptstyle p}}}\frac{\partial \zeta _{i}}{\partial {\varvec{\eta }}}\frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial \zeta _{i}}=\frac{1}{\sigma \tau _{p}^{2}}\mathbf {J_{i}^{\top }}{\varvec{D}}_{i}^{-1}\zeta _{i}, \end{aligned}$$

with $\mathbf {J_{i}}$ as defined in Sect. 3.2. and

$$\begin{aligned} \frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial \sigma }= & {} -\frac{3n_{i}}{2}\frac{1}{\sigma }+\frac{1}{2\sigma ^{2}\tau _{p}^{2}}\zeta _{i}^{\top }{\varvec{D}}_{i}^{-1}\zeta _{i} + \frac{1}{\sigma ^{2}}\mathbf {u}_{i}^{\top }\mathbf {1}{}_{n_{i}}. \end{aligned}$$

Let ${\varvec{\alpha }}$ be the vector of reduced parameters from ${\varvec{\varPsi }}$, the dispersion matrix for $\mathbf {b}_i$. Using the trace properties and differentiating the complete log-likelihood function, we have that

$$\begin{aligned} \frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial {\varvec{\varPsi }}}= & {} \frac{\partial }{\partial {\varvec{\varPsi }}}\left[ -\frac{n}{2} log\bigl |{\varvec{\varPsi }}\bigr |-\frac{1}{2}\text {tr}\{{\varvec{\varPsi }}^{-1}\mathbf {b}_{i}\mathbf {b}_{i}^{\top }\}\right] \\= & {} -\frac{1}{2}\text {tr}\{{\varvec{\varPsi }}^{-1}\}+\frac{1}{2}\text {tr}\{{\varvec{\varPsi }}^{-1}{\varvec{\varPsi }}^{-1}\mathbf {b}_{i}\mathbf {b}_{i}^{\top }\}\\= & {} \;\;\;\, \frac{1}{2}\text {tr}\{{\varvec{\varPsi }}^{-1}(\mathbf {b}_{i}\mathbf {b}_{i}^{\top }-{\varvec{\varPsi }}){\varvec{\varPsi }}^{-1}\}. \end{aligned}$$

Next, taking derivatives with respect to a specific $\alpha _{j}$ from ${\varvec{\alpha }}$ based on the chain rule, we have

$$\begin{aligned} \frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial \alpha _{j}}= & {} \frac{\partial {\varvec{\varPsi }}}{\partial \alpha _{j}}\frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial {\varvec{\varPsi }}}\\= & {} \frac{\partial {\varvec{\varPsi }}}{\partial \alpha _{j}}\frac{1}{2}\text {tr}\{{\varvec{\varPsi }}^{-1}(\mathbf {b}_{i}\mathbf {b}_{i}^{\top }-{\varvec{\varPsi }}){\varvec{\varPsi }}^{-1}\}. \end{aligned}$$

where, using the fact that $\text {tr}\{\mathbf{ABCD }\}=(\mathrm {vec}(\mathbf{A }^{\top }))^{\top }$$({\varvec{D}}^{\top }\otimes \mathbf{B })(\mathrm {vec}(\mathbf{C }))$, (14) can be rewritten as

$$\begin{aligned} \frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial \alpha _{j}}=(\mathrm {vec}(\tfrac{\partial {\varvec{\varPsi }}}{\partial \alpha _{j}}^{\top }))^{\top }\frac{1}{2}({\varvec{\varPsi }}^{-1}\otimes {\varvec{\varPsi }}^{-1})(\mathrm {vec}(\mathbf {b}_{i}\mathbf {b}_{i}^{\top }-{\varvec{\varPsi }})). \end{aligned}$$

(14)

Let $\mathcal {D}_{q}$ be the elimination matrix (Lavielle 2014), which transforms the vectorized ${\varvec{\varPsi }}$ (written as $\text{ vec }({\varvec{\varPsi }})$) into its half-vectorized form vech(${\varvec{\varPsi }}$), so that $\mathcal {D}_{q}\mathrm {vec}({\varvec{\varPsi }})$$=\mathrm {vech}({\varvec{\varPsi }}).$ Using the fact that for all $j=1,\ldots ,\frac{1}{2}q(q+1)$, the vector $(\mathrm {vec}(\frac{\partial {\varvec{\varPsi }}}{\partial \alpha _{j}})^{\top })^{\top }$ corresponds to the j-th row of the elimination matrix $\mathcal {D}_{q}$, we can generalize the derivative in (14) for the vector of parameters ${\varvec{\alpha }}$ as

$$\begin{aligned} \frac{\partial \ell _{ci}({\varvec{\theta }})}{\partial {\varvec{\alpha }}}= & {} \frac{1}{2}\mathcal {D}_{q}({\varvec{\varPsi }}^{-1}\otimes {\varvec{\varPsi }}^{-1})(\mathrm {vec}(\mathbf {b}_{i}\mathbf {b}_{i}^{\top }-{\varvec{\varPsi }})). \end{aligned}$$

Finally, at each iteration, we can compute the empirical information matrix by approximating the score for the observed log-likelihood by the stochastic approximation given in (9).

1.4 A4: Figures

Fig. 8

Soybean data: Point estimates (center solid line) and 95% confidence intervals for model parameters after fitting the QR-NLME model. The interpolated curves are spline-smoothed

Fig. 9

ACTG 315 data: Point estimates (center solid line) and 95% confidence intervals for model parameters after fitting the QR-NLME model. The interpolated curves are spline-smoothed

Fig. 10

Graphical summary for the convergence of the fixed effect estimates, variance components of the random effects, and nuisance parameters performing a median regression $(p=0.50)$ for the soybean data. The vertical dashed line delimits the beginning of the almost sure convergence as defined by the cut-point parameter $c=0.25$

1.5 A5: Sample output from R package qrNLMM()