Frailty modelling approaches for semi-competing risks data

Abstract

In the semi-competing risks situation where only a terminal event censors a non-terminal event, observed event times can be correlated. Recently, frailty models with an arbitrary baseline hazard have been studied for the analysis of such semi-competing risks data. However, their maximum likelihood estimator can be substantially biased in the finite samples. In this paper, we propose effective modifications to reduce such bias using the hierarchical likelihood. We also investigate the relationship between marginal and hierarchical likelihood approaches. Simulation results are provided to validate performance of the proposed method. The proposed method is illustrated through analysis of semi-competing risks data from a breast cancer study.

Appendix

1.1 Appendix A: Marginal-likelihood estimation procedure

For gamma frailty models with $E(u_{i})=1$ and var$(u_{i})=\alpha$, we have an explicit marginal likelihood as follows. Since the second term of h-likelihood in (8) under the gamma frailty is given by

$$\begin{aligned} \ell _{2i}=\ell _{2i}(\alpha ; v_{i})=\alpha ^{-1}(v_i-u_i) +c(\alpha ), \end{aligned}$$

with $c(\alpha )=-\log \Gamma (\alpha ^{-1}) -\alpha ^{-1} \log \alpha$, from (8) and (15) we have that

$$\begin{aligned} m= & {} \sum _{i} [\delta _{i1} \{\log \lambda _{01}(y_{i1}) + x_{i}^T\beta _1 \} + \delta _{i2} (1-\delta _{i1}) \{\log \lambda _{02}(y_{i2})+ x_i^T\beta _2 \}\nonumber \\&+~ \delta _{i1} \delta _{i2} \{\log \lambda _{03}(y_{i2}) + x_i^T \beta _3\}]\nonumber \\&-\sum _{i}[(\alpha ^{-1}+~\delta _{i+})\log (1+\alpha \mu _{i+}) -\log \{\alpha ^{\delta _{i+}} \Gamma (\alpha ^{-1}+\delta _{i+} )/\Gamma (\alpha ^{-1})\} ]\nonumber \\= & {} \sum _{k_1}d_{1(k_1)}\log \lambda _{01k_1}+\sum _{i} \delta _{i1}(x_i^T\beta _1) + \sum _{k_2}d_{2(k_2)}\log \lambda _{02k_2}\nonumber \\&+\sum _{i} \delta _{i2}(1-\delta _{i1})(x_i^T\beta _2) + \sum _{k_3}d_{3(k_3)}\log \lambda _{03k_3}+\sum _{i} \delta _{i1}\delta _{i2}(x_i^T\beta _3)\nonumber \\&-\sum _{i}[(\alpha ^{-1}+~\delta _{i+})\log (1+\alpha \mu _{i+}) -\delta _{i1}\delta _{i2} \log (1+\alpha )], \end{aligned}$$

(A.1)

where $\delta _{i+}= \delta _{i1} + \delta _{i2}$ and $\mu _{i+}=\sum _{j=1}^{3} \mu _{ij}$ with

$$\begin{aligned} \mu _{i1}= & {} \Lambda _{01}(y_{i1})\exp (x_{i}^T \beta _1)=\sum _{k_1} \lambda _{01k_1} I(y_{1(k_{1})} \le y_{i1}) \exp (x_i^T \beta _1),\\ \mu _{i2}= & {} \Lambda _{02}(y_{i2})\exp (x_{i}^T \beta _2)=\sum _{k_2} \lambda _{02k_2} I(y_{2(k_{2})} \le y_{i1}) \exp (x_i^T \beta _2),\\ \mu _{i3}= & {} \Lambda _{03}(y_{i1}, y_{i2})\exp (x_{i}^T \beta _3) =\sum _{k_3} \lambda _{03k_3} I(y_{i1} < y_{3(k_3)}\le y_{i2} ) \exp (x_i^T \beta _3). \end{aligned}$$

In fact, the marginal likelihood (A.1) is the same as that of Xu et al. (2010).

Under the gamma frailty, the score equations for $\beta$ are given by

$$\begin{aligned} \frac{\partial m}{\partial \beta _1}= & {} \sum _i \biggl \{ \delta _{i1}- \biggl (\frac{\alpha ^{-1} + \delta _{i+}}{\alpha ^{-1} + \mu _{i+}} \biggl ) \mu _{i1} \biggl \} x_i, \end{aligned}$$

(A.2)

$$\begin{aligned} \frac{\partial m}{\partial \beta _2}= & {} \sum _i \biggl \{ \delta _{i2}(1-\delta _{i1})- \biggl (\frac{\alpha ^{-1} + \delta _{i+}}{\alpha ^{-1} + \mu _{i+}} \biggl ) \mu _{i2} \biggl \} x_i, \end{aligned}$$

(A.3)

$$\begin{aligned} \frac{\partial m}{\partial \beta _3}= & {} \sum _i \biggl \{ \delta _{i1}\delta _{i2}- \biggl (\frac{\alpha ^{-1} + \delta _{i+}}{\alpha ^{-1} + \mu _{i+}} \biggl ) \mu _{i3} \biggl \} x_i. \end{aligned}$$

(A.4)

In particular, the solutions of $\partial m/\partial \lambda _{0jk_j}=0~(j=1,2,3)$ lead to closed forms:

$$\begin{aligned} {\widetilde{\lambda }}_{0jk_j}(\beta ,\alpha )=\frac{d_{j(k_j)}}{\sum _{~i \in R_{(k_j)}}\exp (x_i^T \beta _j){\tilde{u}}_i}, \end{aligned}$$

(A.5)

where ${\tilde{u}}_i=(\alpha ^{-1}+\delta _{i+})/(\alpha ^{-1}+\mu _{i+})$. We see that the score equations of $(\beta , \lambda _{0j})$ in (A.2)–(A.4) and (A.5) are extensions of those in the shared gamma frailty models (Andersen et al. 1997). Finally, the score equation for the frailty parameter $\alpha$ is given by

$$\begin{aligned} \frac{\partial m}{\partial \alpha }=\sum _i \biggl \{ \delta _{i1}\delta _{i2} (1+\alpha )^{-1} +\alpha ^{-2}\log (1+\alpha \mu _{i+}) -(\alpha ^{-1} +\delta _{i+})\mu _{i+} (1+\alpha \mu _{i+})^{-1} \biggl \}. \end{aligned}$$

Then the estimates of fixed parameters $(\beta ,\alpha , \lambda _{0})$ can be obtained using a numerical iterative method such as the Newton-Raphson method. Note that the maximum likelihood estimating equations, $\partial m/\partial (\beta ,\alpha , \lambda _{0})=0$, by Xu et al. (2010) are equivalent to $\partial m^{*}/\partial (\beta ,\alpha )=0$, where $m^{*}$ is the profile marginal likelihood in (16).

1.2 Appendix B: Comparison of h-likelihood with marginal likelihood

We assume that $\alpha$ is known. Recall that given $(\beta , v)$, the score equations $\partial h/\partial \lambda _{0jk_j}=0~(j=1,2,3)$ provide the non-parametric maximum h-likelihood estimators in Sect. 2.2, i.e.

$$\begin{aligned} \widehat{\lambda }_{0jk_j}(\beta ,v)=\frac{d_{j(k_j)}}{\sum _{~i \in R_{(k_j)}}\exp (x_i^T \beta _j)u_i}. \end{aligned}$$

The maximum h-likelihood estimating equations for $\beta$, under the gamma frailty, become

$$\begin{aligned} \frac{\partial h}{\partial \beta _1} \left| _{{\lambda _{01}={\widehat{\lambda }}_{01}} }\frac{}{} \right.= & {} \sum _i \biggl \{ \delta _{i1}- \mu _{i1}u_i \biggl \} x_i \left| _{{\lambda _{01}={\widehat{\lambda }}_{01}} }=0\frac{}{} \right. , \end{aligned}$$

(B.1)

$$\begin{aligned} \frac{\partial h}{\partial \beta _2}\left| _{{\lambda _{02}={\widehat{\lambda }}_{02}} }\frac{}{} \right.= & {} \sum _i \biggl \{ \delta _{i2}(1-\delta _{i1})-\mu _{i2} u_i \biggl \} x_i \left| _{{\lambda _{02}={\widehat{\lambda }}_{02}} }=0\frac{}{} \right. , \end{aligned}$$

(B.2)

$$\begin{aligned} \frac{\partial h}{\partial \beta _3}\left| _{{\lambda _{03}={\widehat{\lambda }}_{03}} }\frac{}{} \right.= & {} \sum _i \biggl \{ \delta _{i1}\delta _{i2}- \mu _{i3}u_i \biggl \} x_i \left| _{{\lambda _{03}={\widehat{\lambda }}_{03}} }=0\frac{}{} \right. . \end{aligned}$$

(B.3)

From

$$\begin{aligned} \frac{\partial h}{\partial v_i}= (\delta _{i+}-\mu _{i+}u_i) +\alpha ^{-1} -\alpha ^{-1}u_i=0, \end{aligned}$$

we have that

$$\begin{aligned} {\hat{u}}_i=\frac{\alpha ^{-1}+\delta _{i+}}{\alpha ^{-1}+\mu _{i+}}, \end{aligned}$$

(B.4)

which also becomes $E(u_i|y_i^o)$ because the conditional distribution of $u_i$ given the observed data $y_i^o=(y_{i1}, y_{i2}, \delta _{i1}, \delta _{i2})$ is gamma. Here $\delta _{i+}=\delta _{i1}+\delta _{i2}$ and $\mu _{i+}=\mu _{i1} +\mu _{i2}+\mu _{i3}$. From (12) we see that the estimating Eqs. (B.1)–(B.3) with (B.4) are equivalent to the estimating Eqs. (A.2)–(A.4) with (A.5), given by

$$\begin{aligned} \frac{\partial m}{\partial \beta _1} \left| _{{\lambda _{01}={{\widetilde{\lambda }}}_{01}} }\frac{}{} \right.= & {} \sum _i \biggl \{ \delta _{i1}- \mu _{i1} \biggl (\frac{\alpha ^{-1} + \delta _{i+}}{\alpha ^{-1} + \mu _{i+}} \biggl ) \biggl \} x_i,\left| _{{\lambda _{01}={{\widetilde{\lambda }}}_{01}} }=0\frac{}{} \right. \end{aligned}$$

(B.5)

$$\begin{aligned} \frac{\partial m}{\partial \beta _2} \left| _{{\lambda _{02}={{\widetilde{\lambda }}}_{02}} }\frac{}{} \right.= & {} \sum _i \biggl \{ \delta _{i2}(1-\delta _{i1})- \mu _{i2} \biggl (\frac{\alpha ^{-1} + \delta _{i+}}{\alpha ^{-1} + \mu _{i+}} \biggl ) \biggl \} x_i,\left| _{{\lambda _{02}={{\widetilde{\lambda }}}_{02}} }=0\frac{}{} \right. \quad \quad \end{aligned}$$

(B.6)

$$\begin{aligned} \frac{\partial m}{\partial \beta _3} \left| _{{\lambda _{03}={{\widetilde{\lambda }}}_{03}} }\frac{}{} \right.= & {} \sum _i \biggl \{ \delta _{i1}\delta _{i2}- \mu _{i3} \biggl (\frac{\alpha ^{-1} + \delta _{i+}}{\alpha ^{-1} + \mu _{i+}} \biggl ) \biggl \} x_i,\left| _{{\lambda _{03}={{\widetilde{\lambda }}}_{03}} }=0\frac{}{} \right. . \end{aligned}$$

(B.7)

Accordingly, the maximum h-likelihood (MHL) estimator for $\beta$ given $\alpha$ is the same as the marginal maximum likelihood (ML) estimator as shown in the standard gamma frailty models (Ha et al. 2001; Ha and Lee 2003). Note, however, that both methods give different estimators for $\alpha$.

The marginal likelihood does not often have an analytic form (e.g. log-normal frailty model), so that the natural approach to the maximum likelihood estimator (MLE) is to use the EM treating the random effects as missing data. Below we present the comparison of the proposed h-likelihood method with the EM method for obtaining the MLE. The EM equations for fixed parameters $\theta$ can be expressed via the h-likelihood as follows:

$$\begin{aligned} E(\partial h/\partial \theta \vert \mathrm{data})=0, \end{aligned}$$

which is equivalent to the ML equations, i.e. $\partial m/\partial \theta =0$ (Lee and Nelder 1996; Engel and Keen 1996; Ha et al. 2001).

In the semi-competing risks frailty models (5)–(7), the EM equations for $(\beta ,\alpha )$ are given by

$$\begin{aligned} E\left( \partial h/\partial (\beta ,\alpha ) \vert ~y_i^{o},{{\widetilde{\lambda }}}_{0jk_j}^{*} \right) =0. \end{aligned}$$

Here, the EM equations of the baseline hazards $\lambda _{0jk_j}$ are given by

$$\begin{aligned} E(\partial h/\partial \lambda _{0jk_j} |~y_i^{o} )=0, \end{aligned}$$

which lead to

$$\begin{aligned} {\widetilde{\lambda }}_{0jk_j}^{*}(\beta ,v)=\frac{d_{j(k_j)}}{\sum _{~i \in R_{(k_j)}}\exp (x_i^T \beta _j) E(u_{i} |y_i^{o}) }. \end{aligned}$$

Following Ha et al. (2001), in the gamma frailty models, given $\alpha$ the MHL equations for $\beta$

$$\begin{aligned} \partial h_{p}/\partial \beta =(\partial h/\partial \beta )|_{\lambda _0={\widehat{\lambda }}_0}=0{,} \end{aligned}$$

with (B.4) are equivalent to the EM equations

$$\begin{aligned} E(\partial h/\partial \beta \vert ~y_i^{o},{{\widetilde{\lambda }}}_{0jk_j}^{*} )=0{,} \end{aligned}$$

since $E(u_{i} |y_i^{o})$ becomes ${\tilde{u}}_{i}$ in (A.5) and thus ${{\widetilde{\lambda }}}_{0jk_j}^{*}$ is identical to ${\tilde{\lambda }}_{0jk_j}$ in (A.5) as well as to ${\widehat{\lambda }}_{0jk_j}$. However, in general the EM may be difficult to apply because the conditional distribution of $u_{i}$ given $y_i^{o}$ is not trivial to be evaluated. For example, in the log-normal frailty with $v_{i}=\log u_{i} \sim N(0, \alpha )$, the EM equation for $\beta _1$ is given by

$$\begin{aligned} E\left( \partial h/\partial \beta _{1} \vert ~y_i^{o},{{\widetilde{\lambda }}}_{0jk_j}^{*}\right) = \sum _i \biggl \{ \delta _{i1}- {{\widetilde{\mu }}}_{i1}^{*} E(u_i|y_{i}^{o}) \biggl \} x_i =0, \end{aligned}$$

where

$$\begin{aligned} {\widetilde{\mu }}_{i1}^{*}={\widetilde{\Lambda }}_{01}^{*}(y_{i1})\exp (x_{i}^T \beta _1)=\sum _{k_1} {\widetilde{\lambda }}_{01k_1}^{*} I(y_{1(k_{1})} \le y_{i1}) \exp (x_i^T \beta _1). \end{aligned}$$

Note here that the computation of $E(u_i|y_{i}^{o})$ requires a numerical integration.