Penalised logistic regression and dynamic prediction for discrete-time recurrent event data

Abstract

We consider methods for the analysis of discrete-time recurrent event data, when interest is mainly in prediction. The Aalen additive model provides an extremely simple and effective method for the determination of covariate effects for this type of data, especially in the presence of time-varying effects and time varying covariates, including dynamic summaries of prior event history. The method is weakened for predictive purposes by the presence of negative estimates. The obvious alternative of a standard logistic regression analysis at each time point can have problems of stability when event frequency is low and maximum likelihood estimation is used. The Firth penalised likelihood approach is stable but in removing bias in regression coefficients it introduces bias into predicted event probabilities. We propose an alterative modified penalised likelihood, intermediate between Firth and no penalty, as a pragmatic compromise between stability and bias. Illustration on two data sets is provided.

Appendix: logistic regression with separation not detected in R

In the following, y is a vector of length 100, with all elements zero except the first, which is one, and x1 is a vector of 50 zeros followed by 50 ones, representing two equally sized groups. If we attempt to fit the logistic regression

$$\begin{aligned} \pi _x = P\big (Y=1 \big | x\big ) = \mathrm{expit} \big (\beta _0+\beta _1x\big ) \end{aligned}$$

then clearly a perfect fit is obtained at $\hat{\beta }_0=\mathrm{logit}(1/50)=-3.892$ and $\hat{\beta }_1=-\infty$. Some R (version 3.1.2) output, edited to remove unnecessary material (marked by [...], is:

Of most concern is the statement of convergence, which is true because the maximised likelihood has indeed converged: moving either of the coefficients away from their current values leads to no improvement. The fitted probabilities $\hat{\pi }_0$ and $\hat{\pi }_1$ are accurate but clearly $\hat{\beta }_1$ is unrealistic. Uncritical assessment of the results might lead to this problem being missed.

If we use the Firth correction as implemented in Kosmidis’ bias reduction package brglm, we obtain:

Hence the coefficients are stabilised, at the expense of higher values of $\hat{\pi }_0$ and $\hat{\pi }_1$ as expected. Heinze’ package logistf gives the same results.