$L_1$ splitting rules in survival forests

Abstract

The log-rank test is used as the split function in many commonly used survival trees and forests algorithms. However, the log-rank test may have a significant loss of power in some circumstances, especially when the hazard functions or when the survival functions cross each other in the two compared groups. We investigate the use of the integrated absolute difference between the two children nodes survival functions as the splitting rule. Simulations studies and applications to real data sets show that forests built with this rule produce very good results in general, and that they are often better compared to forests built with the log-rank splitting rule.

Appendix

Hazard function formula for DGP 1

$$\begin{aligned} {\left\{ \begin{array}{ll} 0.27t &{} x_1\le 0.5, t\le 2 \\ 0.2(t-2)+5.4 &{} x_1\le 0.5, t>2 \\ 0.1t &{} x_1>0.5, t\le 6 \\ 5.5(t-6)+0.6 &{} x_1>0.5, t>6. \\ \end{array}\right. } \end{aligned}$$

Hazard function formula for DGP 2

$$\begin{aligned} {\left\{ \begin{array}{ll} 0.27t &{} x_1\le 0.5, x_2\le 0.5, t\le 2 \\ 0.2(t-2)+5.4 &{} x_1\le 0.5, x_2\le 0.5, t>2 \\ 0.27t &{} x_1\le 0.5, x_2>0.5, t\le 2 \\ 5.5(t-2)+5.4 &{} x_1\le 0.5, x_2>0.5, t>2 \\ 0.1t &{} x_1>0.5, x_2\le 0.5, t\le 6 \\ 0.2(t-6)+0.6 &{} x_1>0.5, x_2\le 0.5, t>6 \\ 0.1t &{} x_1>0.5,x_2>0.5, t\le 6 \\ 5.5(t-6)+0.6 &{} x_1>0.5, x_2>0.5, t>6. \\ \end{array}\right. } \end{aligned}$$

Simplification of the Lin and Xu (2010) statistic leading to the $L_1^*$ splitting rule

For $i=L,R$, the left and right nodes, denote by $\hat{\sigma }_i^2$ the estimated variance of $\hat{S}_i$ from Greenwood’s formula. To perform a formal test of the equality of the survival functions in the left and right nodes, Lin and Xu (2010) propose the statistic

$$\begin{aligned} \Delta ^* = \frac{\Delta -\hat{E}(\Delta )}{\sqrt{\widehat{Var}(\Delta )}} \end{aligned}$$

where

$$\begin{aligned} \hat{E}(\Delta ) = \sum _{j|t_j<\tau } \{2/\pi (\hat{\sigma }_L^2(t_j) + \hat{\sigma }_R^2(t_j))\}^{1/2} (t_{j+1}-t_j) \end{aligned}$$

and

$$\begin{aligned} \widehat{Var}(\Delta )= & {} \sum _{j|t_j<\tau } (t_{j+1}-t_j)^2 (1-2/\pi ) (\hat{\sigma }_L^2(t_j) + \hat{\sigma }_R^2(t_j)) \\&+ \sum _{j<j'|t_j,t_{j'}<\tau } (t_{j+1}-t_j) (t_{j'+1}-t_{j'}) (1-2/\pi ) \\&\times \{(\hat{\sigma }_L^2(t_j) + \hat{\sigma }_R^2(t_j))(\hat{\sigma }_L^2(t_{j'}) + \hat{\sigma }_R^2(t_{j'}))\}^{1/2} \end{aligned}$$

are estimates of $E(\Delta )$ and $Var(\Delta )$. These estimates arise from a normal approximation for $\hat{S}_L(t)-\hat{S}_R(t)$, and the test statistic $\Delta ^*$ is asymptotically normally distributed under the null hypothesis of equality of the two survival functions. To simplify this statistic in order to speed up computations for tree building, assume that all observations are from the same population with survival function S(t), that is we are under the null hypothesis and there is no censoring. Then $Var(\hat{S}_i(t))=S(t)(1-S(t))/n_i$, for $i=L,R$. In that case,

$$\begin{aligned} \hat{E}(\Delta )= & {} \sqrt{2/\pi } \sqrt{(n_L+n_R)/(n_L n_R)} \sum _{j|t_j<\tau } (S(t_j)(1-S(t_j)))^{1/2} (t_{j+1}-t_j)\\= & {} c_1/\sqrt{n_L n_R} \end{aligned}$$

where $c_1$ is the same constant for all candidate splits. Similarly, $\widehat{Var}(\Delta )=c^2_2/(n_L n_R)$ where $c_2$ is the same constant for all candidate splits. Hence,

$$\begin{aligned} \frac{\Delta -\hat{E}(\Delta )}{\sqrt{\widehat{Var}(\Delta )}} = \frac{\sqrt{n_L n_R} \Delta }{c_2} - \frac{c_1}{c_2}. \end{aligned}$$

But using this last expression is equivalent to using $\sqrt{n_L n_R} \Delta$ as the splitting criterion.