Penalized Cox’s proportional hazards model for high-dimensional survival data with grouped predictors

Abstract

The rapid development of next-generation sequencing technologies has made it possible to measure the expression profiles of thousands of genes simultaneously. Often, there exist group structures among genes manifesting biological pathways and functional relationships. Analyzing such high-dimensional and structural datasets can be computationally expensive and results in the complicated models that are hard to interpret. To address this, variable selection such as penalized methods are often taken. Here, we focus on the Cox’s proportional hazards model to deal with censoring data. Most of the existing penalized methods for Cox’s model are the group lasso methods that show deficiencies, including the over-shrinkage problem. In addition, the contemporary algorithms either exhibit the loss of efficiency or require the group-wise orthonormality assumption. Hence, efficient algorithms for general design matrices are needed to enable practical applications. In this paper, we investigate and comprehensively evaluate three group penalized methods for Cox’s model: the group lasso and two nonconvex penalization methods—group SCAD and group MCP—that have several advantages over the group lasso. These methods are able to perform group selection in both non-overlapping and overlapping cases. We have developed the fast and stable algorithms and a new package grpCox to fit these models without the initial orthonormalization step. The runtime of grpCox is improved significantly over the existing packages, such as grpsurv (for the non-overlapping case), grpregOverlap (overlapping), and SGL. In addition, grpCox is better than grpsurv and comparable with SGL in terms of variable selection performances. Comprehensive studies on both simulation and real-world cancer datasets demonstrate the statistical properties of our grpCox implementations with the group lasso, SCAD, and MCP regularization terms.

Appendices

Appendix 1

We have studied the statistical properties of the estimators: consistency and convergence rate as follows.

The partial likelihood

$$\begin{aligned} \ell _n(\beta ) = -\frac{1}{n} \sum _{i=1}^{D} \Bigg [ \big (\sum _{j=1}^J X_{j}^{(i)}\mathbf {\beta }_j \big ) - \text {log} \bigg ( \sum _{l \in R_i} \text {exp}\big (\sum _{j=1}^J X_{j}^{(l)}\mathbf {\beta }_j \big ) \bigg ) \Bigg ], \end{aligned}$$

where the penalty term $P_{\lambda , \gamma }(\beta )$ can be denoted as $P_{\lambda _n}(\beta )$ since $\gamma$ for group SCAD and group MCP are fixed. Here, $\ell _n(\beta ), \lambda _n$ denote the partial likelihood and tuning parameter changing with the sample size n, respectively.

Let the true parameter be $\beta _0 = \big ( \beta _{01}^T, \beta _{02}^T \big )^T$ where $\beta _{01}$ consists of all nonzero groups and $\beta _{02}$ consists of all remaining zero groups. The objective function is

$$\begin{aligned} {\mathcal {Q}}_n(\beta , \lambda _n)&= \ell _n(\beta _0) + \ell _n^{'}(\beta _0)^T (\beta -\beta _0) \\&\quad + \frac{\tau }{2}(\beta -\beta _0)^T(\beta -\beta _0) + P_{\lambda _n}(\beta ). \end{aligned}$$

Correspondingly, the minimizer of ${\mathcal {Q}}_n(\beta , \lambda _n)$ is $\beta _n = \big ( \beta _{n1}^T, \beta _{n2}^T \big )^T$ where ${\beta _n} = \underset{\beta }{\text {argmin }}{\mathcal {Q}}_n(\beta , \lambda _n)$.

Define $a_n = \text {max} \{ P^{'}_{\lambda _n}(\Vert \beta _{j0}\Vert ): \Vert \beta _{j0}\Vert \ne 0 \}$ and $b_n = \text {max} \{ P^{''}_{\lambda _n}(\Vert \beta _{j0}\Vert ): \Vert \beta _{j0}\Vert \ne 0 \}$.

Theorem 1

(Consistency and convergence rate) If $P_{\lambda _n}(\Vert \beta \Vert )$ simultaneously satisfies two conditions: $a_n = O_p(n^{-1/2})$ and $b_n \rightarrow 0$, then $\beta _n$ is a root-n consistent estimator for $\beta _0$ with rate $n^{-1/2}$, i.e. $\Vert \beta _n - \beta _0\Vert = O_p(n^{-1/2})$.

Table 11 Results for group lasso using different cross-validation methods to select hyperparameters over 100 replications

Table 12 Results for group SCAD using different cross-validation methods to select hyperparameters over 100 replications

Table 13 Results for group MCP using different cross-validation methods to select hyperparameters over 100 replications

Table 14 Results for misspecified group structures over 100 replications

Proof

According to Theorem 3.2 in Andersen and Gill (1982) two results hold

$$\begin{aligned}&-\ell ^{'}(\beta _0) \overset{p}{\rightarrow } n^{-1/2} {\mathcal {N}}(0,\Sigma ) \\&\ell ^{\prime \prime }(\beta ^*) \overset{p}{\rightarrow } n\Sigma \text{ for } \text{ any } \text{ random } \beta ^* \overset{p}{\rightarrow } \beta _0 \\ \text{ Then, }&\ell ^{\prime \prime }(\beta ^*) = n(\Sigma + O_p(1)), \end{aligned}$$

where $\Sigma$ is the positive definite Fisher information matrix.

Consider a constant ball, $B(C) = \{ \beta _0 + \alpha _n\mathbf{u }: \Vert \mathbf{u }\Vert \le C \}$ and its boundary $\partial B(C)$ where $C>0$ and $\alpha _n = n^{-1/2} + a_n$. Therefore, $O_p(\alpha _n) = O_p(a_n) = O_p(n^{-1/2})$. To prove $\Vert \beta _n - \beta _0\Vert = O_p(n^{-1/2})$, it is sufficient to prove that for any $\epsilon >0$, there exists a large constant C such that

$$\begin{aligned} \text {P}\bigg ( \underset{\beta \in \partial B(C)}{\text {sup}} Q_n(\beta , \lambda _n) < Q(\beta _0, \lambda _n) \bigg ) \ge 1-\epsilon . \end{aligned}$$

(15)

This implies that with probability at least $1-\epsilon$ (or goes to 1), $Q_n(\beta , \lambda _n)$ has a local minimum in the ball B(C) for a given $\lambda _n$.

Denote $D_n(\mathbf{u }) = Q_n(\beta , \lambda _n) - Q(\beta _0, \lambda _n)$, we have

$$\begin{aligned} D_n(\mathbf{u })&= \ell ^{'}(\beta _0)^T (\beta -\beta _0) + \frac{\tau }{2}(\beta -\beta _0)^T(\beta -\beta _0) \\&\quad + P_{\lambda _n}(\beta ) - P_{\lambda _n}(\beta _0) = D_1 + D_2. \end{aligned}$$

Consider that

$$\begin{aligned} D_1&= \ell ^{'}(\beta _0)^T (\beta -\beta _0) + \frac{\tau }{2}(\beta -\beta _0)^T(\beta -\beta _0) \\&= O_p(n^{-1/2})\alpha _n\mathbf{u } + \frac{\tau }{2} \alpha _n^2 \mathbf{u }^T\mathbf{u } \\&= O_p(C\alpha _n^2) + O_p(C^2\alpha _n^2). \end{aligned}$$

Consider $D_2$ using Taylor expansion, we have

$$\begin{aligned} D_2&= P_{\lambda _n}(\beta ) - P_{\lambda _n}(\beta _0) \\&= \sum _j P^{'}_{\lambda _n}(\Vert \beta _{j0}\Vert )(\Vert \beta _{j0}+\alpha _n\mathbf{u }_j\Vert -\Vert \beta _{j0}\Vert ) + \frac{1}{2}(\Vert \beta _{j0} + \\&\alpha _n\mathbf{u }_j\Vert -\Vert \beta _{j0}\Vert )^T\big ( P^{\prime \prime }_{\lambda _n}(\Vert \beta _{j0}\Vert )(\Vert \beta _{j0}+\alpha _n\mathbf{u }_j\Vert -\Vert \beta _{j0}\Vert )\\&\le \sum _j a_n\alpha _n \Vert \mathbf{u }_j\Vert + b_n \alpha _n^2 \Vert \mathbf{u }_j\Vert ^2 \\&\le \sum _j \alpha _n^2 C + b_n \alpha _n^2 C^2 = J(\alpha _n^2 C + b_n \alpha _n^2 C^2). \end{aligned}$$

Because $b_n \rightarrow 0$, $D_2 \rightarrow O_p(C\alpha _n^2)$. By choosing a sufficiently large C, $D_1$ dominates $D_2$. Thus, inequality (15) holds.

Appendix 2

We present the simulation studies of the second cross-validation approach described in Section 2.7 to select the tuning parameters $\lambda$ and evaluate its variable selection performance.

In Fig. 8, each dot represents the logarithm of the $\lambda$ values along the solution path, and the error bars provide the confidence intervals for the cross-validation log-partial-likelihood. The left vertical bar indicates the maximum cross-validation partial-log-likelihood using the first method Verweij and Houwelingen (1993) while the right one shows the maximum cross-validation log-partial-likelihood using the second method Ternes et al. (2016).

We continue considering $N=100$ observations and $P=400$ covariates with 40 groups, each with 10 elements. There are two non-zero groups. The coefficient magnitude $|\beta | = 0.5$, the values of the population correlation $\rho$ are 0,  0.2 and 0.5, the censoring rates are 0% and 20%. The results are summarized in Tables 11, 12, and 13 . It can be seen that using the second cross-validation method always results in smaller models than using the first cross-validation method. For group lasso, it produces better variable selection results with much smaller FPR values. For group SCAD and MCP, it often gives better results, but sometimes suppresses too much, e.g., in group MCP case with 20% censoring, $\rho =0.5$. Therefore, the second cross-validation method should be used with caution.

Appendix 3

We present additional settings based on the reviewer’s suggestions: settings with a large number of overlapping covariates and the number of zero groups being more than the number of non-zero groups. More specifically, we have performed an additional experiment using the simulated data with $N=100$, $P=55$, in which there are 10 groups of size 10 and 50% covariates overlap between two successive groups. The “correct” underlying group structure is given by

$$\begin{aligned}&\underset{group 1}{\underbrace{1,\dots ,10}}\text { } \underset{group 2}{\underbrace{6,\dots ,15}}\text { } \underset{group 3}{\underbrace{11,\dots ,20}}\text { } \underset{group 4}{\underbrace{16,\dots ,25}}\text { } \underset{group 5}{\underbrace{21,\dots ,30}}\text { } \\&\underset{group 6}{\underbrace{26,\dots ,35}}\text { }\underset{group 7}{\underbrace{31,\dots ,40}}\text { }\underset{group 8}{\underbrace{36,\dots ,45}}\text { }\underset{group 9}{\underbrace{41,\dots ,50}}\text { }\underset{group 10}{\underbrace{46,\dots ,55}}. \end{aligned}$$

We set the population correlation $\rho =0.5$ with 30% censoring rate. The corresponding coefficients are

$$\begin{aligned}&\underset{group 1-2}{\underbrace{0,\dots ,0}}\text { } \underset{group 3}{\underbrace{0,0,0,0,0,1.5,0,0,-2,0}}\text { } \underset{group 4}{\underbrace{1.5,0,0,-2,0,0,0,0,0,0}}\\&\text { }\underset{group 5-6}{\underbrace{0,\dots ,0}} \text { }\underset{group 7}{\underbrace{0,0,0,0,0,1.4,0,0,0, 1.8}}\\&\text { }\underset{group 8}{\underbrace{1.4,0,0,0, 1.8,0,0,0,0,0}}\text { }\underset{group 9-10}{\underbrace{0,\dots ,0}}. \end{aligned}$$

Then we consider four setups with the misspecified group structures for inference. In the first setup, the number of groups are incorrect because the overlapping groups are collapsed as follows:

$$\begin{aligned}&\underset{group 1}{\underbrace{1,\dots ,10}}\text { } \underset{group 2}{\underbrace{6,\dots ,15}}\text { } \underset{group 3}{\underbrace{11,\dots ,25}}\text { } \underset{group 4}{\underbrace{21,\dots ,30}}\text { } \underset{group 5}{\underbrace{26,\dots ,35}} \\&\underset{group 6}{\underbrace{31,\dots ,45}}\text { }\underset{group 7}{\underbrace{41,\dots ,50}}\text { }\underset{group 8}{\underbrace{46,\dots ,55}}. \end{aligned}$$

In the second setup, the misspecified group structure deviates from the ground truth more significantly will all the overlapping covariates put into one group:

$$\begin{aligned}&\underset{group 1}{\underbrace{1,3,5,7,9,11,13,15}}\text { } \underset{group 2}{\underbrace{2,4,\dots ,12,14,16,17,18,19,20,21,22}} \\&\text { } \underset{group 3}{\underbrace{16,\dots ,25}}\text { } \underset{group 4}{\underbrace{21,\dots ,30}} \text { } \underset{group 5}{\underbrace{26,\dots ,35}}\text { }\underset{group 6}{\underbrace{31,\dots ,45}}\text { }\underset{group 7}{\underbrace{41,\dots ,50}} \\&\text { }\underset{group 8}{\underbrace{46,\dots ,55}}. \end{aligned}$$

Similar as the first setup, the third and fourth setups are defined as follows:

$$\begin{aligned}&\underset{group 1}{\underbrace{1,\dots ,20}}\text { } \underset{group 2}{\underbrace{16,\dots ,25}}\text { } \underset{group 3}{\underbrace{21,\dots ,30}}\text { } \underset{group 4}{\underbrace{26,\dots ,35}}\text { }\underset{group 5}{\underbrace{31,\dots ,45}}\\&\underset{group 6}{\underbrace{41,\dots ,50}}\text { }\underset{group 7}{\underbrace{46,\dots ,55}} \end{aligned}$$

and

$$\begin{aligned}&\underset{group 1}{\underbrace{1,\dots ,10}}\text { } \underset{group 2}{\underbrace{6,\dots ,20}}\text { } \underset{group 3}{\underbrace{16,\dots ,25}}\text { } \underset{group 4}{\underbrace{21,\dots ,30}}\text { } \underset{group 5}{\underbrace{26,\dots ,40}}\\&\underset{group 6}{\underbrace{36,\dots ,45}}\text { }\underset{group 7}{\underbrace{41,\dots ,50}}\text { }\underset{group 8}{\underbrace{46,\dots ,55}} \end{aligned}$$

The results shown in Table 14 confirm our expectation: the setup with the collapsed groups including several non-zero (active) groups produces worse results than the cases with the collapsed groups with none or only one non-zero group. More clearly, the first setup in the table including two collapsed groups (group3 and group5), where each of them consists of two non-zero groups, has the worst variable selection performance. Both the second and third misspecification setups including only one group (group5) that is collapsed from two non-zero groups have almost the same performance, better than the first misspecification setup. The fourth mispecification setup with no misspecified group collapsed from two non-zero groups has the best performance. We hypothesize that the probability of variables being incorrectly selected increases due to the ignorance of the overlapping property of active elements in the collapsed groups and the larger group sizes of these collapsed groups. In other words, FPR increases and then corresponding RMSE increases.