A Bayesian Approach to Sparse Cox Regression in High-Dimentional Survival Analysis

Abstract

Survival prediction and prognostic factor identification play an important role in machine learning research. This paper employs the machine learning regression algorithms for constructing survival model. The paper suggests a new Bayesian framework for feature selection in high-dimensional Cox regression problems. The proposed approach gives a strong probabilistic statement of the shrinkage criterion for feature selection. The proposed regularization gives the estimates that are unbiased, possesses grouping and oracle properties, their maximal risk diverges to a finite value. Experimental results show that the proposed framework is competitive on both simulated data and publicly available real data sets.

The work is supported by grants of the Russian Foundation for Basic Research No. 11-07-00409 and No. 11-07-00634.

Appendix: Proofs

Proof of Theorem 1. The function $l(\varvec{\beta })$ (11) is strictly convex [5]. The penalty term (15) is strictly convex in subspace ${\beta _l} \in [-1/\mu ,1/\mu ], l = 1,...,p.$ In fact, the second derivatives of $p(\varvec{\beta })$ are

$$\begin{aligned} \frac{\partial ^2 p(\varvec{\beta })}{\partial \beta _i\partial \beta _j}= {\left\{ \begin{array}{ll} \frac{1-\mu \beta _i^{2}}{(\mu \beta _i+1)^2}, i=j\\ 0, i\not =j. \end{array}\right. } \end{aligned}$$

(18)

So the Hessian of $p(\varvec{\beta })$ non-negative defined in subspace ${\beta _l} \in [-1/\mu ,1/\mu ], l = 1,...,p.$ Define estimator $\hat{\varvec{\beta }}^*$: let $\hat{\beta }^*_k=\hat{\beta }_k$ for all $i \not = j$, otherwise let $\hat{\beta }_k^*= a\hat{\beta }_i+(1-a)\hat{\beta }_j$ for $a=1/2$. Since $\varvec{x}^{(i)}=\varvec{x}^{(j)}$, $\varvec{\tilde{X}}\varvec{\hat{\beta }^*}= \tilde{\varvec{X}}\varvec{\hat{\beta }}$ and $|\varvec{\tilde{z}} -\tilde{\varvec{X}}\varvec{\hat{\beta }^*} |=|\varvec{\tilde{z}} -\tilde{\varvec{X}}\hat{\varvec{\beta }} |$. However, the penalization function is convex in ${\beta _l} \in [-1/\mu ,1/\mu ], l = 1,...,p.$, that

$$\begin{aligned} p(\hat{\varvec{\beta }}^*) = p(a\hat{\beta }_i+(1-a)\hat{\varvec{\beta }}_j) <a p(\hat{\varvec{\beta _i}})+(1-a)p(\hat{\varvec{\beta }}_j)< p(\hat{\varvec{\beta }}) . \end{aligned}$$

Because $p(\hat{\varvec{\beta }}^*)=p(\hat{\varvec{\beta }})$ and because p(.) is additive, $p(\hat{\varvec{\beta }}^*) < p(\hat{\varvec{\beta }})$ and therefore cannot be the case that $\hat{\varvec{\beta }}$ is a minimizer. Hence $\hat{\beta _i} = \hat{\beta _j}$.

Proof of Theorem 2. By definition,

$$\begin{aligned} \frac{\partial J(\varvec{\beta }|\mu )}{\partial {\beta _k}}\mid _{\beta =\hat{\beta }}=0. \end{aligned}$$

(19)

By (19) (for non-zero $\hat{\beta }_i$ and $\hat{\beta }_j$ ),

$$\begin{aligned} -2\tilde{\varvec{x}}_i^T(\tilde{\varvec{z}}-\tilde{\varvec{X}}\hat{\varvec{\beta }})+(1+1/\mu )\frac{2\mu \hat{\varvec{\beta }_i}}{\mu \hat{\varvec{\beta }_i^2}+1}=0 \end{aligned}$$

(20)

and

$$\begin{aligned} -2\tilde{\varvec{x}}_j^T(\tilde{\varvec{z}}-\tilde{\varvec{X}}\hat{\varvec{\beta }})+(1+1/\mu )\frac{2\mu \hat{\varvec{\beta }_j}}{\mu \hat{\varvec{\beta }_j^2}+1}=0 \end{aligned}$$

(21)

Hence

$$\begin{aligned} \frac{\hat{\varvec{\beta }_i}}{\mu \hat{\varvec{\beta }_i^2}+1} - \frac{\hat{\varvec{\beta }_j}}{\mu \hat{\varvec{\beta }_j^2}+1} = \frac{1}{1+\mu }(\tilde{\varvec{x}}_i - \tilde{\varvec{x}}_j)^T(\tilde{\varvec{z}}-\tilde{\varvec{X}}\hat{\varvec{\beta }})\le \frac{1}{1+\mu }|\tilde{\varvec{x}}_i - \tilde{\varvec{x}}_j||\tilde{\varvec{z}}-\tilde{\varvec{X}}\hat{\varvec{\beta }}|. \end{aligned}$$

(22)

Also, note that $J(\varvec{\hat{\beta }}|\mu )\le J(\varvec{\hat{\beta }=\varvec{0}}|\mu )$, so $|\tilde{\varvec{z}}-\tilde{\varvec{X}}\hat{\varvec{\beta }}|\le |\tilde{\varvec{z}}| = 1$, since $\tilde{\varvec{z}}$ is centered and standardize . Hence,

$$\begin{aligned} \frac{\hat{\varvec{\beta }_i}}{\mu \hat{\varvec{\beta }_i^2}+1} - \frac{\hat{\varvec{\beta }_j}}{\mu \hat{\varvec{\beta }_j^2}+1} \le \frac{1}{1+\mu }|\tilde{\varvec{x}}_i - \tilde{\varvec{x}}_j| =\frac{\sqrt{2(1-\rho )}}{\mu +1} . \end{aligned}$$

(23)