Model-free feature screening for ultrahigh-dimensional data conditional on some variables

Abstract

In this paper, the conditional distance correlation (CDC) is used as a measure of correlation to develop a conditional feature screening procedure given some significant variables for ultrahigh-dimensional data. The proposed procedure is model free and is called conditional distance correlation-sure independence screening (CDC-SIS for short). That is, we do not specify any model structure between the response and the predictors, which is appealing in some practical problems of ultrahigh-dimensional data analysis. The sure screening property of the CDC-SIS is proved and a simulation study was conducted to evaluate the finite sample performances. Real data analysis is used to illustrate the proposed method. The results indicate that CDC-SIS performs well.

Appendix

We first establish the following regularity conditions:

• (C1) Denote the density function of W by $f(\cdot )$, and assume that it has continuous second derivatives. The support of W is assumed to be bounded and is denoted by $\mathcal {W}=[a,b]$ with finite constants a and b.

• (C2) $K(\cdot )$ is a symmetric density function with bounded support and bounded over its support.

• (C3) The random variables $\mathbf X$ and Y satisfy the sub-exponential tail probability uniformly in p. That is, there exists a positive constant $s_0$, such that for $0\le s<s_0$,

  $$\begin{aligned} \sup _{W\in \mathcal {W}}\max _{1\le j \le p}E(\exp (sX_j^2|W))< & {} \infty ,\\ \sup _{W\in \mathcal {W}}E(\exp (s Y^2|W))< & {} \infty , \end{aligned}$$

• (C4) $\min _{j\in \mathcal {{M}}^{*}}{\rho }_{j0}^{*} \ge 2cn^{-\kappa }$ for some constant $c>0$ and $0\le \kappa < 1/2$.

Proof of Theorem 1

The proof consists of three steps. We denote the positive constants c and C as generic constants depending on the context, which can vary from line to line.

• Step 1. For some $0\le \kappa <1/2$, we first prove

  $$\begin{aligned}&\max _{1\le j\le p}\sup _{w\in [a,b]}P(|\hat{\rho }^2 (X_j,Y|W=w)-{\rho }^2 (X_j,Y|W=w)|\nonumber \\ {}&\quad \ge cn^{-\kappa }) \le C \exp \left( -\frac{n^{-\kappa }}{Ch}\right) . \end{aligned}$$

  (7)

  Refer to the Supplemental material for the proof of Step 1.

• Step 2. We prove $P(\max _{1\le j\le p}|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-k}) \le O(np\exp (-{n^{\gamma -\kappa }}/{\xi }))$. Note that

  $$\begin{aligned} P(|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-\kappa })\le & {} P(|\hat{\rho }_{j}^{*}-{\rho }_{j}^{*}|+|{\rho }_{j}^{*}-{\rho }_{j0}^{*}| \ge cn^{-\kappa })\\\le & {} P(|\hat{\rho }_{j}^{*}-{\rho }_{j}^{*}| \ge cn^{-\kappa }/2)+ P(|{\rho }_{j}^{*}-{\rho }_{j0}^{*}| \ge cn^{-\kappa }/2). \end{aligned}$$

  By the definitions of $\hat{\rho }_{j}^{*}$, ${\rho }_{j}^{*}=\frac{1}{n}\sum _{i=1}^n{\rho }_j^2 (W_i)$ with ${\rho }_j^2 (w)=\rho ^2(X_j,Y|W=w)$ and the result of Step 1, we have, for $j=1,2,\ldots ,p$

  $$\begin{aligned} P(|\hat{\rho }_{j}^{*}-{\rho }_{j}^{*}| \ge cn^{-\kappa }/2)= & {} P\left( |\frac{1}{n}\sum _{i=1}^n\hat{\rho }_j^2 (W_i)-\frac{1}{n}\sum _{i=1}^n{\rho }_j^2 (W_i)| \ge cn^{-\kappa }/2\right) \nonumber \\\le & {} \sum _{i=1}^n P(|\hat{\rho }_j^2 (W_i)-{\rho }_j^2 (W_i)| \ge cn^{-\kappa }/2)\nonumber \\\le & {} Cn\exp \left( -\frac{n^{-\kappa }}{Ch}\right) \nonumber \\= & {} O(n\exp (-n^{\gamma -\kappa }/\xi )), \end{aligned}$$

  (8)

  where $\xi$ is a positive constant, and $0\le \kappa <\gamma$. By Hoeffding’s inequality, for $j=1,2,\ldots ,p$, it follows that

  $$\begin{aligned} P(|{\rho }_{j}^{*}-{\rho }_{j0}^{*}| \ge cn^{-\kappa }/2)= & {} P\left( |\frac{1}{n}\sum _{i=1}^n{\rho }_j^2 (W_i)-E{\rho }_j^2 (W_i)| \ge cn^{-\kappa }/2\right) \nonumber \\\le & {} 2\exp (-nc^2n^{-2\kappa }/2))= O(\exp (-n^{1-2\kappa }/\xi )).\nonumber \\ \end{aligned}$$

  (9)

  Eq. (8) dominates Eq. (9). Hence, for $j=1,2,\ldots ,p$, we get

  $$\begin{aligned} P(|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-\kappa }) \le O(n\exp (-{n^{\gamma -\kappa }}/{\xi })). \end{aligned}$$

  We thus have

  $$\begin{aligned} P\left( \max _{1\le j\le p}|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-k}\right) \le O(np\exp (-{n^{\gamma -\kappa }}/{\xi })). \end{aligned}$$

• Step 3. We prove $P(\mathcal {{M}}^{*} \subset \mathcal {\hat{M}}) \ge 1-O(ns_n\exp (-{n^{\gamma -\kappa }}/{\xi }))$. If $\mathcal {{M}}^{*} \not \subset \mathcal {\hat{M}}$, then there exist some $j\in \mathcal {{M}}^{*}$ such that $\hat{\rho }_{j}^{*}<cn^{-\kappa }$, due to $\min _{j\in \mathcal {{M}}^{*}}{\rho }_{j0}^{*} \ge 2cn^{-\kappa }$, $|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-\kappa }$ for some $j\in \mathcal {{M}}^{*}$, indicating that

  $$\begin{aligned} \{\mathcal {{M}}^{*} \not \subset \mathcal {\hat{M}}\} \subset \{|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-\kappa }\quad \text{ for } \text{ some } \; j\in \mathcal {{M}}^{*}\}. \end{aligned}$$

  Consequently,

  $$\begin{aligned} P\{\mathcal {{M}}^{*} \subset \mathcal {\hat{M}}\}\ge & {} P\{\max _{j\in \mathcal {{M}}^{*}}|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|< cn^{-\kappa }\}\\= & {} 1-P\{\max _{j\in \mathcal {{M}}^{*}}|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-\kappa }\}\\\ge & {} 1-s_n P\{|\hat{\rho }_{j}^{*}-{\rho }_{j0}^{*}|\ge cn^{-\kappa }\}\\\ge & {} 1 - O(ns_n\exp (-{n^{\gamma -\kappa }}/{\xi })). \end{aligned}$$

$\square$