A fast iterative algorithm for high-dimensional differential network

Abstract

A differential network is an important tool for capturing the changes in conditional correlations under two sample cases. In this paper, we introduce a fast iterative algorithm to recover the differential network for high-dimensional data. The computational complexity of our algorithm is linear in the sample size and the number of parameters, which is optimal in that it is of the same order as computing two sample covariance matrices. The proposed method is appealing for high-dimensional data with a small sample size. The experiments on simulated and real datasets show that the proposed algorithm outperforms other existing methods.

Appendix

According to the main results of Beck and Teboulle (2009), to complete the proof of Theorem 1, we need to show only that the loss function $L_1(\varDelta )$ is convex, which is the result of the following lemma.

Lemma 1.1 The loss function (2.3) is a smooth convex function, and its gradient is Lipschitz continuous with Lipschitz constant $L=\lambda _{\max }(S_1) \lambda _{\max }(S_2)$, that is

$$\begin{aligned} \Vert \nabla L_1(\varDelta _1) - \nabla L_1(\varDelta _2)\Vert _2 \le L \Vert \varDelta _1 - \varDelta _2\Vert _2, \end{aligned}$$

where $\lambda _{\max }(S_i)$ is the largest eigenvalue of the sample covariance matrix $S_i$ for $i=1,2$.

Proof

Because the loss function (2.3) is defined by

$$\begin{aligned} L_1(\varDelta )= \frac{1}{2} \text{ tr }\{\varDelta ^{\hbox {T}}S_1 \varDelta S_2\}- \text{ tr }\{\varDelta (S_1-S_2)\}, \end{aligned}$$

We can calculate the gradient of $L_1(\varDelta )$

$$\begin{aligned} \nabla L_1(\varDelta ) = S_1\varDelta S_2 - (S_1-S_2), \end{aligned}$$

and the Hessian matrix is $S_2 \otimes S_1$. Because both covariance matrices $S_1$ and $S_2$ are definite positive matrices, the Hessian matrix is a definite positive matrix. Hence, the loss function $L_1(\varDelta )$ is a smooth convex function.

Moreover, for any $\varDelta _1, \varDelta _2 \in \text {dom}(\nabla L_1)$, we have

$$\begin{aligned} \Vert \nabla L_1(\varDelta _1) - \nabla L_1(\varDelta _2)\Vert _2= & {} \Vert S_1 (\varDelta _1-\varDelta _2) S_2 \Vert _2 \\= & {} \Vert (S_2 \otimes S_1) \text{ vec }(\varDelta _1-\varDelta _2) \Vert _2 \\\le & {} \lambda _{\max }(S_2 \otimes S_1) \Vert \text{ vec }(\varDelta _1-\varDelta _2)\Vert _2 \\= & {} \lambda _{\max }(S_1) \lambda _{\max }(S_2) \Vert \varDelta _1-\varDelta _2\Vert _2 . \end{aligned}$$

The proof is now complete. $\square$