A Bayesian Approach for Learning Gene Networks Underlying Disease Severity in COPD

Abstract

In this paper, we propose a Bayesian hierarchical approach to infer network structures across multiple sample groups where both shared and differential edges may exist across the groups. In our approach, we link graphs through a Markov random field prior. This prior on network similarity provides a measure of pairwise relatedness that borrows strength only between related groups. We incorporate the computational efficiency of continuous shrinkage priors, improving scalability for network estimation in cases of larger dimensionality. Our model is applied to patient groups with increasing levels of chronic obstructive pulmonary disease severity, with the goal of better understanding the break down of gene pathways as the disease progresses. Our approach is able to identify critical hub genes for four targeted pathways. Furthermore, it identifies gene connections that are disrupted with increased disease severity and that characterize the disease evolution. We also demonstrate the superior performance of our approach with respect to competing methods, using simulated data.

Appendix

1.1 Details on our MCMC Algorithm

In this section, we provide a detailed description of Step a and Step b of our MCMC algorithm.

Step a. By partitioning $\Omega$ into $V=(\upsilon _{i,j}^2)$, a $p\times p$ symmetric matrix with zeroed diagonal entries and $(\upsilon _{i,j}^2)_{i<j}$ in the upper diagonal entries and setting $S=X'X$, we can focus on the last column and row to acquire

$$\begin{aligned} \Omega = \left( \begin{array}{cc} \Omega _{1,1}&{}\quad \omega _{1,2}\\ \omega _{1,2}' &{}\quad \omega _{2,2} \end{array}\right) , \quad S=\left( \begin{array}{cc} S_{1,1}&{}\quad s_{1,2}\\ s_{1,2}' &{}\quad s_{2,2} \end{array}\right) , \quad V=\left( \begin{array}{cc} V_{1,1} &{}\quad v_{1,2} \\ v_{1,2}' &{}\quad 0 \end{array}\right) . \end{aligned}$$

Changing variables from $(\omega _{1,2}, \omega _{2,2})$ to $(u=\omega _{1,2},\upsilon =\omega _{2,2}-\omega _{1,2}'\Omega ^{-1}\omega _{1,2})$, we have full conditionals

$$\begin{aligned} u|\cdot \sim N(-Cs_{1,2}, C) \quad {\text {and}} \; \upsilon |\cdot \sim {\text {Gamma}}\biggr (\frac{n}{2}+1, \frac{s_{2,2}+\lambda }{2}\biggr ), \end{aligned}$$

where $C=\{(s_{2,2}+\lambda )\Omega _{1,1}^{-1}+{\text {diag}} (v_{1,2}^{-1})\}^{-1}$. Using this method, we can permute any column to attain the full conditional used to generate $\Omega |\mathbf{{G}},X$. Our full conditional on $\mathbf{{G}}$ is then an independent Bernoulli of the form

$$\begin{aligned} P(g_{i,j}=1|\Omega , X)=\frac{N(\omega _{i,j}|0, \upsilon _1^2)\pi }{N(\omega _{i,j}|0, \upsilon _1^2)\pi + N(\omega _{i,j}|0,\upsilon _0^2)(1-\pi )}, \end{aligned}$$

where the quantity $\frac{\pi }{1-\pi }$ is determined by the MRF prior on the graph structure such that

$$\begin{aligned} \frac{\pi }{1-\pi }=\frac{p(G_k'|\nu _{i,j}, \Theta , \{G_m\}_{m\ne k})}{p(G_k|\nu _{i,j}, \Theta , \{G_m\}_{m\ne k})}=\exp \bigg \{-\nu _{i,j}+2\sum _{m\ne k} \theta _{k,m}g_{m,i,j})\bigg \}, \end{aligned}$$

for proposed new graph $G_k'$ which differs from the current graph $G_k$ only in that edge (i, j) is excluded from $G_k'$ and included in $G_k$.

Step b. In order to update $\theta _{k,m}$ and $\gamma _{k,m}$, we must consider the full conditional distribution. Considering only the terms of the joint prior for graphs $G_1, \ldots , G_k$ which include $\theta _{k,m}$, we can see that

$$\begin{aligned} p(G_1, \ldots , G_4|\nu , \Theta )&=\prod _{i<j}C(\nu _{i,j}, \Theta )^{-1}\exp \bigg (\nu _{i,j}{} \mathbf{{1}}^T\mathbf{{g_{i,j}}}+\mathbf{{g_{i,j}}}^T \Theta \mathbf{{g_{i,j}}}\bigg ) \\&\propto \prod _{i<j} C(\nu _{i,j},\Theta )^{-1}\exp \bigg (2\theta _{k,m} g_{k,i,j}g_{m,i,j}\bigg ). \end{aligned}$$

The full conditional distribution of $\theta _{k,m}$ and $\gamma _{k,m}$ can then be written as

$$\begin{aligned} p(\theta _{k,m}, \gamma _{k,m}|\cdot )&= p(G_1, \ldots , G_k|\nu , \Theta )p(\theta _{k,m}|\gamma _{k,m})p(\gamma _{k,m}|w)\\&\propto \biggr (\prod _{i<j} C(\nu _{i,j}, \Theta )^{-1}\exp (2\theta _{k,m}g_{k,i,j}g_{m,i,j})\biggr ) \\&\qquad \times \;\biggr ((1-\gamma _{k,m})\delta _0+\gamma _{k,m}\frac{\beta ^\alpha }{\Gamma (\alpha )}\theta _{k,m}^{\alpha -1}e^{-\beta \theta _{k,m}}\biggr ) \\&\qquad \times \;\biggr (w^{\gamma _{k,m}}(1-w)^{(1-\gamma _{k,m})}\biggr ). \end{aligned}$$

Because the normalizing constant from the joint prior on the graphs is analytically intractable, we use Metropolis–Hastings step to sample from $\theta _{k,m}$ and $\gamma _{k,m}$ for each pair of (k, m), $1\le k<m \le 4$ from the joint full conditional distribution. Each iteration has two steps based on the approach described by [14] to sample from mutually singular distribution mixtures. First, we perform a between-model move. If the current state is $\gamma _{k,m}=1$, we propose $\gamma ^\star _{k,m}=0$ and $\theta ^\star _{k,m}=0$ resulting in the Metropolis–Hastings ratio

$$\begin{aligned} r&=\frac{p(\theta ^\star _{k,m}, \gamma ^\star _{k,m}|\cdot )\times q(\theta _{k,m})}{p(\theta _{k,m}, \gamma _{k,m}|\cdot )} =\frac{\Gamma (\alpha )}{\Gamma (\alpha ^\star )}\frac{(\beta ^\star )^{ \alpha ^\star }}{\beta ^\alpha }(\theta _{k,m})^{\alpha ^\star -\alpha }e^{ (\beta -\beta ^\star )\theta _{k,m}}\\&\quad \times \;\prod _{i<j}\frac{C(\nu _{i,j}, \Theta )\exp (-2\theta _{k,m}g_{k,i,j}g_{m,i,j})}{C(\nu _{i,j}, \Theta ^\star )}\frac{1-w}{w}, \end{aligned}$$

where $\Theta ^\star$ represents the network similarity matrix $\Theta$ with entry $\theta _{k,m}=\theta _{k,m}^\star$. If moving instead from $\gamma _{k,m}=0$ to $\gamma ^\star _{k,m}=1$, the ratio is

$$\begin{aligned} r&=\frac{p(\theta ^\star _{k,m}, \gamma ^\star _{k,m}|\cdot )}{p(\theta _{k,m}, \gamma _{k,m}|\cdot )\times q(\theta _{k,m})} =\frac{\Gamma (\alpha ^\star )}{\Gamma (\alpha )}\frac{\beta ^{\alpha }}{(\beta ^\star )^{\alpha ^\star }}\times (\theta _{k,m})^{\alpha -\alpha ^\star } e^{(\beta ^\star -\beta )\theta _{k,m}} \\&\quad \times \;\prod _{i<j}\frac{C(\nu _{i,j}, \Theta )\exp (-2\theta ^\star _{k,m}g_{k,i,j}g_{m,i,j})}{C(\nu _{i,j}, \Theta ^\star )}\frac{w}{1-w}. \end{aligned}$$

Next, we perform the within-model move if the value of $\gamma _{k,m}$ sampled from the between-model move is 1. Here, we propose a new value using the same proposal density as before, for $\theta _{k,m}$. Our Metropolis–Hastings ratio is

$$\begin{aligned} r&=\frac{p(\theta ^\star _{k,m}, \gamma ^\star _{k,m}|\cdot )\cdot q(\theta _{k,m})}{p(\theta _{k,m}, \gamma _{k,m}|\cdot )\cdot q(\theta ^\star _{k,m})} =\biggr (\frac{\theta ^\star _{k,m}}{\theta _{k,m}}\biggr )^{\alpha -\alpha ^\star } \cdot {e}^{(\beta ^\star -\beta )(\theta ^\star _{k,m}-\theta _{k,m})}\\&\quad \times \;\prod _{i<j} \frac{C(\nu _{i,j},\Theta )\exp (2(\theta ^\star _{k,m}-\theta _{k,m})g_{k,i,j} g_{m,i,j})}{C(\nu _{i,j}, \Theta ^\star )}. \end{aligned}$$

In our last step of the MCMC, we sample from the full conditional distribution of $\nu _{i,j}$. The terms of the joint prior on the graphs including $\nu _{i,j}$ are

$$\begin{aligned} p(G_1, \ldots , G_k|\nu , \Theta )&=\prod _{i<j}C(\nu _{i,j}, \Theta )^{-1}\exp \bigg (\nu _{i,j}{} \mathbf{{1}}^T\mathbf{{g_{i,j}}}+\mathbf{{g_{i,j}}}^T\Theta \mathbf{{g_{i,j}}}\bigg ) \\&\propto C(\nu _{i,j},\Theta )^{-1}\exp \bigg (\nu _{i,j}{} \mathbf{{1}}^T\mathbf{{g_{i,j}}}\bigg ). \end{aligned}$$

Given the prior on $\nu _{i,j}$, we can attain the posterior full conditional given the data and all remaining parameters

$$\begin{aligned} p(\nu _{i,j}|\cdot )&\propto \frac{\exp (a\nu _{i,j})}{(1+e^{\nu _{i,j}})^ {a+b}}C(\nu _{i,j},\Theta )^{-1}\exp \bigg (\nu _{i,j}{} \mathbf{{1}}^T\mathbf{{g_{i,j}}}\bigg )\\&=\frac{\exp (\nu _{i,j}(a+\mathbf{{1}}^T\mathbf{{g_{i,j}}}))}{C(\nu _{i,j}, \Theta )\cdot (1+e^{\nu _{i,j}})^{a+b}}. \end{aligned}$$

We then propose a value $q^\star$ from the density Beta(2, 4) for each pair (i, j) where $1\le i<j\le p$ and set $\nu ^\star = {\text {logit}}(q^\star )$. We can write our proposal density in terms of $\nu ^\star$ as

$$\begin{aligned} q(\nu ^\star )=\frac{1}{B(a^\star , b^\star )}\frac{e^{a^\star \nu ^\star }}{(1+e^{\nu ^\star })^{a^\star + b^\star }}, \end{aligned}$$

with Metropolis–Hastings ratio

$$\begin{aligned} r&=\frac{p(\nu ^\star |\cdot )}{p(\nu _{i,j}|\cdot )}\frac{q(\nu _{i,j})}{q(\nu ^\star )}\\&=\frac{\exp ((\nu ^\star -\nu _{i,j})\cdot (a-a^\star +\mathbf{{1}}^ T\mathbf{{g_{i,j}}}))\cdot C(\nu _{i,j}, \Theta )\cdot (1+e^{\nu _{i,j}})^{a+b-a^\star -b^\star }}{C(\nu ^\star , \Theta )\times (1+e^{\nu ^\star })^{a+b-a^\star -bI^\star }}. \end{aligned}$$

1.2 Case Study: Comparison to the Fused and Joint Graphical Lasso

In this section, we compare the proposed Bayesian approach to the fused and joint graphical lasso in terms of the findings obtained from the analysis of the ECLIPSE dataset. Specifically, we focused on the Reg Auto and GPL pathways. For both the fused and joint graphical lasso, we selected the penalty parameters that minimized the AIC, as recommended by [9]. For the Reg Auto pathway, the fused graphical lasso penalty parameters were selected as $\lambda _1=0.015$ and $\lambda _2=0.0001$, and for the group lasso were selected as $\lambda _1= 0.015$ and $\lambda _2=0$ (this value was selected after an extensive grid search with step size of .0000005). For the GPL pathway, penalty parameters were selected as $\lambda _1=0.02$ and $\lambda _2=0.0005$ for the fused lasso, and $\lambda _1=0.02$ and $\lambda _2=0.0$ for the group lasso. Results are summarized in the two tables below.

Reg auto: method edge count comparison

	Proposed method	Group fused lasso	Joint group lasso
Group 1 edge count	98	159	159
Group 2 edge count	95	155	155
Group 3 edge count	89	155	155
Group 4 edge count	98	146	146
Unique edge count	153	190	190

GPL: method edge count comparison

	Proposed method	Group fused lasso	Joint group lasso
Group 1 edge count	312	560	560
Group 2 edge count	255	553	553
Group 3 edge count	288	545	545
Group 4 edge count	314	536	536
Unique edge count	539	802	802

For the Reg Auto pathway, it can be seen that edge counts were equivalent for the fused lasso and the group lasso. Both lasso methods selected all the possible 190 edges; this illustrates the issue corresponding to high false positive rates for lasso methods and consequently hints at more difficult interpretation of results. Percentage overlap of unique edges for Reg Auto was computed as

$$\begin{aligned} \frac{{\text {Unique Edges in Proposed and Lasso Method}}}{{\text {Unique Lasso Edge Count}}}, \end{aligned}$$

and resulted in an overlap of 80 %. Lasso methods identified the same hub genes as the proposed Bayesian approach, plus ATG10 and ULK3.

Similar conclusions can be derived from the analysis of the GPL pathway. The same edges were selected by both the group and fused lasso for all disease groups; 802 out of 820 possible unique edges were selected. Of the 18 edges remaining which were not selected by the lasso methods, five were selected by our proposed method. This resulted in a percentage overlap of unique edges for GPL 67 %. The lasso methods identified the same hub genes as our proposed method in addition to DGKE, DGKQ, and MBOAT1. Overall, the lasso methods have similar results to our proposed approach, but result in much more dense networks due to their higher false positive rates. The proposed Bayesian approach provides sparser solutions that can be more easily interpreted.