Efficient local updates for undirected graphical models

Abstract

We present a new Bayesian approach for undirected Gaussian graphical model determination. We provide some graph theory results for local updates that facilitate a fast exploration of the graph space. Specifically, we show how to locally update, after either edge deletion or inclusion, the perfect sequence of cliques and the perfect elimination order of the nodes associated to an oriented, directed acyclic version of a decomposable graph. Building upon the decomposable graphical models framework, we propose a more flexible methodology that extends to the class of nondecomposable graphs. Posterior probabilities of edge inclusion are interpreted as a natural measure of edge selection uncertainty. When applied to a protein expression data set, the model leads to fast estimation of the protein interaction network.

Appendix

Proof

(Proposition 1) After removing an edge $e = \{a,b\}$ the two subsets $C_{a}$ or $C_{b}$ are created, and it is assumed without loss of generality that $a$ comes before $b$ in the perfect numbering of the nodes. The running intersection property implies that either $C_{a}, C_{b}$ or both can be subsets of only the cliques directly connected to $C^*$ in the junction tree, thus of only the cliques in $N_a, N_b$ as defined by Eq. (4). We have the following exhaustive cases.

• (a) If neither $C_{a}$ nor $C_{b}$ are subsets of any other clique, the junction tree can be updated by first replacing $C^*$ by the nodes $C_a$ and $C_b$ and connecting $C_{a}$ to $C_{b}$. Then, the existing edges $\{C, C^*\}$ for $C \in N_a$ are replaced with edges $\{C, C_a\}$ and the existing edges $\{C, C^*\}$ for $C \in N_b$ are replaced with edges $\{C, C_b\}$. If there are other edges $\{C, C^*\}$ with $C \in N_{\,\overline{ab}}$ then these can be replaced arbitrarily with edges $\{C, C_a\}$ or $\{C, C_a\}$. It is then straightforward to verify that the running intersection property is retained and that the perfect sequence of the cliques obtained by applying Proposition 2.29 of Lauritzen (1996) coincides with that from Proposition 1.

• (b) If $C_{a} \subset C$, where $C$ precedes $C_a$ in the sequence, then in the previous procedure, the edge $\{C, C_a\}$ must be contracted and $C_a$ must be replaced by $C$. Thus, $C$ will be connected to $C_{b}$ and to all the cliques in $N_a$, and $C_{b}$ will be connected to all the cliques in $N_b$. In this case, the perfect sequence of the cliques derived from Lauritzen (1996, Proposition. 2.29) coincides with that from Proposition 1.

• (c) If $C_{a} \subset C'$ and $C'$ follows $C_a$ in the perfect sequence, the junction tree can be updated by connecting $C_{a}$ to $C_{b}$ and to all the other cliques in $N_a$, and connecting $C_{b}$ to all the cliques in $N_b$. In this case, the perfect sequence of the cliques derived from Lauritzen (1996, Proposition. 2.29) coincides with that from Proposition 1.

• (d) If $C_{a} \subset C$ and $C_{b} \subset C'$, where $C$ precedes $C_a$ and $C'$ follows $C_a$ in the perfect sequence, the junction tree can be updated by connecting $C$ to $C_{b}$ and to all the other cliques in $N_a$, and connecting $C_{b}$ to all the other cliques in $N_b$. In this case, the perfect sequence of the cliques derived from Lauritzen (1996, Proposition 2.29) coincides with that from Proposition 1.

• (e) When $C_{a}$ and $C_{b}$ are subsets of cliques $C'$ and $C''$, respectively, following $C_a$ in the perfect sequence, the junction tree can be updated by connecting $C_{a}$ to $C_b$ and to all the other cliques in $N_a$, and connecting $C_b$ to all the other cliques in $N_b$. Then the perfect sequence of the cliques derived from Lauritzen (1996, Proposition 2.29) coincides with that from Proposition 1. $\square$

Proof

(Proposition 2) At the end of Algorithm 3, we have three possible cases:

Case	Are $C_a, C_b$ in $G'$?	Edge in $J$	Edge in $J'$
(a)	Yes, yes	$\{C_a, C_b\}$	$\{C_a, C^*\}$ and $\{C^*, C_b\}$
(b)	No, yes   yes, no	$\{C_a, C_b\}$	$\{C_a, C^*\}$ or $\{C^*, C_b\}$
(c)	No, no	$\{C_a, C_b\}$	none

In case (a), if $C_a$ and $C_b$ are both maximal cliques in $G'$, the new junction tree $J'$ has a new node $C^*$ on the directed path between $C_a$ and $C_b$. Therefore, the perfect sequence of the cliques is given in Eq. (5). In case (b), if only one of $C_a$ or $C_b$ is a maximal clique in $G'$, the sequence (5) is reduced by possibly removing either $C_a$ or $C_b$. Finally, in case (a), both $C_a$ and $C_b$ are removed.$\square$

Proof

(Proposition 3) In the junction tree $T$, all the cliques in $\mathcal {C}_t$ and $\mathcal {C}_{d}$ are separated from all the remaining cliques by $C_b'$. Then the cliques in $\mathcal {C}_L$ are the first in the new perfect sequence, since they belong to a subtree of $T$ that remains unchanged after the update. By definition, $C_a'$ and $C_b'$ can be the next two cliques in the sequence. Then the set of cliques in $\mathcal {C}_t$ is a subpath between $C_b'$ and one of the leaves of $T$ and must be in reverse order with respect to that in (6); see Fig. 2. Next, by the definition of descendants, the cliques in $\mathcal {C}_d$ can immediately follow $\mathcal {C}_t$ in the same ordering as that of (6). Clearly, the descendants $\mathrm {de}(C_{t_i})$ of each clique $C_{t_i} \in \mathcal {C}_t$ must be ordered as in (6) since the subtree that connects them remains unchanged after updating $J$; whereas the ordering between $\mathrm {de}(C_{t_i})$ is arbitrary since they are separated by $\mathcal {C}_t$. Finally, the cliques in $\mathcal {C}_R$ also belong to a subtree of $T$ that remains unchanged after the tree update, and they are separated, in $T$, from the cliques in $\mathcal {C}_t$ and $\mathcal {C}_d$ by $C_a'$.$\square$