Model-based clustering of multiple networks with a hierarchical algorithm

Abstract

The paper tackles the problem of clustering multiple networks, directed or not, that do not share the same set of vertices, into groups of networks with similar topology. A statistical model-based approach based on a finite mixture of stochastic block models is proposed. A clustering is obtained by maximizing the integrated classification likelihood criterion. This is done by a hierarchical agglomerative algorithm, that starts from singleton clusters and successively merges clusters of networks. As such, a sequence of nested clusterings is computed that can be represented by a dendrogram providing valuable insights on the collection of networks. Using a Bayesian framework, model selection is performed in an automated way since the algorithm stops when the best number of clusters is attained. The algorithm is computationally efficient, when carefully implemented. The aggregation of clusters requires a means to overcome the label-switching problem of the stochastic block model and to match the block labels of the networks. To address this problem, a new tool is proposed based on a comparison of the graphons of the associated stochastic block models. The clustering approach is assessed on synthetic data. An application to a set of ecological networks illustrates the interpretability of the obtained results.

Appendix

1.1 Details on the update of the node labels

Here we present the details on the efficient computation of the ICL changes $\Delta _{m^*, i^*}^{\rightarrow h}$, in the case when moving node $i^*$ to block h does not empty block g.

Changes in the statistics. Let $s^{(m^*)}_{k}$ be the count statistic before the swap and $\vec {s}^{(m^*)}_{k}$ its value after the swap. We use the same notation for all other statistics. Clearly, $\vec {s}^{(m^*)}_{g}=s^{(m^*)}_{g}-1$ and $\vec {s}^{(m^*)}_{h}=s^{(m^*)}_{h}+1$, while the other terms remain unchanged. Define

$$\begin{aligned} \delta _{k,\cdot i^*}=\sum _{i\ne i^*}Z^{(m^*)}_{i,k}A^{(m^*)}_{i,i^*},\qquad \delta _{\ell , i^* \cdot }=\sum _{j\ne i^*}Z^{(m^*)}_{j,\ell }A^{(m^*)}_{i^*,j}. \end{aligned}$$

Then, for any $k,\ell \in \llbracket K\rrbracket$,

$$\begin{aligned} \vec {a}^{(m^*)}_{k,\ell }&=a^{(m^*)}_{k,\ell } -\mathbb {1}_{k=g}\delta _{\ell , i^* \cdot } +\mathbb {1}_{k=h}\delta _{\ell , i^* \cdot } \\&\quad -\mathbb {1}_{\ell =g}\delta _{k, \cdot i^*} +\mathbb {1}_{\ell =h}\delta _{k, \cdot i^*}. \end{aligned}$$

When considering the matrix $(a^{(m^*)}_{k,\ell })_{k,\ell }$, only the g-th and h-th row and the g-th and h-th column change when moving $i^*$ from g to h. We introduce the number of possible dyads from nodes in block k to nodes in block $\ell$ in graph m defined as

$$\begin{aligned} r^{(m)}_{k,\ell } =\sum _{i\ne j}Z^{(m)}_{i,k}Z^{(m)}_{j,\ell } = \left\{ \begin{array}{ll} s^{(m)}_{k}s^{(m)}_{\ell }&{}\quad \text {if }k\ne \ell \\ s^{(m)}_{k}(s^{(m)}_{k}-1)&{}\quad \text {if }k=\ell \end{array}\right. \end{aligned}$$

Then $b^{(m)}_{k,\ell }=r^{(m)}_{k,\ell }-a^{(m)}_{k,\ell }$ and

$$\begin{aligned}&\vec {r}^{(m^*)}_{k,\ell }=r^{(m^*)}_{k,\ell } -s^{(m^*)}_{\ell }\mathbb {1}_{k=g} +s^{(m^*)}_{\ell }\mathbb {1}_{k=h} -s^{(m^*)}_{k}\mathbb {1}_{\ell =g}\\&\quad +s^{(m^*)}_{k}\mathbb {1}_{\ell =h} +2\mathbb {1}_{k=g,\ell =g} -\mathbb {1}_{k=g,\ell =h} -\mathbb {1}_{k=h,\ell =g}. \end{aligned}$$

and $\vec {b}_{k,l}^{(m^*)}=\vec {r}_{k,l}^{(m^*)}-\vec {a}_{k,l}^{(m^*)}$. For any $m\ne m^*$, the statistics remain unchanged, that is, $\vec {a}_{k,l}^{(m)}=a_{k,l}^{(m)}$, $\vec {b}_{k,l}^{(m)}=b_{k,l}^{(m)}$ and $\vec {r}_{k,l}^{(m)}=r_{k,l}^{(m)}$. Finally, we define function $\Psi :\mathbb {R}_+\times {\mathbb {Z}}\rightarrow \mathbb {R}$ as

$$\begin{aligned} \Psi (a,z)&=\log \left( \frac{\Gamma (a+z)}{\Gamma (a)}\right) \mathbb {1}\{a+z>0\}. \end{aligned}$$

First case: K does not change. Suppose that $i^*$ is not the last vertex in block g, that is, $\sum _{m}\sum _iZ^{(m)}_{i,g}>1$. Then, moving node $i^*$ to another block h does not empty block g and the number of blocks K remains unchanged. In this case, the ICL variation is given by

$$\begin{aligned}&\Delta _{m^*, i^*}^{\rightarrow h} \nonumber \\&\quad =\sum _{(k,\ell )\in I_{g,h}} \left\{ \log \left( \frac{\Gamma (\eta +\sum _m \vec {a}_{k,l}^{(m)})\Gamma (\zeta +\sum _m \vec {b}_{k,l}^{(m)})}{\Gamma (\eta +\zeta +\sum _m \vec {r}_{k,l}^{(m)})}\right) \right. \nonumber \\&\quad \quad \left. - \log \left( \frac{\Gamma (\eta +\sum _m a_{k,l}^{(m)})\Gamma (\zeta +\sum _m b_{k,l}^{(m)})}{\Gamma (\eta +\zeta +\sum _m r_{k,l}^{(m)})}\right) \right\} \nonumber \\&\quad \quad + \sum _{k\in \{g,h\}} \left\{ \log \left( \Gamma (\alpha +\sum _m \vec {s}_{k}^{(m)})\right) \right. \nonumber \\&\quad \quad \left. -\log \left( \Gamma (\alpha +\sum _m s_{k}^{(m)})\right) \right\} \nonumber \\&\quad =\sum _{(k,\ell )\in I_{g,h}} \left\{ \Psi \left( \eta +\sum _m a_{k,l}^{(m)}, \vec {a}_{k,l}^{(m^*)}-a_{k,l}^{(m^*)}\right) \right. \nonumber \\&\quad \quad +\left. \Psi \left( \zeta +\sum _m b_{k,l}^{(m)}, \vec {b}_{k,l}^{(m^*)}-b_{k,l}^{(m^*)}\right) \right. \nonumber \\&\quad \quad \left. -\Psi \left( \eta +\zeta +\sum _m r_{k,l}^{(m)}, \vec {r}_{k,l}^{(m^*)}-r_{k,l}^{(m^*)}\right) \right\} \nonumber \\&\quad \quad + \log \left( \frac{ \alpha +\sum _m s_{h}^{(m)}}{\alpha +\sum _m s_{g}^{(m)}-1}\right) , \end{aligned}$$

(6)

where $I_{g,h} =\left\{ (k,\ell )\in \llbracket K\rrbracket ^2, k\in \{g,h\} \text { or } \ell \in \{g,h\} \right\} .$

1.2 Details on the efficient computation of $\Delta _{c,c'}$

Fig. 11

Geographical representation of the clustering of the foodwebs obtained with Mukherjee’s method

Here it is shown how to evaluate $\Delta _{c,c'}$ efficiently. Denote ${\mathcal {U}}_{c\cup c'}$ the cluster labels afte merging clusters c and $c'$, that is, $U_{c\cup c'}^{(m)}=\min \{c,c'\}$ if $m\in I_c\cup I_{c'}$ and $U_{c\cup c'}^{(m)}=U^{(m)}$ otherwise. Likewise, denote ${\mathcal {Z}}_{c\cup c'}$ the node labels after aggregation and relabeling with ${\mathcal {Z}}^{(\ell )}_{c\cup c'} =\{{\hat{\sigma }}_{\ell }({\textbf{Z}}^{(j)}), j \in I_{\ell }\}$ for $\ell \in \{c,c'\}$, where ${\hat{\sigma }}_{\ell }$ are the permutations that match the block labels. For convenience, denote by $\beta (x,y) = \log \left( \frac{\Gamma (x)\Gamma (y)}{\Gamma (x+y)}\right)$ the logarithm of the Beta function of x and y. Moreover, for any $c\in \llbracket C\rrbracket , (k,l)\in \llbracket K_c\rrbracket$, denote

$$\begin{aligned} {\textbf{s}}^{(c)}_{k}= \sum _{m\in I_{c}} s_{k}^{(m)},\quad {\textbf{a}}^{(c)}_{k,l}= \sum _{m\in I_{c}} a_{k,l}^{(m)},\quad {\textbf{b}}^{(c)}_{k,l}= \sum _{m\in I_{c}} b_{k,l}^{(m)}. \end{aligned}$$

Then $\Delta _{c,c'}=\textrm{ICL}^{\text {mix}}({\mathcal {A}}, {\mathcal {U}}_{c\cup c'}, {\mathcal {Z}}_{c\cup c'})- \textrm{ICL}^{\text {mix}}({\mathcal {A}}, {\mathcal {U}}, {\mathcal {Z}})$ is given by

$$\begin{aligned}&\Delta _{c,c'} = \sum _{(k,\ell ) } \beta \left( \eta +{\textbf{a}}^{(c)}_{{\hat{\sigma }}_{c}^{-1}(k),{\hat{\sigma }}_{c}^{-1}(l)}+ {\textbf{a}}^{(c')}_{{\hat{\sigma }}_{c'}^{-1}(k),{\hat{\sigma }}_{c'}^{-1}(l)}\right. \nonumber \\&\quad \quad \left. + {\textbf{b}}^{(c)}_{{\hat{\sigma }}_{c}^{-1}(k),{\hat{\sigma }}_{c}^{-1}(l)}+ {\textbf{b}}^{(c')}_{{\hat{\sigma }}_{c'}^{-1}(k), {\hat{\sigma }}_{c'}^{-1}(l)}\right) \nonumber \\&\quad \quad - \sum _{(k,\ell ) } \beta \left( \eta +{\textbf{a}}^{(c)}_{k,l}, \zeta +{\textbf{b}}^{(c)}_{k,l}\right) - \sum _{(k,\ell ) }\beta \left( \eta + {\textbf{a}}^{(c')}_{k,l}, \zeta +{\textbf{b}}^{(c')}_{k,l}\right) \nonumber \\&\quad \quad +\sum _{k} \log \left( \Gamma (\alpha +{\textbf{s}}_{{\hat{\sigma }}_{c}^{-1}(k)}^{(c)}+{\textbf{s}}_{{\hat{\sigma }}_{c'}^{-1}(k)}^{(c')})\right) - \log \left( \Gamma (\alpha +{\textbf{s}}_{k}^{(c)})\right) \nonumber \\&\qquad -\log \left( \Gamma (\alpha +{\textbf{s}}_{k}^{(c')})\right) +\log \left( \frac{\Gamma (\lambda +|I_{c}|+|I_{c'}|)}{\Gamma (\lambda + |I_{c}|)\Gamma (\lambda +|I_{c'}|)}\right) . \nonumber \\&\quad \quad +\log \left( \frac{\Gamma (K_{c}\alpha + \sum _{m\in I_{c}}n^{(m)})\Gamma (K_{c'}\alpha + \sum _{m\in I_{c'}}n^{(m)})}{\Gamma \left( K_{\max }\alpha + \sum _{m\in I_{c}\cup I_{c'}}n^{(m)}\right) }\right) \nonumber \\&\quad \quad +K_{\min }^2 \beta (\eta ,\zeta ) +K_{\min }\log \left( \Gamma (\alpha )\right) \nonumber \\&\quad \quad + \beta \left( (C-1)\lambda , \lambda \right) + \log \left( \frac{\Gamma (C\lambda +M)}{\Gamma ((C-1)\lambda +M)}\right) , \end{aligned}$$

(7)

where $K_{\max }=\max \{K_{c},K_{c'}\}$ and $K_{\min }=\min \{K_{c},K_{c'}\}$ are the maximal and minimal number of blocks in the clusters c and $c'$.

1.3 Supplement to the analysis of ecological networks

Figure 11 illustrates the clustering of the foodwebs obtained with the alternative graph moments method by Mukherjee et al. (2017). The obtained clustering is virtually very different from the one obtained by our graph clustering procedure.