Variational Bayes model averaging for graphon functions and motif frequencies inference in W-graph models

Abstract

W-graph refers to a general class of random graph models that can be seen as a random graph limit. It is characterized by both its graphon function and its motif frequencies. In this paper, relying on an existing variational Bayes algorithm for the stochastic block models (SBMs) along with the corresponding weights for model averaging, we derive an estimate of the graphon function as an average of SBMs with increasing number of blocks. In the same framework, we derive the variational posterior frequency of any motif. A simulation study and an illustration on a social network complete our work.

Appendix

1.1 Inference of the function W

Proof of Proposition 1

The first part is straightforward, based on a conditioning of the binnings of u and v

$$\begin{aligned} \widetilde{p}(w(u,\,v)|\mathbf{X},\,Q)= & {} \widetilde{p}\left( \pi _{C(u),\,C(v)}|\mathbf{X},\,Q\right) \\= & {} \sum _{q \le \ell } \widetilde{p}\left( \pi _{q, \ell } | \mathbf{X},\, Q,\, C(u) = q,\, C(v) = \ell \right) \\&\quad \widetilde{\Pr }\{C(u) = q,\, C(v) = \ell | \mathbf{X},\,Q\} \\= & {} \sum _{q \le \ell } b\left( w;\, \eta _{q, \ell },\, \zeta _{q, \ell }\right) \\&\quad \widetilde{\Pr }\{C(u) = q,\,C(v) = \ell | \mathbf{X},\, Q\}. \end{aligned}$$

We are now left with the calculation of

$$\begin{aligned} \widetilde{\Pr }\{C(u)= & {} q,\,C(v) = \ell | \mathbf{X},\, Q\} = \widetilde{\Pr }\left\{ \sigma _{q-1} < u < \sigma _q,\, \sigma _{\ell -1}\right. \\< & {} \left. v < \sigma _\ell | \mathbf{X},\, Q\right\} \\= & {} F_{{q-1}, {\ell -1}}(u,\, v;\, \mathbf{a})- F_{{q}, {\ell -1}}(u,\, v;\, \mathbf{a})\\&\quad - F_{{q-1}, {\ell }}(u,\, v;\, \mathbf{a}) \\&\quad + F_{{q}, {\ell }}(u,\, v;\, \mathbf{a}), \end{aligned}$$

where

• $\mathbf{a},\,\varvec{\eta }$ and $\varvec{\zeta }$ are the parameters of the variational Bayes posterior distributions;

• $b(\cdot ;\,\varvec{\eta },\,\varvec{\zeta })$ stands for the pdf of the Beta distribution $\text{ Beta }(\varvec{\eta },\, \varvec{\zeta });$

• $F_{q, \ell }(u,\,v;\,\mathbf{a})$ denotes the joint cdf of $(\sigma _q,\, \sigma _\ell ),$ as defined in (1), when ${\varvec{\alpha }}$ has a Dirichlet distribution $\text{ Dir }(\mathbf{a}).$

The last argument comes from Gouda and Szántai (2010) who give explicit recursions to compute the uni- and bi-variate cdf for the Dirichlet $\text{ Dir }(\mathbf{a}),$ denoted $G_{q}(u;\,\mathbf{a})$ and $G_{q, \ell }(u,\, v;\, \mathbf{a}),$ respectively.

Reminding that the approximate variational posterior of ${\varvec{\alpha }}$ is $\text{ Dir }(\mathbf{a})$ and using a simple property of the Dirichlet distribution

$$\begin{aligned} ({\varvec{\alpha }})\sim & {} \text{ Dir }(\mathbf{a}) \;\Rightarrow \;\left( \sum _{j=1}^q \alpha _j,\,\sum _{j=q+1}^\ell \alpha _j,\,\sum _{j=\ell +1}^Q\alpha _j\right) \\\sim & {} \text{ Dir }\left( \sum _{j=1}^q a_j,\,\sum _{j=q+1}^\ell a_j,\, \sum _{j=\ell +1}^Q a_j\right) , \end{aligned}$$

the calculation of $F_{q, \ell }(u,\,v)$ follows as

$$\begin{aligned} F_{q, \ell }(u,\,v)= & {} \widetilde{\Pr }\left\{ \sigma _q < u,\,\sigma _\ell < v | \mathbf{X},\,Q\right\} \\= & {} \widetilde{\Pr }\left\{ \sigma _q < u,\, 1 - \sigma _\ell > 1 - v | \mathbf{X},\, Q\right\} \\= & {} \widetilde{\Pr }\left\{ \sigma _q < u | \mathbf{X},\,Q\right\} \\&- \Pr \left\{ \sigma _q < u,\, \sigma _\ell < 1 - v | \mathbf{X},\,Q\right\} \\= & {} G_1\left( u;\,\left[ s_q,\,s_\ell -s_q,\,s_Q-s_\ell \right] \right) \\&- G_{1, 3}\left( u,\, 1-v;\,\left[ s_q,\,s_\ell -s_q,\,s_Q-s_\ell \right] \right) , \end{aligned}$$

where the $(s_q)$ are the cumulated parameters: $s_q = \sum _{j=1}^q a_j.$ $\square$

1.2 Motif probability

Proof of Proposition 3

We directly write the approximate variational expectation

$$\begin{aligned} \begin{aligned} \widetilde{{\mathbb {E}}}[\mu (\mathbf{m}) | \mathbf{X},\,Q]&= \int \int {\mathbb {E}}[\mu (\mathbf{m})|{\varvec{\alpha }},\,{\varvec{\pi }}] \widetilde{p}({\varvec{\alpha }},\, {\varvec{\pi }}|{\mathbf{X},\,Q})\text{ d }{\varvec{\alpha }}\text{ d }{\varvec{\pi }}\\&= \int \int \left\{ \sum _{\mathbf{c}} {\mathbb {E}}[\mu (\mathbf{m}) |\mathbf{c},\,{\varvec{\pi }}] p(\mathbf{c}|{\varvec{\alpha }})\right\} \\&\quad \, \widetilde{p}({\varvec{\alpha }},\, {\varvec{\pi }}|{\mathbf{X},\,Q})\text{ d }{\varvec{\alpha }}\text{ d }{\varvec{\pi }}, \end{aligned} \end{aligned}$$

where

$$\begin{aligned}&p(\mathbf{c}|{\varvec{\alpha }}) = \prod _{1 \le a \le k} p\left( c_a|{\varvec{\alpha }}\right) \\&\quad = \prod _{1 \le a \le k}\prod _{1 \le q \le Q}\alpha _{q}^{{\mathbb {I}}\{c_a = q\}} = \prod _{1 \le q \le Q} \alpha _{q}^{n_{q}^{\mathbf{c}}}. \end{aligned}$$

Furthermore, we have

$$\begin{aligned} \begin{aligned} {\mathbb {E}}[\mu (\mathbf{m}) |\mathbf{c},\,{\varvec{\pi }}]&= \Pr \left\{ \prod _{1 \le a<b \le k}X_{{a}{b}}^{m_{ab}} = 1| \mathbf{c},\,{\varvec{\pi }}\right\} \\&= \prod _{1 \le a<b \le k}\Pr \left\{ X_{{a}{b}} = 1 | c_a,\, c_b,\, {\varvec{\pi }}\right\} ^{m_{ab}} \\&= \prod _{1 \le a<b \le k}\prod _{1 \le q ,\,\ell \le Q} \pi _{q\ell }^{{\mathbb {I}}\{c_a=q\}{\mathbb {I}}\{c_b=\ell \}m_{ab}} \\&= \prod _{1 \le q < \ell \le Q}\prod _{a \ne b}\pi _{q\ell }^{{\mathbb {I}}\{c_a=q\}{\mathbb {I}}\{c_b=\ell \}m_{ab}}\\&\quad \prod _{1 \le q \le Q}\prod _{1 \le a<b \le k}\pi _{qq}^{{\mathbb {I}}\{c_a=q\}{\mathbb {I}}\{c_b=q\}m_{ab}}\\&= \prod _{1 \le q \le \ell \le Q} \pi _{q\ell }^{\eta _{q\ell }^{\mathbf{c}}}, \end{aligned} \end{aligned}$$

so we end up with

$$\begin{aligned} \widetilde{{\mathbb {E}}}[\mu (\mathbf{m}) | \mathbf{X},\,Q]= & {} \int \int \sum _{\mathbf{c}} \prod _{1 \le q \le \ell \le Q} \pi _{q\ell }^{\eta _{q\ell }^{\mathbf{c}}} \prod _{1 \le q \le Q}\alpha _{q}^{n_{q}^{\mathbf{c}}}\\&\quad \widetilde{p}({\varvec{\alpha }},{\varvec{\pi }}| {Q})\,\text{ d }{\varvec{\alpha }}\,\text{ d }{\varvec{\pi }}\\= & {} \int \int \sum _{\mathbf{c}}\prod _{1 \le q \le \ell \le Q} \pi _{q\ell }^{\eta _{q\ell }^{\mathbf{c}}} \prod _{1 \le q \le Q} \alpha _{q}^{n_{q}^{\mathbf{c}}}\\&\quad \prod _{1 \le q \le \ell \le Q} \frac{\Gamma (\eta _{q\ell }+\zeta _{q\ell })}{\Gamma (\eta _{q\ell })\Gamma (\zeta _{q\ell })}\pi _{q\ell }^{\eta _{q\ell }-1}\\&\times \left( 1-\pi _{q\ell }\right) ^{\zeta _{q\ell }-1}\\&\quad \frac{\Gamma \left( \sum _{1 \le q \le Q}n_{q}\right) }{\prod _{1 \le q \le Q}\Gamma (n_{q})}\prod _{1 \le q \le Q}\alpha _{q}^{n_{q}-1}\text{ d }{\varvec{\alpha }}\text{ d }{\varvec{\pi }}\\= & {} \sum _{\mathbf{c}} \prod _{1 \le q \le \ell \le Q} \frac{\Gamma (\eta _{q\ell }+\zeta _{q\ell })}{\Gamma (\eta _{q\ell })\Gamma (\zeta _{q\ell })}\int \pi _{q\ell }^{\eta _{q\ell } + n_{q\ell }^{\mathbf{c}}-1}\\&\quad \left( 1-\pi _{q\ell }\right) ^{\zeta _{q\ell }-1} \text{ d }\pi _{q\ell } \\&\quad \frac{\Gamma \left( \sum _{1 \le q \le Q}n_{q}\right) }{\prod _{1 \le q \le Q}\Gamma (n_{q})}\prod _{1 \le q \le Q}\int \alpha _{q}^{n_{q}+n_{q}^{\mathbf{c}}-1}\text{ d }{\varvec{\alpha }}_{q} \\= & {} \sum _{\mathbf{c}} \prod _{1 \le q \le \ell \le Q} \frac{\Gamma (\eta _{q\ell }+\zeta _{q\ell })}{\Gamma (\eta _{q\ell })\Gamma (\zeta _{q\ell })}\\&\quad \frac{\Gamma (\eta _{q\ell }+\eta _{q\ell }^{\mathbf{c}})\Gamma (\zeta _{q\ell })}{\Gamma (\eta _{q\ell }+\eta _{q\ell }^{\mathbf{c}}+\zeta _{q\ell })}\\&\quad \frac{\Gamma \left( \sum _{1 \le q \le Q}n_{q}\right) }{\prod _{1 \le q \le Q}\Gamma (n_{q})}\frac{\prod _{1 \le q \le Q}\Gamma (n_{q}+n_{q}^{\mathbf{c}})}{\Gamma \sum _{1 \le q \le Q} (n_{q}+n_{q}^{\mathbf{c}}) }, \end{aligned}$$

and the proof is completed (Fig. 8). $\square$

Proof of Proposition 2

Because the $Z_i$’s are uniformly distributed over $[0;\,1],$ we have

$\square$