Mixed membership distribution-free model

Abstract

We consider the problem of community detection in overlapping weighted networks, where nodes can belong to multiple communities and edge weights can be finite real numbers. To model such complex networks, we propose a general framework—the mixed membership distribution-free (MMDF) model. MMDF has no distribution constraints of edge weights and can be viewed as generalizations of some previous models, including the well-known mixed membership stochastic blockmodels. Especially, overlapping signed networks with latent community structures can also be generated from our model. We use an efficient spectral algorithm with a theoretical guarantee of convergence rate to estimate community memberships under the model. We also propose the fuzzy weighted modularity to evaluate the quality of community detection for overlapping weighted networks with positive and negative edge weights. We then provide a method to determine the number of communities for weighted networks by taking advantage of our fuzzy weighted modularity. Numerical simulations and real data applications are carried out to demonstrate the usefulness of our mixed membership distribution-free model and our fuzzy weighted modularity.

Appendices

Appendix A Vertex hunting algorithm

Algorithm 2 is the SP algorithm.

Algorithm 2

Successive Projection (SP) [46]

Appendix B Proofs under MMDF

1.1 B.1 Proof of Proposition 1

Proof

This proposition holds immediately by the first statement of Theorem 2.1 [21] since we let P be a full rank matrix and Theorem 2.1 [21] is a distribution-free result such that it always holds without constraining the distribution of A. $\square$

1.2 B.2 Proof of Lemma 1

Proof

Since $\Omega =\Pi \rho P\Pi '=U\Lambda U'$ and $U'U=I_{K}$, we have $U=\Pi \rho P\Pi 'U\Lambda ^{-1}$, i.e., $B=\rho P\Pi ' U\Lambda ^{-1}$. So B is unique. Since $U=\Pi B$, we have $U(\mathcal {I},:)=\Pi (\mathcal {I},:)B=B$ and the lemma follows. $\square$

1.3 B.3 Proof of Theorem 1

Proof

First, we prove the following lemma to provide an upper bound of row-wise eigenspace error $\Vert \hat{U}\hat{U}'-UU'\Vert _{2\rightarrow \infty }$. $\square$

Lemma 2

(Row-wise eigenspace error) Under $MMDF_{n}(K,P,\Pi ,\rho ,\mathcal {F})$, when Assumption 1 holds, suppose $\sigma _{K}(\Omega )\ge C\sqrt{\gamma \rho n\textrm{log}(n)}$ for some $C>0$, with probability at least $1-o(n^{-3})$, we have

$$\begin{aligned} \Vert \hat{U}\hat{U}'-UU'\Vert _{2\rightarrow \infty }=O(\frac{\sqrt{\gamma n\textrm{log}(n)}}{\sigma _{K}(P)\rho ^{0.5} \lambda ^{1.5}_{K}(\Pi '\Pi )}). \end{aligned}$$

Proof

First, we use Theorem 1.4 (the Matrix Bernstein) of [67] to build an upper bound of $\Vert A-\Omega \Vert _{\infty }$. This theorem is given below $\square$

Theorem 2

Consider a finite sequence $\{X_{k}\}$ of independent, random, self-adjoint matrices with dimension d. Assume that each random matrix satisfies

$$\begin{aligned} \mathbb {E}[X_{k}]=0, \mathrm {and~}\Vert X_{k}\Vert \le R~\mathrm {almost~surely}. \end{aligned}$$

Then, for all $t\ge 0$,

$$\begin{aligned} \mathbb {P}(\Vert \sum _{k}X_{k}\Vert \ge t)\le d\cdot \textrm{exp}(\frac{-t^{2}/2}{\sigma ^{2}+Rt/3}), \end{aligned}$$

where $\sigma ^{2}:=\Vert \sum _{k}\mathbb {E}(X^{2}_{k})\Vert$.

Let $x=(x_{1},x_{2},\ldots , x_{n})'$ be any $n\times 1$ vector. For any $i,j\in [n]$, we have $\mathbb {E}[(A(i,j)-\Omega (i,j))x(j)]=0$ and $\Vert (A(i,j)-\Omega (i,j))x(j)\Vert \le \tau \Vert x\Vert _{\infty }$. Set $R=\tau \Vert x\Vert _{\infty }$. Since $\Vert \sum _{j=1}^{n}\mathbb {E}[(A(i,j)-\Omega (i,j))^{2}x^{2}(j)]\Vert =\Vert \sum _{j=1}^{n}x^{2}(j)\mathbb {E}[(A(i,j)-\Omega (i,j))^{2}]\Vert =\Vert \sum _{j=1}^{n}x^{2}(j)\textrm{Var}(A(i,j))\Vert \le \gamma \rho \sum _{j=1}^{n}x^{2}(j)$, by Theorem 2, for any $t\ge 0$ and $i\in [n]$, we have

$$\begin{aligned} \mathbb {P}(|\sum _{j=1}^{n}(A(i,j)-\Omega (i,j))x(j)|>t)\le 2\textrm{exp}(-\frac{t^{2}/2}{\gamma \rho \sum _{j=1}^{n}x^{2}(j)+\frac{Rt}{3}}). \end{aligned}$$

Set x(j) as 1 or $-1$ such that $(A(i,j)-\Omega (i,j))y(j)=|A(i,j)-\Omega (i,j)|$, we have

$$\begin{aligned} \mathbb {P}(\Vert A-\Omega \Vert _{\infty }>t)\le 2\textrm{exp}(-\frac{t^{2}/2}{\gamma \rho n+\frac{Rt}{3}}). \end{aligned}$$

Set $t=\frac{\alpha +1+\sqrt{(\alpha +1)(\alpha +19)}}{3}\sqrt{\gamma \rho n\textrm{log}(n)}$ for any $\alpha >0$. By assumption 1, we have

$$\begin{aligned} \mathbb {P}(\Vert A-\Omega \Vert _{\infty }>t)\le 2\textrm{exp}(-\frac{t^{2}/2}{\gamma \rho n+\frac{Rt}{3}})\le n^{-\alpha }. \end{aligned}$$

By Theorem 4.2 of [68], when $\sigma _{K}(\Omega )\ge 4\Vert A-\Omega \Vert _{\infty }$, we have

$$\begin{aligned} \Vert \hat{U}-U\mathcal {O}\Vert _{2\rightarrow \infty }\le 14\frac{\Vert A-\Omega \Vert _{\infty }}{\sigma _{K}(\Omega )}\Vert U\Vert _{2\rightarrow \infty }, \end{aligned}$$

where $\mathcal {O}$ is a $K\times K$ orthogonal matrix. With probability at least $1-o(n^{-\alpha })$, we have

$$\begin{aligned} \Vert \hat{U}-U\mathcal {O}\Vert _{2\rightarrow \infty }=O(\frac{\Vert U\Vert _{2\rightarrow \infty }\sqrt{\gamma \rho n\textrm{log}(n)}}{\sigma _{K}(\Omega )}). \end{aligned}$$

Since $\hat{U}'\hat{U}=I_{K},U'U=I_{K}$, by basic algebra, we have $\Vert \hat{U}\hat{U}'-UU'\Vert _{2\rightarrow \infty }\le 2\Vert \hat{U}-U\mathcal {O}\Vert _{2\rightarrow \infty }$, which gives

$$\begin{aligned} \Vert \hat{U}\hat{U}'-UU'\Vert _{2\rightarrow \infty }=O(\frac{\Vert U\Vert _{2\rightarrow \infty }\sqrt{\gamma \rho n\textrm{log}(n)}}{\sigma _{K}(\Omega )}). \end{aligned}$$

Since $\sigma _{K}(\Omega )\ge \sigma _{K}(P)\rho \lambda _{K}(\Pi '\Pi )$ by Lemma II.4 of [21] and $\Vert U\Vert ^{2}_{2\rightarrow \infty }\le \frac{1}{\lambda _{K}(\Pi '\Pi )}$ by Lemma 3.1 of [21], where these two lemmas are distribution-free and always hold as long as Eqs. (2), (4), and (5) hold, we have

$$\begin{aligned} \Vert \hat{U}\hat{U}'-UU'\Vert _{2\rightarrow \infty }=O(\frac{\sqrt{\gamma n\textrm{log}(n)}}{\sigma _{K}(P)\rho ^{0.5} \lambda ^{1.5}_{K}(\Pi '\Pi )}). \end{aligned}$$

Set $\alpha =3$, and this claim follows.

Remark 6

Alternatively, Theorem 4.2. of [69] can also be applied to obtain the upper bound of $\Vert \hat{U}\hat{U}'-UU'\Vert _{2\rightarrow \infty }$, and this bound is similar to the one in Lemma 2.

For convenience, set $\varpi =\Vert \hat{U}\hat{U}'-UU'\Vert _{2\rightarrow \infty }$. Since DFSP is the SPACL algorithm without the prune step of [21], the proof of DFSP’s consistency is the same as SPACL except for the row-wise eigenspace error step where we need to consider $\gamma$ which is directly related with distribution $\mathcal {F}$. By Lemma 2 and Equation (3) in Theorem 3.2 of [21] where the proof is distribution-free, there exists a $K\times K$ permutation matrix $\mathcal {P}$ such that

$$\begin{aligned} \textrm{max}_{i\in [n]}\Vert e'_{i}({\hat{\Pi }}-\Pi \mathcal {P})\Vert _{1}=O(\varpi \kappa (\Pi '\Pi )\sqrt{\lambda _{1}(\Pi '\Pi )})=O(\frac{\kappa ^{1.5}(\Pi '\Pi )\sqrt{\gamma n\textrm{log}(n)}}{\sigma _{K}(P)\rho ^{0.5} \lambda _{K}(\Pi '\Pi )}). \end{aligned}$$

1.4 B.4 Proof of Corollary 1

Proof

When $\lambda _{K}(\Pi '\Pi )=O(\frac{n}{K})$ and $K=O(1)$, we have $\kappa (\Pi '\Pi )=O(1)$ and $\lambda _{K}(\Pi '\Pi )=O(n/K)=O(n)$. Then, the corollary follows immediately by Theorem 1. $\square$

Appendix C Extra simulation results

In this part, we consider two extra simulations: imbalanced networks and running time. For imbalanced networks, we study the stability of DFSP and its competitors when there are small-size communities. For running time, we compare the running time for each method by increasing the network size n. For simplicity, we only consider the case when $\mathcal {F}$ is Normal distribution here. When $A(i,j)\sim \textrm{Normal}(\Omega (i,j),\sigma ^{2}_{A})$, let all nodes be pure, $K=2, \rho =1$, and $\sigma ^{2}_{A}=1$. Set P as

$$\begin{aligned}P=\begin{bmatrix} 1&{}-0.2\\ -0.2&{}0.9\\ \end{bmatrix}.\end{aligned}$$

Let the first community has $\delta n$ nodes. So, the second community has $(1-\delta )n$ nodes. Based on the above settings, we consider the following two simulations.

Changing $\delta$: Let $n=200$ or $n=1000$. Let $\delta$ range in $\{0.025, 0.05, 0.075, \ldots , 0.5\}$. For this case, the two evaluation metrics Hamming error and Relative are not suitable for imbalanced networks. To prioritize the ability of DFSP and its competitors to detect the minority communities, we consider the following two metrics who are the smaller the better.

$$\begin{aligned}&\textrm{Clustering}~l_{1}~\textrm{error}=\textrm{min}_{\mathcal {P}\in \{ K\times K\mathrm {~permutation~matrix}\}}\textrm{max}_{k\in [K]}\frac{\Vert {\hat{\Pi }}(:,k)-(\Pi \mathcal {P})(:,k)\Vert _{1}}{\Vert (\Pi \mathcal {P})(:,k)\Vert _{1}},\\&\textrm{Clustering}~l_{2}~\textrm{error}=\textrm{min}_{\mathcal {P}\in \{K\times K\mathrm {~permutation~matrix}\}}\textrm{max}_{k\in [K]}\frac{\Vert {\hat{\Pi }}(:,k)-(\Pi \mathcal {P})(:,k)\Vert _{F}}{\Vert (\Pi \mathcal {P})(:,k)\Vert _{F}}. \end{aligned}$$

Unlike Hamming error which measures the $l_{1}$ difference between $\Pi$ and ${\hat{\Pi }}$ up to a permutation of community labels, Clustering $l_{1}$ error measures the maximum $l_{1}$ difference between the size of the true k-th community and the size of the estimated k-th community up to a permutation of community labels among all K communities. Therefore, Clustering $l_{1}$ error can evaluate the ability of a community detection method to detect the minority communities. Similar arguments hold for the Clustering $l_{2}$ error.

Panels (a–f) of Fig. 5 display numerical results for changing $\delta$. For the case when $n=200$, we find that DFSP and its competitors perform similarly and all of them can successfully detect the minority community when $\delta \in [0.125,0.5]$, i.e., the proportion of community sizes between the largest community and the smallest community locates in [1, 7]. KDFSP successfully estimates the number of communities K when $\delta \in [0.15,0.5]$ while NB and BHac fail to infer K for all cases. For the case when $n=1000$, DFSP and its competitors successfully detect all communities when $\delta \in [0.1, 0.5]$, i.e., the proportion of community sizes between the largest community and the smallest community locates in [1, 9]. KDFSP correctly determines K when $\delta \in [0.05,0.5]$ while its competitors fail to find K.

Changing n: Let $\delta =0.075$ or $\delta =0.1$, i.e., let the proportion of community sizes between the largest community and the smallest community be $\frac{37}{3}$ or 9. Let n range in $\{2000,4000,6000,\ldots ,12000\}$. For simplicity, we only report the averaged Clustering $l_{1}$ error, averaged Clustering $l_{2}$ error, and averaged running time over 100 repetitions for DFSP and its competitors. Figure 6 displays the numerical results. We see that DFSP is better than GeoNMF, SVM-cD, and OCCAM in both estimation accuracy and running time. In particular, DFSP runs much faster than OCCAM. Meanwhile, DFSP performs satisfactorily for its small clustering errors for the two cases $\delta =0.075$ and $\delta =0.1$. By comparing panel (a) and panel (e) (panel (b) and panel (f)), we see that all methods perform poorer for a more imbalanced network and this result is consistent with that of changing $\delta$. By comparing panel (c) and panel (g) (panel (d) and panel (h)), we see that each method takes more time to detect a more imbalanced network.

Fig. 5

Numerical results of changing $\delta$

Fig. 6

Numerical results of changing n