Mining Multiplex Structural Patterns from Complex Networks

Abstract

Wisdom Web of Things (W2T) can be modeled and studied from the perspective of complex networks. The complex network perspective aims to model and characterize complex systems that consist of multiple and interdependent components. Among the studies on complex networks, topological structure analysis is of the most fundamental importance, as it represents a natural route to understand the dynamics, as well as to synthesize or optimize the functions, of networks. A broad spectrum of network structural patterns have been respectively reported in the past decade, such as communities, multipartites, hubs, authorities, outliers, bow ties, and others. In this chapter, we show that many real-world networks demonstrate multiplex structure patterns. A multitude of known or even unknown (hidden) patterns can simultaneously exist in the same network, and moreover they may be overlapped and nested with each other to collaboratively form a heterogeneous, nested or hierarchical organization, in which different connective phenomena can be observed at different granular levels. In addition, we show that such patterns hidden in exploratory networks can be well defined as well as effectively recognized within an unified framework consisting of a set of proposed concepts, models, and algorithms. Our findings provide a strong evidence that many real-world complex systems are driven by a combination of heterogeneous mechanisms that may collaboratively shape their ubiquitous multiplex structures as we currently observe. This work also contributes a mathematical tool for analyzing different sources of networks from a new perspective of unveiling multiplex structure patterns, which will be beneficial to Wisdom Web of Things.

Appendix

Proposition 1

For an indirected network, its feedforward-coupling matrix is equal to its feedback-coupling matrix, i.e., we have: $P=Q$.

Proof

$p_{ij}=P(i\rightarrow j|y=k)$

$=\sum _{k=1}^K m_{ik}P(i\rightarrow j|y=k)$

$=\sum _{k=1}^K (\sum _{l=1}^L b_{il}z_{lk})P(i\rightarrow j|y=k)$

$= \sum _{l=1}^L \sum _{k=1}^K b_{il}z_{lk}\theta _{kj}$

where $i\rightarrow j$ denote the event that node $v_i$ couples with $v_j$, and $y=k$ denote the event that $v_i$ is labeled by cluster k; $m_{ik}=1$ if $v_i$ is labeled by cluster k, otherwise $m_{ik}=0$. So we have:

$P=B_gZ\varTheta .$

$q_{ij}=P(i\dashleftarrow j|y=k)$

$=\sum _{k=1}^K m_{ik}P(i\dashleftarrow j|y=k)$

$=\sum _{k=1}^K (\sum _{l=1}^L b_{il}z_{lk})P(i\dashleftarrow j|y=k)$

$= \sum _{l=1}^L \sum _{k=1}^K b_{il}z_{lk}\delta _{kj}$

where $i\dashleftarrow j$ denote the event that node $v_i$ except to be coupled by $v_j$. So we have:

$Q=B_gZ\varDelta$.

If A is symmetry, from the Eq. 12.4 in the chapter, we have

$\theta _{kj}=\frac{\sum _{l=1}^L\sum _{b_{il}\ne 0} a_{ij}\gamma _{lk}}{\sum _{l=1}^L\sum _{b_{il}\ne 0}\gamma _{lk}}= \frac{\sum _{l=1}^L\sum _{b_{il}\ne 0} a_{ji}\gamma _{lk}}{\sum _{l=1}^L\sum _{b_{il}\ne 0}\gamma _{lk}}=\delta _{kj}$.

So we have $P=Q$.$\square$

Proposition 2

$$\begin{aligned} L(N|X,B_g)=\sum _{l=1}^L\sum _{b_{il}\ne 0}\ln \sum _{k=1}^K \varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji})\omega _k \end{aligned}$$

(12.10)

where $f(x,y)=x^y(1-x)^{1-y}$.

Proof

Let $v=i$ denote the event that a node with linkage structure $<a_{i1},\cdots ,a_{in},a_{1i},\cdots ,a_{ni}>$ will be observed in network N. Let $y=k$ denote the event that the cluster label assigned to a node is equal to k. Let $i\rightarrow _{a_{ij}} j$ denote the event that node $v_i$ link to node $v_j$ or not depending on $a_{ij}$. Let $i\leftarrow _{a_{ji}} j$ denote the event that node $v_i$ will be linked by node $v_j$ or not depending on $a_{ji}$. We have:

$L(N|X,B_g)=\ln \varPi _{i=1}^nP(v=i)=\sum _{i=1}^n\ln P(v=i)$

$=\sum _{i=1}^n\ln P((v=i) \cap (\cup _{k=1}^Ky=k))$

$=\sum _{i=1}^n\ln \sum _{k=1}^K P(v=i,y=k)$

$=\sum _{i=1}^n\ln \sum _{k=1}^K (P(v=i|y=k)P(y=k))$

$=\sum _{i=1}^n\ln \sum _{k=1}^K (P(<a_{i1},\cdots ,a_{in},a_{1i},\cdots ,a_{ni}>|y=k)P(y=k))$

$=\sum _{i=1}^n\ln \sum _{k=1}^K (\varPi _{j=1}^n P(i\rightarrow _{a_{ij}} j|y=k)P(i\leftarrow _{a_{ji}} j|y=k)P(y=k))$

$=\sum _{i=1}^n\ln \sum _{k=1}^K (\varPi _{j=1}^n (\theta _{kj}^{a_{ij}}(1-\theta _{kj})^{1-a_{ij}})(\delta _{kj}^{a_{ji}}(1-\delta _{kj})^{1-a_{ji}})\omega _k)$

$=\sum _{i=1}^n\ln \sum _{k=1}^K (\varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji})\omega _k)$

$=\sum _{l=1}^L\sum _{b_{il}\ne 0}\ln \sum _{k=1}^K (\varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji})\omega _k)$ $\square$

Proposition 3

$$\begin{aligned} L(N,Z|X,B_g)= \sum _{l=1}^L\sum _{b_{il}\ne 0}\sum _{k=1}^K z_{lk}(\sum _{j=1}^n (\ln f(\theta _{kj},a_{ij}) \nonumber \\ +\ln f(\delta _{kj},a_{ji}))+\ln \omega _k) \end{aligned}$$

(12.11)

Proof

Let y(i) denote the cluster label assigned to node i under the given partition Z, we have:

$L(N,Z|X,B_g)$

$=\ln \varPi _{i=1}^n P(v=i,y=y(i))$

$=\sum _{i=1}^n \ln \sum _{k=1}^K m_{ik}P(v=i,y=k)$

$=\sum _{i=1}^n\ln \sum _{k=1}^K m_{ik}P(v=i|y=k)P(y=k)$

$=\sum _{i=1}^n\sum _{k=1}^K \ln (P(v=i|y=k)P(y=k))^{m_{ik}}$

$=\sum _{i=1}^n\sum _{k=1}^K m_{ik}\ln (P(v=i|y=k)P(y=k))$

$=\sum _{i=1}^n \sum _{k=1}^K m_{ik}\ln (\varPi _{j=1}^n (\theta _{kj}^{a_{ij}}(1-\theta _{kj}^{1-a_{ij}})\delta _{kj}^{a_{ji}}(1-\delta _{kj}^{1-a_{ji}}))\omega _k$

$=\sum _{i=1}^n\sum _{k=1}^K m_{ik}(\sum _{j=1}^n(\ln f(\theta _{kj},a_{ij})+ \ln f(\delta _{kj},a_{ji}))+\ln \omega _k)$

$=\sum _{b_{i1\ne 0}}\sum _{k=1}^K z_{1k}(\sum _{j=1}^n(\ln f(\theta _{kj},a_{ij})+ \ln f(\delta _{kj},a_{ji}))+\ln \omega _k)+\cdots$

$+\sum _{b_{iL\ne 0}}\sum _{k=1}^K z_{Lk}(\sum _{j=1}^n(\ln f(\theta _{kj},a_{ij})+ \ln f(\delta _{kj},a_{ji}))+\ln \omega _k)$

$=\sum _{l=1}^L\sum _{b_{il}\ne 0}\sum _{k=1}^K z_{lk}(\sum _{j=1}^n(\ln f(\theta _{kj},a_{ij})+ \ln f(\delta _{kj},a_{ji}))+\ln \omega _k)$.$\square$

Notice that, in the proofs of Propositions 2–3, all probabilities such as $P(y=k|v=i)$ and $P(y=k)$ are discussed under the conditions of X and $B_g$. To simplify the equations, we omit them without losing correctness.

Proposition 4

$$\begin{aligned} \gamma _{lk}=\frac{1}{\sum _{i=1}^n b_{il}}\sum _{b_{il}\ne 0}\frac{\varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji}) \omega _k}{\sum _{k=1}^K \varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji}) \omega _k} \end{aligned}$$

(12.12)

Proof

let $P(y=k|v=i)$ be the probability that node i belongs to cluster k given X and $B_g$. We have:

$\gamma _{lk}=P(y=k|b=l,X,B_g)$

$= \sum _{b_{il}\ne 0}\frac{1}{\sum _{i=1}^n b_{il}}P(y=k|v=i)$

where $\frac{1}{\sum _{i=1}^n b_{il}}$ is the probability of selecting node i from block l.

According to the Bayesian theorem, we have:

$P(y=k|v=i)=\frac{P(y=k)P(v=i|y=k)}{\sum _{k=1}^K P(y=k)P(v=i|y=k)}$.

Based on the proof of Proposition 1, we have:

$P(y=k)P(v=i|y=k)=\varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji}) \omega _k$.

So, we have Eq. 12.12.$\square$

As an approximate version of Eq. 12.12, we have:

$$\begin{aligned} \gamma _{lk}=P(y=k|b=l)=P(y=k|v=i,b_{il}\ne 0 )\nonumber \\ =\frac{P(y=k)P(v=i,b_{il}\ne 0|y=k)}{\sum _{k=1}^K P(y=k)P(v=i,b_{il}\ne 0|y=k)}\,\,\,\nonumber \\ =\frac{\exists _{b_{il}\ne 0}\varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji}) \omega _k}{\sum _{k=1}^K \exists _{b_{il}\ne 0}\varPi _{j=1}^n f(\theta _{kj},a_{ij})f(\delta _{kj},a_{ji}) \omega _k} \end{aligned}$$

(12.13)

where $\exists _{b_{il}\ne 0}$ denotes that randomly selecting a node from block l.

That is, instead of averaging all nodes in the block l, the real value of $\gamma _{lk}$ can be approximately estimated by a randomly selected node from block l.

Correspondingly, an approximate version of the log-likelihood of Eq. 12.10 is given by:

$$\begin{aligned} L(N|X,B_g)=\sum _{l=1}^L N_l(\exists _{b_{il}\ne 0}\ln \sum _{k=1}^K \varPi _{j=1}^n f(\theta _{kj},a_{ij})\nonumber \\ f(\delta _{kj},a_{ji})\omega _k) \end{aligned}$$

(12.14)

where $N_l$ denotes the size of block l.

The time calculating Eqs. 12.12 and 12.10 will be bounded by $O(n^2K)$. While, the time calculating Eqs. 12.13 and 12.14 will be bounded by O(LnK). This will be much efficient for constructing the hierarchical organizations of networks.

Theorem 1

A local optimum of Eq. 12.10 will be guaranteed by recursively calculating Eqs. 12.4 and 12.5 in the chapter.

Proof

From the Proposition 1, we have:

$L(N|X,B_g)$

$=\sum _{i=1}^n \ln P(v=i|X,B_g)$

$=\sum _{i=1}^n \ln \sum _{k=1}^K P(v=i,y=k|X,B_g)$

$=\sum _{i=1}^n \ln \sum _{k=1}^K P(y=k|v=i,X^{(s)},B_g)$

$\frac{P(v=i,y=k|X,B_g)}{P(y=k|v=i,X^{(s)},B_g)}$

(by Jensen’s inequality)

$\ge \sum _{i=1}^n \sum _{k=1}^K P(y=k|v=i,X^{(s)},B_g)\ln \frac{P(v=i,y=k|X,B_g)}{P(y=k|v=i,X^{(s)},B_g)}$

$\equiv G(X,X^{(s)})$.

Furthermore, we have:

$G(X^{(s)},X^{(s)})$

$=\sum _{i=1}^n\sum _{k=1}^K P(y=k|v=i,X^{(s)},B_g)\ln \frac{P(v=i,y=k|X^{(s)},B_g)}{P(y=k|v=i,X^{(s)},B_g)}$

$=\sum _{i=1}^n\sum _{k=1}^K P(y=k|v=i,X^{(s)},B_g)\ln P(v=i|X^{(s)},B_g)$

$=\sum _{i=1}^n \ln P(v=i|X^{(s)},B_g)\sum _{k=1}^K P(y=k|v=i,X^{(s)},B_g)$

$=\sum _{i=1}^n \ln P(v=i|X^{(s)},B_g)$

$= L(N|X^{(s)},B_g)$.

Let $P(y=k|b=l,X^{(s)},B_g)=\gamma _{ik}^{(s)}$, we have:

$G(X,X^{(s)})$

$=\sum _{l=1}^L\sum _{b_{il}\ne 0}\sum _{k=1}^K \gamma _{lk}^{(s)}\ln P(v=i,y=k|X,B_g)-\sum _{l=1}^L\sum _{b_{il}\ne 0}\sum _{k=1}^K \gamma _{ik}^{(s)}\ln P(y=k|v=i,X^{(s)},B_g)$.

So, we have:

$\arg \max G(X,X^{(s)})$

${=}\,\arg \max (\sum _{l=1}^L\sum _{b_{il}\ne 0}\sum _{k=1}^K \gamma _{lk}^{(s)}\ln P(v=i,y=k|X,B_g)\,{-}\,\sum _{l=1}^L\sum _{b_{il}\ne 0}\sum _{k=1}^K \gamma _{ik}^{(s)}\ln P(y=k|v=i,X^{(s)},B_g))$

$=\arg \max (\sum _{l=1}^L\sum _{b_{il}\ne 0}\sum _{k=1}^K (\gamma _{ik}^{(s)}\ln P(v=i,y=k|X,B_g)))$

$=\arg \max E[L(N,Z^{(s)}|X,B_g)]$

$=X^{(s+1)}$.

Recall that, the $\varTheta ^{(s+1)}$, $\varDelta ^{(s+1)}$ and $\varOmega ^{(s+1)}$ of $X^{(s+1)}$ can be computed in terms of $\gamma _{lk}^{(s)}$ by Eq. 12.4 in the chapter. So, we have:

$G(X^{(s+1)},X^{(s)})\ge G(X^{(s)},X^{(s)})=L(N|X^{(s)},B_g)$.

Recall that $L(N|X,B_g) \ge G(X,X^{(s)})$, we have:

$L(N|X^{(s+1)},B_g)\ge G(X^{(s+1)},X^{(s)}) \ge G(X^{(s)},X^{(s)})=L(N|X^{(s)},B_g)$.

That is to say, the $X^{(s+1)}$ obtained in the current iteration will be not worse than $X^{(s)}$ obtained in last iteration. So, we have the theorem.$\square$

Proposition 5

In terms of the parameter of X, $\varTheta$, $\varDelta$, Z and $\varOmega$, we have:

$$\begin{aligned} \varPhi =\varTheta B_gZ D^{-1}, \quad \varPsi =\varDelta B_gZ D^{-1} \end{aligned}$$

(12.15)

where $D=diag(n\varOmega )$.

Proof

We have

$\phi _{pq}=\sum _{i\in C_q}\frac{1}{N_q}\theta _{pi}$

where $i\in C_q$ denotes node i is in the cluster q with a size $N_q$, and $\frac{1}{N_q}$ is the probability of selecting node i from cluster q. Furthermore, we have:

$\phi _{pq}=\frac{1}{n\omega _q}\sum _{i=1}^n\theta _{pi}(B_gZ)_{iq}$.

Similarly, we have:

$\psi _{pq}=\frac{1}{n\omega _q}\sum _{i=1}^n\delta _{pi}(B_gZ)_{iq}$.

So, we have

$\varPhi =\varTheta B_gZ D^{-1}, \quad \varPsi =\varDelta B_gZ D^{-1}$.$\square$

Proposition 6

Let $B_{g_i}$ denotes the blocking model on the $i-th$ layer of the hierarchical organization of network N, we have:

$P(X|N,B_{g_1},\cdots ,B_{g_h})\propto P(N|X,B_{g_h})P(X)^{g_h}$

Proof

$P(X|N,B_{g_1},\cdots ,B_{g_h})$

$=\frac{P(X,N,B_{g_1},\cdots ,B_{g_h})}{P(N,B_{g_1},\cdots ,B_{g_h})}$

$\propto P(X,N,B_{g_1},\cdots ,B_{g_h})$

$=P(N|X,B_{g_1},\cdots ,B_{g_h})P(X,B_{g_1},\cdots ,B_{g_h})$

$=P(N|X,B_{g_1},\cdots ,B_{g_h})P(X|B_{g_1},\cdots ,B_{g_h})$

   $P(B_{g_h}|B_{g_1},\cdots ,B_{g_{h-1}})\cdots P(B_{g_2}|B_{g_1})P(B_{g_1})$

$\propto P(N|X,B_{g_1},\cdots ,B_{g_h})P(X|B_{g_1},\cdots ,B_{g_h})$

Since two nodes from the same block of $B_{g_{i-1}}$ will also be in the same block of $B_{g_i}$, we have:

$P(X|N,B_{g_1},\cdots ,B_{g_h})\propto P(N|X,B_{g_h})P(X|B_{g_h})=P(N|X,B_{g_h})P(X)^{g_h}$ $\square$