Discovering patterns in time-varying graphs: a triclustering approach

Abstract

This paper introduces a novel technique to track structures in time varying graphs. The method uses a maximum a posteriori approach for adjusting a three-dimensional co-clustering of the source vertices, the destination vertices and the time, to the data under study, in a way that does not require any hyper-parameter tuning. The three dimensions are simultaneously segmented in order to build clusters of source vertices, destination vertices and time segments where the edge distributions across clusters of vertices follow the same evolution over the time segments. The main novelty of this approach lies in that the time segments are directly inferred from the evolution of the edge distribution between the vertices, thus not requiring the user to make any a priori quantization. Experiments conducted on artificial data illustrate the good behavior of the technique, and a study of a real-life data set shows the potential of the proposed approach for exploratory data analysis.

Appendix 1: Interpretations of the dissimilarity between two clusters

Interestingly, the dissimilarity given in Definition 3 receives several interpretations. It corresponds to a loss of coding length (when the MODL criterion is interpreted as a description length), a loss of posterior probability of the triclustering given the data (see Proposition 1), and asymptotically to a divergence between probability distributions associated to the clusters (see Proposition 2).

Proposition 1

The exponential of the dissimilarity between two clusters, $c_1$ and $c_2$, gives the inverse ratio between the probability of the simplified triclustering given the data set and the probability of the original triclustering given the data set:

$$\begin{aligned} P(\mathcal {M}|E)=e^{\Delta _{ MODL }(c_1,c_2)} P(\mathcal {M}_{\text{ merge } c_1 \text { and }c_2}|E). \end{aligned}$$

(31)

Asymptotically—i.e when the number of edges tends to infinity - the dissimilarity between two clusters is proportional to a generalized Jensen–Shannon divergence between two distributions that characterize the clusters in the triclustering structure. To simplify the discussion, we give only the definition and result for the case of source clusters, but this can be generalized to the two other cases.

Definition 5

Let $\mathcal {M}$ be a triclustering. For all $i\in \{1,\ldots ,k_S\}$ we denote

$$\begin{aligned} \mathbb {P}^S_i=\left( \frac{\mu _{ijl}}{\mu _{{i}..}}\right) _{1\le j\le k_D, 1\le l\le k_T}. \end{aligned}$$

(32)

The matrix $\mathbb {P}^S_i$ can be interpreted as a probability distribution over $\{1, \ldots , k_D\}\times \{1, \ldots , k_T\}$. It characterizes $c^S_i$ as a cluster of source vertices as seen from clusters of destination vertices and of time stamps.

We denote $\mathbb {P}^S$ the associated marginal probability distribution obtained by

$$\begin{aligned} \mathbb {P}^S=\left( \frac{\sum _{i=1}^{k_S}\mu _{ijl}}{\sum _{i=1}^{k_S}\mu _{{i}..}}\right) _{1\le j\le k_D, 1\le l\le k_T}. \end{aligned}$$

(33)

Obviously, we have

$$\begin{aligned} \mathbb {P}^S=\sum _{i=1}^{k_S}\pi _i\mathbb {P}^S_i, \end{aligned}$$

(34)

where

$$\begin{aligned} \pi _i=\frac{\mu _{{i}..}}{\sum _{k=1}^{k_S}\mu _{{k}..}}. \end{aligned}$$

(35)

Proposition 2

Let $\mathcal {M}$ be a triclustering and let $c^S_i$ and $c^S_k$ be two source clusters. Then

$$\begin{aligned} \dfrac{\Delta _{ MODL }(c^S_i,c^S_k)}{\nu }\underset{\nu \rightarrow +\infty }{\longrightarrow } (\pi _i+\pi _k) JS^{\alpha _i,\alpha _k} (\mathbb {P}^S_i,\mathbb {P}^S_k), \end{aligned}$$

(36)

with

$$\begin{aligned} JS^{\alpha _i,\alpha _k} (\mathbb {P}^S_i,\mathbb {P}^S_k)=\alpha _i KL(\mathbb {P}^S_i || \alpha _i \mathbb {P}^S_i + \alpha _k \mathbb {P}^S_k) + \alpha _k KL(\mathbb {P}^S_k || \alpha _i \mathbb {P}^S_i + \alpha _k \mathbb {P}^S_k), \end{aligned}$$

(37)

and where $\alpha _i$ and $\alpha _k$ are the normalized mixture coefficients such as $\alpha _i = \frac{\pi _i}{\pi _i+\pi _k}$ and $\alpha _k = \frac{\pi _k}{\pi _i+\pi _k}$.

Proof

JS is the generalized Jensen–Shannon Divergence (Lin 1991) and KL, the Kullback–Leibler Divergence. The full proof is left out for brevity and relies on the Stirling approximation: $\log n!= n \log (n) - n + O(\log n)$, when the difference between the criterion value after and before the merge is computed. $\square$

The Jensen–Shannon divergence has some interesting properties: it is a symmetric and non-negative divergence measure between two probability distributions. In addition, the Jensen–Shannon divergence of two identical distributions is equal to zero. While this divergence is not a metric, as it is not sub-additive, it has nevertheless the minimal properties needed to be used as a dissimilarity measure within an agglomerative process in the context of co-clustering (Slonim and Tishby 1999).