Multiple change points detection and clustering in dynamic networks

Abstract

The increasing amount of data stored in the form of dynamic interactions between actors necessitates the use of methodologies to automatically extract relevant information. The interactions can be represented by dynamic networks in which most existing methods look for clusters of vertices to summarize the data. In this paper, a new framework is proposed in order to cluster the vertices while detecting change points in the intensities of the interactions. These change points are key in the understanding of the temporal interactions. The model used involves non-homogeneous Poisson point processes with cluster-dependent piecewise constant intensity functions and common discontinuity points. A variational expectation maximization algorithm is derived for inference. We show that the pruned exact linear time method, originally developed for change points detection in univariate time series, can be considered for the maximization step. This allows the detection of both the number of change points and their location. Experiments on artificial and real datasets are carried out, and the proposed approach is compared with related methods.

Appendix: Proofs

1.1 A.1 Proof of Proposition 1

Proof

Notice, first, that the central factor on the r.h.s. of the equality in Eq. (3) can be written as

$$\begin{aligned} \prod _{m=1}^M \lambda _{Z_{i_m}Z_{j_m}}(\nu _m)=\prod _{m=1}^M\prod _{j>i}^N\left( \lambda _{Z_i Z_j}(\nu _m)\right) ^{{\mathbf {1}}_{\mathcal {A}^{(i,j)}}(\nu _m)}, \end{aligned}$$

where ${\mathbf {1}}_{{\mathcal {G}}}(\cdot )$ is the indicator function on a set ${\mathcal {G}}$ and $\mathcal {A}^{(ij)}$ has been defined in (1). By inverting the product on the right-hand side, because of the indicator function we get

$$\begin{aligned}&\prod _{j>i}^N \prod _{m=1}^M \left( \lambda _{Z_i Z_j}(\nu _m)\right) ^{{\mathbf {1}}_{\mathcal {A}^{(i,j)}}(\nu _m)}=\prod _{j>i}^N \prod _{m=1}^{M^{(i,j)}}\lambda _{Z_iZ_j}(\nu _m^{(i,j)})\\&\quad =\prod _{j>i}^N \prod _{m=1}^{M^{(i,j)}}\left[ \prod _{k,g}^K (\lambda _{kg}(\nu _m^{(i,j)}))^{Z_{ik}Z_{jg}}\right] , \end{aligned}$$

where $\nu _m^{(i,j)}$ are the interaction times in the set $\mathcal {A}^{(i,j)}$, whose cardinality is $M^{(i,j)}$. Thanks (5), the following holds

$$\begin{aligned}&\prod _{j>i}^N \prod _{m=1}^{M^{(i,j)}}\left[ \prod _{k,g}^K (\lambda _{kg}(\nu _m^{(i,j)}))^{Z_{ik}Z_{jg}}\right] \nonumber \\&\quad =\prod _{j>i}^N \prod _{m=1}^{M^{(i,j)}}\left[ \prod _{k,g}^K \prod _{d=1}^{D} \lambda _{kgd}^{Z_{ik}Z_{jg} {\mathbf {1}}_{[\eta _{d-1}, \eta _d[}(\nu _m^{(i,j)})}\right] \nonumber \\&\quad =\prod _{k,g}^K \prod _{d=1}^{D}\lambda _{kgd}^{\sum _{j>i}^N Z_{ik}Z_{jg}(M^{(i,j)}(\eta _d) - M^{(i,j)}(\eta _{d-1}))}. \end{aligned}$$

(19)

Note that the last equality employs the definition of counting process

$$\begin{aligned} M^{(i,j)}(t)=\sum _{m=1}^{M^{(i,j)}}{\mathbf {1}}_{]0,t]}(\nu _m^{(i,j)}). \end{aligned}$$

By replacing (19) into (3) and using that

$$\begin{aligned} \Lambda _{Z_iZ_j}(T)=\sum _{k,g}^K \Lambda _{kg}(T)Z_{ik}Z_{jg}=\sum _{d=1}^D \sum _{k,g}^K \lambda _{kgd}\Delta _d Z_{ik}Z_{jg}, \end{aligned}$$

it suffices to take the logarithm of the likelihood and the proposition is proven. $\square$

1.2 A.2 Proof of Proposition 3

Proof

The following objective function is taken into account

$$\begin{aligned} \mathcal {L} \left( q(Z);K,\eta , D, {\varvec{\lambda }}, \pi \right) + \sum _{i=1}^N l_i\left( \sum _{k=1}^K\tau _{ik} -1\right) . \end{aligned}$$

This function has to be maximized with respect to both $\tau$ and the N Lagrange multipliers $l_1, \ldots , l_N$, introduced to take into account the normality of the lines of $\tau$. The most difficult step consists in taking the partial derivative of the objective function with respect to $\tau _{i_0k_0}$. We first focus on those terms of $\mathcal {L}(\cdot )$ depending on d

$$\begin{aligned}&Q(\tau ,\theta ):=-\sum _{d=1}^{D}\sum _{k,g}^K \left( \lambda _{kgd}\Delta _d \left( \sum _{i=1}^N\sum _{j>i}^N \tau _{ik} \tau _{jg}\right) \right. \\&\left. \quad - \log (\lambda _{kgd})\left( \sum _{i=1}^N\sum _{j>i}^N \tau _{ik}\tau _{jg}X_{ij}^{(d)}\right) \right) . \end{aligned}$$

Hence

$$\begin{aligned}&\frac{\partial Q(\tau ,\theta )}{\partial \tau _{i_0k_0}}=\sum _{d=1}^{D} \frac{\partial }{\partial \tau _{i_0k_0}}\left( -\sum _{i=1}^N\sum _{j>i}^N \sum _{k,g}^K\tau _{ik} \tau _{jg} \lambda _{kgd}\Delta _d \right) \\&\qquad + \sum _{d=1}^{D} \frac{\partial }{\partial \tau _{i_0 k_0}}\left( \sum _{i=1}^N\sum _{j>i}^N \sum _{k,g}^K\tau _{ik}\tau _{jg}X_{ij}^{(d)}\log (\lambda _{kgd})\right) \\&\quad =-\sum _{d=1}^{D}\left[ \sum _{j>i_0}^N \sum _{g=1}^K \tau _{jg}\Delta _d \lambda _{k_0 g d} + \sum _{j<i_0}^N \sum _{g=1}^K\tau _{jg}\Delta _d \lambda _{g k_0 d}\right] \\&\qquad +\sum _{d=1}^{D}\left[ \sum _{j>i_0}^N \sum _{g=1}^K \tau _{jg} X_{i_0 j}^{(d)}\log (\lambda _{k_0 g d}) + \sum _{j<i_0}^N \sum _{g=1}^K\tau _{jg} X_{j i_0}^{(d)}\log (\lambda _{g k_0 d})\right] \\&\quad =-\sum _{d=1}^{D}\left[ \sum _{j\ne i_0}^N \sum _{g=1}^K \tau _{jg} \Delta _d\lambda _{k_0 g d} - \sum _{j \ne i_0}^N \sum _{g=1}^K\tau _{jg} X_{i_0 j}^{(d)}\log (\lambda _{k_0 g d})\right] , \end{aligned}$$

where the last equality comes from the symmetry (i.e., interactions are undirected) of the frontal slices of tensors X and ${\varvec{\lambda }}$. Notice that the last term in the above equation is the function inside the exponential in Proposition 3. The remaining terms of $\mathcal {L}(\cdot )$, not involving d, can be differentiated straightforward. Imposing the partial derivatives of $\mathcal {L}(\cdot )$ equal to zero and using the above equation lead to the following system

$$\begin{aligned} {\left\{ \begin{array}{ll} \log (\tau _{i_0 k_0})= \log (\pi _{k_0})- \frac{\partial Q(\tau , \theta )}{\partial \tau _{i_0k_0}} + l_{i_0} -1 &{} \\ \sum _{k_0=1}^K \tau _{i_0 k_0}=1 &{}\qquad \forall (i_0,k_0). \end{array}\right. } \end{aligned}$$

The solution is obtained straightforward after some manipulations, and this concludes the proof. $\square$

1.3 A.3 Proof of Proposition 4

Proof

The following definitions are introduced to keep the notation uncluttered

$$\begin{aligned} S_{kg}:&=\sum _{j>i}^N \tau _{ik}\tau _{jg}\\ Y_{kg}^{[s,t[}:&=\sum _{j>i}^N\tau _{ik}\tau _{jg}(M^{(i,j)}(t)-M^{(i,j)}(s)) \qquad \forall s<t. \end{aligned}$$

Moreover, for every $t_{u_e}<t_{u_f}<t_{u_g}$, the following shorthand notation is used

$$\begin{aligned} \Delta ^{e,f}:=t_{u_f}-t_{u_e} \end{aligned}$$

and similarly for $\Delta ^{f,g}$ and $\Delta ^{e,g}$. Hence, we get

$$\begin{aligned}&{\mathcal {G}}([t_{u_e}, t_{u_g}[)=\sum _{k,g}\left[ \max _{\lambda _{kg}^{e,g} \in ]0, +\infty [}\left( -\lambda _{kg}^{e,g}\Delta ^{e,g}S_{kg} + \log (\lambda _{kg}^{e,g})Y_{kg}^{[t_{u_e}, t_{u_g}[}\right) \right] \\&\quad =\sum _{k,g}\left[ \max _{\lambda _{kg}^{e,g} \in ]0, +\infty [}\left( -\lambda _{kg}^{e,g}\Delta ^{e,f}S_{kg} + \log (\lambda _{kg}^{e,g})Y_{kg}^{[t_{u_e}, t_{u_f}[}\right) \right] \\&\qquad +\sum _{k,g}\left[ \max _{\lambda _{kg}^{e,g} \in ]0, +\infty [}\left( -\lambda _{kg}^{e,g}\Delta ^{f,g}S_{kg} + \log (\lambda _{kg}^{e,g})Y_{kg}^{[t_{u_f}, t_{u_g}[}\right) \right] \\&\quad \le \sum _{k,g}\left[ \max _{\lambda _{kg}^{e,f} \in ]0, +\infty [}\left( -\lambda _{kg}^{e,f}\Delta ^{e,f}S_{kg} + \log (\lambda _{kg}^{e,f})Y_{kg}^{[t_{u_e}, t_{u_f}[}\right) \right] \\&\qquad +\sum _{k,g}\left[ \max _{\lambda _{kg}^{f,g} \in ]0, +\infty [}\left( -\lambda _{kg}^{f,g}\Delta ^{f,g}S_{kg} + \log (\lambda _{k,g}^{f,g})Y_{kg}^{[t_{u_f}, t_{u_g}[}\right) \right] \\&\quad = {\mathcal {G}}([t_{u_e}, t_{u_f}[) + {\mathcal {G}}([t_{u_f}, t_{u_g}[), \end{aligned}$$

where the first and the last equalities come from the definition of ${\mathcal {G}}(\cdot )$. This concludes the proof. $\square$