Skip to main content

The Bethe Hessian and Information Theoretic Approaches for Online Change-Point Detection in Network Data

Sankhya A volume 84, pages 283–320 (2022)Cite this article

Abstract

Sequences of networks are currently a common form of network data sets. Identification of structural change-points in a network data sequence is a natural problem. The problem of change-point detection can be classified into two main types - offline change-point detection and online or sequential change-point detection. In this paper, we propose three different algorithms for online change-point detection based on certain cusum statistics for network data with community structures. For two of the proposed algorithms, we use information theoretic measures to construct the statistic for the estimation of a change-point. In the third algorithm, we use eigenvalues of the Bethe Hessian matrix to construct the statistic for the estimation of a change-point. We show the consistency property of the estimated change-point theoretically under networks generated from the multi-layer stochastic block model and the multi-layer degree-corrected block model. We also conduct an extensive simulation study to demonstrate the key properties of the algorithms as well as their efficacy.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1
Figure 2
Figure 3
Figure 4

References

  • Aminikhanghahi, S. and Cook, D.J. (2017). A survey of methods for time series change point detection. Knowl. Inf. Syst. 51, 339–367.

    Google Scholar 

  • Angel, O., Friedman, J. and Hoory, S. (2015). The non-backtracking spectrum of the universal cover of a graph. Trans. Am. Math. Soc. 367, 4287–4318.

    MathSciNet  MATH  Google Scholar 

  • Bai, J. and Perron, P. (1998). Estimating and testing linear models with multiple structural changes. Econometrica 47–78.

  • Bao, W. and Michailidis, G. (2018). Core community structure recovery and phase transition detection in temporally evolving networks. Sci. Rep. 8, 1–16.

    Google Scholar 

  • Bhamidi, S., Jin, J., Nobel, A. et al. (2018). Change point detection in network models: Preferential attachment and long range dependence. Ann. Appl. Probab. 28, 35–78.

    MathSciNet  MATH  Google Scholar 

  • Bhattacharjee, M., Banerjee, M. and Michailidis, G. (2018). Change point estimation in a dynamic stochastic block model. arXiv:1812.03090.

  • Bhattacharyya, S. and Chatterjee, S. (2020). Consistent recovery of communities from sparse multi-relational networks: a scalable algorithm with optimal recovery conditions. Complex networks XI, pp. 92–103. Springer.

  • Bhattacharyya, S. and Chatterjee, S. (2020). General community detection with optimal recovery conditions for multi-relational sparse networks with dependent layers.

  • Bickel, P.J. and Sarkar, P. (2016). Hypothesis testing for automated community detection in networks. J. R. Stat. Soc. Ser. B Stat. Methodol. 78, 253–273.

    MathSciNet  MATH  Google Scholar 

  • Bleakley, K. and Vert, J.P. (2011).

  • Blonder, B., Wey, T.W., Dornhaus, A., James, R. and Sih, A. (2012). Temporal dynamics and network analysis. Methods Ecol. Evol. 3, 958–972.

    Google Scholar 

  • Bordenave, C., Lelarge, M. and Massoulié, L. (2015). Non-backtracking spectrum of random graphs: community detection and non-regular ramanujan graphs. In 2015 IEEE 56Th annual symposium on foundations of computer science, pp. 1347–1357. IEEE.

  • Bosc, M., Heitz, F., Armspach, J.P., Namer, I., Gounot, D. and Rumbach, L. (2003). Automatic change detection in multimodal serial mri: application to multiple sclerosis lesion evolution. Neuroimage 20, 643–656.

    Google Scholar 

  • Brodsky, E. and Darkhovsky, B.S. (2013). Nonparametric methods in change point problems, vol. 243 Springer Science & Business Media.

  • Bruna, J. and Li, X. (2017). Community detection with graph neural networks. Stat. 1050, 27.

    Google Scholar 

  • Van de Bunt, G.G., Van Duijn, M.A. and Snijders, T.A. (1999). Friendship networks through time: an actor-oriented dynamic statistical network model. Comput. Math. Organ. Theory 5, 167–192.

    MATH  Google Scholar 

  • Cape, J., Tang, M. and Priebe, C.E. (2017). The kato–temple inequality and eigenvalue concentration with applications to graph inference. Electron. J. Stat. 11, 3954–3978.

    MathSciNet  MATH  Google Scholar 

  • Celik, T. (2009). Unsupervised change detection in satellite images using principal component analysis and k-means clustering. IEEE Geosci. Remote Sens. Lett. 6, 772–776.

    Google Scholar 

  • Celik, T. (2010). Image change detection using gaussian mixture model and genetic algorithm. J. Vis. Commun. Image Represen. 21, 965–974.

    Google Scholar 

  • Chen, H. et al. (2019). Sequential change-point detection based on nearest neighbors. Ann. Stat. 47, 1381–1407.

    MathSciNet  MATH  Google Scholar 

  • Chen, J. and Gupta, A.K. (2011). Parametric statistical change point analysis: with applications to genetics, medicine, and finance. Springer Science & Business Media.

  • Chen, S., Ilany, A., White, B.J., Sanderson, M.W. and Lanzas, C. (2015). Spatial-temporal dynamics of high-resolution animal networks: what can we learn from domestic animals? PloS one 10(6).

  • Cho, H. and Fryzlewicz, P. (2015). Multiple-change-point detection for high dimensional time series via sparsified binary segmentation. J. R. Stat. Soc. Ser. B Stat. Methodol. 77, 475–507.

    MathSciNet  MATH  Google Scholar 

  • Coste, S. and Zhu, Y. (2019). Eigenvalues of the non-backtracking operator detached from the bulk. arXiv:1907.05603.

  • Cribben, I., Haraldsdottir, R., Atlas, L.Y., Wager, T.D. and Lindquist, M.A. (2012). Dynamic connectivity regression: determining state-related changes in brain connectivity. Neuroimage 61, 907–920.

    Google Scholar 

  • Dall’Amico, L. and Couillet, R. (2019). Community detection in sparse realistic graphs: Improving the bethe hessian. In ICASSP 2019-2019 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp. 2942–2946. IEEE.

  • Dall’Amico, L., Couillet, R. and Tremblay, N. (2019). Revisiting the bethe-hessian: improved community detection in sparse heterogeneous graphs. In Advances in neural information processing systems, pp. 4039–4049.

  • Dall’Amico, L., Couillet, R. and Tremblay, N. (2020). Optimal laplacian regularization for sparse spectral community detection. In ICASSP 2020-2020 IEEE International conference on acoustics, speech and signal processing (ICASSP, pp. 3237–3241. IEEE.

  • Ferraz Costa, A., Yamaguchi, Y., Juci Machado Traina, A., Traina, Jr C. and Faloutsos, C. (2015). Rsc: Mining and modeling temporal activity in social media. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 269–278. ACM.

  • Gao, C. and Lafferty, J. (2017). Testing network structure using relations between small subgraph probabilities. arXiv:1704.06742.

  • Gates, M.C. and Woolhouse, M.E. (2015). Controlling infectious disease through the targeted manipulation of contact network structure. Epidemics 12, 11–19.

    Google Scholar 

  • Girshick, M.A. and Rubin, H. (1952). A bayes approach to a quality control model. Ann. Math. Stat., 114–125.

  • Gulikers, L., Lelarge, M. and Massoulié, L. (2016). Non-backtracking spectrum of degree-corrected stochastic block models. arXiv:1609.02487.

  • Harchaoui, Z., Vallet, F., Lung-Yut-Fong, A. and Cappé, O. (2009). A regularized kernel-based approach to unsupervised audio segmentation. In 2009 IEEE International conference on acoustics, speech and signal processing, pp. 1665–1668. IEEE.

  • Hashimoto, K.I. (1989). Zeta functions of finite graphs and representations of p-adic groups. In Automorphic forms and geometry of arithmetic varieties, pp. 211–280. Elsevier.

  • Hocking, T.D., Schleiermacher, G., Janoueix-Lerosey, I., Boeva, V., Cappo, J., Delattre, O., Bach, F. and Vert, J.P. (2013). Learning smoothing models of copy number profiles using breakpoint annotations. BMC Bioinform. 14, 164.

    Google Scholar 

  • Hogg, T. and Lerman, K. (2012). Social dynamics of digg. EPJ Data Sci. 1, 5.

    Google Scholar 

  • Holme, P. (2015). Modern temporal network theory: a colloquium. Eur. Phys. J. B 88, 234.

    Google Scholar 

  • Holme, P. and Saramäki, J. (2012). Temporal networks. Phys. Rep. 519, 97–125.

    Google Scholar 

  • Jacobs, A.Z., Way, S.F., Ugander, J. and Clauset, A. (2015). Assembling thefacebook: Using heterogeneity to understand online social network assembly. In Proceedings of the ACM Web Science Conference, pp. 1–10.

  • Jin, J., Ke, Z. and Luo, S. (2018). Network global testing by counting graphlets. In International conference on machine learning, pp. 2333–2341.

  • Kasetkasem, T. and Varshney, P.K. (2002). An image change detection algorithm based on markov random field models. IEEE Trans. Geosci. Remote Sens.40, 1815–1823.

    Google Scholar 

  • Kolmogorov, A.N. (1950). Unbiased estimates. Izvestiya Rossiiskoi Akademii Nauk. Seriya Matematicheskaya 14, 303–326.

    Google Scholar 

  • Krings, G., Karsai, M., Bernhardsson, S., Blondel, V.D. and Saramäki, J. (2012). Effects of time window size and placement on the structure of an aggregated communication network. EPJ Data Sci. 1, 4.

    Google Scholar 

  • Krzakala, F., Moore, C., Mossel, E., Neeman, J., Sly, A., Zdeborová, L. and Zhang, P. (2013). Spectral redemption in clustering sparse networks. Proc. Natl. Acad. Sci. 110, 20935–20940.

    MathSciNet  MATH  Google Scholar 

  • Lahiri, M. and Berger-Wolf, T.Y. (2007). Structure prediction in temporal networks using frequent subgraphs. In 2007 IEEE Symposium on computational intelligence and data mining, p. 35–42. IEEE.

  • Lavielle, M. and Teyssiere, G. (2007). Adaptive detection of multiple change-points in asset price volatility. In Long memory in economics, pp. 129–156. Springer.

  • Le, C.M. and Levina, E. (2015). Estimating the number of communities in networks by spectral methods. arXiv:1507.00827.

  • Lei, J., Rinaldo, A. et al. (2015). Consistency of spectral clustering in stochastic block models. Ann. Stat. 43, 215–237.

    MathSciNet  MATH  Google Scholar 

  • Lévy-Leduc, C., Roueff, F. et al. (2009). Detection and localization of change-points in high-dimensional network traffic data. Ann. Appl. Stat. 3, 637–662.

    MathSciNet  MATH  Google Scholar 

  • Lorden, G. et al. (1971). Procedures for reacting to a change in distribution. Ann. Math. Stat. 42, 1897–1908.

    MathSciNet  MATH  Google Scholar 

  • Masuda, N. and Holme, P. (2017). Temporal network epidemiology. Springer.

  • Matteson, D.S. and James, N.A. (2014). A nonparametric approach for multiple change point analysis of multivariate data. J. Am. Stat. Assoc. 109, 334–345.

    MathSciNet  MATH  Google Scholar 

  • Meilă, M. (2007). Comparing clusterings-an information based distance. J. Multi. Anal. 98, 873–895.

    MathSciNet  MATH  Google Scholar 

  • Mislove, A.E. (2009). Online social networks: measurement, analysis, and applications to distributed information systems. Ph.D thesis.

  • Omodei, E., De Domenico, M.D. and Arenas, A. (2015). Characterizing interactions in online social networks during exceptional events. Front. Phys.3, 59.

    Google Scholar 

  • Padilla, O.H.M., Yu, Y. and Priebe, C.E. (2019). Change point localization in dependent dynamic nonparametric random dot product graphs. arXiv:1911.07494.

  • Page, E.S. (1954). Continuous inspection schemes. Biometrika 41, 100–115.

    MathSciNet  MATH  Google Scholar 

  • Page, E.S. (1957). On problems in which a change in a parameter occurs at an unknown point. Biometrika 44, 248–252.

    MATH  Google Scholar 

  • Panisson, A., Gauvin, L., Barrat, A. and Cattuto, C. (2013). Fingerprinting temporal networks of close-range human proximity. In 2013 IEEE International conference on pervasive computing and communications workshops (PERCOM workshops), pp. 261–266. IEEE.

  • Park, H.J. and Friston, K. (2013). Structural and functional brain networks: from connections to cognition. Science 342, 1238411.

    Google Scholar 

  • Park, Y., Priebe, C.E. and Youssef, A. (2013). Anomaly detection in time series of graphs using fusion of graph invariants. IEEE J. Select. Top. Signal Process. 7, 67–75.

    Google Scholar 

  • Peel, L. and Clauset, A. (2015). Detecting change points in the large-scale structure of evolving networks. In AAAI, pp. 2914–2920.

  • Peixoto, T.P. (2015). Inferring the mesoscale structure of layered, edge-valued, and time-varying networks, Vol. 92.

  • Peixoto, T.P. and Gauvin, L. (2018). Change points, memory and epidemic spreading in temporal networks. Scient. Rep. 8, 15511.

    Google Scholar 

  • Picard, F., Robin, S., Lavielle, M., Vaisse, C. and Daudin, J.J. (2005). A statistical approach for array cgh data analysis. BMC Bioinform. 6, 27.

    Google Scholar 

  • Popović, M., Štefančić, H., Sluban, B., Novak, P.K., Grčar, M., Mozetič, I., Puliga, M. and Zlatić, V. (2014). Extraction of temporal networks from term co-occurrences in online textual sources. PloS one 9, e99515.

    Google Scholar 

  • Radke, R.J., Andra, S., Al-Kofahi, O. and Roysam, B. (2005). Image change detection algorithms: a systematic survey. IEEE Trans. Image Process. 14, 294–307.

    MathSciNet  Google Scholar 

  • Ranshous, S., Shen, S., Koutra, D., Harenberg, S., Faloutsos, C. and Samatova, N.F. (2015). Anomaly detection in dynamic networks: a survey. Wiley Interdiscip. Rev. Comput. Stat. 7, 223–247.

    MathSciNet  Google Scholar 

  • Reeves, J., Chen, J., Wang, X.L., Lund, R. and Lu, Q.Q. (2007). A review and comparison of changepoint detection techniques for climate data. J. Appl. Meteorol. Climatol. 46, 900–915.

    Google Scholar 

  • Rigbolt, K.T., Prokhorova, T.A., Akimov, V., Henningsen, J., Johansen, P.T., Kratchmarova, I., Kassem, M., Mann, M., Olsen, J.V. and Blagoev, B. (2011). System-wide temporal characterization of the proteome and phosphoproteome of human embryonic stem cell differentiation. Sci. Signal. 4, rs3–rs3.

    Google Scholar 

  • Rocha, L.E., Liljeros, F. and Holme, P. (2010). Information dynamics shape the sexual networks of internet-mediated prostitution. Proc. Natl. Acad. Sci.107, 5706–5711.

    MATH  Google Scholar 

  • Rocha, L.E., Liljeros, F. and Holme, P. (2011). Simulated epidemics in an empirical spatiotemporal network of 50,185 sexual contacts. PLos computational biology 7(3).

  • Roy, S., Atchadé, Y. and Michailidis, G. (2017). Change point estimation in high dimensional markov random-field models. J. R. Stat. Soc. Ser. B Stat. Methodol. 79, 1187–1206.

    MathSciNet  MATH  Google Scholar 

  • Saade, A., Krzakala, F. and Zdeborová, L. (2014). Spectral clustering of graphs with the bethe hessian. Advances in neural information processing systems, pp. 406–414.

  • Saade, A., Krzakala, F. and Zdeborová, L. (2014). Spectral density of the non-backtracking operator on random graphs. EPL Europhys. Lett. 107, 50005.

    Google Scholar 

  • Salathé, M., Kazandjieva, M., Lee, J.W., Levis, P., Feldman, M.W. and Jones, J.H. (2010). A high-resolution human contact network for infectious disease transmission. Proc. Natl. Acad. Sci. 107, 22020–22025.

    Google Scholar 

  • Shiryaev, A.N. (1963). On optimum methods in quickest detection problems. Theory Probab. App. 8, 22–46.

    MATH  Google Scholar 

  • Siegmund, D. (2013). Change-points: from sequential detection to biology and back. Seq. Anal. 32, 2–14.

    MathSciNet  MATH  Google Scholar 

  • Sikdar, S., Ganguly, N. and Mukherjee, A. (2016). Time series analysis of temporal networks. Eur. Phys. J. B 89, 11.

    Google Scholar 

  • Sporns, O. (2013). Structure and function of complex brain networks. Dialogues Clin. Neurosci. 15, 247.

    Google Scholar 

  • Staudacher, M., Telser, S., Amann, A., Hinterhuber, H. and Ritsch-Marte, M. (2005). A new method for change-point detection developed for on-line analysis of the heart beat variability during sleep. Phys. A Stat. Mech. Appl. 349, 582–596.

    Google Scholar 

  • Stopczynski, A., Sekara, V., Sapiezynski, P., Cuttone, A., Madsen, M.M., Larsen, J.E. and Lehmann, S. (2014). Measuring large-scale social networks with high resolution. PloS one 9, e95978.

    Google Scholar 

  • Thompson, W.H., Brantefors, P. and Fransson, P. (2017). From static to temporal network theory: Applications to functional brain connectivity. Netw. Neurosci. 1, 69–99.

    Google Scholar 

  • Viswanath, B., Mislove, A., Cha, M. and Gummadi, K.P. (2009). On the evolution of user interaction in facebook. In Proceedings of the 2nd ACM workshop on Online social networks, pp. 37–42.

  • Wang, D., Yu, Y. and Rinaldo, A. (2018). Optimal change point detection and localization in sparse dynamic networks. arXiv:1809.09602.

  • Wang, Y., Chakrabarti, A., Sivakoff, D. and Parthasarathy, S. (2017). Fast change point detection on dynamic social networks. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 2992–2998. AAAI Press.

  • Wang, Y.R., Bickel, P.J. et al. (2017). Likelihood-based model selection for stochastic block models. Ann. Stat. 45, 500–528.

    MathSciNet  MATH  Google Scholar 

  • Watanabe, Y. and Fukumizu, K. (2009). Graph zeta function in the bethe free energy and loopy belief propagation. Advances in neural information processing systems, pp. 2017–2025.

  • Wills, P. and Meyer, F.G. (2019). Change point detection in a dynamic stochastic blockmodel. International conference on complex networks and their applications, pp. 211–222. Springer.

  • Yang, P., Dumont, G. and Ansermino, J.M. (2006). Adaptive change detection in heart rate trend monitoring in anesthetized children. IEEE Trans. Biomed. Eng. 53, 2211–2219.

    Google Scholar 

  • Yedidia, J.S., Freeman, W.T. and Weiss, Y. (2003). Understanding belief propagation and its generalizations. Explor. Artif. Intell. New Millennium8, 236–239.

    Google Scholar 

  • Zhang, X., Shao, S., Stanley, H.E. and Havlin, S. (2014). Dynamic motifs in socio-economic networks. EPL Europhys. Lett. 108, 58001.

    Google Scholar 

  • Zhao, L., Wang, G.J., Wang, M., Bao, W., Li, W. and Stanley, H.E. (2018). Stock market as temporal network. Physic. A Stat. Mech. Appl.506, 1104–1112.

    Google Scholar 

  • Zhao, Z., Chen, L. and Lin, L. (2019). Change-point detection in dynamic networks via graphon estimation. arXiv:1908.01823.

Download references

Author information

Authors and Affiliations

  1. City University of New York - Bronx Community College, New York, NY, USA

    Neil Hwang

  2. Oregon State University, Corvallis, OR, USA

    Jiarui Xu & Sharmodeep Bhattacharyya

  3. City University of New York - City College and Graduate Center, New York, NY, USA

    Shirshendu Chatterjee

Authors
  1. Neil Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiarui Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shirshendu Chatterjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sharmodeep Bhattacharyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sharmodeep Bhattacharyya.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

N. Hwang was partially supported by the Rich Internship awarded by the Department of Mathematics, City College of New York, CUNY, in Summer 2019, and S. Chatterjee was partially supported by the PSC-CUNY Enhanced Research Award #62781-00 50

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 6.11 MB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hwang, N., Xu, J., Chatterjee, S. et al. The Bethe Hessian and Information Theoretic Approaches for Online Change-Point Detection in Network Data. Sankhya A 84, 283–320 (2022). https://doi.org/10.1007/s13171-021-00248-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13171-021-00248-1

Keywords

  • Change-point detection
  • Bethe hessian operator
  • Spectral clustering
  • Community detection
  • Sparse networks
  • Variation of information.

AMS (2000) subject classification

  • Primary 62H30
  • 62F12
  • Secondary 91D30