The Bethe Hessian and Information Theoretic Approaches for Online Change-Point Detection in Network Data

Abstract

Sequences of networks are currently a common form of network data sets. Identification of structural change-points in a network data sequence is a natural problem. The problem of change-point detection can be classified into two main types - offline change-point detection and online or sequential change-point detection. In this paper, we propose three different algorithms for online change-point detection based on certain cusum statistics for network data with community structures. For two of the proposed algorithms, we use information theoretic measures to construct the statistic for the estimation of a change-point. In the third algorithm, we use eigenvalues of the Bethe Hessian matrix to construct the statistic for the estimation of a change-point. We show the consistency property of the estimated change-point theoretically under networks generated from the multi-layer stochastic block model and the multi-layer degree-corrected block model. We also conduct an extensive simulation study to demonstrate the key properties of the algorithms as well as their efficacy.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2
Figure 3
Figure 4

References

  1. Aminikhanghahi, S. and Cook, D.J. (2017). A survey of methods for time series change point detection. Knowl. Inf. Syst. 51, 339–367.

    Article  Google Scholar 

  2. Angel, O., Friedman, J. and Hoory, S. (2015). The non-backtracking spectrum of the universal cover of a graph. Trans. Am. Math. Soc. 367, 4287–4318.

    MathSciNet  MATH  Article  Google Scholar 

  3. Bai, J. and Perron, P. (1998). Estimating and testing linear models with multiple structural changes. Econometrica 47–78.

  4. Bao, W. and Michailidis, G. (2018). Core community structure recovery and phase transition detection in temporally evolving networks. Sci. Rep. 8, 1–16.

    Google Scholar 

  5. Bhamidi, S., Jin, J., Nobel, A. et al. (2018). Change point detection in network models: Preferential attachment and long range dependence. Ann. Appl. Probab. 28, 35–78.

    MathSciNet  MATH  Article  Google Scholar 

  6. Bhattacharjee, M., Banerjee, M. and Michailidis, G. (2018). Change point estimation in a dynamic stochastic block model. arXiv:1812.03090.

  7. Bhattacharyya, S. and Chatterjee, S. (2020). Consistent recovery of communities from sparse multi-relational networks: a scalable algorithm with optimal recovery conditions. Complex networks XI, pp. 92–103. Springer.

  8. Bhattacharyya, S. and Chatterjee, S. (2020). General community detection with optimal recovery conditions for multi-relational sparse networks with dependent layers.

  9. Bickel, P.J. and Sarkar, P. (2016). Hypothesis testing for automated community detection in networks. J. R. Stat. Soc. Ser. B Stat. Methodol. 78, 253–273.

    MathSciNet  MATH  Article  Google Scholar 

  10. Bleakley, K. and Vert, J.P. (2011).

  11. Blonder, B., Wey, T.W., Dornhaus, A., James, R. and Sih, A. (2012). Temporal dynamics and network analysis. Methods Ecol. Evol. 3, 958–972.

    Article  Google Scholar 

  12. Bordenave, C., Lelarge, M. and Massoulié, L. (2015). Non-backtracking spectrum of random graphs: community detection and non-regular ramanujan graphs. In 2015 IEEE 56Th annual symposium on foundations of computer science, pp. 1347–1357. IEEE.

  13. Bosc, M., Heitz, F., Armspach, J.P., Namer, I., Gounot, D. and Rumbach, L. (2003). Automatic change detection in multimodal serial mri: application to multiple sclerosis lesion evolution. Neuroimage 20, 643–656.

    Article  Google Scholar 

  14. Brodsky, E. and Darkhovsky, B.S. (2013). Nonparametric methods in change point problems, vol. 243 Springer Science & Business Media.

  15. Bruna, J. and Li, X. (2017). Community detection with graph neural networks. Stat. 1050, 27.

    Google Scholar 

  16. Van de Bunt, G.G., Van Duijn, M.A. and Snijders, T.A. (1999). Friendship networks through time: an actor-oriented dynamic statistical network model. Comput. Math. Organ. Theory 5, 167–192.

    MATH  Article  Google Scholar 

  17. Cape, J., Tang, M. and Priebe, C.E. (2017). The kato–temple inequality and eigenvalue concentration with applications to graph inference. Electron. J. Stat. 11, 3954–3978.

    MathSciNet  MATH  Article  Google Scholar 

  18. Celik, T. (2009). Unsupervised change detection in satellite images using principal component analysis and k-means clustering. IEEE Geosci. Remote Sens. Lett. 6, 772–776.

    Article  Google Scholar 

  19. Celik, T. (2010). Image change detection using gaussian mixture model and genetic algorithm. J. Vis. Commun. Image Represen. 21, 965–974.

    Article  Google Scholar 

  20. Chen, H. et al. (2019). Sequential change-point detection based on nearest neighbors. Ann. Stat. 47, 1381–1407.

    MathSciNet  MATH  Google Scholar 

  21. Chen, J. and Gupta, A.K. (2011). Parametric statistical change point analysis: with applications to genetics, medicine, and finance. Springer Science & Business Media.

  22. Chen, S., Ilany, A., White, B.J., Sanderson, M.W. and Lanzas, C. (2015). Spatial-temporal dynamics of high-resolution animal networks: what can we learn from domestic animals? PloS one 10(6).

  23. Cho, H. and Fryzlewicz, P. (2015). Multiple-change-point detection for high dimensional time series via sparsified binary segmentation. J. R. Stat. Soc. Ser. B Stat. Methodol. 77, 475–507.

    MathSciNet  MATH  Article  Google Scholar 

  24. Coste, S. and Zhu, Y. (2019). Eigenvalues of the non-backtracking operator detached from the bulk. arXiv:1907.05603.

  25. Cribben, I., Haraldsdottir, R., Atlas, L.Y., Wager, T.D. and Lindquist, M.A. (2012). Dynamic connectivity regression: determining state-related changes in brain connectivity. Neuroimage 61, 907–920.

    Article  Google Scholar 

  26. Dall’Amico, L. and Couillet, R. (2019). Community detection in sparse realistic graphs: Improving the bethe hessian. In ICASSP 2019-2019 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp. 2942–2946. IEEE.

  27. Dall’Amico, L., Couillet, R. and Tremblay, N. (2019). Revisiting the bethe-hessian: improved community detection in sparse heterogeneous graphs. In Advances in neural information processing systems, pp. 4039–4049.

  28. Dall’Amico, L., Couillet, R. and Tremblay, N. (2020). Optimal laplacian regularization for sparse spectral community detection. In ICASSP 2020-2020 IEEE International conference on acoustics, speech and signal processing (ICASSP, pp. 3237–3241. IEEE.

  29. Ferraz Costa, A., Yamaguchi, Y., Juci Machado Traina, A., Traina, Jr C. and Faloutsos, C. (2015). Rsc: Mining and modeling temporal activity in social media. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 269–278. ACM.

  30. Gao, C. and Lafferty, J. (2017). Testing network structure using relations between small subgraph probabilities. arXiv:1704.06742.

  31. Gates, M.C. and Woolhouse, M.E. (2015). Controlling infectious disease through the targeted manipulation of contact network structure. Epidemics 12, 11–19.

    Article  Google Scholar 

  32. Girshick, M.A. and Rubin, H. (1952). A bayes approach to a quality control model. Ann. Math. Stat., 114–125.

  33. Gulikers, L., Lelarge, M. and Massoulié, L. (2016). Non-backtracking spectrum of degree-corrected stochastic block models. arXiv:1609.02487.

  34. Harchaoui, Z., Vallet, F., Lung-Yut-Fong, A. and Cappé, O. (2009). A regularized kernel-based approach to unsupervised audio segmentation. In 2009 IEEE International conference on acoustics, speech and signal processing, pp. 1665–1668. IEEE.

  35. Hashimoto, K.I. (1989). Zeta functions of finite graphs and representations of p-adic groups. In Automorphic forms and geometry of arithmetic varieties, pp. 211–280. Elsevier.

  36. Hocking, T.D., Schleiermacher, G., Janoueix-Lerosey, I., Boeva, V., Cappo, J., Delattre, O., Bach, F. and Vert, J.P. (2013). Learning smoothing models of copy number profiles using breakpoint annotations. BMC Bioinform. 14, 164.

    Article  Google Scholar 

  37. Hogg, T. and Lerman, K. (2012). Social dynamics of digg. EPJ Data Sci. 1, 5.

    Article  Google Scholar 

  38. Holme, P. (2015). Modern temporal network theory: a colloquium. Eur. Phys. J. B 88, 234.

    Article  Google Scholar 

  39. Holme, P. and Saramäki, J. (2012). Temporal networks. Phys. Rep. 519, 97–125.

    Article  Google Scholar 

  40. Jacobs, A.Z., Way, S.F., Ugander, J. and Clauset, A. (2015). Assembling thefacebook: Using heterogeneity to understand online social network assembly. In Proceedings of the ACM Web Science Conference, pp. 1–10.

  41. Jin, J., Ke, Z. and Luo, S. (2018). Network global testing by counting graphlets. In International conference on machine learning, pp. 2333–2341.

  42. Kasetkasem, T. and Varshney, P.K. (2002). An image change detection algorithm based on markov random field models. IEEE Trans. Geosci. Remote Sens.40, 1815–1823.

    Article  Google Scholar 

  43. Kolmogorov, A.N. (1950). Unbiased estimates. Izvestiya Rossiiskoi Akademii Nauk. Seriya Matematicheskaya 14, 303–326.

    Google Scholar 

  44. Krings, G., Karsai, M., Bernhardsson, S., Blondel, V.D. and Saramäki, J. (2012). Effects of time window size and placement on the structure of an aggregated communication network. EPJ Data Sci. 1, 4.

    Article  Google Scholar 

  45. Krzakala, F., Moore, C., Mossel, E., Neeman, J., Sly, A., Zdeborová, L. and Zhang, P. (2013). Spectral redemption in clustering sparse networks. Proc. Natl. Acad. Sci. 110, 20935–20940.

    MathSciNet  MATH  Article  Google Scholar 

  46. Lahiri, M. and Berger-Wolf, T.Y. (2007). Structure prediction in temporal networks using frequent subgraphs. In 2007 IEEE Symposium on computational intelligence and data mining, p. 35–42. IEEE.

  47. Lavielle, M. and Teyssiere, G. (2007). Adaptive detection of multiple change-points in asset price volatility. In Long memory in economics, pp. 129–156. Springer.

  48. Le, C.M. and Levina, E. (2015). Estimating the number of communities in networks by spectral methods. arXiv:1507.00827.

  49. Lei, J., Rinaldo, A. et al. (2015). Consistency of spectral clustering in stochastic block models. Ann. Stat. 43, 215–237.

    MathSciNet  MATH  Google Scholar 

  50. Lévy-Leduc, C., Roueff, F. et al. (2009). Detection and localization of change-points in high-dimensional network traffic data. Ann. Appl. Stat. 3, 637–662.

    MathSciNet  MATH  Article  Google Scholar 

  51. Lorden, G. et al. (1971). Procedures for reacting to a change in distribution. Ann. Math. Stat. 42, 1897–1908.

    MathSciNet  MATH  Article  Google Scholar 

  52. Masuda, N. and Holme, P. (2017). Temporal network epidemiology. Springer.

  53. Matteson, D.S. and James, N.A. (2014). A nonparametric approach for multiple change point analysis of multivariate data. J. Am. Stat. Assoc. 109, 334–345.

    MathSciNet  MATH  Article  Google Scholar 

  54. Meilă, M. (2007). Comparing clusterings-an information based distance. J. Multi. Anal. 98, 873–895.

    MathSciNet  MATH  Article  Google Scholar 

  55. Mislove, A.E. (2009). Online social networks: measurement, analysis, and applications to distributed information systems. Ph.D thesis.

  56. Omodei, E., De Domenico, M.D. and Arenas, A. (2015). Characterizing interactions in online social networks during exceptional events. Front. Phys.3, 59.

    Article  Google Scholar 

  57. Padilla, O.H.M., Yu, Y. and Priebe, C.E. (2019). Change point localization in dependent dynamic nonparametric random dot product graphs. arXiv:1911.07494.

  58. Page, E.S. (1954). Continuous inspection schemes. Biometrika 41, 100–115.

    MathSciNet  MATH  Article  Google Scholar 

  59. Page, E.S. (1957). On problems in which a change in a parameter occurs at an unknown point. Biometrika 44, 248–252.

    MATH  Article  Google Scholar 

  60. Panisson, A., Gauvin, L., Barrat, A. and Cattuto, C. (2013). Fingerprinting temporal networks of close-range human proximity. In 2013 IEEE International conference on pervasive computing and communications workshops (PERCOM workshops), pp. 261–266. IEEE.

  61. Park, H.J. and Friston, K. (2013). Structural and functional brain networks: from connections to cognition. Science 342, 1238411.

    Article  Google Scholar 

  62. Park, Y., Priebe, C.E. and Youssef, A. (2013). Anomaly detection in time series of graphs using fusion of graph invariants. IEEE J. Select. Top. Signal Process. 7, 67–75.

    Article  Google Scholar 

  63. Peel, L. and Clauset, A. (2015). Detecting change points in the large-scale structure of evolving networks. In AAAI, pp. 2914–2920.

  64. Peixoto, T.P. (2015). Inferring the mesoscale structure of layered, edge-valued, and time-varying networks, Vol. 92.

  65. Peixoto, T.P. and Gauvin, L. (2018). Change points, memory and epidemic spreading in temporal networks. Scient. Rep. 8, 15511.

    Article  Google Scholar 

  66. Picard, F., Robin, S., Lavielle, M., Vaisse, C. and Daudin, J.J. (2005). A statistical approach for array cgh data analysis. BMC Bioinform. 6, 27.

    Article  Google Scholar 

  67. Popović, M., Štefančić, H., Sluban, B., Novak, P.K., Grčar, M., Mozetič, I., Puliga, M. and Zlatić, V. (2014). Extraction of temporal networks from term co-occurrences in online textual sources. PloS one 9, e99515.

    Article  Google Scholar 

  68. Radke, R.J., Andra, S., Al-Kofahi, O. and Roysam, B. (2005). Image change detection algorithms: a systematic survey. IEEE Trans. Image Process. 14, 294–307.

    MathSciNet  Article  Google Scholar 

  69. Ranshous, S., Shen, S., Koutra, D., Harenberg, S., Faloutsos, C. and Samatova, N.F. (2015). Anomaly detection in dynamic networks: a survey. Wiley Interdiscip. Rev. Comput. Stat. 7, 223–247.

    MathSciNet  Article  Google Scholar 

  70. Reeves, J., Chen, J., Wang, X.L., Lund, R. and Lu, Q.Q. (2007). A review and comparison of changepoint detection techniques for climate data. J. Appl. Meteorol. Climatol. 46, 900–915.

    Article  Google Scholar 

  71. Rigbolt, K.T., Prokhorova, T.A., Akimov, V., Henningsen, J., Johansen, P.T., Kratchmarova, I., Kassem, M., Mann, M., Olsen, J.V. and Blagoev, B. (2011). System-wide temporal characterization of the proteome and phosphoproteome of human embryonic stem cell differentiation. Sci. Signal. 4, rs3–rs3.

    Article  Google Scholar 

  72. Rocha, L.E., Liljeros, F. and Holme, P. (2010). Information dynamics shape the sexual networks of internet-mediated prostitution. Proc. Natl. Acad. Sci.107, 5706–5711.

    MATH  Article  Google Scholar 

  73. Rocha, L.E., Liljeros, F. and Holme, P. (2011). Simulated epidemics in an empirical spatiotemporal network of 50,185 sexual contacts. PLos computational biology 7(3).

  74. Roy, S., Atchadé, Y. and Michailidis, G. (2017). Change point estimation in high dimensional markov random-field models. J. R. Stat. Soc. Ser. B Stat. Methodol. 79, 1187–1206.

    MathSciNet  MATH  Article  Google Scholar 

  75. Saade, A., Krzakala, F. and Zdeborová, L. (2014). Spectral clustering of graphs with the bethe hessian. Advances in neural information processing systems, pp. 406–414.

  76. Saade, A., Krzakala, F. and Zdeborová, L. (2014). Spectral density of the non-backtracking operator on random graphs. EPL Europhys. Lett. 107, 50005.

    Article  Google Scholar 

  77. Salathé, M., Kazandjieva, M., Lee, J.W., Levis, P., Feldman, M.W. and Jones, J.H. (2010). A high-resolution human contact network for infectious disease transmission. Proc. Natl. Acad. Sci. 107, 22020–22025.

    Article  Google Scholar 

  78. Shiryaev, A.N. (1963). On optimum methods in quickest detection problems. Theory Probab. App. 8, 22–46.

    MATH  Article  Google Scholar 

  79. Siegmund, D. (2013). Change-points: from sequential detection to biology and back. Seq. Anal. 32, 2–14.

    MathSciNet  MATH  Article  Google Scholar 

  80. Sikdar, S., Ganguly, N. and Mukherjee, A. (2016). Time series analysis of temporal networks. Eur. Phys. J. B 89, 11.

    Article  Google Scholar 

  81. Sporns, O. (2013). Structure and function of complex brain networks. Dialogues Clin. Neurosci. 15, 247.

    Article  Google Scholar 

  82. Staudacher, M., Telser, S., Amann, A., Hinterhuber, H. and Ritsch-Marte, M. (2005). A new method for change-point detection developed for on-line analysis of the heart beat variability during sleep. Phys. A Stat. Mech. Appl. 349, 582–596.

    Article  Google Scholar 

  83. Stopczynski, A., Sekara, V., Sapiezynski, P., Cuttone, A., Madsen, M.M., Larsen, J.E. and Lehmann, S. (2014). Measuring large-scale social networks with high resolution. PloS one 9, e95978.

    Article  Google Scholar 

  84. Thompson, W.H., Brantefors, P. and Fransson, P. (2017). From static to temporal network theory: Applications to functional brain connectivity. Netw. Neurosci. 1, 69–99.

    Article  Google Scholar 

  85. Viswanath, B., Mislove, A., Cha, M. and Gummadi, K.P. (2009). On the evolution of user interaction in facebook. In Proceedings of the 2nd ACM workshop on Online social networks, pp. 37–42.

  86. Wang, D., Yu, Y. and Rinaldo, A. (2018). Optimal change point detection and localization in sparse dynamic networks. arXiv:1809.09602.

  87. Wang, Y., Chakrabarti, A., Sivakoff, D. and Parthasarathy, S. (2017). Fast change point detection on dynamic social networks. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 2992–2998. AAAI Press.

  88. Wang, Y.R., Bickel, P.J. et al. (2017). Likelihood-based model selection for stochastic block models. Ann. Stat. 45, 500–528.

    MathSciNet  MATH  Google Scholar 

  89. Watanabe, Y. and Fukumizu, K. (2009). Graph zeta function in the bethe free energy and loopy belief propagation. Advances in neural information processing systems, pp. 2017–2025.

  90. Wills, P. and Meyer, F.G. (2019). Change point detection in a dynamic stochastic blockmodel. International conference on complex networks and their applications, pp. 211–222. Springer.

  91. Yang, P., Dumont, G. and Ansermino, J.M. (2006). Adaptive change detection in heart rate trend monitoring in anesthetized children. IEEE Trans. Biomed. Eng. 53, 2211–2219.

    Article  Google Scholar 

  92. Yedidia, J.S., Freeman, W.T. and Weiss, Y. (2003). Understanding belief propagation and its generalizations. Explor. Artif. Intell. New Millennium8, 236–239.

    Google Scholar 

  93. Zhang, X., Shao, S., Stanley, H.E. and Havlin, S. (2014). Dynamic motifs in socio-economic networks. EPL Europhys. Lett. 108, 58001.

    Article  Google Scholar 

  94. Zhao, L., Wang, G.J., Wang, M., Bao, W., Li, W. and Stanley, H.E. (2018). Stock market as temporal network. Physic. A Stat. Mech. Appl.506, 1104–1112.

    Article  Google Scholar 

  95. Zhao, Z., Chen, L. and Lin, L. (2019). Change-point detection in dynamic networks via graphon estimation. arXiv:1908.01823.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sharmodeep Bhattacharyya.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

N. Hwang was partially supported by the Rich Internship awarded by the Department of Mathematics, City College of New York, CUNY, in Summer 2019, and S. Chatterjee was partially supported by the PSC-CUNY Enhanced Research Award #62781-00 50

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 6.11 MB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hwang, N., Xu, J., Chatterjee, S. et al. The Bethe Hessian and Information Theoretic Approaches for Online Change-Point Detection in Network Data. Sankhya A (2021). https://doi.org/10.1007/s13171-021-00248-1

Download citation

Keywords

  • Change-point detection
  • Bethe hessian operator
  • Spectral clustering
  • Community detection
  • Sparse networks
  • Variation of information.

AMS (2000) subject classification

  • Primary 62H30
  • 62F12
  • Secondary 91D30