Skip to main content

Fast and Attributed Change Detection on Dynamic Graphs with Density of States

  • Conference paper
  • First Online:
Advances in Knowledge Discovery and Data Mining (PAKDD 2023)

Abstract

How can we detect traffic disturbances from international flight transportation logs, or changes to collaboration dynamics in academic networks? These problems can be formulated as detecting anomalous change points in a dynamic graph. Current solutions do not scale well to large real world graphs, lack robustness to large amount of node additions / deletions and overlook changes in node attributes. To address these limitations, we propose a novel spectral method: Scalable Change Point Detection (SCPD). SCPD generates an embedding for each graph snapshot by efficiently approximating the distribution of the Laplacian spectrum at each step. SCPD can also capture shifts in node attributes by tracking correlations between attributes and eigenvectors. Through extensive experiments using synthetic and real world data, we show that SCPD (a) achieves state-of-the-art performance, (b) is significantly faster than the state-of-the-art methods and can easily process millions of edges in a few CPU minutes, (c) can effectively tackle a large quantity of node attributes, additions or deletions and (d) discovers interesting events in large real world graphs. Code is publicly available at https://github.com/shenyangHuang/SCPD.git.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://zenodo.org/record/3974209/#.Yf62HepKguU.

  2. 2.

    https://www.worldometers.info/coronavirus/country/china/.

References

  1. Barabási, A.L.: Philosophical transactions of the royal society a: mathematical, physical and engineering sciences. Netw. Sci. 371(1987), 20120375 (2013)

    Google Scholar 

  2. Benson, A.R., Abebe, R., Schaub, M.T., Jadbabaie, A., Kleinberg, J.: Simplicial closure and higher-order link prediction. PNAS (2018)

    Google Scholar 

  3. Bro, R.: Parafac. tutorial and applications. Chemometrics and intelligent laboratory systems 38(2), 149–171 (1997)

    Google Scholar 

  4. Dong, K., Benson, A.R., Bindel, D.: Network density of states. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 1152–1161 (2019)

    Google Scholar 

  5. Eswaran, D., Faloutsos, C.: Sedanspot: Detecting anomalies in edge streams. In: 2018 IEEE International Conference on Data Mining (ICDM). IEEE (2018)

    Google Scholar 

  6. Eswaran, D., Faloutsos, C., Guha, S., Mishra, N.: Spotlight: Detecting anomalies in streaming graphs. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2018)

    Google Scholar 

  7. Guha, S., Mishra, N., Roy, G., Schrijvers, O.: Robust random cut forest based anomaly detection on streams. In: International conference on machine learning. pp. 2712–2721. PMLR (2016)

    Google Scholar 

  8. Hall, K.M.: An r-dimensional quadratic placement algorithm. Manag. Sci. 17(3), 219–229 (1970)

    Article  Google Scholar 

  9. Harshman, R.A., et al.: Foundations of the PARAFAC procedure: models and conditions for an explanatory multimodal factor analysis (1970)

    Google Scholar 

  10. Holland, P.W., Laskey, K.B., Leinhardt, S.: Stochastic blockmodels: first steps. Social Netw. 5(2), 109–137 (1983)

    Article  MathSciNet  Google Scholar 

  11. Huang, L., Graven, A.J., Bindel, D.: Density of states graph kernels. In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM) (2021)

    Google Scholar 

  12. Huang, S., Hitti, Y., Rabusseau, G., Rabbany, R.: Laplacian change point detection for dynamic graphs. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2020)

    Google Scholar 

  13. Idé, T., Kashima, H.: Eigenspace-based anomaly detection in computer systems. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 440–449. ACM (2004)

    Google Scholar 

  14. Koutra, D., Papalexakis, E.E., Faloutsos, C.: Tensorsplat: Spotting latent anomalies in time. In: 2012 16th Panhellenic Conference on Informatics. IEEE

    Google Scholar 

  15. Olive, X.: Traffic, a toolbox for processing and analysing air traffic data. J. Open Source Softw. 4(39), 1518 (2019)

    Article  Google Scholar 

  16. Peel, L., Clauset, A.: Detecting change points in the large-scale structure of evolving networks. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)

    Google Scholar 

  17. Ranshous, S., Harenberg, S., Sharma, K., Samatova, N.F.: A scalable approach for outlier detection in edge streams using sketch-based approximations. In: 2016 SIAM International Conference on Data Mining. SIAM (2016)

    Google Scholar 

  18. Sawlani, S., Zhao, L., Akoglu, L.: Fast attributed graph embedding via density of states. In: 2021 IEEE International Conference on Data Mining (ICDM) (2021)

    Google Scholar 

  19. Schäfer, M., Strohmeier, M., Lenders, V., Martinovic, I., Wilhelm, M.: Bringing up opensky: A large-scale ads-b sensor network for research. In: IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks. pp. 83–94. IEEE (2014)

    Google Scholar 

  20. Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.J.P., Wang, K.: An overview of microsoft academic service (MAS) and applications. In: the 24th International Conference on World Wide Web. ACM Press (2015)

    Google Scholar 

  21. Spielman, D.A.: Spectral graph theory and its applications. In: 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07). IEEE (2007)

    Google Scholar 

  22. Von Luxburg, U.: A tutorial on spectral clustering. Statist. comput. 17, 395–416 (2007). https://doi.org/10.1007/s11222-007-9033-z

  23. Wang, Y., Chakrabarti, A., Sivakoff, D., Parthasarathy, S.: Fast change point detection on dynamic social networks. arXiv preprint arXiv:1705.07325 (2017)

  24. Weisfeiler, B., Leman, A.: The reduction of a graph to canonical form and the algebra which appears therein. NTI Series 2(9), 12–16 (1968)

    Google Scholar 

Download references

Acknowledgement

This research was supported by the CIFAR AI chair program, NSERC PGS Doctoral Award and FRQNT Doctoral Award.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shenyang Huang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, S., Danovitch, J., Rabusseau, G., Rabbany, R. (2023). Fast and Attributed Change Detection on Dynamic Graphs with Density of States. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13935. Springer, Cham. https://doi.org/10.1007/978-3-031-33374-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33374-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33373-6

  • Online ISBN: 978-3-031-33374-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics