Abstract
How can we detect traffic disturbances from international flight transportation logs, or changes to collaboration dynamics in academic networks? These problems can be formulated as detecting anomalous change points in a dynamic graph. Current solutions do not scale well to large real world graphs, lack robustness to large amount of node additions / deletions and overlook changes in node attributes. To address these limitations, we propose a novel spectral method: Scalable Change Point Detection (SCPD). SCPD generates an embedding for each graph snapshot by efficiently approximating the distribution of the Laplacian spectrum at each step. SCPD can also capture shifts in node attributes by tracking correlations between attributes and eigenvectors. Through extensive experiments using synthetic and real world data, we show that SCPD (a) achieves state-of-the-art performance, (b) is significantly faster than the state-of-the-art methods and can easily process millions of edges in a few CPU minutes, (c) can effectively tackle a large quantity of node attributes, additions or deletions and (d) discovers interesting events in large real world graphs. Code is publicly available at https://github.com/shenyangHuang/SCPD.git.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barabási, A.L.: Philosophical transactions of the royal society a: mathematical, physical and engineering sciences. Netw. Sci. 371(1987), 20120375 (2013)
Benson, A.R., Abebe, R., Schaub, M.T., Jadbabaie, A., Kleinberg, J.: Simplicial closure and higher-order link prediction. PNAS (2018)
Bro, R.: Parafac. tutorial and applications. Chemometrics and intelligent laboratory systems 38(2), 149–171 (1997)
Dong, K., Benson, A.R., Bindel, D.: Network density of states. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 1152–1161 (2019)
Eswaran, D., Faloutsos, C.: Sedanspot: Detecting anomalies in edge streams. In: 2018 IEEE International Conference on Data Mining (ICDM). IEEE (2018)
Eswaran, D., Faloutsos, C., Guha, S., Mishra, N.: Spotlight: Detecting anomalies in streaming graphs. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2018)
Guha, S., Mishra, N., Roy, G., Schrijvers, O.: Robust random cut forest based anomaly detection on streams. In: International conference on machine learning. pp. 2712–2721. PMLR (2016)
Hall, K.M.: An r-dimensional quadratic placement algorithm. Manag. Sci. 17(3), 219–229 (1970)
Harshman, R.A., et al.: Foundations of the PARAFAC procedure: models and conditions for an explanatory multimodal factor analysis (1970)
Holland, P.W., Laskey, K.B., Leinhardt, S.: Stochastic blockmodels: first steps. Social Netw. 5(2), 109–137 (1983)
Huang, L., Graven, A.J., Bindel, D.: Density of states graph kernels. In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM) (2021)
Huang, S., Hitti, Y., Rabusseau, G., Rabbany, R.: Laplacian change point detection for dynamic graphs. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2020)
Idé, T., Kashima, H.: Eigenspace-based anomaly detection in computer systems. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 440–449. ACM (2004)
Koutra, D., Papalexakis, E.E., Faloutsos, C.: Tensorsplat: Spotting latent anomalies in time. In: 2012 16th Panhellenic Conference on Informatics. IEEE
Olive, X.: Traffic, a toolbox for processing and analysing air traffic data. J. Open Source Softw. 4(39), 1518 (2019)
Peel, L., Clauset, A.: Detecting change points in the large-scale structure of evolving networks. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Ranshous, S., Harenberg, S., Sharma, K., Samatova, N.F.: A scalable approach for outlier detection in edge streams using sketch-based approximations. In: 2016 SIAM International Conference on Data Mining. SIAM (2016)
Sawlani, S., Zhao, L., Akoglu, L.: Fast attributed graph embedding via density of states. In: 2021 IEEE International Conference on Data Mining (ICDM) (2021)
Schäfer, M., Strohmeier, M., Lenders, V., Martinovic, I., Wilhelm, M.: Bringing up opensky: A large-scale ads-b sensor network for research. In: IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks. pp. 83–94. IEEE (2014)
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.J.P., Wang, K.: An overview of microsoft academic service (MAS) and applications. In: the 24th International Conference on World Wide Web. ACM Press (2015)
Spielman, D.A.: Spectral graph theory and its applications. In: 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07). IEEE (2007)
Von Luxburg, U.: A tutorial on spectral clustering. Statist. comput. 17, 395–416 (2007). https://doi.org/10.1007/s11222-007-9033-z
Wang, Y., Chakrabarti, A., Sivakoff, D., Parthasarathy, S.: Fast change point detection on dynamic social networks. arXiv preprint arXiv:1705.07325 (2017)
Weisfeiler, B., Leman, A.: The reduction of a graph to canonical form and the algebra which appears therein. NTI Series 2(9), 12–16 (1968)
Acknowledgement
This research was supported by the CIFAR AI chair program, NSERC PGS Doctoral Award and FRQNT Doctoral Award.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, S., Danovitch, J., Rabusseau, G., Rabbany, R. (2023). Fast and Attributed Change Detection on Dynamic Graphs with Density of States. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13935. Springer, Cham. https://doi.org/10.1007/978-3-031-33374-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-33374-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33373-6
Online ISBN: 978-3-031-33374-3
eBook Packages: Computer ScienceComputer Science (R0)