Advertisement

EigenPulse: Detecting Surges in Large Streaming Graphs with Row Augmentation

  • Jiabao Zhang
  • Shenghua LiuEmail author
  • Wenjian YuEmail author
  • Wenjie Feng
  • Xueqi Cheng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11440)

Abstract

How can we spot dense blocks in a large streaming graph efficiently? Anomalies such as fraudulent attacks, spamming, and DDoS attacks, can create dense blocks in a short time window, emerging a surge of density in a streaming graph. However, most existing methods detect dense blocks in a static graph or a snapshot of dynamic graphs, which need to inefficiently rerun the algorithms for a streaming graph. Moreover, some works on streaming graphs are either consuming much time on updating algorithm for every incoming edge, or spotting the whole snapshot of a graph instead of the attacking sub-block.

Therefore, we propose a row-augmented matrix with sliding window to model a streaming graph, and design the AugSVD algorithm for computation- and memory-efficient singular decomposition. EigenPulse is then proposed to spot the density surges in streaming graphs based on the singular spectrum. We theoretically analyze the robustness of our method. Experiments on real datasets with injections show our performance and efficiency compared with the state-of-the-art baseline.

Keywords

Surge detection Streaming graphs Sliding window 

Notes

Acknowledgments

This material is based upon work supported by the Strategic Priority Research Program of CAS (XDA19020400), NSF of China (61772498, 61872206, 61425016, 91746301), and the Beijing NSF (4172059).

References

  1. 1.
    Akoglu, L., Tong, H., Koutra, D.: Graph based anomaly detection and description: a survey. Data Min. Knowl. Discov. 29(3), 626–688 (2015)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Eswaran, D., Faloutsos, C., Guha, S., Mishra, N.: Spotlight: detecting anomalies in streaming graphs. In: SIGKDD, pp. 1378–1386. ACM (2018)Google Scholar
  3. 3.
    Gao, J., Zhou, C., Yu, J.X.: Toward continuous pattern detection over evolving large graph with snapshot isolation. In: VLDB (2016)Google Scholar
  4. 4.
    Guha, S., Mishra, N., Roy, G., Schrijvers, O.: Robust random cut forest based anomaly detection on streams. In: ICML (2016)Google Scholar
  5. 5.
    Halko, N., Martinsson, P.G., Tropp, J.A.: Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53, 217–288 (2011)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hooi, B., Song, H.A., Beutel, A., Shah, N., Shin, K., Faloutsos, C.: Fraudar: bounding graph fraud in the face of camouflage. In: KDD. ACM (2016)Google Scholar
  7. 7.
    Jiang, M., Beutel, A., Cui, P., Hooi, B., Yang, S., Faloutsos, C.: A general suspiciousness metric for dense blocks in multimodal data. In: ICDM. IEEE (2015)Google Scholar
  8. 8.
    Koutra, D., Shah, N., Vogelstein, J.T., Gallagher, B., Faloutsos, C.: Deltacon: principled massive-graph similarity function with attribution. ACM Trans. Knowl. Discov. Data (TKDD) 10, 28 (2016)Google Scholar
  9. 9.
    Liu, S., Hooi, B., Faloutsos, C.: Holoscope: topology-and-spike aware fraud detection. In: CIKM, pp. 1539–1548. ACM (2017)Google Scholar
  10. 10.
    Prakash, B.A., Sridharan, A., Seshadri, M., Machiraju, S., Faloutsos, C.: EigenSpokes: surprising patterns and scalable community chipping in large graphs. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (eds.) PAKDD 2010. LNCS (LNAI), vol. 6119, pp. 435–448. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-13672-6_42CrossRefGoogle Scholar
  11. 11.
    Shin, K., Hooi, B., Faloutsos, C.: M-Zoom: fast dense-block detection in tensors with quality guarantees. In: Frasconi, P., Landwehr, N., Manco, G., Vreeken, J. (eds.) ECML PKDD 2016. LNCS (LNAI), vol. 9851, pp. 264–280. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46128-1_17CrossRefGoogle Scholar
  12. 12.
    Shin, K., Hooi, B., Kim, J., Faloutsos., C.: D-cube: dense-block detection in terabyte-scale tensors. In: WSDM (2017)Google Scholar
  13. 13.
    Shin, K., Hooi, B., Kim, J., Faloutsos, C.: Densealert: incremental dense-subtensor detection in tensor streams. In: KDD. ACM (2017)Google Scholar
  14. 14.
    Sun, J., Tao, D., Faloutsos, C.: Beyond streams and graphs: dynamic tensor analysis. In: KDD. ACM (2006)Google Scholar
  15. 15.
    Yongsub Lim, M.J., Kang, U.: Memory-efficient and accurate sampling for counting local triangles in graph streams: from simple to multigraphs. In: TKDD. ACM (2018)Google Scholar
  16. 16.
    Yu, W., Gu, Y., Li, J., Liu, S., Li, Y.: Single-pass PCA of large high-dimensional data. In: IJCAI, pp. 3350–3356 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.CAS Key Laboratory of Network Data Science and Technology, Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.BNRist, Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations