Skip to main content
SpringerLink
Log in

Two-stage anomaly detection algorithm via dynamic community evolution in temporal graph

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Detecting anomalies from a massive amount of user behavioral data is often liken to finding a needle in a haystack. While tremendous efforts have been devoted to anomaly detection from temporal graphs, existing studies rarely consider community evolution and evolutionary paths simultaneously, and analyze those characteristics for the purpose of anomaly detection. Therefore, we propose a two-stage anomaly detection (TSAD) framework to detect anomalies. In this study, we suggest detecting the community evolution events from a sequence of snapshot graphs by constructing an evolution bipartite graph and designing community similarity scores. We then propose a novel anomaly detection method combining community evolution-based anomaly detection and evolutionary path-based anomaly detection. An anomalous score is designed to detect anomalous community evolution events by extracting the characteristics of evolution communities in the community evolution-based anomaly detection method. Moreover, to reduce the false alarm rate, we propose evolutionary path-based anomaly detection to further detect the abnormality of the identified normal evolutionary paths by extracting the characteristics of the identified anomalous evolutionary paths based on community evolution-based anomaly detection. We conduct extensive experiments on real-world datasets and demonstrate that TSAD consistently outperforms competitive baseline methods in anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Bródka P, Saganowski S (2013) Ged: the method for group evolution discovery in social networks. Soc Netw Anal Min 3(1):1–14

    Article  Google Scholar 

  2. Kioucheab AE, Lagraac S, Amroucheb K, Seba H (2020) A simple graph embedding for anomaly detection in a stream of heterogeneous labeled graphs. Pattern Recognition

  3. Cazabet R, Amblard F (2014) Dynamic Community Detection. Springer, New York, pp 404–414

  4. Chelmis C, Dani R (2017) Assist: Automatic summarization of significant structural changes in large temporal graphs. In: Proceedings of the 2017 ACM on web science conference, WebSci ’17. Association for computing machinery, New York, pp 201–205

  5. Coscia M, Giannotti F, Pedreschi D (2011) A classification for community discovery methods in complex networks. Stat Anal Data Min ASA Data Sci J:4

  6. Eswaran D, Faloutsos C (2018) Sedanspot: Detecting anomalies in edge streams. In: 2018 IEEE International conference on data mining (ICDM), pp 953–958

  7. Eswaran D, Faloutsos C, Guha S, Mishra N (2018) Spotlight: Detecting anomalies in streaming graphs. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’18. Association for computing machinery, New York, pp 1378–1386

  8. Christos F, Neil S, Danai K, Joshua V, Gallagher T (2016) Deltacon: Principled massive-graph similarity function with attribution. ACM Trans Knowl Discov Data 10(3):28.1

  9. Fortunato S (2009) Community detection in graphs. Phys Rep 486(3-5)

  10. Holme P, Saram?ki J (2012) Temporal networks. Phys Rep 519(3):97–125

  11. Hooi B, Shin K, Song HA, Beutel A, Shah N, Faloutsos C (2017) Graph-based fraud detection in the face of camouflage. ACM Trans Knowl Discov Data 11(4)

  12. Hooi B, Song HA, Beutel A, Shah N, Shin K, Faloutsos C (2016) Fraudar: Bounding graph fraud in the face of camouflage. In: Proceedings of the 22Nd ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’16. ACM, New York, pp 895–904

  13. Hu M, Xu G, Ma C, Daneshmand M (2019) Detecting review spammer groups in dynamic review networks. In: Proceedings of the ACM turing celebration conference - China, ACM TURC ’19. Association for computing machinery

  14. Hu Y, Yang B, Lv C (2016) A local dynamic method for tracking communities and their evolution in dynamic networks. Knowl-Based Syst 110:176–190

    Article  Google Scholar 

  15. Jiang M, Cui P, Beutel A, Faloutsos C, Yang S (2014) Inferring strange behavior from connectivity pattern in social networks. In: Advances in knowledge discovery and data mining. Springer International Publishing, Cham, pp 126–138

  16. Jiang M, Cui P, Beutel A, Faloutsos C, Yang S (2016) Catching synchronized behaviors in large networks: A graph mining approach. ACM Trans Knowl Discov Data 10(4)

  17. Jurgovsky J, Granitzer M, Ziegler K, Calabretto S, Portier PE, He-Guelton L, Caelen O (2018) Sequence classification for credit-card fraud detection. Expert Syst Appl 100:234– 245

    Article  Google Scholar 

  18. Liu S, Hooi B, Faloutsos C (2017) Holoscope: Topology-and-spike aware fraud detection. In: Proceedings of the 2017 ACM on conference on information and knowledge management, CIKM ’17. ACM, New York, pp 1539–1548

  19. Liu S, Hooi B, Faloutsos C (2018) A contrast metric for fraud detection in rich graphs. IEEE Trans Knowl Data Eng:1–1

  20. Lu Z, Johan W, Arye N (2018) Community detection in complex networks via clique conductance. Sci Rep 8(1):5982

    Article  Google Scholar 

  21. Macha M, Akoglu L (2018) Explaining anomalies in groups with characterizing subspace rules. Data Min Knowl Discov 32(5):1444–1480

    Article  MathSciNet  Google Scholar 

  22. Manzoor E, Milajerdi SM, Akoglu L (2016) Fast memory-efficient anomaly detection in streaming heterogeneous graphs. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’16. Association for computing machinery, New York, pp 1035–1044

  23. Meng J, Beutel A, Peng C, Hooi B, Yang S, Faloutsos C (2016) Spotting suspicious behaviors in multimodal data: a general metric and algorithms. IEEE Trans Knowl Data Eng 28(8):2187–2200

    Article  Google Scholar 

  24. Mohammadmosaferi KK, Naderi H (2020) Evolution of communities in dynamic social networks: An efficient map-based approach. Expert Syst Appl 147:113221–

  25. Palla G, Barabasi AL, Vicsek T (2007) Quantifying social group evolution. Nature 446:664–7

    Article  Google Scholar 

  26. Peixoto T, Rosvall M (2017) Modeling sequences and temporal networks with dynamic community structures. Nat Commun:8

  27. Perozzi B, Akoglu L (2018) Discovering communities and anomalies in attributed graphs: interactive visual exploration and summarization. ACM Trans Knowl Discov Data 12(2)

  28. Ranshous S, Harenberg S, Sharma K, Samatova NF (2016) A scalable approach for outlier detection in edge streams using sketch-based approximations. In: Proceedings of the 2016 SIAM international conference on data mining

  29. Rayana S, Akoglu L (2015) Collective opinion spam detection: Bridging review networks and metadata. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’15. Association for Computing Machinery, New York, pp 985–994

  30. Rodrigues J (2014) Copycatch: stopping group attacks by spotting lockstep behavior in social networks. Comput Rev 55(8):509–509

    Google Scholar 

  31. Rossetti G, Cazabet R (2018) Community discovery in dynamic networks: a survey. ACM Comput Surv 51(2)

  32. Sachpenderis N, Koloniari G (2018) Determining interesting communities in evolving social networks, pp 249–254

  33. Shah N, Beutel A, Gallagher B, Faloutsos C (2014) Spotting suspicious link behavior with fbox: an adversarial perspective. In: 2014 IEEE International conference on data mining, pp 959–964

  34. Shah N, Beutel A, Hooi B, Akoglu L, Gunnemann S, Makhija D, Kumar M, Faloutsos C (2015) Edgecentric: Anomaly detection in edge-attributed networks. In: 2016 IEEE 16Th international conference on data mining workshops (ICDMW)

  35. Shin K, Eliassi-Rad T, Faloutsos C (2018) Patterns and anomalies in k-cores of real-world graphs with applications. Knowl Inf Syst 54(3):677–710

    Article  Google Scholar 

  36. Shin K, Hooi B, Faloutsos C (2016) M-zoom: Fast dense-block detection in tensors with quality guarantees. In: Joint european conference on machine learning and knowledge discovery in databases

  37. Shin K, Hooi B, Faloutsos C (2018) Fast, accurate, and flexible algorithms for dense subtensor mining. ACM Trans Knowl Discov Data 12(3)

  38. Shin K, Hooi B, Kim J, Faloutsos C (2017) D-cube: Dense-block detection in terabyte-scale tensors. In: Proceedings of the tenth ACM international conference on web search and data mining, WSDM ’17, pp 681–689

  39. Shin K, Hooi B, Kim J, Faloutsos C (2017) Densealert: Incremental dense-subtensor detection in tensor streams. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’17. Association for computing machinery, New York, pp 1057–1066

  40. Teng W, Fang C, Lin D, Felix S (2015) Localizing temporal anomalies in large evolving graphs. In: SDM, pp 927–935

  41. Traag VA, Waltman L, Eck NV (2019) From louvain to leiden: guaranteeing well-connected communities. Sci Rep 9(1)

  42. Van Vlasselaer V, Eliassi-Rad T, Akoglu L, Snoeck M, Baesens B (2017) Gotcha! network-based fraud detection for social security fraud. Manag Sci 63(9):3090–3110

    Article  Google Scholar 

  43. Wagenseller P, Wang F, Wu W (2018) Size matters: a comparative analysis of community detection algorithms. IEEE Trans Comput Soc Syst 5(4):951–960

    Article  Google Scholar 

  44. Wang H, Qiao C (2019) A nodes’ evolution diversity inspired method to detect anomalies in dynamic social networks. IEEE Trans Knowl Data Eng:1–1

  45. Yang Z, Algesheimer R, Tessone C (2016) A comparative analysis of community detection algorithms on artificial networks. Sci Rep:6

  46. Yoon M, Hooi B, Shin K, Faloutsos C (2020) Fast and accurate anomaly detection in dynamic graphs with a two-pronged approach

Download references

Acknowledgements

This work was partially supported by National Key R&D Program of China (2019YFB2101804). Dr. Guannan Liu’s work was supported by National Natural Science Foundation of China (NSFC) under Grant 92046025 and 71701007.

Author information

Authors and Affiliations

  1. Beihang University, Beijing, China

    Yan Jiang & Guannan Liu

Authors
  1. Yan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guannan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guannan Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, Y., Liu, G. Two-stage anomaly detection algorithm via dynamic community evolution in temporal graph. Appl Intell 52, 12222–12240 (2022). https://doi.org/10.1007/s10489-021-03109-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03109-4

Keywords

Search

Navigation