Skip to main content
SpringerLink
Log in

Unsupervised group-based crowd dynamic behavior detection and tracking in online video sequences

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Analysis of video sequences of public places is an important topic in video surveillance systems. Due to the high probability of occurring abnormal behavior in crowded scene, the main purpose of many surveillance systems is to monitor the crowd movement, and detection of abnormalities. To speed up this process and also for error reduction, it is highly important to use automated and intelligent tools in surveillance systems, as an alternative to the human operator. This study presents an unsupervised and online algorithm for analysis of dynamic crowd behavior, which uses the proposed features, with the capability to analyze crowds over time and reveal different behaviors of the crowd groups. In the proposed algorithm, prominent points are initially tracked. These key points are processed by the proposed system that includes removing the fixed points, employing proposed features of the moving points, automated determination of neighborhood, the similarity of the invariant neighbors. Group clustering is done automatically and the classification stage is conducted without the training phase. The dynamic behavior of the crowd is examined using the features and the extracted group properties and different states in the scene are diagnosed by dynamic thresholding. Experimental evaluation of the proposed method on several databases shows that it is performed properly in video sequences and it is able to detect various abnormal behaviors in the crowd scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

Data available on request from the authors.

References

  1. Zhou B, Wang X, Tang X (2012) Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: IEEE conference on computer vision and pattern recognition, pp 2871–2878

  2. Wu S, Moore BE, Shah M (2010) Chaotic invariants of lagrangian particle trajectories for anomaly detection in crowded scenes. In: IEEE computer society conference on computer vision and pattern recognition, pp 2054–2060

  3. Ferryman J, Shahrokni A (2009) Pets2009: dataset and challenge. In: 12th IEEE international workshop on performance evaluation of tracking and surveillance, pp 1–6

  4. Available: http://ee.cuhk.edu.hk/en-gb/~jshao/CUHKcrowd_files/cuhk_crowd_dataset.htm

  5. Li T, Chang H, Wang M, Ni B, Hong R, Yan S (2015) Crowded scene analysis: a survey. IEEE Trans Circuits Syst Video Technol 25:367–386

    Article  Google Scholar 

  6. Sodemann AA, Ross MP, Borghetti BJ (2012) A review of anomaly detection in automated surveillance. IEEE Trans Syst Man Cybern Part C (Appl Rev) 42:1257–1272

    Article  Google Scholar 

  7. Hao Y, Xu Z-J, Liu Y, Wang J, Fan J-L (2019) Effective crowd anomaly detection through spatio-temporal texture analysis. Int J Autom Comput 16:27–39

    Article  Google Scholar 

  8. Sabih M, Vishwakarma DK (2021) Crowd anomaly detection with LSTMs using optical features and domain knowledge for improved inferring. Vis Comput 38(5):1719–1730. https://doi.org/10.1007/s00371-021-02100-x

    Article  Google Scholar 

  9. Zou Y, Zhao X, Liu Y (2018) Measuring crowd collectiveness by macro-microscopic motion consistency. IEEE Trans Multimed 20:3311–3323. https://doi.org/10.1109/TMM.2018.2832601

    Article  Google Scholar 

  10. Ding Y, Sun Y, Li Z (2023) Who is partner: a new perspective on data association of multi-object tracking. Image Vision Comput 2023:104737

    Article  Google Scholar 

  11. Xue Y, Li Y, Liu S, Zhang X, Qian X (2021) Crowd scene analysis encounters high density and scale variation. IEEE Trans Image Process 30:2745–2757

    Article  Google Scholar 

  12. Yan R, Xie L, Tang J, Shu X, Tian Q (2020) HiGCIN: hierarchical graph-based cross inference network for group activity recognition. IEEE Trans Pattern Anal Machine Intell 45(6):6955–6968

    Article  Google Scholar 

  13. Xin L, Wang P, Chan C-Y, Chen J, Li SE, Cheng B (2018) Intention-aware long horizon trajectory prediction of surrounding vehicles using dual LSTM networks. In: 2018 21st international conference on intelligent transportation systems (ITSC), pp 1441–1446.

  14. Zhang T, Yang Z, Jia W, Yang B, Yang J, He X (2016) A new method for violence detection in surveillance scenes. Multimed Tools Appl 75:7327–7349

    Article  Google Scholar 

  15. Solmaz B, Moore BE, Shah M (2012) Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Trans Pattern Anal Mach Intell 34:2064–2070

    Article  Google Scholar 

  16. Shao J, Loy CC, Wang X (2017) Learning scene-independent group descriptors for crowd understanding. IEEE Trans Circuits Syst Video Technol 27:1290–1303. https://doi.org/10.1109/TCSVT.2016.2539878

    Article  Google Scholar 

  17. Zhou B, Tang X, Wang X (2012) Coherent filtering: Detecting coherent motions from crowd clutters. In: European conference on computer vision, pp 857–871

  18. Mingliang X, Li C, Lv P, Lin N, Hou R, Zhou B (2018) An efficient method of crowd aggregation computation in public areas. IEEE Trans Circuits Syst Video Technol 28(10):2814–2825. https://doi.org/10.1109/TCSVT.2017.2731866

    Article  Google Scholar 

  19. Ullah H, Uzair M, Ullah M, Khan A, Ahmad A, Khan W (2017) Density independent hydrodynamics model for crowd coherency detection. Neurocomputing 242:28–39

    Article  Google Scholar 

  20. Chen X-H, Lai J-H (2019) Detecting abnormal crowd behaviors based on the div-curl characteristics of flow fields. Pattern Recogn 88:342–355

    Article  Google Scholar 

  21. Wu S, Wong H-S, Yu Z (2014) A Bayesian model for crowd escape behavior detection. IEEE Trans Circuits Syst Video Technol 24:85–98

    Article  Google Scholar 

  22. Dehghan A, Idrees H, Zamir AR, Shah M (2014) Automatic detection and tracking of pedestrians in videos with various crowd densities. In: Weidmann U, Kirsch U, Schreckenberg M (eds) Pedestrian and evacuation dynamics 2012. Springer International Publishing, Cham, pp 3–19. https://doi.org/10.1007/978-3-319-02447-9_1

    Chapter  Google Scholar 

  23. Davchev T, Burke M, Ramamoorthy S (2020) Learning structured representations of spatial and interactive dynamics for trajectory prediction in crowded scenes. IEEE Robot Autom Letter 6:707–714

    Article  Google Scholar 

  24. Qiao Z, Zhao L, Gu L, Jiang X, Li R, Ge L (2021) Research on abnormal pedestrian trajectory detection of dynamic crowds in public scenarios. IEEE Sens J 21:23046–23054

    Article  Google Scholar 

  25. Chen D-Y, Huang P-C (2011) Motion-based unusual event detection in human crowds. J Vis Commun Image Represent 22:178–186

    Article  Google Scholar 

  26. Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. In: IEEE conference on computer vision and pattern recognition pp 935–942

  27. Pennisi A, Bloisi DD, Iocchi L (2016) Online real-time crowd behavior detection in video sequences. Comput Vis Image Underst 144:166–176

    Article  Google Scholar 

  28. Dutta S, Burman S, Mazumdar A, Roy ND (2020) Crowd behavior analysis and alert system using image processing. In: Mandal JK, Bhattacharya D (eds) Emerging Technology in Modelling and Graphics. Springer, Singapore, pp 721–729

    Chapter  Google Scholar 

  29. Qin T, Li P, Shen S (2018) Vins-mono: a robust and versatile monocular visual-inertial state estimator. IEEE Trans Rob 34:1004–1020

    Article  Google Scholar 

  30. Wang H, Oneata D, Verbeek J, Schmid C (2016) A robust and efficient video representation for action recognition. Int J Comput Vision 119:219–238

    Article  MathSciNet  Google Scholar 

  31. Fradi H, Dugelay J-L (2012) Low level crowd analysis using frame-wise normalized feature for people counting. In: IEEE international workshop on information forensics and security (WIFS), pp 246–251

  32. Srivastava S, Ng KK, Delp EJ (2011) Crowd flow estimation using multiple visual features for scenes with changing crowd densities. In: 2011 8th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 60–65

  33. Zhou B, Tang X, Zhang H, Wang X (2014) Measuring crowd collectiveness. IEEE Trans Pattern Anal Mach Intell 36:1586–1599

    Article  Google Scholar 

  34. Li J, Cheng K, Wang S, Morstatter F, Trevino RP, Tang J et al (2017) Feature selection: a data perspective. ACM Comput Surv (CSUR) 50:1–45

    Google Scholar 

  35. Available: http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi

  36. Jiang X, Ma J, Xiao G, Shao Z, Guo X (2021) A review of multimodal image matching: methods and applications. Inf Fusion 73:22–71

    Article  Google Scholar 

  37. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60:91–110

    Article  Google Scholar 

  38. Rosten E, Drummond T (2005) Fusing points and lines for high performance tracking. In: ICCV, pp 1508–1515

  39. Dutta KK, GS AK (2015) Double handed Indian Sign Language to speech and text. In: 2015 3rd international conference on image information processing (ICIIP), pp 374–377

  40. Bouwmans T, El Baf F, Vachon B (2008) Background modeling using mixture of gaussians for foreground detection-a survey. Recent Patents Comput Sci 1:219–237

    Article  Google Scholar 

  41. Xu Y, Dong J, Zhang B, Xu D (2016) Background modeling methods in video analysis: a review and comparative evaluation. CAAI Trans Intell Technol 1:43–60

    Article  Google Scholar 

  42. Xiong G, Cheng J, Wu X, Chen Y-L, Ou Y, Xu Y (2012) An energy model approach to people counting for abnormal crowd behavior detection. Neurocomputing 83:121–135

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, University of Guilan, Rasht, Guilan, Iran

    Atefeh Ghorbanpour & Manoochehr Nahvi

Authors
  1. Atefeh Ghorbanpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manoochehr Nahvi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manoochehr Nahvi.

Ethics declarations

Conflict of interest

Authors certified that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghorbanpour, A., Nahvi, M. Unsupervised group-based crowd dynamic behavior detection and tracking in online video sequences. Pattern Anal Applic 27, 55 (2024). https://doi.org/10.1007/s10044-024-01279-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10044-024-01279-8

Keywords

Search

Navigation