Skip to main content
SpringerLink
Log in

Evolving graph-based video crowd anomaly detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Detecting anomalous crowd behavioral patterns from videos is an important task in video surveillance and maintaining public safety. In this work, we propose a novel architecture to detect anomalous patterns of crowd movements via graph networks. We represent individuals as nodes and individual movements with respect to other people as the node-edge relationship of an evolving graph network. We then extract the motion information of individuals using optical flow between video frames and represent their motion patterns using graph edge weights. In particular, we detect the anomalies in crowded videos by modeling pedestrian movements as graphs and then by identifying the network bottlenecks through a max-flow/min-cut pedestrian flow optimization scheme (MFMCPOS). The experiment demonstrates that the proposed framework achieves superior detection performance compared to other recently published state-of-the-art methods. Considering that our proposed approach has relatively low computational complexity and can be used in real-time environments, which is crucial for present day video analytics for automated surveillance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The datasets and code generated during and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Rao, A.S., Gubbi, J., Marusic, S., Palaniswami, M.: Estimation of crowd density by clustering motion cues. The Visual Computer 31, 1533–1552 (2015)

    Article  Google Scholar 

  2. Li, L., Huang, W., Gu, I.Y.-H., Luo, R., Tian, Q.: An efficient sequential approach to tracking multiple objects through crowds for real-time intelligent cctv systems. IEEE Trans. on Systems, Man, and Cybernetics, Part B 38, 1254–1269 (2008)

    Article  Google Scholar 

  3. Yang, M., Rashidi, L., Rajasegarar, S., Leckie, C.: Graph stream mining based anomalous event analysis. In: Pacific Rim International Conference on Artificial Intelligence, pp. 891–903. Springer (2018)

  4. Yang, M., Rashidi, L., Rajasegarar, S., Leckie, C., Rao, A.S., Palaniswami, M.: Crowd activity change point detection in videos via graph stream mining. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, pp. 215–223 (2018)

  5. Moshtaghi, M., Rajasegarar, S., Leckie, C., Karunasekera, S.: An efficient hyperellipsoidal clustering algorithm for resource-constrained environments. Pattern Recognition 44(9), 2197–2209 (2011)

    Article  Google Scholar 

  6. Bouguet, J.-Y.: Pyramidal implementation of the affine Lucas Kanade feature tracker description of the algorithm. Intel Corporation 5, 4 (2001)

    Google Scholar 

  7. Mo, X., Monga, V., Bala, R., Fan, Z.: Adaptive sparse representations for video anomaly detection. IEEE Trans. on Circuits and Systems for Video Technology 24, 631–645 (2014)

    Article  Google Scholar 

  8. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942 (2009). IEEE

  9. Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1975–1981 (2010). IEEE

  10. Kim, J., Grauman, K.: Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2928. IEEE (2009)

  11. Bansod, S.D., Nandedkar, A.V.: Crowd anomaly detection and localization using histogram of magnitude and momentum. The Visual Computer 36(3), 609–620 (2020)

    Article  Google Scholar 

  12. Farooq, M.U., Saad, M.N.M., Khan, S.D.: Motion-shape-based deep learning approach for divergence behavior detection in high-density crowd. The Visual Computer 38(5), 1553–1577 (2022)

    Article  Google Scholar 

  13. Sabih, M., Vishwakarma, D.K.: Crowd anomaly detection with LSTMs using optical features and domain knowledge for improved inferring. The Visual Computer 38(5), 1719–1730 (2022)

    Article  Google Scholar 

  14. Patel, A.S., Vyas, R., Vyas, O., Ojha, M., Tiwari, V.: Motion-compensated online object tracking for activity detection and crowd behavior analysis. The Visual Computer 1–21 (2022)

  15. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  16. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  17. Xu, D., Ricci, E., Yan, Y., Song, J., Sebe, N.: Learning deep representations of appearance and motion for anomalous event detection. In: Proceedings of the British Machine Vision Conference 2015, BMVC, pp. 801–812 (2015)

  18. Sabokrou, M., Fayyaz, M., Fathy, M., Moayed, Z., Klette, R.: Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes. Computer Vision and Image Understanding (2018)

  19. Morais, R., Le, V., Tran, T., Saha, B., Mansour, M., Venkatesh, S.: Learning regularity in skeleton trajectories for anomaly detection in videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11996–12004 (2019)

  20. Dubey, S., Boragule, A., Gwak, J., Jeon, M.: Anomalous event recognition in videos based on joint learning of motion and appearance with multiple ranking measures. Applied Sciences 11(3), 1344 (2021)

    Article  Google Scholar 

  21. Ravanbakhsh, M., Nabi, M., Mousavi, H., Sangineto, E., Sebe, N.: Plug-and-play CNN for crowd motion analysis: An application in abnormal event detection. In: Winter Conf. on Applications of Computer Vision (WACV), pp. 1689–1698. IEEE (2018)

  22. Xu, K., Sun, T., Jiang, X.: Video anomaly detection and localization based on an adaptive intra-frame classification network. IEEE Transactions on Multimedia 22(2), 394–406 (2019)

    Article  Google Scholar 

  23. Lucas, B.: An iterative image registration technique with an application to stereo vision. In: DARPA Image Understanding Workshop, pp. 121–130 (1981)

  24. Zhou, B., Wang, X., Tang, X.: Random field topic model for semantic region analysis in crowded scenes from tracklets. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3441–3448. IEEE (2011)

  25. Tomasi, C., Kanade, T.: Detection and tracking of point features. CMU Technical Report (1991)

  26. Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: 9th European Conference on Computer Vision, pp. 404–417. Springer (2006)

  27. Zeppelzauer, M., Zaharieva, M., Mitrovic, D., Breiteneder, C.: A novel trajectory clustering approach for motion segmentation. In: Intern Conf. on Multimedia Modeling, pp. 433–443. Springer (2010)

  28. Chaker, R., Al Aghbari, Z., Junejo, I.N.: Social network model for crowd anomaly detection and localization. Pattern Recognition 61, 266–281 (2017)

    Article  Google Scholar 

  29. Dantzig, G., Fulkerson, D.R.: On the max flow min cut theorem of networks. Linear Inequalities and Related Systems 38, 225–231 (2003)

    Google Scholar 

  30. Schroeder, J., Guedes, A., Duarte Jr, E.P.: Computing the minimum cut and maximum flow of undirected graphs. RelatórioTécnico RT-DINF 003/2004 (2004)

  31. Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes. IEEE Trans. on Pattern Analysis and Machine Intelligence 36, 18–32 (2014)

    Article  Google Scholar 

  32. Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 FPS in Matlab. In: IEEE International Conference on Computer Vision, pp. 2720–2727 (2013)

  33. Reddy, V., Sanderson, C., Lovell, B.C.: Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, pp. 55–61. IEEE (2011)

  34. Cong, Y., Yuan, J., Liu, J.: Sparse reconstruction cost for abnormal event detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3449–3456. IEEE (2011)

  35. Liu, W., Luo, W., Lian, D., Gao, S.: Future frame prediction for anomaly detection–a new baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6536–6545 (2018)

  36. Luo, W., Liu, W., Lian, D., Tang, J., Duan, L., Peng, X., Gao, S.: Video anomaly detection with sparse coding inspired deep neural networks. IEEE transactions on pattern analysis and machine intelligence 43(3), 1070–1084 (2019)

    Article  Google Scholar 

  37. Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–742 (2016)

  38. Smeureanu, S., Ionescu, R.T., Popescu, M., Alexe, B.: Deep appearance features for abnormal behavior detection in video. In: International Conference on Image Analysis and Processing, pp. 779–789. Springer (2017)

  39. Luo, W., Liu, W., Gao, S.: Remembering history with convolutional lstm for anomaly detection. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 439–444. IEEE (2017)

  40. Ionescu, R.T., Smeureanu, S., Popescu, M., Alexe, B.: Detecting abnormal events in video using narrowed normality clusters. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1951–1960 (2019)

  41. Luo, W., Liu, W., Gao, S.: A revisit of sparse coding based anomaly detection in stacked RNN framework. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 341–349 (2017)

Download references

Acknowledgements

The authors are very grateful to Editor and the anonymous reviewers for their valuable comments and suggestions that improved the presentation and quality of this paper highly. This work was supported by the Natural Science Foundation of China under Grants 12201523 and also supported by the Fundamental Research Funds for the Central Universities under Grants No. 2682021CX078.

Author information

Authors and Affiliations

  1. School of Mathematics, Southwest Jiaotong University, Chengdu, 610000, China

    Meng Yang, Shucong Tian & Zhengchun Zhou

  2. School of Systems Engineering, National University of Defense Technology, Changsha, 410073, China

    Yanghe Feng

  3. Department of Electrical and Electronic Engineering, The University of Melbourne, Melbourne, 3010, Australia

    Aravinda S. Rao

  4. School of Information Technology, Deakin University, Geelong, 3125, Australia

    Sutharshan Rajasegarar

Authors
  1. Meng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanghe Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aravinda S. Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sutharshan Rajasegarar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shucong Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhengchun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanghe Feng.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, M., Feng, Y., Rao, A.S. et al. Evolving graph-based video crowd anomaly detection. Vis Comput 40, 303–318 (2024). https://doi.org/10.1007/s00371-023-02783-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02783-4

Keywords

Search

Navigation