Abstract
As a significant research hotspot in the field of computer vision, video anomaly detection plays an essential role in ensuring public safety. Anomaly detection remains a challenging task given the complex situation in public areas and the large random distribution of crowds. The density of people in the same scene varies greatly due to the instability of the pedestrian volume. Specifically, the characteristics of crowd distribution mainly include low density, small aggregation and dispersion, or large aggregation and severe occlusion. Considering the large difference between high-density and low-density crowd characteristics, we propose an anomaly detection algorithm based on scene classification in order to obtain better anomaly detection result. Firstly, we propose a novel scene classification method, which uses pre-trained YoloV4 model to detect the number of people in the video frames and generate heatmaps, and extracts pixel features through the Double-Canny algorithm to represent the occlusion degree of the crowd. Furthermore, K-Means clustering is used to adaptively divide the scene into sparse and dense. Secondly, the Generative Adversarial Network (GAN) based on prediction and reconstruction is introduced to detect anomalies respectively, and the final accuracy is achieved by combining the detection accuracy of both networks. Finally, experiments on three benchmark datasets demonstrate the competitive performance of our method with the state-of-the-art methods.
Similar content being viewed by others
Data availability
We provide original and editable data appearing in the submitted article, including figures, tables and experimental results.
Code availability
We are pleased to share code that is used in work submitted for publication. Authors' contributions: All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Hongjun Li, Xulin Shen, Xiaohu Sun, Yunlong Wang, Chaobo Li, Junjie Chen. The first draft of the manuscript was written by Xulin Shen and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
References
Abati D, Porrello A, Calderara S et al (2019) Latent space autoregression for novelty detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 481–490. IEEE Computer Society, Long Beach, USA
Alafif Tarik et al (2022) Generative adversarial network based abnormal behavior detection in massive crowd videos: a hajj case study. J Ambient Intell Humaniz Comput 13(8):4077–4088
Alanazi AA, Bilal M (2019) Crowd density estimation using novel feature descriptor. arXiv preprint arXiv:1905.05891
Bhuiyan MR, Abdullah J, Hashim N et al (2022) A deep crowd density classification model for Hajj pilgrimage using fully convolutional neural network. PeerJ Comput Sci 8:e895
Bhuiyan R, Abdullah J, Hashim N et al (2022) Deep dilated convolutional neural network for crowd density image classification with dataset augmentation for hajj pilgrimage. Sensors 22(14):5102
Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934
Chen D, Yue L, Chang X et al (2021) NM-GAN: Noise-modulated generative adversarial network for video anomaly detection. Pattern Recognit 116:107969
Gong S, Bourennane EB (2019) A method based on texture feature and edge detection for people counting in a crowded area. In: Digital Image and Signal Processing
Gong D, Liu L, Le V et al (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1705–1714. Institute of Electrical and Electronics Engineers Inc, Seoul, Korea
Goodfellow IJ, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial networks. Adv Neural Inf Process Syst 3:2672–2680
Hussain N, Yatim HSM, Hussain NL et al (2011) CDES: A pixel-based crowd density estimation system for Masjid al-Haram. Saf Sci 49(6):824–833
Huynh VS, Tran VH, Huang CC (2019) Iuml: Inception u-net based multi-task learning for density level classification and crowd density estimation. In: IEEE International Conference on Systems, Man and Cybernetics (SMC), pp 3019–3024. IEEE
Jia D, Zhang C, Zhang B (2021) Crowd density classification method based on pixels and texture features. Mach Vis Appl 32(2):1–22
Jiang X, Zhang L, Xu M et al (2020) Attention scaling for crowd counting. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 4705–4714. IEEE Computer Society, Seattle, USA
Lamba S, Nain N (2017) A large scale crowd density classification using spatio-temporal local binary pattern. In: 13th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp 296–302. IEEE
Lamba S, Nain N (2019) A texture based mani-fold approach for crowd density estimation using Gaussian Markov Random Field. Multimedia Tools Apply 78(5):5645–5664
Lazaridis L, Dimou A, Daras P (2018) Abnormal behavior detection in crowded scenes using density heatmaps and optical flow. In: 26th European Signal Processing Conference (EUSIPCO), pp 2060–2064. European Signal Processing Conference, Rome
Lee S, Kim HG, Ro YM (2020) BMAN: bidirectional multi-scale aggregation networks for abnormal event detection. IEEE Trans Image Process 29:2395–2408
Lei Z, Deng F, Yang X (2019) Spatial temporal balanced generative adversarial autoencoder for anomaly detection. In: Proceedings of the 2019 International Conference on Image, Video and Signal Processing, pp 1–7
Li N, Chang F, Liu C (2020) Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes. IEEE Trans Multimed 23:203–215
Liu W, Luo W, Lian D et al (2018) Future frame prediction for anomaly detection - a new baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6536–6545. IEEE Computer Society, Salt Lake City
Marsden M, McGuinness K, Little S et al (2017) Resnetcrowd: A residual deep learning architecture for crowd counting, violent behaviour detection and crowd density level classification. In: 14th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 1–7. IEEE
Naeem H, Cheng X, Ullah F et al (2022) A deep convolutional neural network stacked ensemble for malware threat classification in internet of things. J Circuits Syst Comput 31:2250302
Ravanbakhsh M, Sangineto E, Nabi M et al (2019) Training adversarial discriminators for cross-channel abnormal event detection in crowds. In: 19th IEEE Winter Conference on Applications of Computer Vision, pp 1896–1904. Institute of Electrical and Electronics Engineers Inc, Hilton Waikoloa Village
Sabokrou M, Khalooei M, Fathy M et al (2018) Adversarially learned one-class classifier for novelty detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3379–3388. IEEE Computer Society, Salt Lake City, USA
Samriya JK, Tiwari R, Cheng X et al (2022) Network intrusion detection using ACO-DNN model with DVFS based energy optimization in cloud framework. Sustain Comput Inform Syst 35:100746
Song H, Sun C, Wu X et al (2019) Learning normal patterns via adversarial attention-based autoencoder for abnormal event detection in videos. IEEE Trans Multimedia 22(8):2138–2148
Wang P, Wang P, Fan E (2021) Violence detection and face recognition based on deep learning. Pattern Recogn Lett 142:20–24
Xiong G, Cheng J, Wu X et al (2012) An energy model approach to people counting for abnormal crowd behavior detection. Neurocomputing 83:121–135
Xu M, Ge Z, Jiang X et al (2019) Depth information guided crowd counting for complex crowd scenes. Pattern Recogn Lett 125:563–569
Zhu L, Li C, Yang Z et al (2020) Crowd density estimation based on classification activation map and patch density level. Neural Comput Appl 32(9):5105–5116
Funding
This work is supported in part by National Natural Science Foundation of China under Grant 61871241, Grant 61971245 and Grant 61976120, in part by Jiangsu Industry University Research Cooperation Project BY2021349, in part by Nantong Science and Technology Program JC2021131 and in part by Postgraduate Research and Practice Innovation Program of Jiangsu Province KYCX21_3084 and KYCX22_3340.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest/Competing interests
None.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, H., Shen, X., Sun, X. et al. Video anomaly detection based on scene classification. Multimed Tools Appl 82, 45345–45365 (2023). https://doi.org/10.1007/s11042-023-15328-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-15328-7