Abstract
With the popularity of surveillance video, people’s increased security awareness and demand for risk warning, video anomaly detection is gradually gaining widespread attention. Since the amount of data in the background is larger than that of the foreground object, the background information of video frames is more easily noticed by the model, while the abnormal behaviors that cannot be easily detected mostly occur in the foreground. At the same time, if the background information is processed separately from the foreground information, it will inevitably increase the complexity of the model. In this paper, we propose a novel video anomaly detection based on scene object. In the training phase, we put foreground objects on the canvas for training and the memory module remembers various normal patterns of the foreground objects, so that the model can be more sensitive to the anomalies in the foreground objects during detection. Our method is able to achieve AUC 74.01% and 30 FPS on ShanghaiTech dataset. At the same time, several advanced video anomaly detection algorithms are compared, which all demonstrate the superiority of our method.
Similar content being viewed by others
Data availability
We provide original and editable data appearing in the submitted article, including figures, tables and experimental results.
Code availability
We are pleased to share code that is used in work submitted for publication.
References
Bahrami M, Pourahmadi M, Vafaei A, Shayesteh MR (2021) A comparative study between single and multi-frame anomaly detection and localization in recorded video streams. J Vis Commun Image Represent 79(1):1–10
Chang, YP, Tu, ZG, Xie, W, Yuan, JS (2020) Clustering driven deep autoencoder for video anomaly detection. In: European Conference on Computer Vision, Glasgow, pp 329–345
Chang YP, Tu ZG, Xie W, Luo B, Zhang SF, Sui HG, Yuan JS (2022) Video anomaly detection with spatio-temporal dissociation. Pattern Recogn 122(1):1–12
Feng, JC, Hong, FT, Zheng, WS (2021) MIST: multiple instance self-training framework for video anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 14009–14018. Virtual
Gong, D, Liu, LQ, Le, V, Saha, B, Mansour, MR, Venkatesh, S, Hengel, AV (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, pp 1705–1714
He KM, Zhang XY, Ren SQ Sun (2017) Simple online and Realtime tracking with a deep association metric. In: IEEE International Conference on Image Processing, Beijing, pp 3645–3649
Lee S, Kim HG, Choi DH, Kim H, Ro YM (2021) Video prediction recalling Long-term motion context via memory alignment learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3053–3062. Virtual
Li B, Zhang S, Li K(2017)Towards a multi-layers anomaly detection framework for analyzing network traffic. Concurrency and Computation: Practice and Experience 29(14):1-13
Li B, Leroux S, Simoens P (2021) Decoupled appearance and motion learning for efficient anomaly detection in surveillance video. Comput Vis Image Underst 210(9):1–10
Li Q, Yang R, Xiao F, Bhanu B, Zhang F (2022) Attention-based anomaly detection in multi-view surveillance videos. Knowl-Based Syst 252(9):109348
Li DH, Nie XS, Li XF, Zhang Y, Yin YL (2022) Context-related video anomaly detection via generative adversarial network. Pattern Recogn Lett 156(4):183–189
Li CB, Li HJ, Zhang GA (2023) Future frame prediction based on generative assistant discriminative network for anomaly detection. Appl Intell 53(1):542–559
Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollar P, Zitnick CL (2014) Microsoft COCO: Common objects in context. In: European Conference on Computer Vision, Zurich, pp 740–755
Lin TY, Goyal P, Girshick R, He K, Dollar P (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 42(2):318–317
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: Single shot multibox detector. In: European Conference on Computer Vision, Amsterdam, pp 21–37
Liu W, Luo WX, Lian DZ, Gao SH (2018) Future frame prediction for anomaly detection–a new baseline. In: IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, pp 6536–6545
Liu ZA, Nie YW, Long CJ, Zhang Q, Li GQ (2021) A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction. In: International Conference on Computer Vision, Montreal, pp 13588–13597
Luo, WX, Liu, W, Gao, SH (2017) A revisit of sparse coding based anomaly detection in stacked RNN framework. In: International Conference on Computer Vision, Venice, pp 341–349
Lv H, Chen C, Cui Z, Xu CY, Li Y, Yang J (2021) Learning normal dynamics in videos with meta prototype network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 15420–15429. Virtual
Nazia A, Prateek KR, Maheshkumar HK (2022) A3N: Attention-based adversarial autoencoder network for detecting anomalies in video sequence. J Vis Commun Image Represent 87(8):103598
Olaf R., Philipp F, Thomas B (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, Munich, pp 234–241
Park H, Noh J, Ham B (2020) Learning memory-guided normality for anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Seattle, pp 14360–14369
Ren SQ, He KM, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149
Saligrama V, Konrad J, Jodoin P (2010) Video anomaly identification. IEEE Signal Process Mag 27(5):18–33
Sun P, Zhang RF, Jiang Y, Kong T, Xu CF, Zhan W, Tomizuka M, Li L, Yuan ZH, Wang CH, Luo P (2021) Sparse R-CNN: end-to-end object detection with learnable proposals. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 14449–14458. Virtual
Ultralytics/ yolov5 (2020) Available online: https://github.com/ultralytics/yolov5 (accessed on 25 June 2020)
Wojke N, Bewley A, Paulus D (2017) Simple online and Realtime tracking with a deep association metric. In: IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, pp 3645–3649
Ye M, Peng X, Gan W, Wu W, Qiao Y (2019) Anopcn:Video anomaly detection via deep predictive coding network. In: Proceedings of the 27th ACM International Conference on Multimedia, New York, pp 1805-1813
Zhang Y, Hutchinson P, Lieven NAJ, Nunez-Yanez J (2020) Remaining useful life estimation using long short-term memory neural networks and deep fusion. IEEE Access 8(1):19033–19045
Zhong YH, Chen X, Jiang JY, Ren F (2022) A cascade reconstruction model with generalization ability evaluation for anomaly detection in videos. Pattern Recogn 122(2):1–13
Funding
This work is supported in part by National Natural Science Foundation of China under Grant 61871241, Grant 61971245 and Grant 61976120, in part by Jiangsu Industry University Research Cooperation Project BY2021349, in part by Nantong Science and Technology Program JC2021131 and in part by Postgraduate Research and Practice Innovation Program of Jiangsu Province KYCX21_3084 and KYCX22_3340.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Hongjun Li, Jinyi Chen, Xiezhou Huang, Yuxing Zhang,Yunlong Du and Junjie Chen. The first draft of the manuscript was written by Jinyi Chen and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflicts of interest/competing interests
None.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, H., Chen, J., Huang, X. et al. FOAD: a novel video anomaly detection focusing on objects. Multimed Tools Appl 83, 20637–20651 (2024). https://doi.org/10.1007/s11042-023-16429-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-16429-z