Abstract
Weakly supervised video anomaly detection is to distinguish anomalies from normal scenes and events in videos, under the setting that we only know whether there are abnormal events in a video, but the specific occurrence time of abnormal events is unknown. It is generally modeled as a MIL (multiple instance learning) problem, where video-level labels are provided to train an anomaly detector to obtain frame-level labels for videos. However, most existing methods generally overlook temporal information in abnormal videos (positive bags), and only use one sample (snippet) in the positive bag to train. The positive bag may include more useful information with high possibility. Therefore, we propose the Weakly Supervised Video Anomaly Detection Approach with Temporal and Positive Features, which consider both the temporal information and more positive samples for video anomaly detection. Its contributions can be summarized as follows: (1) we consider more temporal information and introduced the attention mechanism in our network, we use both local and global snippets’ features to enhance the temporal representation ability of these features. (2) We use more positive (abnormal) samples and its features in bags to train our model, so that more complementary and relevant information will make our model more robust and effective. (3) We consider not only the differences between normal samples and abnormal samples but also between abnormal samples and abnormal samples, which can help our proposed approach to excavate positive (abnormal) samples’ information more efficiently and adequately. Experimental results demonstrate the effectiveness of our proposed methods in the UCF-Crime and ShanghaiTech dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sultani, W., Chen, C., Shah, M.: Real-world anomaly detection in surveillance videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6479–6488 (2018)
Unusual crowd activity dataset of University of Minnesota. http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi
Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes (PAMI). IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 18–32 (2013)
Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 FPS in MATLAB. In: IEEE International Conference on Computer Vision (ICCV), pp. 2720–2727 (2013)
Liu, W., Lian, D., Luo, W., Gao, S.: Future frame prediction for anomaly detection - a new baseline. In: IEEE International Conference on Computer Vision (ICCV) (2018)
He, C., Shao, J., Sun, J.: An anomaly-introduced learning method for abnormal event detection. Multimedia Tools Appl. 77(22), 29573–29588 (2017). https://doi.org/10.1007/s11042-017-5255-z
Zhong, J., Li, N., Kong, W., Liu, S., Li, T.H., Li, G.: Graph convolutional label noise cleaner: train a plug-and-play action classifier for anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1237–1246 (2019)
Wu, P., et al.: Not only look, but also listen: learning multimodal violence detection under weak supervision. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12375, pp. 322–339. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58577-8_20
Tian, Y., Pang, G., Chen, Y., Singh, R., Verjans, J.W., Carneiro, G.: Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In: IEEE International Conference on Computer Vision (ICCV) (2021)
Wan, B., Fang, Y., Xia, X., Mei, J.: Weakly supervised video anomaly detection via center-guided discriminative learning. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2020)
Feng, J.-C., Hong, F.-T., Zheng, W.-S.: MIST: multiple instance self-training framework for video anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14009–14018 (2021)
Del Giorno, A., Bagnell, J.A., Hebert, M.: A discriminative framework for anomaly detection in large videos. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 334–349. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_21
Ionescu, R., Smeureanu, S., Alexe, B., Popescu, M.: Unmasking the abnormal events in video. In: IEEE International Conference on Computer Vision (ICCV) (2017)
Wang, S., Zeng, Y., Liu, Q., Zhu, C., Zhu, E., Yin, J.: Detecting abnormality without knowing normality: a two-stage approach for unsupervised video abnormal event detection. In: ACM International Conference on Multimedia (ACM MM) (2018)
Liu, W., Lian, D., Luo, W., Gao, S.: Future frame prediction for anomaly detection - a new baseline. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Ye, M., Peng, X., Gan, W., Wu, W., Qiao, Y.: AnoPCN: video anomaly detection via deep predictive coding network. In: ACM International Conference on Multimedia (ACM MM), pp. 1805–1813 (2019)
Zhong, J., Li, N., Kong, W., Liu, S., Li, T.H., Li, G.: Graph convolutional label noise cleaner: train a plug-and-play action classifier for anomaly detection. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1237–1246 (2019)
Bromley, J., Guyon, I., Lecun, Y., Sckinger, E., Shah, R.: Signature verification using a “Siamese” time delay neural network. In: Neural Information Processing Systems (NeurIPS) (1994)
Hong, F., Huang, X., Li, W., Zheng, W.: Mini-Net: multiple instance ranking network for video highlight detection. arXiv preprint arXiv:2007.09833 (2020)
Nguyen, P., Liu, T., Prasad, G., Han, B.: Weakly supervised action localization by sparse temporal pooling network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Sohn, K.: Improved deep metric learning with multiclass n-pair loss objective. In: Neural Information Processing Systems (NeurIPS) (2016)
Song, H., Xiang, Y., Jegelka, S., Savarese, S.: Deep metric learning via lifted structured feature embedding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Liu, W., Luo, W., Li, Z., Zhao, P., Gao, S., et al.: Margin learning embedded prediction for video anomaly detection with a few anomalies. In: International Joint Conference on Artificial Intelligence (IJCAI) (2019)
Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6299–6308 (2017)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Cao, Y., Xu, J., Lin, S., et al.: GCNet: non-local networks meet squeeze-excitation networks and beyond. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). IEEE (2020)
Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 733–742 (2016)
Wang, J., Cherian, A.: GODS: generalized one-class discriminative subspaces for anomaly detection. In: IEEE International Conference on Computer Vision (CVPR), pp. 8201–8211 (2019)
Wan, B., Fang, Y., Xia, X., Mei, J.: Weakly supervised video anomaly detection via center-guided discriminative learning. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2020)
Georgescu, M.-I., Barbalau, A., Ionescu, R.T., Khan, F.S., Popescu, M., Shah, M.: Anomaly detection in video via self-supervised and multi-task learning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12742–12752 (2021)
Wang, Z., Zou, Y., Zhang, Z.: Cluster attention contrast for video anomaly detection. In: ACM International Conference on Multimedia (ACM MM), pp. 2463–2471 (2020)
Cai, R., Zhang, H., Liu, W., Gao, S., Hao, Z.: Appearance-motion memory consistency network for video anomaly detection. In: Association for the Advancement of Artificial Intelligence (AAAI), pp. 938–946 (2021)
Zaheer, M.Z., Mahmood, A., Astrid, M., Lee, S.-I.: CLAWS: clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12367, pp. 358–376. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58542-6_22
Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3d convolutional networks. In: IEEE international conference on Computer Vision (ICCV), pp. 4489–4497 (2015)
Acknowledgments
This work was supported by the grants from the National Natural Science Foundation of China (61925201, 62132001, U21B2025) and the National Key R &D Program of China (2021YFF0901502).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pi, R., He, X., Peng, Y. (2022). Weakly Supervised Video Anomaly Detection with Temporal and Abnormal Information. In: Yu, S., et al. Pattern Recognition and Computer Vision. PRCV 2022. Lecture Notes in Computer Science, vol 13536. Springer, Cham. https://doi.org/10.1007/978-3-031-18913-5_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-18913-5_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18912-8
Online ISBN: 978-3-031-18913-5
eBook Packages: Computer ScienceComputer Science (R0)