Skip to main content
SpringerLink
Log in

Grey-adversary perceptual network for anomaly detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The task of anomaly detection in surveillance videos is challenging due to the sparsity and diversity. In order to perceive more discriminative features and further improve performance, a grey-adversary perceptual network is proposed for anomaly detection. Our method is designed as a combination of frame prediction stage and frame optimization stage. The former stage introduces a grey perceptual unit based on Deng’s grey relation, which perceives encoding features from encoder and outputs perceptual features for decoder, improving the capacity of anomaly perception. The latter one designs a discrimination network to learn more detailed features for small abnormal regions, and the grey absolute relation is imported to enhance the robustness against illumination, reducing the false detection. The training stage is performed by adversarial learning. Extensive experiments on UCSD Ped2, CUHK Avenue and ShanghaiTech datasets reach the averaged AUC of 97.6%, 88.9% and 73.7%, respectively. The comparison results with state-of-the-art methods demonstrate that the effectiveness and advantage of our method in anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

We provide original and editable data appearing in the submitted article, including figures, tables and experimental results.

Code availability

We are pleased to share code that is used in work submitted for publication.

References

  1. Tran TM, Vu TN, Vo ND et al (2022) Anomaly analysis in images and videos: A comprehensive review. ACM Comput Surv 55(7):1–37

    Google Scholar 

  2. Jian M, Wang R, Xu H et al (2023) Robust seed selection of foreground and background priors based on directional blocks for saliency-detection system. Multimed Tools Appl 82(1):427–451

    Google Scholar 

  3. Zhou Z, Sun Y, Sun Q et al (2023) Only once attack: fooling the tracker with adversarial template. IEEE Trans Circuits Syst Video Technol 33(7):3173–3184

    Google Scholar 

  4. Wang WQ, Chang FL, Mi HD (2021) Intermediate fused network with multiple timescales for anomaly detection. Neurocomputing 433:37–49

    Google Scholar 

  5. Chauhan S, Singh M, Aggarwal AK (2021) Data science and data analytics: artificial intelligence and machine learning integrated based approach. Data Sci Data Anal: Opp Challenge 1:1–16

    Google Scholar 

  6. Ionescu RT, Smeureanu S, Alexe B et al (2017) Unmasking the abnormal events in video. In Proc. IEEE Int Conf Comput Vision 2895–2903

  7. Jian M, Wang J, Yu H et al (2021) Visual saliency detection by integrating spatial position prior of object with background cues. Expert Syst Appl 168:114219

    Google Scholar 

  8. Nayak R, Pati UC, Das SK (2021) A comprehensive review on deep learning-based methods for video anomaly detection. Image Vis Comput 106:104078

  9. Jian M, Wang J, Yu H et al (2021) Integrating object proposal with attention networks for video saliency detection. Inf Sci 576:817–830

    MathSciNet  Google Scholar 

  10. Aggarwal AK (2020) Enhancement of GPS position accuracy using machine vision and deep learning techniques. J Comput Sci 16(5):651–659

    Google Scholar 

  11. Lv H, Zhou C, Cui Z et al (2021) Localizing anomalies from weakly-labeled videos. IEEE Trans Image Process 30:4505–4515

    Google Scholar 

  12. Yu J, Lee Y, Yow KC et al (2022) Abnormal event detection and localization via adversarial event prediction. IEEE Trans Neural Netw Learning Syst 33(8):3572–3586

    Google Scholar 

  13. Cai YH, Liu JQ, Guo YJ et al (2021) Video anomaly detection with multi-scale feature and temporal information fusion. Neurocomputing 423:264–273

    Google Scholar 

  14. Yan S, Smith JS, Lu W et al (2020) Abnormal event detection from videos using a two-stream recurrent variational autoencoder. IEEE Trans Cogn Dev Syst 12(1):30–42

    Google Scholar 

  15. Song H, Sun C, Wu X et al (2020) Learning normal patterns via adversarial attention-based autoencoder for abnormal event detection in videos. IEEE Trans Multimedia 22(8):2138–2148

    Google Scholar 

  16. Zavrtanik V, Kristan M, Skočaj D (2021) Reconstruction by inpainting for visual anomaly detection. Pattern Recogn 112:107706

    Google Scholar 

  17. Park H, Noh J, Ham B (2020) Learning memory-guided normality for anomaly detection. In Proc. IEEE Conf Comp Vision Pattern Recog 14360–14369

  18. Fang Z, Zhou JT, Xiao Y et al (2021) Multi-encoder towards effective anomaly detection in videos. IEEE Trans Multimedia 23:4106–4116

    Google Scholar 

  19. Li N, Chang F, Liu C (2021) Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes. IEEE Trans Multimedia 23:203–215

    Google Scholar 

  20. Luo W, Liu W, Lian D et al (2021) Video anomaly detection with sparse coding inspired deep neural networks. IEEE Trans Pattern Anal Mach Intell 43(3):1070–1084

    Google Scholar 

  21. Li S, Fang J, Xu H, Xue J (2021) Video frame prediction by deep multi-branch mask network. IEEE Trans Circuits Syst Video Technol 31(4):1283–1295

    Google Scholar 

  22. Luo W, Liu W, Lian D et al (2022) Future frame prediction network for video anomaly detection. IEEE Trans Pattern Anal Mach Intell 44(11):7505–7520

    Google Scholar 

  23. Khan FS, Georgescu MI, Popescu M et al (2021) A background-agnostic framework with adversarial training for abnormal event detection in video. IEEE Trans Pattern Anal Mach Intell 44(9):4505–4523

    Google Scholar 

  24. Jian M, Lam KM, Dong J et al (2015) Visual-patch-attention-aware saliency detection. IEEE Trans Cybernetics 45(8):1575–1586

    Google Scholar 

  25. Jian M, Zhang W, Yu H et al (2018) Saliency detection based on directional patches extraction and principal local color contrast. J Vis Commun Image Represent 57:1–11

    Google Scholar 

  26. Wang Y, Liu T, Zhou J et al (2023) Video anomaly detection based on spatio-temporal relationships among objects. Neurocomputing 532:141–151

    Google Scholar 

  27. Zhong Y, Chen X, Hu Y et al (2022) Bidirectional spatio-temporal feature learning with multiscale evaluation for video anomaly detection. IEEE Trans Circuits Syst Video Technol 32(12):8285–8296

    Google Scholar 

  28. Zhang Y, Nie X, He R et al (2021) Normality learning in multispace for video anomaly detection. IEEE Trans Circuits Syst Video Technol 31(9):3694–3706

    Google Scholar 

  29. Li D, Nie X, Li X et al (2022) Context-related video anomaly detection via generative adversarial network. Pattern Recogn Lett 156:183–189

    Google Scholar 

  30. Guo A, Guo L, Zhang R et al (2022) Self-trained prediction model and novel anomaly score mechanism for video anomaly detection. Image Vis Comput 119:104391

    Google Scholar 

  31. Leng J, Tan M, Gao X et al (2022) Anomaly warning: Learning and memorizing future semantic patterns for unsupervised ex-ante potential anomaly prediction. In Proc. ACM Int Conf Multimedia 6746–6754

  32. Wang W, Chang F, Liu C (2022) Mutuality-oriented reconstruction and prediction hybrid network for video anomaly detection. Signal Image Video Process 16:1747–1754

    Google Scholar 

  33. De Paepe D, Van Yperen-De Deyne A, Defever J, Van Hoecke S (2022) An incremental grey-box current regression model for anomaly detection of resistance mash seam welding in steel mills. Appl Sci 12(2):913

  34. Qin Y, Lyu J, Jiang L, Li L (2016) Traffic anomaly detection algorithm for wireless sensor networks based on improved exploitation of the GM(1,1) model. Int J Distrib Sens Netw 12(7):2181256

    Google Scholar 

  35. Zhao LT, Yang T, Yan R et al (2022) Anomaly detection of the blast furnace smelting process using an improved multivariate statistical process control model. Process Saf Environ Prot 166:617–627

    Google Scholar 

  36. Li HJ, Li CB, Hu W et al (2021) GRGAL: a grey relational generative adversarial learning method for image denoising. Journal of Grey System 33(1):30–42

    MathSciNet  Google Scholar 

  37. Guo X, Sahu AK, Sahu NK, Sahu AK (2022) A novel integrated computational TRIFMRG approach with grey relational analysis toward parametric evaluation of weld bead geometry of ms-grade: IS 2062. Grey Syst Theory Appl 12(1):117–141

    Google Scholar 

  38. Thukral R, Arora AS, Kumar A et al (2022) Denoising of thermal images using deep neural network. In Proc. Int Conf Recent Trends Comput 827–833

  39. Thukral R, Kumar A, Arora AS (2019) Effect of different thresholding techniques for denoising of EMG signals by using different wavelets. In Proc. Int Conf Intell Commun Comput Tech 161–165

  40. Miswan NH, Chan CS, Ng CG (2021) Hospital readmission prediction based on improved feature selection using grey relational analysis and LASSO. Grey Syst Theory Appl 11(4):796–812

    Google Scholar 

  41. Li L, Wang L, Luo H et al (2021) Towards effective link prediction: A hybrid similarity model. J Intell Fuzzy Syst 40(3):4013–4026

    Google Scholar 

  42. Ning X, An Y, Ju L et al (2023) Real-time online prediction of surface settlement considering spatiotemporal characteristics during foundation excavation. Autom Constr 150:104831

    Google Scholar 

  43. Liu SF (1991) The three axioms of buffer operator and their application. J Grey Syst 3(1):39–48

    MathSciNet  Google Scholar 

  44. Li P, Xu Z, Liu J et al (2023) Social network group decision-making for probabilistic linguistic information based on GRA. Comput Ind Eng 175:108861

    Google Scholar 

  45. Li C, Li H, Sun X, Zhang G (2022) Grey relational frame prediction method for anomaly detection. J Grey Syst 34(1):1–16

    Google Scholar 

  46. Gong D, Liu LQ, Le V et al (2019) Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proc. IEEE Int Conf Comput Vision 1705–1714

  47. Liu W, Luo W, Lian D, Gao S (2018) Future frame prediction for anomaly detection-A new baseline. In Proc. IEEE Conf Comp Vision Pattern Recog 6536–6545

  48. Lu Y, Kumar KM, Nabavi SS et al (2019) Future frame prediction using convolutional VRNN for anomaly detection. In Proc. Int Conf Adv Video Signal-Based Surveill 1–8

  49. Li C, Li H, Zhang G (2023) Future frame prediction based on generative assistant discriminative network for anomaly detection. Appl Intell 53:542–559

    Google Scholar 

  50. Slavic G, Baydoun M, Campo D et al (2022) Multilevel anomaly detection through variational autoencoders and bayesian models for self-aware embodied agents. IEEE Trans Multimedia 24:1399–1414

    Google Scholar 

  51. Huang C, Liu Y, Zhang Z et al (2022) Hierarchical graph embedded pose regularity learning via spatio-temporal transformer for abnormal behavior detection. In Proc. ACM Int Conf Multimedia 307–315

  52. Cho M, Kim T, Kim WJ et al (2022) Unsupervised video anomaly detection via normalizing flows with implicit latent features. Pattern Recogn 129:108703

    Google Scholar 

Download references

Funding

This work is supported by National Natural Science Foundation of China (NO.61871241, NO.61971245, NO.61976120); Nanjing University State Key Lab. for Novel Software Technology (KFKT2019B15); Nantong Science and Technology Program (JC2021131); Postgraduate Research and Practice Innovation Program of Jiangsu Province (KYCX22_3340).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Nantong University, Nantong, 226019, Jiangsu, People’s Republic of China

    Chaobo Li, Hongjun Li & Guoan Zhang

  2. State Key Lab. for Novel Software Technology, Nanjing University, Nanjing, 210023, Jiangsu, People’s Republic of China

    Hongjun Li

Authors
  1. Chaobo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongjun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Chaobo Li, Hongjun Li, and Guoan Zhang. The first draft of the manuscript was written by Chaobo Li and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hongjun Li.

Ethics declarations

Conflicts of interest/Competing interests

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Li, H. & Zhang, G. Grey-adversary perceptual network for anomaly detection. Multimed Tools Appl 83, 41273–41291 (2024). https://doi.org/10.1007/s11042-023-17253-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17253-1

Keywords

Search

Navigation