Skip to main content
SpringerLink
Log in

Anomalous human activity detection in videos using Bag-of-Adapted-Models-based representation

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Anomalous human activity detection is highly essential due to its numerous uses in both public and private safety. Autoencoders-based normality modeling approaches detect some anomalous events as normal activities. We propose an effective strategy Bag-of-Adapted-Model (BoAM)-based approaches that utilize multiple Adapted Gaussian Mixture Models (AGMMs). In the first approach, two base class GMMs are constructed for normal and anomalous event classes. Then, adapted GMMs are constructed using training snippets of normal and anomalous events by adapting the corresponding base class GMMs to derive class-specific characteristics in AGMM. For a given video segment, likelihood-based embeddings provide improved discrimination between normal and anomalous event classes. BoAM can also be created using only normal occurrences of training data, and then, the representations fed to OC-SVM in order to detect outliers. Another variant of BoAM that used Universal Background Model (UBM) as a base model (BoAM_UBM) for adaptation gives comparable performance. Results over three standard datasets proved the consistent improvement. The proposed embeddings are suitable for detecting abnormalities even with smaller amount of anomalous training data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Code availability

The code supporting this work is available from the corresponding author upon valid request.

References

  1. Baby SA, Vinod B, Chinni C, Mitra K (2017) Dynamic vision sensors for human activity recognition. In: 2017 4th IAPR Asian conference on pattern recognition (ACPR), IEEE, p 316–321

  2. Bendersky M, Garcia-Pueyo L, Harmsen J, Josifovski V, Lepikhin D (2014) Up next: retrieval methods for large scale related video suggestion. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, p 1769–1778

  3. Chandrakala S, Deepak K, Vignesh L (2022) Bag-of-event-models based embeddings for detecting anomalies in surveillance videos. Expert Syst Appl 190:116168

    Article  Google Scholar 

  4. Cheng M, Cai K, Li M (2021) Rwf-2000: an open large scale video database for violence detection. In: 2020 25th International conference on pattern recognition (ICPR), IEEE, p 4183–4190

  5. Deepak K, Chandrakala S, Mohan CK (2021) Residual spatiotemporal autoencoder for unsupervised video anomaly detection. Signal Image Video Process 15(1):215–222

    Article  Google Scholar 

  6. Dong F, Zhang Y (2020) Nie X Dual discriminator generative adversarial network for video anomaly detection. IEEE Access 8:88170–88176

    Article  Google Scholar 

  7. Du T, Bourdev L, Fergus R, Torresani L, Paluri M (2014) C3d: generic features for video analysis. Eprint Arxiv

  8. Fang Z, Liang J, Zhou JT, Xiao Y, Yang F (2020) Anomaly detection with bidirectional consistency in videos. IEEE Trans Neural Netw Learn Syst 33:1079–1092

    Article  Google Scholar 

  9. Gauvain JL, Lee CH (1994) Maximum a posteriori estimation for multivariate gaussian mixture observations of markov chains. IEEE Trans Speech Audio Process 2(2):291–298

    Article  Google Scholar 

  10. Gong D, Liu L, Le V, Saha B, Mansour MR, Venkatesh S, Hengel AVD (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE international conference on computer vision, p 1705–1714

  11. Hanson A, Pnvr K, Krishnagopal S, Davis L (2018) Bidirectional convolutional lstm for the detection of violence in videos. In: Proceedings of the European conference on computer vision (ECCV)

  12. Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 733–742

  13. Hassner T, Itcher Y, Kliper-Gross O (2012) Violent flows: real-time detection of violent crowd behavior. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops, p 1–6. https://doi.org/10.1109/CVPRW.2012.6239348

  14. Huang C, Wen J, Xu Y, Jiang Q, Yang J, Wang Y, Zhang D (2022) Self-supervised attentive generative adversarial networks for video anomaly detection. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2022.3159538

    Article  Google Scholar 

  15. Huang C, Wu Z, Wen J, Xu Y, Jiang Q, Wang Y (2021) Abnormal event detection using deep contrastive learning for intelligent video surveillance system. IEEE Trans Ind Inform 18(8):5171–5179

    Article  Google Scholar 

  16. Kaltsa V, Briassouli A, Kompatsiaris I, Hadjileontiadis LJ, Strintzis MG (2015) Swarm intelligence for detecting interesting events in crowded environments. IEEE Trans Image Process 24(7):2153–2166

    Article  MathSciNet  MATH  Google Scholar 

  17. Kamoona AM, Gosta AK, Bab-Hadiashar A, Hoseinnezhad R (2020) Multiple instance-based video anomaly detection using deep temporal encoding-decoding. arXiv preprint arXiv:2007.01548

  18. Khan MUK, Park HS, Kyung CM (2018) Rejecting motion outliers for efficient crowd anomaly detection. IEEE Trans Inf Forensics Secur 14(2):541–556

    Article  Google Scholar 

  19. Kim J, Grauman K (2009) Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, p 2921–2928

  20. Leyva R, Sanchez V, Li C (2017) Video anomaly detection with compact feature sets for online performance. IEEE Trans Image Process 26(7):3463–3478. https://doi.org/10.1109/TIP.2017.2695105

    Article  MathSciNet  MATH  Google Scholar 

  21. Leyva R, Sanchez V, Li CT (2017) The lv dataset: a realistic surveillance video dataset for abnormal event detection. In: 2017 5th International workshop on biometrics and forensics (IWBF), IEEE, p 1–6

  22. Li G, Chung W (2018) Combined eeg-gyroscope-tdcs brain machine interface system for early management of driver drowsiness. IEEE Trans Hum Mach Syst 48(1):50–62. https://doi.org/10.1109/THMS.2017.2759808

    Article  Google Scholar 

  23. Li J, Jiang X, Sun T, Xu K (2019) Efficient violence detection using 3d convolutional neural networks. In: 2019 16th IEEE international conference on advanced video and signal based surveillance (AVSS), IEEE, p 1–8

  24. Li N, Chang F (2019) Video anomaly detection and localization via multivariate gaussian fully convolution adversarial autoencoder. Neurocomputing 369:92–105

    Article  Google Scholar 

  25. Li W, Mahadevan V, Vasconcelos N (2013) Anomaly detection and localization in crowded scenes. IEEE Trans Pattern Anal Mach Intell 36(1):18–32

    Google Scholar 

  26. Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 fps in matlab. In: Proceedings of the IEEE international conference on computer vision, p 2720–2727

  27. Luo W, Liu W, Gao S (2017) Remembering history with convolutional lstm for anomaly detection. In: 2017 IEEE international conference on multimedia and expo (ICME), IEEE, p 439–444

  28. Luo W, Liu W, Lian D, Tang J, Duan L, Peng X, Gao S (2019) Video anomaly detection with sparse coding inspired deep neural networks. IEEE Trans Pattern Anal Mach Intell 43(3):1070–1084

    Article  Google Scholar 

  29. Nguyen TN, Roy S, Meunier J (2021) Smithnet: strictness on motion-texture coherence for anomaly detection. IEEE Trans Neural Netw Learn Syst 33(6):2287–2300

    Article  MathSciNet  Google Scholar 

  30. Nievas E.B, Suarez O.D, García GB, Sukthankar R (2011) Violence detection in video using computer vision techniques. In: International conference on computer analysis of images and patterns, Springer, p 332–339

  31. Ramachandra B, Jones M, Vatsavai R (2020) Learning a distance function with a siamese network to localize anomalies in videos. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, p 2598–2607

  32. Reddy V, Sanderson C, Lovell BC (2011) Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: CVPR 2011 workshops, IEEE, p 55–61

  33. Rodriguez M, Orrite C, Medrano C, Makris D (2016) One-shot learning of human activity with an map adapted gmm and simplex-hmm. IEEE Trans Cybern 47(7):1769–1780

    Article  Google Scholar 

  34. Roshtkhari MJ, Levine MD (2013) An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions. Comput Vis Image Underst 117(10):1436–1452

    Article  Google Scholar 

  35. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  36. Song W, Zhang D, Zhao X, Yu J, Zheng R, Wang A (2019) A novel violent video detection scheme based on modified 3d convolutional neural networks. IEEE Access 7:39172–39179

    Article  Google Scholar 

  37. Sovan B, Babu RV (2013) Real time anomaly detection in h. 264 compressed videos. In: Computer vision, pattern recognition, image processing and graphics (NCVPRIPG), 2013 4th national conference, p 1–4

  38. Sudhakaran S, Lanz O (2017) Learning to detect violent videos using convolutional long short-term memory. CoRR abs/1709.06531

  39. Sultani W, Chen C, Shah M (2018) Real-world anomaly detection in surveillance videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 6479–6488

  40. Tian Y, Pang G, Chen Y, Singh R, Verjans JW, Carneiro G (2021) Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In: Proceedings of the IEEE/CVF international conference on computer vision, p 4975–4986

  41. Tziakos I, Cavallaro A, Xu LQ (2010) Local abnormality detection in video using subspace learning. In: 2010 7th IEEE international conference on advanced video and signal based surveillance, IEEE, p 519–525

  42. Uijlings J, Duta IC, Sangineto E, Sebe N (2015) Video classification with densely extracted hog/hof/mbh features: an evaluation of the accuracy/computational efficiency trade-off. Int J Multimed Inf Retr 4(1):33–44

    Article  Google Scholar 

  43. Wang J, Zhang G, Zhang K, Zhao Y, Wang Q, Li X (2020) Detection of small aerial object using random projection feature with region clustering. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.3018120

    Article  Google Scholar 

  44. Wang L, Xiong Y, Wang Z, Qiao Y, Lin D, Tang X, Van Gool L (2016) Temporal segment networks: towards good practices for deep action recognition. In: European conference on computer vision, Springer, p 20–36

  45. Xing P, Li Z (2023) Visual anomaly detection via partition memory bank module and error estimation. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2023.3237562

    Article  Google Scholar 

  46. Xu D, Yan Y, Ricci E, Sebe N (2017) Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput Vis Image Underst 156:117–127

    Article  Google Scholar 

  47. Yang MY, Liao W, Cao Y, Rosenhahn B (2018) Video event recognition and anomaly detection by combining gaussian process and hierarchical dirichlet process models. Photogramm Eng Remote Sens 84(4):203–214

    Article  Google Scholar 

  48. Yuan Y, Xiong Z, Wang Q (2019)VSSA-NET: vertical spatial sequence attention network for traffic sign detection. CoRR abs/1905.01583. http://arxiv.org/abs/1905.01583

  49. Shuai Yuan, Chengli Sun, Haoge Yang (2017) Recognition of aircraft engine sound based on gmm-ubm model. MATEC Web Conf. 128:05011. https://doi.org/10.1051/matecconf/201712805011

    Article  Google Scholar 

  50. Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S, Vinyals O, Monga R, Toderici G (2015) Beyond short snippets: deep networks for video classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 4694–4702

  51. Zhang S, Gong M, Xie Y, Qin A, Li H, Gao Y, Ong YS (2022) Influence-aware attention networks for anomaly detection in surveillance videos. IEEE Trans Circuits Syst Video Technol 32:5427–5437

    Article  Google Scholar 

  52. Zhang T, Jia W, He X, Yang J (2017) Discriminative dictionary learning with motion weber local descriptor for violence detection. IEEE Trans Circuits Syst Video Technol 27(3):696–709

    Article  Google Scholar 

  53. Zhang T, Jia W, Yang B, Yang J, He X, Zheng Z (2017) Mowld: a robust motion image descriptor for violence detection. Multimed Tools Appl 76(1):1419–1438

    Article  Google Scholar 

  54. Zhang T, Yang Z, Jia W, Yang B, Yang J, He X (2016) A new method for violence detection in surveillance scenes. Multimed Tools Appl 75(12):7327–7349

    Article  Google Scholar 

  55. Zhao Y, Deng B, Shen C, Liu Y, Lu H, Hua XS (2017) Spatio-temporal autoencoder for video anomaly detection. In: Proceedings of the 25th ACM international conference on Multimedia, p 1933–1941

  56. Zhong JX, Li N, Kong W, Liu S, Li TH, Li G (2019) Graph convolutional label noise cleaner: train a plug-and-play action classifier for anomaly detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p 1237–1246

  57. Zhong Y, Chen X, Hu Y, Tang P, Ren F (2022) Bidirectional spatio-temporal feature learning with multi-scale evaluation for video anomaly detection. IEEE Trans Circuits Syst Video Technol 32:6058–6072

    Article  Google Scholar 

  58. Zhu H, Liu B, Lu Y, Li W, Yu N (2018) Real-time anomaly detection with hmof feature. In: Proceedings of the 2018 the 2nd international conference on video and image processing, ACM, p 49–54

  59. Zhu Y, Newsam S (2019) Motion-aware feature for improved video anomaly detection. arXiv preprint arXiv:1907.10211

Download references

Funding

This work is supported by Cognitive Science Research Initiative, Department of Science & Technology, Government of India, vide No.DST/CSRI/2017/131(G).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shiv Nadar University, Chennai, India

    S. Chandrakala

Authors
  1. S. Chandrakala
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Single author.

Corresponding author

Correspondence to S. Chandrakala.

Ethics declarations

Conflicts of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chandrakala, S. Anomalous human activity detection in videos using Bag-of-Adapted-Models-based representation. Pattern Anal Applic 26, 1101–1112 (2023). https://doi.org/10.1007/s10044-023-01177-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-023-01177-5

Keywords

Search

Navigation