Skip to main content
SpringerLink
Log in

Hierarchical detection of abnormal behaviors in video surveillance through modeling normal behaviors based on AUC maximization

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Detection of abnormal behaviors is a challenging issue as there are disagreements over how abnormal behaviors should be defined. In this paper, a new method is proposed to estimate a model of normal behaviors and consequently to detect abnormal behaviors. Estimating a model of normal behaviors constitutes a main part of the training phase in detecting abnormal behaviors and is one of the main issues in the field. To estimate such a model, the histogram of the oriented optical flow (HOF) is first extracted as a local-based feature, and then, the proposed features of speed, variance of the trajectory and deviation from the trajectory are extracted as object-based features. Spectral clustering is used to cluster behavioral features that are similar. To design a classifier for estimating the model of normal behaviors in each cluster, a new method is proposed based on maximization of the area under the receiver operating characteristic curve (AUC). This new method does not need manual labeling for estimation of the model, and then, it carries out the estimation process in a semi-supervised fashion. In this paper, a method for hierarchical detection of abnormal behaviors based on the priority of the behavioral features is also proposed in the testing phase of the study. To this purpose, the existence of an abnormal behavior is hierarchically detected through using the HOF feature and the three object-based features of speed, variance of the trajectory and deviation from the trajectory. The distinguishing characteristic of the proposed method is that, as soon as the abnormal behavior is detected along the hierarchical application of the features, the process stops. The experimental results show that the proposed method can effectively detect the abnormal behaviors in several databases and achieve comparable performance to the state-of-the-art methods for detection of abnormal behaviors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Amraee S, Vafaei A, Jamshidi K, Adibi P (2018a) Abnormal event detection in crowded scenes using one-class SVM. SIViP 12(6):1115–1123. https://doi.org/10.1007/s11760-018-1267-z

    Article  Google Scholar 

  • Amraee S, Vafaei A, Jamshidi K, Adibi P (2018b) Anomaly detection and localization in crowded scenes using connected component analysis. Multimed Tools Appl 77(12):14767–14782. https://doi.org/10.1007/s11042-017-5061-7

    Article  Google Scholar 

  • Calders T, Jaroszewicz S (2007) Efficient AUC optimization for classification. In: Knowledge discovery databases, pp 42–53

  • Ermis EB, Saligrama V, Jodoin PM, Konrad J (2008) Abnormal behavior detection and behavior matching for networked cameras. In: 2008 2nd ACM/IEEE international conference on distributed smart cameras, ICDSC 2008

  • Fu Z, Hu W, Tan T (2005) Similarity based vehicle trajectory clustering and anomaly detection. In: Proceedings—international conference on image processing, ICIP, 2005, vol 2, pp 602–605

  • Gong S, Xiang T (2003) Recognition of group activities using dynamic probabilistic networks. In: Proceedings of the ninth IEEE international conference on computer vision, ICCV, vol 2, pp 742–749

  • Gu X, Cui J, Zhu Q (2014) Abnormal crowd behavior detection by using the particle entropy. Optik (Stuttgart) 125(14):3428–3433

    Article  Google Scholar 

  • Hu Y, Zhang Y, Davis LS (2013) Unsupervised abnormal crowd activity detection using semiparametric scan statistic. In: IEEE computer society conference on computer vision and pattern recognition workshops, 2013, vol 1, no c, pp 767–774

  • Jiang H, Cai Y, Wang H, Chen X (2015) Trajectory-based anomalous behaviour detection for intelligent traffic surveillance. IET Intell Transp Syst 9(8):810–816

    Article  Google Scholar 

  • Jin D, Zhu S, Sun X, Liang Z, Xu G (2016) Optical flow and spatio-temporal gradient based abnormal behavior detection. In: Proceedings of the 28th Chinese control decision conference CCDC 2016, pp 1532–1537

  • Kaltsa V, Briassouli A, Kompatsiaris I, Strintzis MG (2018) Multiple Hierarchical Dirichlet Processes for anomaly detection in traffic. Comput Vis Image Underst 169:28–39

    Article  Google Scholar 

  • Kim J, Grauman K (2009) Observe locally, infer globally: a space–time mrf for detecting abnormal activities with incremental updates. In: Proceedings of IEEE conference on computer vision and pattern recognition, June 2009

  • Kim Y, Toh KA, Teoh ABJ, Eng HL, Yau WY (2012) An online AUC formulation for binary classification. Pattern Recognit 45(6):2266–2279

    Article  Google Scholar 

  • Kosmopoulos DI, Voulodimos AS (2010) Behavior recognition from multiple views, pp 345–350

  • Li S, Liu C, Yang Y (2018) Anomaly detection based on maximum a posteriori. Pattern Recogn Lett 107:91–97. https://doi.org/10.1016/j.patrec.2017.09.001

    Article  Google Scholar 

  • Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 FPS in MATLAB. In: Proceedings of the IEEE international conference on computer vision, 2013, pp 2720–2727

  • Lucas BD, Takeo K (1981) An iterative image registration technique with an application to stereo vision. In: Internal joint conference on artificial intelligence (IJCAI), 1981, pp 674–679

  • Marrocco C, Duin RPW, Tortorella F (2008) Maximizing the area under the ROC curve by pairwise feature combination. Pattern Recognit 41(6):1961–1974

    Article  Google Scholar 

  • Morris RJ (2000) Statistical models of object interaction. Int J Comput Vis 37(2):209–215

    Article  Google Scholar 

  • Narasimhan MG, Sowmya Kamath S (2018) Dynamic video anomaly detection and localization using sparse denoising autoencoders. Multimed Tools Appl 77(11):13173–13195. https://doi.org/10.1007/s11042-017-4940-2

    Article  Google Scholar 

  • Oliver N, Rosario B, Pentland A (1999) A Bayesian computer vision system for modeling human interactions. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 1999, vol 1542, no 8, pp 255–272

  • Perš J, Sulić V, Kristan M, Perše M, Polanec K, Kovačič S (2010) Histograms of optical flow for efficient representation of body motion. Pattern Recognit Lett 31(11):1369–1376

    Article  Google Scholar 

  • Planck M, Von Luxburg U (2006) A tutorial on spectral clustering a tutorial on spectral clustering. Stat Comput 17(March):395–416

    Google Scholar 

  • Ricamato MT, Tortorella F (2011) Partial AUC maximization in a linear combination of dichotomizers. Pattern Recognit 44(10–11):2669–2677

    Article  Google Scholar 

  • Sabokrou M, Fayyaz M, Fathy M, Moayed Z, Klette R (2018) Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes. Comput Vis Image Underst 172:88–97

    Article  Google Scholar 

  • Saligrama V, Konrad J, Jodoin PM (2010) Video anomaly identification. IEEE Signal Process Mag 27(5):18–33

    Article  Google Scholar 

  • Shen M, Jiang X, Sun T (2018) Anomaly detection based on nearest neighbor search with locality-sensitive B-tree. Neurocomputing 289:55–67

    Article  Google Scholar 

  • Toh KA, Kim J, Lee S (2008) Maximizing area under ROC curve for biometric scores fusion. Pattern Recognit 41(11):3373–3392

    Article  Google Scholar 

  • Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  • Wang T, Snoussi H (2013) Histograms of optical flow orientation for abnormal events detection. In: IEEE international workshop on performance evaluation of tracking and surveillance, PETS, 2013, pp 45–52

  • Wang T, Snoussi H (2014) Detection of abnormal visual events via global optical flow orientation histogram. IEEE Trans Inf Forensics Secur 9(6):988–998

    Article  Google Scholar 

  • Wang J, Xia L (2018) Abnormal behavior detection in videos using deep learning. Clust Comput 2:1–11

    Google Scholar 

  • Xu D, Yan Y, Ricci E, Sebe N (2017) Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput Vis Image Underst 156:117–127

    Article  Google Scholar 

  • Yang W, Gao Y, Cao L (2013) TRASMIL: a local anomaly detection framework based on trajectory segmentation and multi-instance learning. Comput Vis Image Underst 117(10):1273–1286

    Article  Google Scholar 

  • Yin J, Meng Y (2009) Abnormal behavior recognition using self-adaptive hidden markov models. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 5627. LNCS, pp 337–346

  • Yuan Y, Wang D, Wang Q (2017) Anomaly detection in traffic scenes via spatial-aware motion reconstruction. IEEE Trans Intell Transp Syst 18(5):1198–1209

    Article  Google Scholar 

  • Zen G, Ricci E (2011) Earth mover prototypes: a convex learning approach for discovering activity patterns in dynamic scenes mining behaviors in complex scenes. In: Proceedings of IEEE conference on computer vision and pattern recognition, 2011

  • Zhang Z, Zhou S, Shen W, Zeng D (2014) Unusual event detection in crowded scenes. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), Sept 2014, pp 1300–1304

  • Zhang Y, Lu H, Zhang L, Ruan X (2016) Combining motion and appearance cues for anomaly detection. Pattern Recognit 51:443–452

    Article  Google Scholar 

  • Zhu Y, Nayak NM, Roy-Chowdhury AK (2013) Context-aware activity recognition and anomaly detection in video. IEEE J Sel Top Signal Process 7(1):91–101

    Article  Google Scholar 

  • Zhu S, Hu J, Shi Z (2016) Local abnormal behavior detection based on optical flow and spatio-temporal gradient. Multimed Tools Appl 75(15):9445–9459

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Damghan University, Damghan, Semnan, Iran

    Asghar Feizi

Authors
  1. Asghar Feizi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Asghar Feizi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest

Ethical standards

None of the authors of this paper has a financial or personal relationship with other people or organizations that could inappropriately influence or bias the content of the paper.

Human and animal rights disclosure

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feizi, A. Hierarchical detection of abnormal behaviors in video surveillance through modeling normal behaviors based on AUC maximization. Soft Comput 24, 10401–10413 (2020). https://doi.org/10.1007/s00500-019-04544-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04544-9

Keywords

Search

Navigation