Skip to main content
SpringerLink
Log in

An efficient subsequence search for video anomaly detection and localization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel framework for anomaly event detection and localization in crowded scenes. For anomaly detection, one-class support vector machine with Bayesian derivation is applied to detect unusual events. We also propose a novel event representation, called subsequence, which refers to a time series of spatial windows in proximity. Unlike recent works encoded an event with a 3D bounding box which may contain irrelevant information, e.g. background, a subsequence can concisely capture the unstructured property of an event. To efficiently locate anomalous subsequences in a video space, we propose the maximum subsequence search. The proposed search algorithm integrates local anomaly scores into a global consistent detection so that the start and end of an abnormal event can be determined under false and missing detections. Experimental results on two public datasets show that our method is robust to the illumination change and achieve at least 80% localization rate which approximately doubles the accuracy of recent works. This study concludes that anomaly localization is crucial in finding abnormal events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. The implementation time is 3.8s/frame on a computer with 2GB RAM/2.6GHz CPU [10].

References

  1. Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30:555–560

    Article  Google Scholar 

  2. Antic B, Ommer B (2011) Video parsing for abnormality detection. In: Int. Conf. Comput. Vis., pp 2415–2422

  3. Benezeth Y, Jodoin P, Saligrama V, Rosenberger C (2009) Abnormal events detection based on spatio-temporal co-occurences. In: Conf. Comput. Vis. Pattern Recognit., pp 2458–2465

  4. Benezeth Y, Jodoin PM, Saligrama V (2011) Abnormality detection using low-level co-occurring events. Pattern Recognit Lett 32:423–431

    Article  Google Scholar 

  5. Bertini M, Bimbo AD, Seidenari L (2012) Multi-scale and real-time non-parametric approach for anomaly detection and localization. Comput Vis Image Underst 116:320–329

    Article  Google Scholar 

  6. Boiman O, Irani M (2005) Detecting irregularities in images and in video. In: Int. Conf. Comput. Vis., pp 462–469

  7. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv(CSUR) 41:41

  8. Chen YT, Lin YC, Fang WH (2010) A video-based human fall detection system for the smart home. J Chin Eng 33:681–690

    Article  Google Scholar 

  9. Cheng KW, Chen YT, Fang WH (2013) Abnormal crowd behavior detection and localization using maximum sub-sequence search. In: ACM/IEEE Int. Workshop Anal. Retr. Track. Event Motion Imag. Stream, pp 49–58

  10. Cong Y, Yuan J, Liu J (2011) Sparse reconstruction cost for abnormal event detection. In: Conf. Comput. Vis. Pattern Recognit., pp 3449–3456

  11. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Conf. Comput. Vis. Pattern Recognit., pp 886–893

  12. Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: Eur. Conf. Comput. Vis., pp 428–441

  13. Duque D, Santos H, Cortez P (2007) Prediction of abnormal behaviors for intelligent video surveillance systems. In: Comput. Intell. Data Min., pp 362–367

  14. Everingham M, Gool LV, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88:303–338

    Article  Google Scholar 

  15. Fei-Fei L, Fergus R, Torralba A (2007) Recognizing and learning object categories. In: Antonio Torralba’s website http://people.csail.mit.edu/torralba/shortCourseRLOC/

  16. Helbing D, Molnár P (1995) Social force model for pedestrian dynamics. Phys Rev E 51:4282–4286

    Article  Google Scholar 

  17. Hu DH, Zhang X, Yin J, Zheng VW, Yang Q (2009) Abnormal activity recognition based on hdp-hmm models. In: Int. Jt. Conf. Artif. Intell., pp 1715–1720

  18. Kettnaker VM (2003) Time-dependent hmms for visual intrusion detection. In: Comput. Vis. Pattern Recognit. Workshop, p 34

  19. Kim J, Grauman K (2009) Observe locally, infer globally: A space-time mrf for detecting abnormal activities with incremental updates. In: Conf. Comput. Vis. Pattern Recognit., pp 2921–2928

  20. Krüger V, Kragic D, Geib C (2007) The meaning of action: a review on action recognition and mapping. Adv Robot 21:1473–1501

    Google Scholar 

  21. Kwon J, Lee KM (2012) A unified framework for event summarization and rare event detection. In: Conf. Comput. Vis. Pattern Recognit., pp 1266–1273

  22. LK, KN (2009) Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: Conf. Comput. Vis. Pattern Recognit., pp 1446–1453

  23. Lampert CH, Blaschko MB, Hofmann T (2008) Beyond sliding windows: Object localization by efficient subwindow search. In: Conf. Comput. Vis. Pattern Recognit., pp 1–8

  24. Lee CK, Ho MF, Wen WS, Huang CL (2006) Abnormal event detection in video using n-cut clustering. In: J. Intell. Inf. Hiding Multimedia Signal Process., pp 407–410

  25. Li W, Mahadevan V, Vasconcelos N (2013) Anomaly detection and localization in crowded scenes. IEEE Trans Pattern Anal Mach Intell 36:18–32

    Google Scholar 

  26. Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 fps in matlab. In: Int. Conf. Comput. Vis

  27. Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. In: Conf. Comput. Vis. Pattern Recognit., pp 1975–1981

  28. Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. In: Conf. Comput. Vis. Pattern Recognit., pp 935–942

  29. Nater F, Grabner H, Van Gool L (2010) Exploiting simple hierarchies for unsupervised human behavior analysis. In: Conf. Comput. Vis. Pattern Recognit., pp 2014–2020

  30. Popoola O, Wang K (2012) Video-based abnormal human behavior recognition xa review. IEEE Trans Syst Man Cybern Soc:865–878

  31. Rohstkhari MJ, Levine MD (2013) Human activity recognition in videos using a single example. Image Vis Comput 31:864–876

    Article  Google Scholar 

  32. Rohstkhari MJ, Levine MD (2013) Online dominant and anomalous behavior detection in videos. In: Conf. Comput. Vis. Pattern Recognit., pp 2611–2618

  33. Rublee E, Rabaud V, Konolige K, Bradski GR (2011) Orb: An efficient alternative to sift or surf. In: Int. Conf. Comput. Vis., pp 2564–2571

  34. Saligrama V, Chen Z (2012) Video anomaly detection based on local statistical aggregates. In: Conf. Comput. Vis. Pattern Recognit., pp 2112–2119

  35. Scholkopt B, Platt J, Shawe-Taylor J, Smola A, Williamson R (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13:1443–1471

    Article  MATH  Google Scholar 

  36. Stauffer C, Grimson W (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22:747–757

    Article  Google Scholar 

  37. Tran D, Yuan J (2011) Optimal spatio-temporal path discovery for video event detection. In: Conf. Comput. Vis. Pattern Recognit., pp 3321–3328

  38. Tran D, Yang J, Forsyth D (2013) Video event detection: from subvolume localization to spatio-temporal path search. IEEE Trans Pattern Anal Mach Intell 36:404–416

    Article  Google Scholar 

  39. Voulodimos AS, Kosmopoulos DI, Doulamis ND, Varvarigou TA (2014) A top-down event-driven approach for concurrent activity recognition. Multimed Tool Appl 69:293–311

    Article  Google Scholar 

  40. Wang T, Snoussi H (2013) Histograms of optical flow orientation for abnormal events detection. In: IEEE Int. Workshop Perform. Eval. Track. Surveill., pp 45–52

  41. Wiliem A, Madasu V, Boles W, Yarlagadda P (2008) Detecting uncommon trajectories. In: Digit. Image Comput. Tech. Appl., pp 398–404

  42. Xiang T, Gong S (2008) Video behavior profiling for anomaly detection. IEEE Trans Pattern Anal Mach Intell 30:893–908

    Article  Google Scholar 

  43. Yin J, Yang Q, Pan J (2008) Sensor-based abnormal human-activity detection. IEEE Trans Knowl Data Eng 20:1082–1090

    Article  Google Scholar 

  44. Yuan J, Liu Z, Wu Y (2009) Discriminative subvolume search for efficient action detection. In: Conf. Comput. Vis. Pattern Recognit., pp 2442–2449

  45. Zaharescu A, Wildes R (2010) Anomalous behaviour detection using spatiotemporal oriented energies, subset inclusion histogram comparison and event-driven processing. In: Eur. Conf. Comput. Vis., pp 563–576

  46. Zhang D, Gatica-Perez D, Bengio S, McCowan I (2005) Semi-supervised adapted hmms for unusual event detection. In: Conf. Comput. Vis. Pattern Recognit., pp 611–618

  47. Zivkovic Z (2004) Improved adaptive gaussian mixture model for background subtraction. In: Int. Conf. Pattern Recognit., pp 28–31

Download references

Acknowledgment

This research was supported by National Science Council of R.O.C. under contract 101-2221-E-011-163.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, Republic of China

    Kai-Wen Cheng, Yie-Tarng Chen & Wen-Hsien Fang

Authors
  1. Kai-Wen Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yie-Tarng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wen-Hsien Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai-Wen Cheng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, KW., Chen, YT. & Fang, WH. An efficient subsequence search for video anomaly detection and localization. Multimed Tools Appl 75, 15101–15122 (2016). https://doi.org/10.1007/s11042-015-2453-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2453-4

Keywords

Search

Navigation