Skip to main content
SpringerLink
Log in

Detecting Abnormal Behavioral Patterns in Crowd Scenarios

  • Chapter
  • First Online:
Toward Robotic Socially Believable Behaving Systems - Volume II

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 106))

Abstract

This Chapter presents a framework for the the task of abnormality detection in crowded scenes based on the analysis of trajectories, build up upon a novel video descriptor, called Histogram of Oriented Tracklets. Unlike standard approaches that employ low level motion features, e.g. optical flow, to form video descriptors, we propose to exploit mid-level features extracted from long-range motion trajectories called tracklets, which have been successfully applied for action modeling and video analysis. Following standard procedure, a video sequence is divided into spatio-temporal cuboids within which we collect statistics of the tracklets passing through them. Specifically, tracklets orientation and magnitude are quantized in a two-dimensional histogram which encodes the actual motion patterns in each cuboid. These histograms are then fed into machine learning models (e.g., Latent Dirichlet allocation and Support Vector Machines) to detect abnormal behaviors in video sequences. The evaluation of the proposed descriptor on different datasets, namely UCSD, BEHAVE, UMN and Violence in Crowds, yields compelling results in abnormality detection, by setting new state-of-the-art and outperforming former descriptors based on the optical flow, dense trajectories and social force models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available at http://www.ces.clemson.edu/~stb/klt/.

  2. 2.

    Available at http://www.svcl.ucsd.edu/projects/anomaly/.

  3. 3.

    Available at http://groups.inf.ed.ac.uk/vision/behavedata/interactoins/.

  4. 4.

    http://mha.cs.umn.edu/movies/crowdactivity-all.avi.

  5. 5.

    Available at http://www.openu.ac.il/home/hassner/data/violentflows/.

  6. 6.

    The Equal Error Rate is the value of false positive rate when the ROC curve intersects the line connecting (0, 1)–(1, 0).

References

  1. Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30:555–560

    Article  Google Scholar 

  2. Alvar M, Torsello A, Sanchez-Miralles A, Armingol JM (2014) Abnormal behavior detection using dominant sets. Mach Vis Appl 32(6):1–18

    Google Scholar 

  3. Bera A, Manocha D (2014) Realtime multilevel crowd tracking using reciprocal velocity obstacles. arXiv:1402.2826

  4. Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  5. Blunsden S, Fisher R (2010) The behave video dataset: ground truthed video for multi-person behavior classification. Ann Br Mach Vis Assoc 4:1–12

    Google Scholar 

  6. Boiman O, Irani M (2007) Detecting irregularities in images and in video. Int J Comput Vis 74:17–31

    Article  Google Scholar 

  7. Chen K, Gong S, Xiang T, Loy CC (2013) Cumulative attribute space for age and crowd density estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2467–2474

    Google Scholar 

  8. Cong Y, Yuan J, Liu J (2011) Sparse reconstruction cost for abnormal event detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3449–3456

    Google Scholar 

  9. Cui X, Liu Q, Gao M, Metaxas DN (2011) Abnormal detection using interaction energy potentials. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3161–3167

    Google Scholar 

  10. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 1, pp 886–893

    Google Scholar 

  11. Fu Z, Hu W, Tan T (2005) Similarity based vehicle trajectory clustering and anomaly detection. In: Proceedings of the IEEE international conference on image processing, vol 2, pp II–602

    Google Scholar 

  12. Haque M, Murshed M (2010) Panic-driven event detection from surveillance video stream without track and motion features. In: Proceedings of the IEEE international conference on multimedia and expo, pp 173–178

    Google Scholar 

  13. Hassner T, Itcher Y, Kliper-Gross O (2012) Violent flows: real-time detection of violent crowd behavior. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1–6

    Google Scholar 

  14. Helbing D, Molnar P (1995) Social force model for pedestrian dynamics. Phys Rev E 51:4282

    Article  Google Scholar 

  15. Hu M, Ali S, Shah M (2008) Detecting global motion patterns in complex videos. In: Proceedings of the international conference on pattern recognition, pp 1–5

    Google Scholar 

  16. Junior SJ et al (2010) Crowd analysis using computer vision techniques. IEEE Signal Process Mag 27:66–77

    Google Scholar 

  17. Kim J, Grauman K (2009) Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2928

    Google Scholar 

  18. Kim J, Grauman K (2009) Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2928

    Google Scholar 

  19. Kratz L, Nishino K (2009) Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1446–1453

    Google Scholar 

  20. Kratz L, Nishino K (2010) Tracking with local spatio-temporal motion patterns in extremely crowded scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 693–700

    Google Scholar 

  21. Krausz B, Bauckhage C (2011) Analyzing pedestrian behavior in crowds for automatic detection of congestions. In: Proceedings of the IEEE international conference on computer vision workshops, pp 144–149

    Google Scholar 

  22. Krausz B, Bauckhage C (2012) Loveparade 2010: automatic video analysis of a crowd disaster. Comput Vis Image Underst 116(3):307–319

    Article  Google Scholar 

  23. Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Google Scholar 

  24. Li W, Mahadevan V, Vasconcelos N (2014) Anomaly detection and localization in crowded scenes. IEEE Trans Pattern Anal Mach Intell 36:18–32

    Article  Google Scholar 

  25. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  26. Lucas BD, Kanade T et al (1981) An iterative image registration technique with an application to stereo vision. In: IJCAI, vol 81, pp 674–679

    Google Scholar 

  27. Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1975–1981

    Google Scholar 

  28. Marques JS, Jorge PM, Abrantes AJ, Lemos J (2003) Tracking groups of pedestrians in video sequences. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshop, vol 9, pp 101–101

    Google Scholar 

  29. Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 935–942

    Google Scholar 

  30. Mousavi H, Mohammadi S, Perina A, Chellali R, Murino V (2015) Analyzing tracklets for the detection of abnormal crowd behavior. In: Proceedings of the IEEE Winter conference on applications of computer vision, pp 148–155

    Google Scholar 

  31. Mousavi H, Nabi M, Kiani H, Perina A, Murino V (2015) Crowd motion monitoring using tracklet-based commotion measure. In: Proceedings of the IEEE international conference on image processing

    Google Scholar 

  32. Mousavi H, Nabi M, Kiani Galogahi H, Perina A, Murino V (2015) Abnormality detection with improved histogram of oriented tracklets. In: Proceedings of the international conference on image analysis and processing

    Google Scholar 

  33. Piciarelli C, Micheloni C, Foresti GL (2008) Trajectory-based anomalous event detection. IEEE Trans Circuits Syst Video Technol 18:1544–1554

    Article  Google Scholar 

  34. Rabaud V, Belongie S (2006) Counting crowded moving objects. In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Google Scholar 

  35. Rabaud V, Belongie S (2006) Counting crowded moving objects. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 705–711

    Google Scholar 

  36. Raptis M, Soatto S (2010) Tracklet descriptors for action modeling and video analysis. In: Proceedings of the IEEE European conference on computer vision

    Google Scholar 

  37. Reddy V, Sanderson C, Lovell BC (2011) Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops

    Google Scholar 

  38. Regazzoni C, Tesei A, Murino V (1993) A real-time vision system for crowding monitoring. In: Proceedings of the IECON’93, International conference on industrial electronics, control, and instrumentation, pp 1860–1864

    Google Scholar 

  39. Rittscher J, Tu PH, Krahnstoever N (2005) Simultaneous estimation of segmentation and shape. In: CVPR, vol 2, pp 486–493. IEEE

    Google Scholar 

  40. Shi J, Tomasi C (1994) Good features to track. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 593–600

    Google Scholar 

  41. Solmaz B, Moore BE, Shah M (2012) Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Trans Pattern Anal Mach Intell 34:2064–2070

    Article  Google Scholar 

  42. Tomasi C, Kanade T (1991) Detection and tracking of point features

    Google Scholar 

  43. Wang B, Ye M, Li X, Zhao F, Ding J (2012) Abnormal crowd behavior detection using high-frequency and spatio-temporal features. Mach Vis Appl 23:501–511

    Article  Google Scholar 

  44. Wang H, Kläser A, Schmid C, Liu C-L (2011) Action recognition by dense trajectories. In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Google Scholar 

  45. Wang T, Snoussi H (2012) Histograms of optical flow orientation for visual abnormal events detection. In: AVSS, pp 13–18

    Google Scholar 

  46. Wang X, Ma X, Grimson WEL (2009) Unsupervised activity perception in crowded and complicated scenes using hierarchical bayesian models. IEEE Trans Pattern Anal Mach Intell 31:539–555

    Article  Google Scholar 

  47. Wijermans N, Jorna R, Jager W, Vliet TV (2007) Modelling crowd dynamics, influence factors related to the probability of a riot. In: Proceedings of the fourth European social simualtion association conference (ESSA). Toulouse University of Social Sciences

    Google Scholar 

  48. Wu S, Moore BE, Shah M (2010) Chaotic invariants of lagrangian particle trajectories for anomaly detection in crowded scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2054–2060

    Google Scholar 

  49. Wu S, San Wong H (2012) Crowd motion partitioning in a scattered motion field. IEEE Trans Syst Man Cybern Part B: Cybern 42:1443–1454

    Article  Google Scholar 

  50. Wu S, Wong H-S, Yu Z (2014) A Bayesian model for crowd escape behavior detection. IEEE Trans Circuits Syst Video Technol 24(1):85–98

    Article  Google Scholar 

  51. Xiang T, Gong S (2008) Video behavior profiling for anomaly detection. IEEE Trans Pattern Anal Mach Intell 30:893–908

    Article  Google Scholar 

  52. Yeffet L, Wolf L (2009) Local trinary patterns for human action recognition. In: Proceedings of the IEEE international conference on computer vision

    Google Scholar 

  53. Zhang Y, Qin L, Ji R, Yao H, Huang Q (2014) Social attribute-aware force model: exploiting richness of interaction for abnormal crowd detection

    Google Scholar 

  54. Zhao T, Nevatia R (2003) Bayesian human segmentation in crowded situations. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 2, pp II–459

    Google Scholar 

  55. Zhong H, Shi J, Visontai M (2004) Detecting unusual activity in video. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 2

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Pattern Analysis and Computer Vision Department, Istituto Italiano di Tecnologia, Genoa, Italy

    Hossein Mousavi, Hamed Kiani Galoogahi, Alessandro Perina & Vittorio Murino

  2. Department of Computer Science, University of Verona, Verona, Italy

    Vittorio Murino

Authors
  1. Hossein Mousavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hamed Kiani Galoogahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessandro Perina
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vittorio Murino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vittorio Murino .

Editor information

Editors and Affiliations

  1. Department of Psychology, Seconda Università di Napoli and IIASS, Caserta, Italy

    Anna Esposito

  2. Data Sci. Inst., Faculty of Sci. & Tech., Bournemouth University, Poole, United Kingdom

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Mousavi, H., Galoogahi, H.K., Perina, A., Murino, V. (2016). Detecting Abnormal Behavioral Patterns in Crowd Scenarios. In: Esposito, A., Jain, L. (eds) Toward Robotic Socially Believable Behaving Systems - Volume II . Intelligent Systems Reference Library, vol 106. Springer, Cham. https://doi.org/10.1007/978-3-319-31053-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-31053-4_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-31052-7

  • Online ISBN: 978-3-319-31053-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation