Motion anomaly detection and trajectory analysis in visual surveillance

Abstract

Motion anomaly detection through video analysis is important for delivering autonomous situation awareness in public places. Surveillance scene segmentation and representation is the preliminary step to implementation anomaly detection. Surveillance scene can be represented using Region Association Graph (RAG), where nodes represent regions and edges denote connectivity among the regions. Existing RAG-based analysis algorithms assume simple anomalies such as moving objects visit statistically unimportant or abandoned regions. However, complex anomalies such as an object encircles within a particular region (Type-I) or within a set of regions (Type-II). In this paper, we extract statistical features from a given set of object trajectories and train multi-class support vector machines (SVM) to deal with each type of anomaly. In the testing phase, a given test trajectory is categorized as normal or anomalous with respect to the trained models. Performance evaluation of the proposed algorithm has been carried out on public as well as our own datasets. We have recorded sensitivity as high as 86% and fall-out rate as low as 9% in experimental evaluation of the proposed technique. We have carried out comparative analysis with state-of-the-art techniques to benchmark the method. It has been observed that the proposed model is consistent and highly accurate across challenging datasets.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Abe S (2005) Support vector machines for pattern classification, vol 53. Springer-Verlag London Limited, London

  2. 2.

    Alon N, Yuster R, Zwick U (1997) Finding and counting given length cycles. Algorithmica 17:209–223

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Antic B, Ommer B (2011) Video parsing for abnormality detection. In: Proceedings of the IEEE international conference on computer, pp 2415–2422

  4. 4.

    Batista J, Peixoto P, Fernandes C, Ribeiro M (2006) A dual-stage robust vehicle detection and tracking for real-time traffic monitoring. In: Proceedings of the intelligent transportation systems conference, pp 528–535

  5. 5.

    Boers Y, Driessen H, Torstensson J, Trieb M, Karlsson R, Gustafsson F (2006) Track-before-detect algorithm for tracking extended targets. IEE Proc Radar Sonar Navi 153(4):345–351

    Article  Google Scholar 

  6. 6.

    Brasnett P, Mihaylova L, Canagarajah N, Bull D (2007) Sequential Monte Carlo tracking by fusing multiple cues in video sequences. Image Vis Comput 25(8):1217–1227

    Article  Google Scholar 

  7. 7.

    Brun L, Cappellania B, Saggese A, Vento M (2014) Detection of anomalous driving behaviors by unsupervised learning of graphs. In: Proceedings of the IEEE international conference on advanced video and signal based surveillance, pp 405–410

  8. 8.

    Brun L, Saggese A, Vento M (2014) Dynamic scene understanding for behavior analysis based on string kernels. IEEE Trans Circ Syst Vid Technol 24(10):1669–1681

    Article  Google Scholar 

  9. 9.

    Cheng K, Chen Y, Fang W (2015) Video anomaly detection and localization using hierarchical feature representation and gaussian process regression. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2909–2917

  10. 10.

    Cong Y, Yuan J, Tang Y (2013) Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Trans Inform Forens Secur 8(10):1590–1599

    Article  Google Scholar 

  11. 11.

    Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low-and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002

    Article  Google Scholar 

  12. 12.

    Dinh T, Vo N, Medioni G (2011) Context tracker: exploring supporters and distracters in unconstrained environments. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1177–1184

  13. 13.

    Dogra D, Ahmed A, Bhaskar H (2015) Interest area localization using trajectory analysis in surveillance scenes. In: Proceedings of the international conference on computer vision theory and applications, pp 31–38

  14. 14.

    Dogra D, Reddy R, Subramanyam K, Ahmed A, Bhaskar H (2015) Scene representation and anomalous activity detection using weighted region association graph. In: Proceedings of the international conference on computer vision theory and applications, pp 17–25

  15. 15.

    Ekman M (2008) Particle filters and data association for multi-target tracking. In: Proceedings of the 11th international conference on information fusion, pp 1–8

  16. 16.

    Elhamod M, Levine M (2013) Automated real-time detection of potentially suspicious behavior in public transport areas. IEEE Trans Intell Transp Syst 14(2):688–699

    Article  Google Scholar 

  17. 17.

    Flum J, Grohe M (2004) The parameterized complexity of counting problems. SIAM J Comput 33(4):892–922

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Gowsikhaa D, Abirami S, Baskaran R (2014) Automated human behavior analysis from surveillance videos: a survey. Artif Intell Rev 42(4):747–765

    Article  Google Scholar 

  19. 19.

    Hamid R, Johnson A, Batta S, Bobick A, Isbell C, Coleman G (2005) Detection and explanation of anomalous activities: representing activities as bags of event n-grams. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, vol 1, pp 1031– 1038

  20. 20.

    Han M, Sethi A, Hua W, Gong Y (2004) A detection-based multiple object tracking method. In: Proceedings of the IEEE international conference on image processing, vol 5, pp 3065–3068

  21. 21.

    Hospedales T, Li J, Gong S, Xiang T (2011) Identifying rare and subtle behaviors: a weakly supervised joint topic model. IEEE Trans Pattern Anal Mach Intell 33(12):2451–2464

    Article  Google Scholar 

  22. 22.

    Jiang Z (2010) Support vector machines for multi-class pattern recognition based on improved voting strategy. In: Proceedings of the Chinese control and decision conference, pp 517–520

  23. 23.

    Khan Z, Gu I (2010) Joint feature correspondences and appearance similarity for robust visual object tracking. IEEE Trans Inform Forens Secur 5(3):591–606

    Article  Google Scholar 

  24. 24.

    Khan Z, Balch T, Dellaert F (2004) A rao-blackwellized particle filter for eigentracking. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 980–986

  25. 25.

    Kiryati N, Raviv T, Ivanchenko Y, Rochel S (2008) Real-time abnormal motion detection in surveillance video. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–4

  26. 26.

    Krishna M, Denzler J (2014) A combination of generative and discriminative models for fast unsupervised activity recognition from traffic scene videos. In: Proceedings of the IEEE winter conference on applications of computer vision, pp 640–645

  27. 27.

    Krishna M, Bodesheim P, Körner M, Denzler J (2014) Temporal video segmentation by event detection: a novelty detection approach. Pattern Recog Image Anal 24(2):243–255

    Article  Google Scholar 

  28. 28.

    Ko T (2008) A survey on behavior analysis in video surveillance for homeland security applications. In: Proceedings of the IEEE applied imagery pattern recognition workshop, pp 1–8

  29. 29.

    Kuettel D, Breitenstein M, Van Gool L, Ferrari V (2010) What’s going on? Discovering spatio-temporal dependencies in dynamic scenes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1951–1958

  30. 30.

    Laxhammar R (2014) Chapter 4 - anomaly detection. In: Balasubramanian VN, Ho S, Vovk V (eds) Conformal prediction for reliable machine learning. Morgan Kaufmann, Boston, pp 71–97

  31. 31.

    Laxhammar R, Falkman G (2014) Online learning and sequential anomaly detection in trajectories. IEEE Trans Pattern Anal Mach Intell 36(6):1158–1173

    Article  MATH  Google Scholar 

  32. 32.

    Leyva R, Sanchez V, Li C (2014) Video anomaly detection based on wake motion descriptors and perspective grids. In: Proceedings of the IEEE international workshop on information forensics and security, pp 209–214

  33. 33.

    Li X, Breckon T (2007) Combining motion segmentation and feature based tracking for object classification and anomaly detection. In: Proceedings of the 4th European conference on visual media production, pp 12–12

  34. 34.

    Lia N, Wua X, Xud D, Guoa H, Fenga W (2015) Spatio-temporal context analysis within video volumes for anomalous-event detection and localization. Elsevier Neurocomput 155(1):309–319

    Article  Google Scholar 

  35. 35.

    Lin K, Chen S, Chen C, Lin D, Hung Y (2015) Abandoned object detection via temporal consistency modeling and back-tracing verification for visual surveillance. IEEE Trans Inform Forens Secur 10(7):1359–1370

    Article  Google Scholar 

  36. 36.

    Liu Y, Zhang X, Cui J, Wu C, Aghajan H, Zha H (2010) Visual analysis of child-adult interactive behaviors in video sequences. In: Proceedings of the international conference on virtual systems and multimedia, pp 26–33

  37. 37.

    Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. In: Proceedings of the international conference on pattern recognition, pp 898–901

  38. 38.

    Liu Y, Nie L, Han L, Zhang L, Rosenblum D (2015) Action2Activity: recognizing complex activities from sensor data. In: Proceedings of the international conference on artificial intelligence, pp 1617–1623

  39. 39.

    Liu Y, Nie L, Liu L, Rosenblum D (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  40. 40.

    Liu L, Cheng L, Liu Y, Jia Y, Rosenblum D (2016) Recognizing complex activities by a probabilistic interval-based model. In: Proceedings of the AAAI conference on artificial intelligence, vol 30, pp 1266–1272

  41. 41.

    Liu Y, Liang Y, Liu S, Rosenblum D, Zheng Y (2016) Predicting urban water quality with ubiquitous data. arXiv:1610.09462

  42. 42.

    Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum D (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of the 25th international joint conference on artificial intelligence, pp. 1–7

  43. 43.

    Liu Y, Zhang L, Nie L, Yan Y, Rosenblum D (2016) Fortune teller: predicting your career path. In: Proceedings of the AAAI conference on artificial intelligence, pp 201–207

  44. 44.

    Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2017) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed Tools Appl 76(8):10701–10719

    Article  Google Scholar 

  45. 45.

    Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1975–1981

  46. 46.

    Morris B, Trivedi M (2008) A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans Circ Syst Vid Technol 18(8):1114–1127

    Article  Google Scholar 

  47. 47.

    Nater F, Grabner H, Gool L (2011) Temporal relations in videos for unsupervised activity analysis. In: Proceedings of the British machine vision conference, pp 21.1–21.11

  48. 48.

    Nayak N, Zhu Y, Roy-Chowdhury A (2013) Exploiting spatio-temporal scene structure for wide-area activity analysis in unconstrained environments. IEEE Trans Inform Forens Secur 8(10):1610–1619

    Google Scholar 

  49. 49.

    Niu W, Long J, Han D, Wang Y (2004) Human activity detection and recognition for video surveillance. In: Proceedings of the IEEE international conference on multimedia and expo, vol 1, pp 719–722

  50. 50.

    Oron S (2012) Locally orderless tracking. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1940–1947

  51. 51.

    Ouivirach K, Gharti S, Dailey M (2013) Incremental behavior modeling and suspicious activity detection. Pattern Recogn 46(3):671–680

    Article  Google Scholar 

  52. 52.

    Pan J, Hu B, Zhang J (2008) Robust and accurate object tracking under various types of occlusions. IEEE Trans Circ Syst Vid Technol 18(2):223–236

    Article  Google Scholar 

  53. 53.

    Preotiuc-Pietro D, Liu Y, Hopkins D, Ungar L (2017) Beyond binary labels: political ideology prediction of Twitter users. n: Proceedings of the 55th annual meeting of the association for computational linguistics, vol 1, pp 729–740

  54. 54.

    Roshtkhari M, Levine M (2013) Online dominant and anomalous behavior detection in videos. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2611–2618

  55. 55.

    Saleh S, Suandi S, Ibrahim H (2015) Recent survey on crowd density estimation and counting for visual surveillance. Eng Appl Artif Intel 41:103–114

    Article  Google Scholar 

  56. 56.

    Särkkä S, Vehtari A, Lampinen J (2007) Rao-blackwellized particle filter for multiple target tracking. Inform Fus 8(1):2–15

    Article  Google Scholar 

  57. 57.

    Saligrama V, Chen Z (2012) Video anomaly detection based on local statistical aggregates. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2112–2119

  58. 58.

    Sebanja I, Megherbi D (2010) Automatic detection and recognition of traffic road signs for intelligent autonomous unmanned vehicles for urban surveillance and rescue. In: Proceedings of the IEEE international conference on technologies for homeland security, pp 132–138

  59. 59.

    Silveira G, Malis E (2007) Real-time visual tracking under arbitrary illumination changes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–6

  60. 60.

    Smeulders A, Chu D, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468

    Article  Google Scholar 

  61. 61.

    Takai M (2010) Detection of suspicious activity and estimate of risk from human behavior shot by surveillance camera. In: Proceedings of the second world congress on nature and biologically inspired computing, pp 298–304

  62. 62.

    Tzanidou G, Zafar I, Edirisinghe E (2013) Carried object detection in videos using color information. IEEE Trans Inform Forens Secur 8(10):1620–1631

    Article  Google Scholar 

  63. 63.

    Veeraraghavan H, Schrater P, Papanikolopoulos N (2006) Robust target detection and tracking through integration of motion, color, and geometry. Comput Vis Image Underst 103(2):121–138

    Article  Google Scholar 

  64. 64.

    Wang X, Ma K, Ng G, Grimson W (2008) Trajectory analysis and semantic region modeling using a nonparametric Bayesian model. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–8

  65. 65.

    Wenger B, Mandayam S, Violante P, Drake K (2010) Detection of anomalous events in shipboard video using moving object segmentation and tracking. In: Proceedings of the IEEE AUTOTESTCON, pp 1–6

  66. 66.

    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):1–45

    Article  Google Scholar 

  67. 67.

    Zhang J, Liu Z (2008) Detecting abnormal motion of pedestrian in video. In: Proceedings of the international conference on information and automation, pp 81–85

  68. 68.

    Zhao B, Fei-Fei L, Xing E (2011) Online detection of unusual events in videos via dynamic sparse coding. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 3313–3320

  69. 69.

    Zhou B, Wang X, Tang X (2012) Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2871–2878

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Debi Prosad Dogra.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chebiyyam, M., Reddy, R.D., Dogra, D.P. et al. Motion anomaly detection and trajectory analysis in visual surveillance. Multimed Tools Appl 77, 16223–16248 (2018). https://doi.org/10.1007/s11042-017-5196-6

Download citation

Keywords

  • Visual surveillance
  • Anomalous activity detection
  • Abnormal behavior classification
  • Trajectory analysis