Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 13, pp 16223–16248 | Cite as

Motion anomaly detection and trajectory analysis in visual surveillance

  • Manaswi Chebiyyam
  • Rohit Desam Reddy
  • Debi Prosad DograEmail author
  • Harish Bhaskar
  • Lyudmila Mihaylova
Article
  • 256 Downloads

Abstract

Motion anomaly detection through video analysis is important for delivering autonomous situation awareness in public places. Surveillance scene segmentation and representation is the preliminary step to implementation anomaly detection. Surveillance scene can be represented using Region Association Graph (RAG), where nodes represent regions and edges denote connectivity among the regions. Existing RAG-based analysis algorithms assume simple anomalies such as moving objects visit statistically unimportant or abandoned regions. However, complex anomalies such as an object encircles within a particular region (Type-I) or within a set of regions (Type-II). In this paper, we extract statistical features from a given set of object trajectories and train multi-class support vector machines (SVM) to deal with each type of anomaly. In the testing phase, a given test trajectory is categorized as normal or anomalous with respect to the trained models. Performance evaluation of the proposed algorithm has been carried out on public as well as our own datasets. We have recorded sensitivity as high as 86% and fall-out rate as low as 9% in experimental evaluation of the proposed technique. We have carried out comparative analysis with state-of-the-art techniques to benchmark the method. It has been observed that the proposed model is consistent and highly accurate across challenging datasets.

Keywords

Visual surveillance Anomalous activity detection Abnormal behavior classification Trajectory analysis 

References

  1. 1.
    Abe S (2005) Support vector machines for pattern classification, vol 53. Springer-Verlag London Limited, LondonGoogle Scholar
  2. 2.
    Alon N, Yuster R, Zwick U (1997) Finding and counting given length cycles. Algorithmica 17:209–223MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Antic B, Ommer B (2011) Video parsing for abnormality detection. In: Proceedings of the IEEE international conference on computer, pp 2415–2422Google Scholar
  4. 4.
    Batista J, Peixoto P, Fernandes C, Ribeiro M (2006) A dual-stage robust vehicle detection and tracking for real-time traffic monitoring. In: Proceedings of the intelligent transportation systems conference, pp 528–535Google Scholar
  5. 5.
    Boers Y, Driessen H, Torstensson J, Trieb M, Karlsson R, Gustafsson F (2006) Track-before-detect algorithm for tracking extended targets. IEE Proc Radar Sonar Navi 153(4):345–351CrossRefGoogle Scholar
  6. 6.
    Brasnett P, Mihaylova L, Canagarajah N, Bull D (2007) Sequential Monte Carlo tracking by fusing multiple cues in video sequences. Image Vis Comput 25(8):1217–1227CrossRefGoogle Scholar
  7. 7.
    Brun L, Cappellania B, Saggese A, Vento M (2014) Detection of anomalous driving behaviors by unsupervised learning of graphs. In: Proceedings of the IEEE international conference on advanced video and signal based surveillance, pp 405–410Google Scholar
  8. 8.
    Brun L, Saggese A, Vento M (2014) Dynamic scene understanding for behavior analysis based on string kernels. IEEE Trans Circ Syst Vid Technol 24(10):1669–1681CrossRefGoogle Scholar
  9. 9.
    Cheng K, Chen Y, Fang W (2015) Video anomaly detection and localization using hierarchical feature representation and gaussian process regression. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2909–2917Google Scholar
  10. 10.
    Cong Y, Yuan J, Tang Y (2013) Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Trans Inform Forens Secur 8(10):1590–1599CrossRefGoogle Scholar
  11. 11.
    Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low-and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002CrossRefGoogle Scholar
  12. 12.
    Dinh T, Vo N, Medioni G (2011) Context tracker: exploring supporters and distracters in unconstrained environments. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1177–1184Google Scholar
  13. 13.
    Dogra D, Ahmed A, Bhaskar H (2015) Interest area localization using trajectory analysis in surveillance scenes. In: Proceedings of the international conference on computer vision theory and applications, pp 31–38Google Scholar
  14. 14.
    Dogra D, Reddy R, Subramanyam K, Ahmed A, Bhaskar H (2015) Scene representation and anomalous activity detection using weighted region association graph. In: Proceedings of the international conference on computer vision theory and applications, pp 17–25Google Scholar
  15. 15.
    Ekman M (2008) Particle filters and data association for multi-target tracking. In: Proceedings of the 11th international conference on information fusion, pp 1–8Google Scholar
  16. 16.
    Elhamod M, Levine M (2013) Automated real-time detection of potentially suspicious behavior in public transport areas. IEEE Trans Intell Transp Syst 14(2):688–699CrossRefGoogle Scholar
  17. 17.
    Flum J, Grohe M (2004) The parameterized complexity of counting problems. SIAM J Comput 33(4):892–922MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Gowsikhaa D, Abirami S, Baskaran R (2014) Automated human behavior analysis from surveillance videos: a survey. Artif Intell Rev 42(4):747–765CrossRefGoogle Scholar
  19. 19.
    Hamid R, Johnson A, Batta S, Bobick A, Isbell C, Coleman G (2005) Detection and explanation of anomalous activities: representing activities as bags of event n-grams. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, vol 1, pp 1031– 1038Google Scholar
  20. 20.
    Han M, Sethi A, Hua W, Gong Y (2004) A detection-based multiple object tracking method. In: Proceedings of the IEEE international conference on image processing, vol 5, pp 3065–3068Google Scholar
  21. 21.
    Hospedales T, Li J, Gong S, Xiang T (2011) Identifying rare and subtle behaviors: a weakly supervised joint topic model. IEEE Trans Pattern Anal Mach Intell 33(12):2451–2464CrossRefGoogle Scholar
  22. 22.
    Jiang Z (2010) Support vector machines for multi-class pattern recognition based on improved voting strategy. In: Proceedings of the Chinese control and decision conference, pp 517–520Google Scholar
  23. 23.
    Khan Z, Gu I (2010) Joint feature correspondences and appearance similarity for robust visual object tracking. IEEE Trans Inform Forens Secur 5(3):591–606CrossRefGoogle Scholar
  24. 24.
    Khan Z, Balch T, Dellaert F (2004) A rao-blackwellized particle filter for eigentracking. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 980–986Google Scholar
  25. 25.
    Kiryati N, Raviv T, Ivanchenko Y, Rochel S (2008) Real-time abnormal motion detection in surveillance video. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–4Google Scholar
  26. 26.
    Krishna M, Denzler J (2014) A combination of generative and discriminative models for fast unsupervised activity recognition from traffic scene videos. In: Proceedings of the IEEE winter conference on applications of computer vision, pp 640–645Google Scholar
  27. 27.
    Krishna M, Bodesheim P, Körner M, Denzler J (2014) Temporal video segmentation by event detection: a novelty detection approach. Pattern Recog Image Anal 24(2):243–255CrossRefGoogle Scholar
  28. 28.
    Ko T (2008) A survey on behavior analysis in video surveillance for homeland security applications. In: Proceedings of the IEEE applied imagery pattern recognition workshop, pp 1–8Google Scholar
  29. 29.
    Kuettel D, Breitenstein M, Van Gool L, Ferrari V (2010) What’s going on? Discovering spatio-temporal dependencies in dynamic scenes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1951–1958Google Scholar
  30. 30.
    Laxhammar R (2014) Chapter 4 - anomaly detection. In: Balasubramanian VN, Ho S, Vovk V (eds) Conformal prediction for reliable machine learning. Morgan Kaufmann, Boston, pp 71–97Google Scholar
  31. 31.
    Laxhammar R, Falkman G (2014) Online learning and sequential anomaly detection in trajectories. IEEE Trans Pattern Anal Mach Intell 36(6):1158–1173CrossRefzbMATHGoogle Scholar
  32. 32.
    Leyva R, Sanchez V, Li C (2014) Video anomaly detection based on wake motion descriptors and perspective grids. In: Proceedings of the IEEE international workshop on information forensics and security, pp 209–214Google Scholar
  33. 33.
    Li X, Breckon T (2007) Combining motion segmentation and feature based tracking for object classification and anomaly detection. In: Proceedings of the 4th European conference on visual media production, pp 12–12Google Scholar
  34. 34.
    Lia N, Wua X, Xud D, Guoa H, Fenga W (2015) Spatio-temporal context analysis within video volumes for anomalous-event detection and localization. Elsevier Neurocomput 155(1):309–319CrossRefGoogle Scholar
  35. 35.
    Lin K, Chen S, Chen C, Lin D, Hung Y (2015) Abandoned object detection via temporal consistency modeling and back-tracing verification for visual surveillance. IEEE Trans Inform Forens Secur 10(7):1359–1370CrossRefGoogle Scholar
  36. 36.
    Liu Y, Zhang X, Cui J, Wu C, Aghajan H, Zha H (2010) Visual analysis of child-adult interactive behaviors in video sequences. In: Proceedings of the international conference on virtual systems and multimedia, pp 26–33Google Scholar
  37. 37.
    Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. In: Proceedings of the international conference on pattern recognition, pp 898–901Google Scholar
  38. 38.
    Liu Y, Nie L, Han L, Zhang L, Rosenblum D (2015) Action2Activity: recognizing complex activities from sensor data. In: Proceedings of the international conference on artificial intelligence, pp 1617–1623Google Scholar
  39. 39.
    Liu Y, Nie L, Liu L, Rosenblum D (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115CrossRefGoogle Scholar
  40. 40.
    Liu L, Cheng L, Liu Y, Jia Y, Rosenblum D (2016) Recognizing complex activities by a probabilistic interval-based model. In: Proceedings of the AAAI conference on artificial intelligence, vol 30, pp 1266–1272Google Scholar
  41. 41.
    Liu Y, Liang Y, Liu S, Rosenblum D, Zheng Y (2016) Predicting urban water quality with ubiquitous data. arXiv:1610.09462
  42. 42.
    Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum D (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of the 25th international joint conference on artificial intelligence, pp. 1–7Google Scholar
  43. 43.
    Liu Y, Zhang L, Nie L, Yan Y, Rosenblum D (2016) Fortune teller: predicting your career path. In: Proceedings of the AAAI conference on artificial intelligence, pp 201–207Google Scholar
  44. 44.
    Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2017) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed Tools Appl 76(8):10701–10719CrossRefGoogle Scholar
  45. 45.
    Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1975–1981Google Scholar
  46. 46.
    Morris B, Trivedi M (2008) A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans Circ Syst Vid Technol 18(8):1114–1127CrossRefGoogle Scholar
  47. 47.
    Nater F, Grabner H, Gool L (2011) Temporal relations in videos for unsupervised activity analysis. In: Proceedings of the British machine vision conference, pp 21.1–21.11Google Scholar
  48. 48.
    Nayak N, Zhu Y, Roy-Chowdhury A (2013) Exploiting spatio-temporal scene structure for wide-area activity analysis in unconstrained environments. IEEE Trans Inform Forens Secur 8(10):1610–1619Google Scholar
  49. 49.
    Niu W, Long J, Han D, Wang Y (2004) Human activity detection and recognition for video surveillance. In: Proceedings of the IEEE international conference on multimedia and expo, vol 1, pp 719–722Google Scholar
  50. 50.
    Oron S (2012) Locally orderless tracking. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1940–1947Google Scholar
  51. 51.
    Ouivirach K, Gharti S, Dailey M (2013) Incremental behavior modeling and suspicious activity detection. Pattern Recogn 46(3):671–680CrossRefGoogle Scholar
  52. 52.
    Pan J, Hu B, Zhang J (2008) Robust and accurate object tracking under various types of occlusions. IEEE Trans Circ Syst Vid Technol 18(2):223–236CrossRefGoogle Scholar
  53. 53.
    Preotiuc-Pietro D, Liu Y, Hopkins D, Ungar L (2017) Beyond binary labels: political ideology prediction of Twitter users. n: Proceedings of the 55th annual meeting of the association for computational linguistics, vol 1, pp 729–740Google Scholar
  54. 54.
    Roshtkhari M, Levine M (2013) Online dominant and anomalous behavior detection in videos. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2611–2618Google Scholar
  55. 55.
    Saleh S, Suandi S, Ibrahim H (2015) Recent survey on crowd density estimation and counting for visual surveillance. Eng Appl Artif Intel 41:103–114CrossRefGoogle Scholar
  56. 56.
    Särkkä S, Vehtari A, Lampinen J (2007) Rao-blackwellized particle filter for multiple target tracking. Inform Fus 8(1):2–15CrossRefGoogle Scholar
  57. 57.
    Saligrama V, Chen Z (2012) Video anomaly detection based on local statistical aggregates. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2112–2119Google Scholar
  58. 58.
    Sebanja I, Megherbi D (2010) Automatic detection and recognition of traffic road signs for intelligent autonomous unmanned vehicles for urban surveillance and rescue. In: Proceedings of the IEEE international conference on technologies for homeland security, pp 132–138Google Scholar
  59. 59.
    Silveira G, Malis E (2007) Real-time visual tracking under arbitrary illumination changes. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–6Google Scholar
  60. 60.
    Smeulders A, Chu D, Cucchiara R, Calderara S, Dehghan A, Shah M (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468CrossRefGoogle Scholar
  61. 61.
    Takai M (2010) Detection of suspicious activity and estimate of risk from human behavior shot by surveillance camera. In: Proceedings of the second world congress on nature and biologically inspired computing, pp 298–304Google Scholar
  62. 62.
    Tzanidou G, Zafar I, Edirisinghe E (2013) Carried object detection in videos using color information. IEEE Trans Inform Forens Secur 8(10):1620–1631CrossRefGoogle Scholar
  63. 63.
    Veeraraghavan H, Schrater P, Papanikolopoulos N (2006) Robust target detection and tracking through integration of motion, color, and geometry. Comput Vis Image Underst 103(2):121–138CrossRefGoogle Scholar
  64. 64.
    Wang X, Ma K, Ng G, Grimson W (2008) Trajectory analysis and semantic region modeling using a nonparametric Bayesian model. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–8Google Scholar
  65. 65.
    Wenger B, Mandayam S, Violante P, Drake K (2010) Detection of anomalous events in shipboard video using moving object segmentation and tracking. In: Proceedings of the IEEE AUTOTESTCON, pp 1–6Google Scholar
  66. 66.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):1–45CrossRefGoogle Scholar
  67. 67.
    Zhang J, Liu Z (2008) Detecting abnormal motion of pedestrian in video. In: Proceedings of the international conference on information and automation, pp 81–85Google Scholar
  68. 68.
    Zhao B, Fei-Fei L, Xing E (2011) Online detection of unusual events in videos via dynamic sparse coding. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 3313–3320Google Scholar
  69. 69.
    Zhou B, Wang X, Tang X (2012) Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 2871–2878Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.School of Electrical SciencesIIT BhubaneswarBhubaneswarIndia
  2. 2.Zero One Infinity Consulting (ZOIC) Services Ltd.MississaugaCanada
  3. 3.Department of Automatic Control and Systems EngineeringUniversity of SheffieldSheffieldUK

Personalised recommendations