Skip to main content
SpringerLink
Log in

Deep learning approaches for video-based anomalous activity detection

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

The pervasive use of cameras at indoor and outdoor premises on account of recording the activities has resulted into deluge of long video data. Such surveillance videos are characterized by single or multiple entities (persons, objects) performing sequential/concurrent activities. It is often interesting to detect suspicious behavior of such entities in an automated manner without any intervention of human personnel, and to this end, anomalous activity detection from surveillance videos is an important research domain in Computer Vision. Detecting the anomalous activities from videos is very challenging due to equivocal nature of anomalies, context at which events took place, lack of ample size of anomalous ground truth training data and also other factors associated with variation in environment conditions, illumination conditions and working status of capturing cameras. Though automated visual surveillance is one of the highly sought-after research domains, use of deep learning techniques for anomalous activity detection is still in nascent stage. Deep learning models like convolution neural networks, auto-encoders, Long Short Term Memory network models have achieved remarkable performance on different domains like image classification, object detection, speech processing, and expediting towards achieving excellence in anomaly detection tasks. This paper aims at studying and analyzing deep learning techniques for video-based anomalous activity detection. As outcome of the study, the graphical taxonomy has been put forth based on kinds of anomalies, level of anomaly detection, and anomaly measurement for anomalous activity detection. The focus has been given on various anomaly detection frameworks having deep learning techniques as their core methodology. Deep learning approaches from both the perspectives of accuracy oriented anomaly detection and real-time processing oriented anomaly detection are compared. This paper also sheds light upon research issues and challenges, application domains, benchmarked datasets and future directions in the domain of deep learning based anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7

Similar content being viewed by others

References

  1. 360-Degree camera Market: https://www.researchnester.com/reports/360-degree-camera-market-global-demand-analysis-opportunity-outlook-2024/385

  2. Adam, A., Rivlin, E., Shimshoni, I., Reinitz, D.: Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans. Pattern Anal. Mach. Intell. 30, 555–560 (2008)

    Article  Google Scholar 

  3. Anomalous behaviour dataset: http://vision.eecs.yorku.ca/research/anomalous-behaviour-data/

  4. Arashloo, S.R., Kittler, J., Christmas, W.: An anomaly detection approach to face spoofing detection: a new formulation and evaluation protocol. IEEE Access. 5, 13868–13882 (2017)

    Article  Google Scholar 

  5. BEHAVE: http://groups.inf.ed.ac.uk/vision/BEHAVEDATA/INTERACTIONS/

  6. Bertini, M., Del Bimbo, A., Seidenari, L.: Multi-scale and real-time non-parametric approach for anomaly detection and localization. Comput. Vis. Image Underst. 116, 320–329 (2012)

    Article  Google Scholar 

  7. Biswas, S. & Babu, R. V.: Real time anomaly detection in H. 264 compressed videos. in Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG), 2013 Fourth National Conference on 1–4 (2013)

  8. Candamo, J., Shreve, M., Goldgof, D.B., Sapper, D.B., Kasturi, R.: Understanding transit scenes: A survey on human behavior-recognition algorithms. IEEE Trans. Intell. Transp. Syst. 11, 206–224 (2010)

    Article  Google Scholar 

  9. CAVIAR: http://homepages.inf.ed.ac.uk/rbf/CAVIAR/ (2002)

  10. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: A survey. ACM Comput. Surv. 41, 15 (2009)

    Article  Google Scholar 

  11. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: British Machine Vision Conference, {BMVC} 2014, Nottingham, UK, September 1–5, 2014 (2014)

    Google Scholar 

  12. Cheng, K.-W., Chen, Y.-T., Fang, W.-H.: Gaussian process regression-based video anomaly detection and localization with hierarchical feature representation. IEEE Trans. Image Process. 24, 5288–5301 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  13. Cheng, K.-W., Chen, Y.-T., Fang, W.-H.: An efficient subsequence search for video anomaly detection and localization. Multimed. Tools Appl. 75, 15101–15122 (2016)

    Article  Google Scholar 

  14. Chong, Y. S. & Tay, Y. H.: Modeling video-based anomaly detection using deep architectures: Challenges and possibilities. in Control Conference (ASCC), 2015 10th Asian 1–8 (2015)

  15. Cong, Y., Yuan, J. & Liu, J.: Sparse reconstruction cost for abnormal event detection. in CVPR 2011 3449–3456 (2011). https://doi.org/10.1109/CVPR.2011.5995434

  16. Cong, Y., Yuan, J., Liu, J.: Abnormal event detection in crowded scenes using sparse representation. Pattern Recognit. 46, 1851–1864 (2013)

    Article  Google Scholar 

  17. Creusot, C. & Munawar, A.: Real-time small obstacle detection on highways using compressive RBM road reconstruction. in Intelligent Vehicles Symposium (IV), 2015 IEEE 162–167 (2015)

  18. CUHK Avenue, http://www.cse.cuhk.edu.hk/leojia/projects/detectabnormal/dataset.html

  19. Data generated by surveillance cameras: http://www.securityinfowatch.com/news/12160483/data-generated-by-new-surveillance-cameras-to-increase-exponentially-in-the-coming-years

  20. de Leo, C., Manjunath, B.S.: Multicamera video summarization and anomaly detection from activity motifs. ACM Trans. Sen. Netw. 10(27), 1–27:30 (2014)

    Article  Google Scholar 

  21. Del Giorno, A., Bagnell, J. A. & Hebert, M. A Discriminative Framework for Anomaly Detection in Large Videos. in Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part V (eds. Leibe, B., Matas, J., Sebe, N. & Welling, M.) 334–349 (Springer International Publishing, 2016). https://doi.org/10.1007/978-3-319-46454-1_21

  22. Distante, A., Marino, F., Mazzeo, P. L.: Nitti, M. & Stella, E. Automatic Method and System for Visual Inspection of Railway Infrastructure. (2009)

  23. Fan, Y., Levine, M.D., Wen, G., Qiu, S.: A deep neural network for real-time detection of falling humans in naturally occurring scenes. Neurocomputing. 260, 43–58 (2017)

    Article  Google Scholar 

  24. Fang, Z., et al.: Abnormal event detection in crowded scenes based on deep learning. Multimed. Tools Appl. 75, 14617–14639 (2016)

    Article  Google Scholar 

  25. Feng, Y., Yuan, Y. & Lu, X.: Deep representation for abnormal event detection in crowded scenes. in Proceedings of the 2016 ACM on Multimedia Conference 591–595 (ACM, 2016).

  26. Feng, Y., Yuan, Y., Lu, X.: Learning deep event models for crowd anomaly detection. Neurocomputing. 219, 548–556 (2017)

    Article  Google Scholar 

  27. Fujitsu’s Intelligent transportation system: http://www.fujitsu.com/cn/en/about/resources/news/press-releases/2016/frdc-0401.html

  28. Gan, C., Wang, N., Yang, Y., Yeung, D.-Y., Hauptmann, A.: G. DevNet: A Deep Event Network for multimedia event detection and evidence recounting. in 2015 I.E. Conference on Computer Vision and Pattern Recognition (CVPR). 2568–2577 (2015). https://doi.org/10.1109/CVPR.2015.7298872

  29. Gao, L., Guo, Z., Zhang, H., Xu, X., Shen, H.T.: Video captioning with attention-based LSTM and semantic consistency. IEEE Trans. Multimed. 19, 2045–2055 (2017)

    Article  Google Scholar 

  30. Girshick, R: Fast r-cnn. IEEE international conference on computer vision, 1440–1448 (2015)

  31. Goodfellow, I. et al. Generative Adversarial Nets. in Advances in Neural Information Processing Systems 27 (eds. Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D. & Weinberger, K. Q.) 2672–2680 (Curran Associates, Inc., 2014)

  32. Guo, Y., Zhang, J., Gao, L.: Exploiting long-term temporal dynamics for video captioning. World Wide Web. (2018)

  33. Haritaoglu, I., Harwood, D., Davis, L.S.: W4: real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell. 22, 809–830 (2000)

    Article  Google Scholar 

  34. Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A. K. & Davis, L. S.: Learning temporal regularity in video sequences. in Computer Vision and Pattern Recognition (CVPR), 2016 I.E. Conference on 733–742 (2016)

  35. He, C., Shao, J., Sun, J.: An anomaly-introduced learning method for abnormal event detection. Multimed. Tools Appl. (2017). https://doi.org/10.1007/s11042-017-5255-z

  36. Hendel, A., Weinshall, D., Peleg, S.: Identifying Surprising Events in Videos Using Bayesian Topic Models. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) Computer Vision -- ACCV 2010: 10th Asian Conference on Computer Vision, Queenstown, New Zealand, November 8–12, 2010, Revised Selected Papers, Part III, pp. 448–459. Springer, Berlin Heidelberg (2011). https://doi.org/10.1007/978-3-642-19318-7_35

    Chapter  Google Scholar 

  37. Hinami, R., Mei, T., Satoh, S.: Joint detection and recounting of abnormal events by learning deep generic knowledge. The IEEE International Conference on Computer Vision (ICCV). 2017, (2017)

  38. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)

    Article  Google Scholar 

  39. Hou, J., Wu, X., Yu, F. & Jia, Y.: Multimedia event detection via deep spatial-temporal neural networks. in 2016 I.E. International Conference on Multimedia and Expo (ICME) 1–6 (2016). https://doi.org/10.1109/ICME.2016.7552981

  40. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man, Cybern. Part C Applications Rev. 34, 334–352 (2004)

    Article  Google Scholar 

  41. Hu, X., Hu, S., Huang, Y., Zhang, H., Wu, H.: Video anomaly detection using deep incremental slow feature analysis network. IET Comput. Vis. 10, 258–267 (2016)

    Article  Google Scholar 

  42. i-Lids bag and vehicle detection challenge: http://www.eecs.qmul.ac.uk/~andrea/avss2007_d.html

  43. Isola, P., Zhu, J.-Y., Zhou, T. & Efros, A. A.: Image-to-image translation with conditional adversarial networks. arXiv Prepr. (2017)

  44. Junior, J.C.S.J., Musse, S.R., Jung, C.R.: Crowd Analysis Using Computer Vision Techniques. IEEE Signal Process. Mag. 27, 66–77 (2010)

    Google Scholar 

  45. Kaltsa, V., Briassouli, A., Kompatsiaris, I., Hadjileontiadis, L.J., Strintzis, M.G.: Swarm intelligence for detecting interesting events in crowded environments. IEEE Trans. image Process. 24, 2153–2166 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  46. Kok, V.J., Lim, M.K., Chan, C.S.: Crowd behavior analysis: A review where physics meets biology. Neurocomputing. 177, 342–362 (2016)

    Article  Google Scholar 

  47. Krizhevsky, A., Sutskever, I. & Hinton, G. E.: Imagenet classification with deep convolutional neural networks. in Advances in neural information processing systems 1097–1105 (2012)

  48. Leach, M.J.V., Sparks, E.P., Robertson, N.M.: Contextual anomaly detection in crowded surveillance scenes. Pattern Recogn. Lett. 44, 71–79 (2014)

    Article  Google Scholar 

  49. Leyva, R., Sanchez, V., Li, C.-T.: Video anomaly detection with compact feature sets for online performance. IEEE Trans. Image Process. 26, 3463–3478 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  50. Leyva, R., Sanchez, V., Li, C.-T.: The LV dataset: a realistic surveillance video dataset for abnormal event detection. In: Biometrics and Forensics (IWBF), 2017 5th International Workshop on 1–6 (2017)

    Google Scholar 

  51. Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes. IEEE Trans. Pattern Anal. Mach. Intell. 36, 18–32 (2014)

    Article  Google Scholar 

  52. Li, T., et al.: Crowded scene analysis: A survey. IEEE Trans. Circuits Syst. Video Technol. 25, 367–386 (2015)

    Article  Google Scholar 

  53. Li, N., Wu, X., Xu, D., Guo, H., Feng, W.: Spatio-temporal context analysis within video volumes for anomalous-event detection and localization. Neurocomputing. 155, 309–319 (2015)

    Article  Google Scholar 

  54. Li, X., Zhou, Z., Chen, L., Gao, L.: Residual attention-based LSTM for video captioning. World Wide Web. (2018). https://doi.org/10.1007/s11280-018-0531-z

  55. Lu, C., Shi, J. & Jia, J.: Abnormal event detection at 150 fps in matlab. in Computer Vision (ICCV), 2013 I.E. International Conference on 2720–2727 (2013)

  56. Luo, W., Liu, W., Gao, S.: A revisit of sparse coding based anomaly detection in stacked RNN framework. in The IEEE International Conference on Computer Vision (ICCV). (2017)

  57. Mahadevan, V., Li, W., Bhalodia, V. & Vasconcelos, N.: Anomaly detection in crowded scenes. in Computer Vision and Pattern Recognition (CVPR), 2010 I.E. Conference on 1975–1981 (2010)

  58. Makhzani, A., Frey, B.J.: A winner-take-all method for training sparse convolutional autoencoders. Adv. Neural Inf. Proces. Syst. 2791–2799 (2015)

  59. Marsden, M., McGuinness, K., Little, S. & O’Connor, N. E.: Holistic features for real-time crowd behaviour anomaly detection. in Image Processing (ICIP), 2016 I.E. International Conference on 918–922 (2016)

  60. Medel, J. R. & Savakis, A. E.: Anomaly detection in video using predictive convolutional long short-term memory networks. CoRR abs/1612.0, (2016)

  61. Menotti, D., et al.: Deep representations for Iris, face, and fingerprint spoofing detection. IEEE Trans. Inf. Forensics Secur. 10, 864–879 (2015)

    Article  Google Scholar 

  62. MIT Traffic dataset, http://www.ee.cuhk.edu.hk/~xgwang/MITtraffic.html (2018)

  63. Mo, X., Monga, V., Bala, R., Fan, Z.: Adaptive sparse representations for video anomaly detection. IEEE Trans. Circuits Syst. Video Technol. 24, 631–645 (2014)

    Article  Google Scholar 

  64. Munawar, A., Vinayavekhin, P. & De Magistris, G.: Spatio-temporal anomaly detection for industrial robots through prediction in unsupervised feature space. in Applications of Computer Vision (WACV), 2017 I.E. Winter Conference on 1017–1025 (2017)

  65. Nakahata, M.T., Thomaz, L.A., da Silva, A.F., da Silva, E.A.B., Netto, S.L.: Anomaly detection with a moving camera using spatio-temporal codebooks. Multidimens. Syst. Signal Process. 29, 1025–1054 (2018)

    Article  MathSciNet  Google Scholar 

  66. Narasimhan, M. G. & Sowmya Kamath S.: Dynamic video anomaly detection and localization using sparse denoising autoencoders. Multimed. Tools Appl. (2017). https://doi.org/10.1007/s11042-017-4940-2

  67. Ogawa, T., Hiraoka, D., Ito, S., Ito, M. & Fukumi, M.: Improvement in detection of abandoned object by pan-tilt camera. in Knowledge and Smart Technology (KST), 2016 8th International Conference on 152–157 (2016)

  68. Olson, C. C. & Doster, T.: A Novel Detection Paradigm and Its Comparison to Statistical and Kernel-Based Anomaly Detection Algorithms for Hyperspectral Imagery. in 2017 I.E. Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 302–308 (2017). https://doi.org/10.1109/CVPRW.2017.43

  69. Pathak, A. R., Pandey, M., Rautaray, S. & Pawar, K.: Assessment of object detection using deep convolutional neural networks. in Intelligent Computing and Information and Communication (eds. Bhalla, S., Bhateja, V., Chandavale, A. A., Hiwale, A. S. & Satapathy, S. C.) 457–466 (Springer Singapore, 2018)

  70. Pathak, A. R., Pandey, M. & Rautaray, S.: Deep learning approaches for detecting objects from images: a review. in Progress in Computing, Analytics and Networking (eds. Pattnaik, P. K., Rautaray, S. S., Das, H. & Nayak, J.) 491–499 (Springer Singapore, 2018)

  71. PETS dataset: http://www.cvg.reading.ac.uk/PETS2009/a.html

  72. Popoola, O.P., Wang, K.: Video-based abnormal human behavior recognition—A review. IEEE Trans. Syst. Man, Cybern. Part C Applications Rev. 42, 865–878 (2012)

    Article  Google Scholar 

  73. QMUL: QMUL junction dataset: http://www.eecs.qmul.ac.uk/~sgg/QMUL_Junction_Datasets/Junction/Junction.html

  74. Ravanbakhsh, M., Nabi, M., Mousavi, H., Sangineto, E. & Sebe, N.: Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection. CoRR abs/1610.0, (2016)

  75. Ravanbakhsh, M., Sangineto, E., Nabi, M. & Sebe, N.: training adversarial discriminators for cross-channel abnormal event detection in crowds. CoRR abs/1706.0, (2017)

  76. Redmon, J., Divvala, S., Girshick, R. & Farhadi, A. You only look once: Unified, real-time object detection. in Proceedings of the IEEE conference on computer vision and pattern recognition 779–788 (2016)

  77. Reed, I.S., Yu, X.: Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution. IEEE Trans. Acoust. 38, 1760–1770 (1990)

    Article  Google Scholar 

  78. Roshtkhari, M.J., Levine, M.D.: An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions. Comput. Vis. Image Underst. 117, 1436–1452 (2013)

    Article  Google Scholar 

  79. Sabokrou, M., Fathy, M., Hoseini, M. & Klette, R.: Real-time anomaly detection and localization in crowded scenes. in Proceedings of the IEEE conference on computer vision and pattern recognition workshops 56–62 (2015)

  80. Sabokrou, M., Fathy, M., Hoseini, M.: Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder. Electron. Lett. 52, 1122–1124 (2016)

    Article  Google Scholar 

  81. Sabokrou, M., Fayyaz, M., Fathy, M., Klette, R.: Deep-cascade: cascading 3d deep neural networks for fast anomaly detection and localization in crowded scenes. IEEE Trans. Image Process. 26, 1992–2004 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  82. Sabokrou, M., Fathy, M., Moayed, Z., Klette, R.: Fast and accurate detection and localization of abnormal behavior in crowded scenes. Mach. Vis. Appl. 28, 965–985 (2017)

    Article  Google Scholar 

  83. Sabokrou, M., Fayyaz, M., Fathy, M., Moayed, Z., Klette, R.: Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes. Comput. Vis. Image Underst. (2018). https://doi.org/10.1016/j.cviu.2018.02.006

  84. Saligrama, V. & Chen, Z.: Video anomaly detection based on local statistical aggregates. in Computer Vision and Pattern Recognition (CVPR), 2012 I.E. Conference on 2112–2119 (2012)

  85. Saligrama, V., Konrad, J., Jodoin, P.-M.: Video anomaly identification. IEEE Signal Process. Mag. 27, 18–33 (2010)

    Article  Google Scholar 

  86. Shah, M., Javed, O., Shafique, K.: Automated visual surveillance in realistic scenarios. IEEE Multimed. 14, 30–39 (2007)

    Article  Google Scholar 

  87. Shao, M., Fu, Y.: Deeply Self-Taught Multi-View Video Analytics Machine for Situation Awareness. in AFA Cyber Workshop. White Paper. (2015)

  88. Simonyan, K. & Zisserman, A. : Two-stream convolutional networks for action recognition in videos. in Advances in neural information processing systems 568–576 (2014)

  89. Sjarif, N.N.A., Shamsuddin, S.M., Hashim, S.Z.: Detection of abnormal behaviors in crowd scene: a review. Int. J. Adv. Soft Comput. Appl. 4, 1–33 (2012)

    Google Scholar 

  90. Sodemann, A.A., Ross, M.P., Borghetti, B.J.: A review of anomaly detection in automated surveillance. IEEE Trans. Syst. Man, Cybern. Part C Applications Rev. 42, 1257–1272 (2012)

    Article  Google Scholar 

  91. Song, J., et al.: Self-supervised video hashing with hierarchical binary auto-encoder. IEEE Trans. Image Process. 27, 3210–3221 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  92. Sun, L., Ai, H., Lao, S.: Localizing activity groups in videos. Comput. Vis. Image Underst. 144, 144–154 (2016)

    Article  Google Scholar 

  93. Sun, J., Shao, J., He, C.: Abnormal event detection for video surveillance using deep one-class learning. Multimed. Tools Appl. (2017). https://doi.org/10.1007/s11042-017-5244-2

  94. Thida, M., Yong, Y.L., Climent-Pérez, P., Eng, H., Remagnino, P.: A Literature Review on Video Analytics of Crowded Scenes. In: Atrey, P.K., Kankanhalli, M.S., Cavallaro, A. (eds.) Intelligent Multimedia Surveillance: Current Trends and Research, pp. 17–36. Springer, Berlin Heidelberg (2013). https://doi.org/10.1007/978-3-642-41512-8_2

    Chapter  Google Scholar 

  95. Tian, Y., et al.: IBM smart surveillance system (S3): event based video surveillance system with an open and extensible framework. Mach. Vis. Appl. 19, 315–327 (2008)

    Article  Google Scholar 

  96. Tran, H. T. M. & Hogg, D. C.: Anomaly detection using a convolutional winner-take-all Autoencoder. in Proceedings of the British Machine Vision Conference 2017 (2017)

  97. Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3d convolutional networks. In: Computer Vision (ICCV), 2015 I.E. International Conference on 4489–4497 (2015)

    Google Scholar 

  98. TRECVID Multimedia Event Recounting (MER) Evaluation Plan: https://www.nist.gov/sites/default/files/documents/itl/iad/mig/MER_TRECVID_evaluation_spec_v23.pdf

  99. Tudor Ionescu, R., Smeureanu, S., Alexe, B., Popescu, M.: Unmasking the abnormal events in video. in The IEEE International Conference on Computer Vision (ICCV). (2017)

  100. UCSD dataset: http://www.svcl.ucsd.edu/projects/anomaly/dataset.html

  101. UMN dataset: http://mha.cs.umn.edu/proj_events.shtml#crowd

  102. Video Surveillance Market: http://www.transparencymarketresearch.com/video-surveillance-vsaas-market.html

  103. VIOLENT-FLOWS dataset: http://www.openu.ac.il/home/hassner/data/violentflows/

  104. Vishwakarma, S., Agrawal, A.: A survey on activity recognition and behavior understanding in video surveillance. Vis. Comput. 29, 983–1009 (2013)

    Article  Google Scholar 

  105. Wang, J., Xu, Z.: Spatio-temporal texture modelling for real-time crowd anomaly detection. Comput. Vis. Image Underst. 144, 177–187 (2016)

    Article  Google Scholar 

  106. Wang, B., Ye, M., Li, X., Zhao, F., Ding, J.: Abnormal crowd behavior detection using high-frequency and spatio-temporal features. Mach. Vis. Appl. 23, 501–511 (2012)

    Article  Google Scholar 

  107. Wang, Q., Wan, J. & Yuan, Y.: Deep metric learning for crowdedness regression. IEEE Trans. Circuits Syst. Video Technol. PP, 1 (2017)

  108. Wang, X., Gao, L., Wang, P., Sun, X., Liu, X.: Two-stream 3-D convNet fusion for action recognition in videos with arbitrary size and length. IEEE Trans. Multimed. 20, 634–644 (2018)

    Article  Google Scholar 

  109. Wang, X., et al.: Deep appearance and motion learning for egocentric activity recognition. Neurocomputing. 275, 438–447 (2018)

    Article  Google Scholar 

  110. Weizmann dataset: http://www.wisdom.weizmann.ac.il/~vision/Irregularities.html

  111. Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.P.: Pfinder: Real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19, 780–785 (1997)

    Article  Google Scholar 

  112. Wu, H., Shao, J., Xu, X., Shen, F. & Shen, H. T. A system for spatiotemporal anomaly localization in surveillance videos. in Proceedings of the 2017 ACM on Multimedia Conference 1225–1226 (ACM, 2017). https://doi.org/10.1145/3123266.3127912

  113. Xie, S., Guan, Y.: Motion instability based unsupervised online abnormal behaviors detection. Multimed. Tools Appl. 75, 7423–7444 (2016)

    Article  Google Scholar 

  114. Xu, D., Ricci, E., Yan, Y., Song, J. & Sebe, N. Learning Deep Representations of Appearance and Motion for Anomalous Event Detection. CoRR abs/1510.0, (2015)

  115. Xu, D., Yan, Y., Ricci, E., Sebe, N.: Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput. Vis. Image Underst. 156, 117–127 (2017)

    Article  Google Scholar 

  116. Yoffie, David B.: "Mobileye: The Future of Driverless Cars." Harvard Business School Case 715–421, October 2014. (Revised October 2015)

  117. Yogameena, B., Nagananthini, C.: Computer vision based crowd disaster avoidance system: A survey. Int. J. Disaster Risk Reduct. 22, 95–129 (2017)

    Article  Google Scholar 

  118. Yu, R., Qiu, H., Wen, Z., Lin, C., Liu, Y.: A survey on social media anomaly detection. SIGKDD Explor. Newsl. 18, 1–14 (2016)

    Article  Google Scholar 

  119. Zeiler, M. D. & Fergus, R.: Visualizing and understanding convolutional networks. in European conference on computer vision 818–833 (2014)

  120. Zhao, B., Fei-Fei, L. & Xing, E. P.: Online detection of unusual events in videos via dynamic sparse coding. in CVPR 2011 3313–3320 (2011). https://doi.org/10.1109/CVPR.2011.5995524

  121. Zhao, Y. et al. Spatio-temporal AutoEncoder for video anomaly detection. in Proceedings of the 2017 ACM on Multimedia Conference 1933–1941 (ACM, 2017). https://doi.org/10.1145/3123266.3123451

  122. Zhou, S., et al.: Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process. Image Commun. 47, 358–368 (2016)

    Article  Google Scholar 

  123. Zhu, Y., Nayak, N.M., Roy-Chowdhury, A.K.: Context-aware activity recognition and anomaly detection in video. IEEE J. Sel. Top. Signal Process. 7, 91–101 (2013)

    Article  Google Scholar 

  124. Zitouni, M.S., Bhaskar, H., Dias, J., Al-Mualla, M.E.: Advances and trends in visual crowd analysis: A systematic survey and evaluation of crowd modelling techniques. Neurocomputing. 186, 139–159 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

Authors would like to thank anonymous reviewers for their valuable comments and guidance. This work has been supported by the Center of Excellence for Signal and Image Processing (CoE - SIP), College of Engineering Pune, India.

Author information

Authors and Affiliations

  1. Department of Computer Engineering & IT, College of Engineering Pune (COEP), Pune, India

    Karishma Pawar & Vahida Attar

Authors
  1. Karishma Pawar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vahida Attar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karishma Pawar.

Additional information

This article belongs to the Topical Collection: Special Issue on Deep vs. Shallow: Learning for Emerging Web-scale Data Computing and Applications

Guest Editors: Jingkuan Song, Shuqiang Jiang, Elisa Ricci, and Zi Huang

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pawar, K., Attar, V. Deep learning approaches for video-based anomalous activity detection. World Wide Web 22, 571–601 (2019). https://doi.org/10.1007/s11280-018-0582-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-018-0582-1

Keywords

Search

Navigation