Skip to main content
SpringerLink
Log in

MAG-Net: A Memory Augmented Generative Framework for Video Anomaly Detection Using Extrapolation

  • Conference paper
  • First Online:
Computer Vision and Image Processing (CVIP 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1568))

Included in the following conference series:

  • 843 Accesses

Abstract

Anomaly detection is one of the popular problem in computer vision as it is elementary to several computer vision applications. However, robust detection of spatiotemporal anomalies with dependencies across multiple frames is equally challenging. Most reconstruction error based unsupervised anomaly detection approaches struggle in case of intraclass dissimilarities and interclass similarities. Robust reconstruction capability of CNN often results in misclassification. To this end, a memory augmented generative encoder-decoder network (MAG-Net) is proposed for anomaly detection. A memory element is inserted between encoder and decoder to store diverse representations of normal feature space. A query is compared with multiple memory items representing normality space to ensure low reconstruction loss for normal frames and vice versa. MAG-Net also employs a residual block in the encoder stream to preserve visual features by allowing feedback from a layer to its successors. Residual block contains channel and pixel attention blocks to further accentuate spatiotemporal representations. Evaluation is done by comparing MAG-Net with other methods on benchmark datasets like UCSD ped-2 and Avenue. Experimental results validate the robustness of the proposed framework .

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hinami, R., Mei, T., Satoh, S.: Joint detection and recounting of abnormal events by learning deep generic knowledge. In: Proceedings of the IEEE International Conference on Computer Vision (2017)

    Google Scholar 

  2. Biradar, K., Dube, S., Vipparthi, S.K.: DEARESt: deep convolutional aberrant behavior detection in real-world scenarios. In: 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS). IEEE (2018)

    Google Scholar 

  3. Biradar, K.M., et al.: Challenges in time-stamp aware anomaly detection in traffic videos. arXiv preprint arXiv:1906.04574 (2019)

  4. Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11363, pp. 622–637. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_39

    Chapter  Google Scholar 

  5. Zenati, H., et al.: Efficient GAN-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)

  6. Chouhan, N., Khan, A.: Network anomaly detection using channel boosted and residual learning based deep convolutional neural network. Appl. Soft Comput. 83, 105612 (2019)

    Article  Google Scholar 

  7. Gong, D., et al.: Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

    Google Scholar 

  8. Park, H., Noh, J., Ham, B.: Learning memory-guided normality for anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  9. Xu, J., Denman, S., Fookes, C., Sridharan, S.: Unusual event detection in crowded scenes using bag of LBPs in spatio-temporal patches. In: Proceedings of DICTA, pp. 549–554 (2011)

    Google Scholar 

  10. Wang, X., Ma, X., Grimson, E.: Unsupervised activity perception by hierarchical Bayesian models. In: Proceedings of CVPR, pp. 1–8 (2007)

    Google Scholar 

  11. Chen, D.Y., Huang, P.C.: Motion-based unusual event detection in human crowds. J. Vis. Commun. Image Represent. 22(2), 178–186 (2011)

    Article  Google Scholar 

  12. Cheng, K., Chen, Y., Fang, W.: Video anomaly detection and localization using hierarchical feature representation and gaussian process regression. In: Proceedings of CVPR, pp. 2909–2917 (2015)

    Google Scholar 

  13. Xu, D., Ricci, E., Yan, Y., Song, J., Sebe, N.: Learning deep representations of appearance and motion for anomalous event detection. In: Proceedings of BMVC (2015)

    Google Scholar 

  14. Adam, A., Rivlin, E., Shimshoni, I., Reinitz, D.: Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 555–560 (2008)

    Article  Google Scholar 

  15. Sabokrou, M., Fathy, M., Hoseini, M., Klette, R.: Real-time anomaly detection and localization in crowded scenes. In: Proceedings of CVPR Workshops, pp. 56–62 (2015)

    Google Scholar 

  16. Wu, S., Wong, H.S., Yu, Z.: A Bayesian model for crowd escape behavior detection. IEEE Trans. Cir. Syst. Video Technol. 24(1), 85–98 (2014)

    Article  Google Scholar 

  17. Jiang, F., Wu, Y., Katsaggelos, A.K.: A dynamic hierarchical clustering method for trajectory-based unusual video event detection. IEEE Trans. Image Process. 18(4), 907–913 (2009)

    Article  MathSciNet  Google Scholar 

  18. Yuan, Y., Fang, J., Wang, Q.: Online anomaly detection in crowd scenes via structure analysis. IEEE Trans. Cybern. 45(3), 548–561 (2015)

    Article  Google Scholar 

  19. Lin, H., Deng, J.D., Woodford, B.J., Shahi, A.: Online weighted clustering or real-time abnormal event detection in video surveillance. In: Proceeding of ACMMM, pp. 536–540 (2016)

    Google Scholar 

  20. Zhou, S., Shen, W., Zeng, D., Fang, M., Wei, Y., Zhang, Z.: Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process. Image Communicat. 47, 358–368 (2016)

    Article  Google Scholar 

  21. Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision (2015)

    Google Scholar 

  22. Malhotra, P., et al.: Long short term memory networks for anomaly detection in time series. In: Proceedings, vol. 89 (2015)

    Google Scholar 

  23. Luo, W., Liu, W., Gao, S.: Remembering history with convolutional LSTM for anomaly detection. In: 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE (2017)

    Google Scholar 

  24. Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: Proceedings of CVPR, pp. 733–742 (2016)

    Google Scholar 

  25. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: International Conference on Learning Representations (ICLR) (2014)

    Google Scholar 

  26. Qin, X., et al.: FFA-Net: feature fusion attention network for single image dehazing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07 (2020)

    Google Scholar 

  27. Liu, W., et al.: Future frame prediction for anomaly detection-a new baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  28. Sultani, W., Chen, C., Shah, M.: Real-world anomaly detection in surveillance videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  29. Singh, D., Mohan, D.K.: Deep spatio-temporal representation for detection of road accidents using stacked autoencoder. IEEE Trans. Intell. Transp. Syst. 20(3), 879–887 (2018)

    Article  Google Scholar 

  30. Kim, J., Grauman, K.: Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE (2009)

    Google Scholar 

  31. Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: CVPR (2010)

    Google Scholar 

  32. Xu, D., Yan, Y., Ricci, E., Sebe, N.: Detecting anomalous events in videos by learning deep representations of appearance and motion. In: CVIU (2017)

    Google Scholar 

  33. Tudor Ionescu, R., Smeureanu, S., Alexe, B., Popescu, M.: Unmasking the abnormal events in video. In: ICCV (2017)

    Google Scholar 

  34. Luo, W., Liu, W., Gao, S.: A revisit of sparse coding based anomaly detection in stacked RNN framework. In: ICCV (2017)

    Google Scholar 

  35. Ravanbakhsh, M., Nabi, M., Sangineto, E., Marcenaro, L., Regazzoni, C., Sebe, N.: Abnormal event detection in videos using generative adversarial nets. In: ICIP (2017)

    Google Scholar 

  36. Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 FPS in MATLAB. In: ICCV (2013)

    Google Scholar 

  37. Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes. IEEE TPAMI 36, 18–32 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Malviya National Institute of Technology, Jaipur, 302017, India

    Sachin Dube, Kuldeep Biradar, Santosh Kumar Vipparthi & Dinesh Kumar Tyagi

Authors
  1. Sachin Dube
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kuldeep Biradar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Santosh Kumar Vipparthi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dinesh Kumar Tyagi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Santosh Kumar Vipparthi .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Roorkee, Roorkee, India

    Balasubramanian Raman

  2. Indian Institute of Technology Ropar, Ropar, India

    Subrahmanyam Murala

  3. Jadavpur University, Kolkata, India

    Ananda Chowdhury

  4. Indian Institute of Technology Ropar, Ropar, India

    Abhinav Dhall

  5. Indian Institute of Technology Ropar, Ropar, India

    Puneet Goyal

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dube, S., Biradar, K., Vipparthi, S.K., Tyagi, D.K. (2022). MAG-Net: A Memory Augmented Generative Framework for Video Anomaly Detection Using Extrapolation. In: Raman, B., Murala, S., Chowdhury, A., Dhall, A., Goyal, P. (eds) Computer Vision and Image Processing. CVIP 2021. Communications in Computer and Information Science, vol 1568. Springer, Cham. https://doi.org/10.1007/978-3-031-11349-9_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-11349-9_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-11348-2

  • Online ISBN: 978-3-031-11349-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation