Skip to main content

Advertisement

SpringerLink
Log in

Abnormal behavior detection algorithm based on multi-branch convolutional fusion neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The recognition of abnormal behavior in surveillance video is the focus of current research, which has high research value and broad application possibilities. Its main applications are in the fields of intelligent surveillance, intelligent security, and smart cities, and it is of great significance to study the recognition of abnormal behaviors. Because of the complexity of human movement and the variability of the external environment, the recognition and detection of abnormal behaviors have some challenges. The recognition and detection of abnormal human behaviors in surveillance video still needs further research and development. This paper uses the multi-branch convolutional neural network to extract the spatial features of video frames for the first time, and as an encoder to pass the condensed features to the Gated Recurrent Unit (GRU), which extracts Temporal features from multiple video frames. And then the Gated Recurrent Unit output the result as the decoder. We did a series of comparative experiments on UCF-Crime dataset. And finally, we achieved an accuracy of 86.78% in the test set. The experimental results show that our multi-branch convolutional fusion neural network is better than previous surveillance video abnormal behavior recognition algorithms. At the same time, in order to verify the generalization performance and efficiency of the algorithm, we also conducted an experimental validation on the UCF-101 dataset in this paper, and the results show that the algorithm in this paper can also show a high accuracy rate on the UCF-101 dataset, and the speed of the algorithm is almost close to that of the C3D method with improved accuracy rate, making it possible to develop simple recognition applications based on the algorithm studied in this paper subsequently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Cheng KW, Chen YT, Fang WH (2015) Gaussian process regression-based video anomaly detection and localization with hierarchical feature representation. IEEE Trans Image Process 24(12):5288–5301

  2. Chong YS, Tay YH (2017) Abnormal event detection in videos using spatiotemporal autoencoder. In: International symposium on neural networks (ISNN), pp 189–196

  3. Chung J, Gulcehre C, Cho KH et al (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. Eprint Arxiv

  4. CNN.Smart cameras Spot Shady Behavior [EB/OL] (2007–03–26). http://edition.cnn.com/2007/TECH/science/03/26/fs.behaviorcameras/

  5. Cong Y, Yuan J, Liu J (2011) Sparse reconstruction cost for abnormal event detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3449–3456

  6. Ji S, Xu W, Yang M, Yu K (2010) 3D convolutional neural networks for human action recognition. In: 27th International Conference on Machine Learning (ICML), pp 495–502

  7. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition[J]. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 770–778

  8. Hentoff N. 40 years of growing surveillance [EB/OL] (2015-10-20). https://www.wnd.com/2015/10/40-years-of-nsa-surveillance

  9. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

  10. Khurram S, Amir RZ, Mubarak S (2012) UCF101 dataset. https://www.crcv.ucf.edu/research/data-sets/ucf101

  11. Ji S, Xu W, Yang M, Yu K (2013) 3D convolutional neural networks for human action recognition[J]. IEEE Trans Pattern Anal Mach Intell 35(1):221–231

    Article  Google Scholar 

  12. Jia D et al (2009) ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 248–255

  13. Kim J, Grauman K (2009) Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2921–2928

  14. Kratz L, Nishino K (2009) Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1446–1453

  15. Li W, Mahadevan V, Vasconcelos N (2013) Anomaly detection and localization in crowded scenes. IEEE Trans Pattern Anal Mach Intell 36(1):18–32

  16. Liu L, Wang S, Hu B, Qiong Q, Junhao W, Rosenblum David S (2018) Learning structures of interval-based Bayesian networks in probabilistic generative model for human complex activity recognition[J]. Pattern Recogn 81:545–561

    Article  Google Scholar 

  17. Luo W, Liu W, Gao S (2017) A revisit of sparse coding based anomaly detection in stacked rnn framework. In: IEEE international conference on computer vision (ICCV), pp 341–349

  18. Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 935–942

  19. Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. Adv Neural Inform Process Syst 1(January):568–576

  20. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations (ICLR)

  21. Sultani W, Chen C, Shah M (2018) Real-world anomaly detection in surveillance videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 6479–6488

  22. Szegedy C, Vanhoucke V, Ioffe S, et al (2016) Rethinking the inception architecture for computer vision[C]. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2818–2826

  23. Xie S et al (2017) Aggregated residual transformations for deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 5987–5995

  24. Zaremba W, Sutskever I, Vinyals O (2014) Recurrent neural network regularization. Eprint Arxiv

  25. Zhao B, Fei-Fei L, Xing EP (2011) Online detection of unusual events in videos via dynamic sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3313–3320

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (61971007 & 61571013).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, North China University of Technology, Beijing, People’s Republic of China

    Zheng Xu & Yuanyao Lu

Authors
  1. Zheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanyao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanyao Lu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Z., Lu, Y. Abnormal behavior detection algorithm based on multi-branch convolutional fusion neural network. Multimed Tools Appl 82, 22723–22740 (2023). https://doi.org/10.1007/s11042-023-14501-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14501-2

Keywords

Search

Navigation