Effective Crowd Anomaly Detection Through Spatio-temporal Texture Analysis

  • Yu Hao
  • Zhi-Jie XuEmail author
  • Ying Liu
  • Jing Wang
  • Jiu-Lun Fan
Open Access
Research Article


Abnormal crowd behaviors in high density situations can pose great danger to public safety. Despite the extensive installation of closed-circuit television (CCTV) cameras, it is still difficult to achieve real-time alerts and automated responses from current systems. Two major breakthroughs have been reported in this research. Firstly, a spatial-temporal texture extraction algorithm is developed. This algorithm is able to effectively extract video textures with abundant crowd motion details. It is through adopting Gabor-filtered textures with the highest information entropy values. Secondly, a novel scheme for defining crowd motion patterns (signatures) is devised to identify abnormal behaviors in the crowd by employing an enhanced gray level co-occurrence matrix model. In the experiments, various classic classifiers are utilized to benchmark the performance of the proposed method. The results obtained exhibit detection and accuracy rates which are, overall, superior to other techniques.


Crowd behavior spatial-temporal texture gray level co-occurrence matrix information entropy 



This research is funded by Chinese National Natural Science Foundation (No. 61671377) and Shaanxi Smart City Technology Project of Xianyang (No. 2017k01-25–5).


  1. [1]
    B. Solmaz, B. E. Moore, M. Shah. Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 10, pp. 2064–2070, 2012. DOI: 10.1109/TPAMI.2012.123.CrossRefGoogle Scholar
  2. [2]
    B. Krausz, C. Bauckhage. Loveparade 2010: Automatic video analysis of a crowd disaster. Computer Vision and Image Understanding, vol. 116, no. 3, pp. 307–319, 2012. DOI: 10.1016/j.cviu.2011.08.006.CrossRefGoogle Scholar
  3. [3]
    X. Y. Cui, Q. S. Liu, M. C. Gao, D. N. Metaxas. Abnormal detection using interaction energy potentials. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 3161–3167, 2011. DOI: 10.1109/CVPR.2011.5995558.Google Scholar
  4. [4]
    D. D. Ma, Q. Wang, Y. Yuan. Anomaly detection in crowd scene via online learning. In Proceedings of the 14th International Conference on Internet Multimedia Computing and Service, ACM, Xiamen, China, pp. 158–162, 2014. DOI: 10.1145/2632856.2632862.Google Scholar
  5. [5]
    R. Raghavendra, A. Del Bue, M. Cristani, V. Murino. Optimizing interaction force for global anomaly detection in crowded scenes. In Proceedings of IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, pp. 136–143, 2011. DOI: 10.1109/ICCVW.2011. 6130235.Google Scholar
  6. [6]
    A. Zeroual, N. Messai, S. Kechida, F. Hamdi. A piecewise switched linear approach for traffic flow modeling. International Journal of Automation and Computing, vol. 14, no. 6, pp. 729–741, 2017. DOI: 10.1007/s11633-017-1060-4.CrossRefGoogle Scholar
  7. [7]
    Z. C. Song, Y. Z. Ge, H. Duan, X. G. Qiu. Agent-based simulation systems for emergency management. International Journal of Automation and Computing, vol. 13, no. 2, pp. 89–98, 2016. DOI: 10.1007/s11633-016-0958-6.CrossRefGoogle Scholar
  8. [8]
    G. Csurka, C. R. Dance, L. X. Fan, J. Willamowski, C. Bray. Visual categorization with bags of keypoints. In Proceedings of ECCV International Workshop on Statistical Learning in Computer Vision, Grenoble, France, pp. 145–146, 2004.Google Scholar
  9. [9]
    D. Helbing, P. Molnar. Social force model for pedestrian dynamics. Physical Review E, vol. 51, no. 5, pp. 4282–4294, 1995. DOI: 10.1103/PhysRevE.51.4282.CrossRefGoogle Scholar
  10. [10]
    W. Yan, Z. Zou, J. B. Xie, T. Liu, P. Q. Li. The detecting of abnormal crowd activities based on motion vector. Optik, vol. 166, pp. 248–256, 2018. DOI: 10.1016/j.ijleo.2017.11.187.CrossRefGoogle Scholar
  11. [11]
    Y. Hao, Y. Liu, J. L. Fan. A crowd behavior feature descriptor based on optical flow field. Journal of Xi’an University of Posts and Telecommunications, vol. 21, no. 6, pp. 55–59, 2016. DOI: 10.13682/j.issn.2095-6533.2016.06.011. (In Chinese)Google Scholar
  12. [12]
    S. A. Niyogi, E. H. Adelson. Analyzing and recognizing walking figures in XYT. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 469–474, 1994. DOI: 10.1109/CVP R. 1994.323868.Google Scholar
  13. [13]
    M. Kass, A. Witkin, D. Terzopoulon. Snakes: Active contour models. International Journal of Computer Vision, vol. 1, no. 4, pp. 321–331, 1988. DOI: 10.1007/BF00133570.CrossRefGoogle Scholar
  14. [14]
    J. Wang, Z. J. Xu. STV-based video feature processing for action recognition. Signal Processing, vol. 93, no. 8, pp. 2151–2168, 2012. DOI: 10.1016/j.sigpro.2012.06.009.CrossRefGoogle Scholar
  15. [15]
    C. Van Gemeren, R. Poppe, R. C. Veltkamp. Hands-on: deformable pose and motion models for spatiotemporal localization of fine-grained dyadic interactions. EURASIP Journal on Image and Video Processing, vol. 2018, Article number 16, 2018. DOI: 10.1186/s13640-018-0255-0.Google Scholar
  16. [16]
    X. F. Ji, Q. Q. Wu, Z. J. Ju, Y. Y. Wang. Study of human action recognition based on improved spatio-temporal features. International Journal of Automation and Computing, vol. 11, no. 5, pp. 500–509, 2014. DOI: 10.1007/s11633-014-0831-4.CrossRefGoogle Scholar
  17. [17]
    E. H. Aldelson, J. R. Bergen. Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, vol. 2, no. 2, pp. 284–299, 1985. DOI: 10.1364/JOSAA.2.000284.CrossRefGoogle Scholar
  18. [18]
    Y. Iwashita, M. Petrou. Person identification from spatiotemporal volumes. In Proceedings of the 23rd International Conference Image and Vision Computing, IEEE, Christchurch, New Zealand, 2008. DOI: 10.1109/IVCNZ.2008.4762086.Google Scholar
  19. [19]
    R. C. Bolles, H. H. Baker, D. H. Marimont. Epipolar-plane image analysis: an approach to determining structure from motion. International Journal of Computer Vision, vol. 1, no. 1, pp. 7–55, 1987. DOI: 10.1007/BF00128525.CrossRefGoogle Scholar
  20. [20]
    H. H. Baker, R. C. Bolles. Generalizing epipolar-plane image analysis on the spatiotemporal surface. In Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, Ann Arbor, USA, pp. 33–49, 1988. DOI: 10.1109/CVPR.1988.196209.Google Scholar
  21. [21]
    G. Kuhne G, S. Richter, M. Beier. Motion-based segmentation and contour-based classification of video objects. In Proceedings of the 9th ACM international conference on Multimedia, Ottawa, Canada, pp. 41–50, 2001. DOI: 10.1145/500141.500150.Google Scholar
  22. [22]
    C. W. Ngo, T. C. Pong, R. T. Chin. Detection of gradual transitions through temporal slice analysis. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, USA, pp. 41–46, 1999. DOI: 10.1109/CVPR.1999.786914.Google Scholar
  23. [23]
    Y. Hao, Z. J. Xu, J. Wang, Y. Liu, J. L. Fan. An approach to detect crowd panic behavior using flow-based feature. In Proceedings of the 22nd International Conference on Automation and Computing, IEEE, Colchester, UK, pp. 462–466, 2016. DOI: 10.1109/IConAC.2016.7604963.Google Scholar
  24. [24]
    J. H. Xiang, H. Fan, J. Xu. Abnormal behavior detection based on spatial-temporal features. In Proceedings of International Conference on Machine Learning and Cybernetics, IEEE, Tianjin, China, pp. 871–876, 2013. DOI: 10.1109/ICMLC.2013.6890406.Google Scholar
  25. [25]
    H. H. Alqaysi, S. Sasi. Detection of abnormal behavior in dynamic crowded gatherings. In Proceedings of IEEE Applied Imagery Pattern Recognition Workshop, IEEE, Washington DC, USA, pp. 1–6, 2013. DOI: 10.1109/AIP R. 2013.6749309.Google Scholar
  26. [26]
    C. Li, Z. J. Han, Q. X. Ye, J. B. Jiao. Abnormal behavior detection via sparse reconstruction analysis of trajectory. In Proceedings of the 6th International Conference on Image and Graphics, IEEE, Hefei, China, pp. 807–810, 2011. DOI: 10.1109/ICIG.2011.104.Google Scholar
  27. [27]
    C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, vol. 27, no. 3, pp. 379–423, 1948. DOI: 10.1002/j.1538-7305.1948.tb01338.x.MathSciNetCrossRefzbMATHGoogle Scholar
  28. [28]
    K. He, S. X. Wang. Study on denoising of fractal signal based on Shannon entropy. In Proceedings of International Conference on Neural Networks and Signal Processing, IEEE, Nanjing, China, pp. 751–755, 2003. DOI:10. 1109/ICNNSP.2003.1279384.Google Scholar
  29. [29]
    S. Liang, Y. Ma, Y. Y. Huang, J. Guo, C. F. Jia. The scheme of detecting encoded malicious web pages based on information entropy. In Proceedings of the 10th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, IEEE, Fukuoka, Japan, pp. 310–312, 2016. DOI: 10.1109/IMIS.2016.82.Google Scholar
  30. [30]
    Z. J. Zhang, X. N. Wang, L. Sun. Mobile payment anomaly detection mechanism based on information entropy. IET Networks, vol. 5, no. 1, pp. 1–7, 2014. DOI: 10.1049/iet-net.2014.0101.CrossRefGoogle Scholar
  31. [31]
    D. K. Panda, S. Meher. Hierarchical background subtraction algorithm using Gabor filter. In Proceedings of IEEE International Conference on Electronics, Computing and Communication Technologies, Bangalore, India, pp. 1–6, 2015. DOI: 10.1109/CONECCT.2015.7383876.Google Scholar
  32. [32]
    D. X. Zhou, H. Zhang, N. Ray. Texture based background subtraction. In Proceedings of IEEE International Conference on Information and Automation, Changsha, China, pp. 20–23, 2008. DOI: 10.1109/ICINFA.2008.4608070.Google Scholar
  33. [33]
    J. C. S. J. Junior, S. R. Musse, C. R. Jung. Crowd analysis using computer vision techniques. IEEE Signal Processing Magazine, vol. 27, no. 5, pp. 66–77, 2010. DOI: 10.1109/ MSP.2010.937394.Google Scholar
  34. [34]
    R. M. Haralick, K. Shanmugam, I. Dinstein. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-3, no. 6, pp. 610–621, 1973. DOI: 10.1109/TSMC.1973.4309314.CrossRefGoogle Scholar
  35. [35]
    Y. Benezeth, P. M. Jodoin, V. Saligrama, C. Rosenberger. Abnormal events detection based on spatio-temporal Cooccurences. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, pp. 2458–2465, 2009. DOI: 10.1109/CVPR.2009.5206686.Google Scholar
  36. [36]
    J. B. Shen, X. G. Jin, C. Zhou, H. L. Zhao. Dynamic textures using wavelet analysis. In Proceedings of International Conference, Springer, Berlin Heidelberg, Germany, pp. 1070–1073, 2006. DOI: 10.1007/11736639—132.Google Scholar
  37. [37]
    The GLCM Tutorial. [Online], Available:, June 28, 2018.
  38. [38]
    S. B. Nikam, S. Agarwal. Wavelet energy signature and GLCM features-based fingerprint anti-spoofing. In Proceedings of the International Conference on Wavelet Analysis and Pattern Recognition, IEEE, Hong Kong, China, pp. 717–723, 2008. DOI: 10.1109/ICWAPR.2008.4635872.Google Scholar
  39. [39]
    G. Mukherjee, A. Chatterjee, B. Tudu. Study on the potential of combined GLCM features towards medicinal plant classification. In Proceedings of the 2nd International Conference on Control, Instrumentation, Energy and Communication, IEEE, Kolkata, India, pp. 98–102, 2016. DOI: 10.1109/CIEC.2016.7513746.Google Scholar
  40. [40]
    R. K. Ranjan, A. Agrawal. Video summary based on Fsift, Tamura textural and middle level semantic feature. Procedia Computer Science, vol. 89, pp. 870–876, 2016. DOI: 10.1016/j.procs.2016.06.075.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyXi’an University of Posts and TelecommunicationsXi’anChina
  2. 2.School of Computing and EngineeringUniversity of HuddersfieldHuddersfieldUK
  3. 3.Faculty of Arts Computing Engineering and SciencesSheffield Hallam UniversitySheffieldUK

Personalised recommendations