Effective Crowd Anomaly Detection Through Spatio-temporal Texture Analysis
Abnormal crowd behaviors in high density situations can pose great danger to public safety. Despite the extensive installation of closed-circuit television (CCTV) cameras, it is still difficult to achieve real-time alerts and automated responses from current systems. Two major breakthroughs have been reported in this research. Firstly, a spatial-temporal texture extraction algorithm is developed. This algorithm is able to effectively extract video textures with abundant crowd motion details. It is through adopting Gabor-filtered textures with the highest information entropy values. Secondly, a novel scheme for defining crowd motion patterns (signatures) is devised to identify abnormal behaviors in the crowd by employing an enhanced gray level co-occurrence matrix model. In the experiments, various classic classifiers are utilized to benchmark the performance of the proposed method. The results obtained exhibit detection and accuracy rates which are, overall, superior to other techniques.
KeywordsCrowd behavior spatial-temporal texture gray level co-occurrence matrix information entropy
This research is funded by Chinese National Natural Science Foundation (No. 61671377) and Shaanxi Smart City Technology Project of Xianyang (No. 2017k01-25–5).
- X. Y. Cui, Q. S. Liu, M. C. Gao, D. N. Metaxas. Abnormal detection using interaction energy potentials. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 3161–3167, 2011. DOI: 10.1109/CVPR.2011.5995558.Google Scholar
- D. D. Ma, Q. Wang, Y. Yuan. Anomaly detection in crowd scene via online learning. In Proceedings of the 14th International Conference on Internet Multimedia Computing and Service, ACM, Xiamen, China, pp. 158–162, 2014. DOI: 10.1145/2632856.2632862.Google Scholar
- R. Raghavendra, A. Del Bue, M. Cristani, V. Murino. Optimizing interaction force for global anomaly detection in crowded scenes. In Proceedings of IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, pp. 136–143, 2011. DOI: 10.1109/ICCVW.2011. 6130235.Google Scholar
- G. Csurka, C. R. Dance, L. X. Fan, J. Willamowski, C. Bray. Visual categorization with bags of keypoints. In Proceedings of ECCV International Workshop on Statistical Learning in Computer Vision, Grenoble, France, pp. 145–146, 2004.Google Scholar
- Y. Hao, Y. Liu, J. L. Fan. A crowd behavior feature descriptor based on optical flow field. Journal of Xi’an University of Posts and Telecommunications, vol. 21, no. 6, pp. 55–59, 2016. DOI: 10.13682/j.issn.2095-6533.2016.06.011. (In Chinese)Google Scholar
- S. A. Niyogi, E. H. Adelson. Analyzing and recognizing walking figures in XYT. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 469–474, 1994. DOI: 10.1109/CVP R. 1994.323868.Google Scholar
- C. Van Gemeren, R. Poppe, R. C. Veltkamp. Hands-on: deformable pose and motion models for spatiotemporal localization of fine-grained dyadic interactions. EURASIP Journal on Image and Video Processing, vol. 2018, Article number 16, 2018. DOI: 10.1186/s13640-018-0255-0.Google Scholar
- Y. Iwashita, M. Petrou. Person identification from spatiotemporal volumes. In Proceedings of the 23rd International Conference Image and Vision Computing, IEEE, Christchurch, New Zealand, 2008. DOI: 10.1109/IVCNZ.2008.4762086.Google Scholar
- H. H. Baker, R. C. Bolles. Generalizing epipolar-plane image analysis on the spatiotemporal surface. In Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, Ann Arbor, USA, pp. 33–49, 1988. DOI: 10.1109/CVPR.1988.196209.Google Scholar
- G. Kuhne G, S. Richter, M. Beier. Motion-based segmentation and contour-based classification of video objects. In Proceedings of the 9th ACM international conference on Multimedia, Ottawa, Canada, pp. 41–50, 2001. DOI: 10.1145/500141.500150.Google Scholar
- C. W. Ngo, T. C. Pong, R. T. Chin. Detection of gradual transitions through temporal slice analysis. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, USA, pp. 41–46, 1999. DOI: 10.1109/CVPR.1999.786914.Google Scholar
- Y. Hao, Z. J. Xu, J. Wang, Y. Liu, J. L. Fan. An approach to detect crowd panic behavior using flow-based feature. In Proceedings of the 22nd International Conference on Automation and Computing, IEEE, Colchester, UK, pp. 462–466, 2016. DOI: 10.1109/IConAC.2016.7604963.Google Scholar
- J. H. Xiang, H. Fan, J. Xu. Abnormal behavior detection based on spatial-temporal features. In Proceedings of International Conference on Machine Learning and Cybernetics, IEEE, Tianjin, China, pp. 871–876, 2013. DOI: 10.1109/ICMLC.2013.6890406.Google Scholar
- H. H. Alqaysi, S. Sasi. Detection of abnormal behavior in dynamic crowded gatherings. In Proceedings of IEEE Applied Imagery Pattern Recognition Workshop, IEEE, Washington DC, USA, pp. 1–6, 2013. DOI: 10.1109/AIP R. 2013.6749309.Google Scholar
- C. Li, Z. J. Han, Q. X. Ye, J. B. Jiao. Abnormal behavior detection via sparse reconstruction analysis of trajectory. In Proceedings of the 6th International Conference on Image and Graphics, IEEE, Hefei, China, pp. 807–810, 2011. DOI: 10.1109/ICIG.2011.104.Google Scholar
- K. He, S. X. Wang. Study on denoising of fractal signal based on Shannon entropy. In Proceedings of International Conference on Neural Networks and Signal Processing, IEEE, Nanjing, China, pp. 751–755, 2003. DOI:10. 1109/ICNNSP.2003.1279384.Google Scholar
- S. Liang, Y. Ma, Y. Y. Huang, J. Guo, C. F. Jia. The scheme of detecting encoded malicious web pages based on information entropy. In Proceedings of the 10th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, IEEE, Fukuoka, Japan, pp. 310–312, 2016. DOI: 10.1109/IMIS.2016.82.Google Scholar
- D. K. Panda, S. Meher. Hierarchical background subtraction algorithm using Gabor filter. In Proceedings of IEEE International Conference on Electronics, Computing and Communication Technologies, Bangalore, India, pp. 1–6, 2015. DOI: 10.1109/CONECCT.2015.7383876.Google Scholar
- D. X. Zhou, H. Zhang, N. Ray. Texture based background subtraction. In Proceedings of IEEE International Conference on Information and Automation, Changsha, China, pp. 20–23, 2008. DOI: 10.1109/ICINFA.2008.4608070.Google Scholar
- J. C. S. J. Junior, S. R. Musse, C. R. Jung. Crowd analysis using computer vision techniques. IEEE Signal Processing Magazine, vol. 27, no. 5, pp. 66–77, 2010. DOI: 10.1109/ MSP.2010.937394.Google Scholar
- Y. Benezeth, P. M. Jodoin, V. Saligrama, C. Rosenberger. Abnormal events detection based on spatio-temporal Cooccurences. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, pp. 2458–2465, 2009. DOI: 10.1109/CVPR.2009.5206686.Google Scholar
- J. B. Shen, X. G. Jin, C. Zhou, H. L. Zhao. Dynamic textures using wavelet analysis. In Proceedings of International Conference, Springer, Berlin Heidelberg, Germany, pp. 1070–1073, 2006. DOI: 10.1007/11736639—132.Google Scholar
- The GLCM Tutorial. [Online], Available: https://doi.org/www.fp.ucalgary.ca/mhallbey/tutorial.htm, June 28, 2018.
- S. B. Nikam, S. Agarwal. Wavelet energy signature and GLCM features-based fingerprint anti-spoofing. In Proceedings of the International Conference on Wavelet Analysis and Pattern Recognition, IEEE, Hong Kong, China, pp. 717–723, 2008. DOI: 10.1109/ICWAPR.2008.4635872.Google Scholar
- G. Mukherjee, A. Chatterjee, B. Tudu. Study on the potential of combined GLCM features towards medicinal plant classification. In Proceedings of the 2nd International Conference on Control, Instrumentation, Energy and Communication, IEEE, Kolkata, India, pp. 98–102, 2016. DOI: 10.1109/CIEC.2016.7513746.Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://doi.org/creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.