Abnormal Events and Behavior Detection in Crowd Scenes Based on Deep Learning and Neighborhood Component Analysis Feature Selection

  • Alaa Atallah AlmazroeyEmail author
  • Salma Kammoun Jarraya
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1153)


In the last few years, surveillance cameras have been massively distributed both indoors and outdoors in public, due to security concerns creating a need to monitor unexpected actions or activities in a scene. An intelligent automated approach is highly required to detect anomalies from the scene as well, to save the time and cost required by laborers to detect the anomalies manually from monitor screens. In this research, we propose a deep learning-based approach to detect abnormal events and behavior from surveillance videos in crowd scenes. Thus, by using the keyframe extractor to localize the keyframes that contain important information from video frames. The selected keyframes are used to compute the optical flow values of magnitudes, orientations, and velocities of each keyframe to generate several 2D templates. Then, the obtained 2D templates are supplied to a pre-trained model ‘AlexNet’ to extract high-level features. The Neighborhood Component Analysis (NCA) feature selection method is applied to select the appropriate features, then use these features to generate a classification model via Support Vector Machine (SVM) classifier. Results are evaluated on several public datasets, along with a new dataset that we built it that contains different videos covering abnormal events and behaviors. The obtained results proved that the proposed method outperforms other methods.


Abnormal events detection Keyframes selection Neighborhood Component Analysis Video surveillance 


  1. 1.
    Yuan, Y., Feng, Y., Lu, X.: Statistical hypothesis detector for abnormal event detection in crowded scenes. IEEE Trans. Cybern. 47(11), 3597–3608 (2016)CrossRefGoogle Scholar
  2. 2.
    Li, Y., et al.: Key frames extraction of human motion capture data based on cosine similarity. Vectors 11(12), 1 (2017)Google Scholar
  3. 3.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)CrossRefGoogle Scholar
  4. 4.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  5. 5.
    Yang, W., Wang, K., Zuo, W.: Neighborhood component feature selection for high-dimensional data. JCP 7(1), 161–168 (2012)Google Scholar
  6. 6.
    Mahadevan, V., et al.: Anomaly detection in crowded scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE (2010)Google Scholar
  7. 7.
    Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 FPS in MATLAB. In: Proceedings of the IEEE International Conference on Computer Vision (2013)Google Scholar
  8. 8.
    Nievas, E.B., et al.: Violence detection in video using computer vision techniques. In: International Conference on Computer Analysis of Images and Patterns. Springer, Heidelberg (2011)Google Scholar
  9. 9.
    Ghrab, N.B., Fendri, E., Hammami, M.: Abnormal events detection based on trajectory clustering. In: 2016 13th International Conference on Computer Graphics, Imaging and Visualization (CGiV). IEEE (2016)Google Scholar
  10. 10.
    Coşar, S., et al.: Toward abnormal trajectory and event detection in video surveillance. IEEE Trans. Circ. Syst. Video Technol. 27(3), 683–695 (2016)CrossRefGoogle Scholar
  11. 11.
    Li, N., et al.: Spatio-temporal context analysis within video volumes for anomalous-event detection and localization. Neurocomputing 155, 309–319 (2015)CrossRefGoogle Scholar
  12. 12.
    Cho, S.-H., Kang, H.-B.: Abnormal behavior detection using hybrid agents in crowded scenes. Pattern Recogn. Lett. 44, 64–70 (2014)CrossRefGoogle Scholar
  13. 13.
    Liu, W., et al.: Future frame prediction for anomaly detection–a new baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  14. 14.
    Li, F., Yang, W., Liao, Q.: An efficient anomaly detection approach in surveillance video based on oriented GMM. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE (2016)Google Scholar
  15. 15.
    Deecke, L., et al.: Image anomaly detection with generative adversarial networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Heidelberg (2018)Google Scholar
  16. 16.
    Kumaran, S.K., Dogra, D.P., Roy, P.P.: Anomaly Detection in Road Traffic Using Visual Surveillance: A Survey. arXiv preprint arXiv:1901.08292 (2019)
  17. 17.
    Zhu, Y., Nayak, N.M., Roy-Chowdhury, A.K.: Context-aware activity recognition and anomaly detection in video. IEEE J. Sel. Top. Sig. Process. 7(1), 91–101 (2012)CrossRefGoogle Scholar
  18. 18.
    Liou, C.-Y., et al.: Autoencoder for words. Neurocomputing 139, 84–96 (2014)CrossRefGoogle Scholar
  19. 19.
    Sabokrou, M., et al.: Real-time anomaly detection and localization in crowded scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2015)Google Scholar
  20. 20.
    Zhou, X.-G., Zhang, L.-Q.: Abnormal event detection using recurrent neural network. In: 2015 International Conference on Computer Science and Applications (CSA). IEEE (2015)Google Scholar
  21. 21.
    Feng, Y., Yuan, Y., Lu, X.: Deep representation for abnormal event detection in crowded scenes. In: Proceedings of the 24th ACM International Conference on Multimedia. ACM (2016)Google Scholar
  22. 22.
    Xu, D., et al.: Detecting anomalous events in videos by learning deep representations of appearance and motion. Comput. Vis. Image Underst. 156, 117–127 (2017)CrossRefGoogle Scholar
  23. 23.
    Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: International Symposium on Neural Networks. Springer, Heidelberg (2017)Google Scholar
  24. 24.
    Yang, B., et al.: Anomalous behaviors detection in moving crowds based on a weighted convolutional autoencoder-long short-term memory network. IEEE Trans. Cogn. Dev. Syst. 11, 473–482 (2018)Google Scholar
  25. 25.
    Sabokrou, M., et al.: Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes. Comput. Vis. Image Underst. 172, 88–97 (2018)CrossRefGoogle Scholar
  26. 26.
    Keçeli, A., Kaya, A.: Violent activity detection with transfer learning method. Electron. Lett. 53(15), 1047–1048 (2017)CrossRefGoogle Scholar
  27. 27.
    Wei, H., et al.: Crowd abnormal detection using two-stream Fully Convolutional Neural Networks. In: 2018 10th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA). IEEE (2018)Google Scholar
  28. 28.
    Yan, M., Jiang, X., Yuan, J.: 3D convolutional generative adversarial networks for detecting temporal irregularities in videos. In: 2018 24th International Conference on Pattern Recognition (ICPR). IEEE (2018)Google Scholar
  29. 29.
    Sudhakaran, S., Lanz, O.: Learning to detect violent videos using convolutional long short-term memory. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE (2017)Google Scholar
  30. 30.
    Hanson, A., et al.: Bidirectional convolutional LSTM for the detection of violence in videos. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)Google Scholar
  31. 31.
    Robnik-Šikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 53(1–2), 23–69 (2003)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Alaa Atallah Almazroey
    • 1
    Email author
  • Salma Kammoun Jarraya
    • 1
    • 2
  1. 1.Computer Science Department, Faculty of Computing and Information TechnologyKing Abdulaziz UniversityJeddahKingdom of Saudi Arabia
  2. 2.MIR@CL LaboratorySfax UniversitySfaxTunisia

Personalised recommendations