Abstract
In a visual surveillance system, predicting crowd behavior has recently emerged as a crucial problem for crowd management and monitoring. Specifically, potential dangers and disasters can be avoided by correctly detecting crowd behavior. In this paper, we propose an approach to forecast crowd behavior using a deep learning framework and multiclass Support Vector Machine (SVM). We extract spatio-temporal descriptors using 3D Convolutional Neural Network (3DCNN) based on crowd emotions. In particular, the learned emotion based descriptors help to build the semantic ambiguity in classifying crowd behavior. The effectiveness of our approach is validated with 3 benchmark datasets: Motion Emotion Dataset (MED), ViolentFlows and UMN. The obtained results prove that our approach is successful in predicting crowd behavior in challenging situations. Our system also outperforms existing methods that use local feature descriptors, which reveals that emotions from spatio-temporal features are beneficial for the correct anticipation of crowd behavior.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Detection of Unusual Crowd Activity. http://mha.cs.umn.edu/proj_events.shtml/crowd. Accessed 7 Apr 2018
Baig, M.W., Barakova, E.I., Marcenaro, L., Rauterberg, M., Regazzoni, C.S.: Crowd emotion detection using dynamic probabilistic models. In: del Pobil, A.P., Chinellato, E., Martinez-Martin, E., Hallam, J., Cervera, E., Morales, A. (eds.) SAB 2014. LNCS, vol. 8575, pp. 328–337. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08864-8_32
Blondel, M., Fujino, A., Ueda, N.: Large-scale multiclass support vector machine training via euclidean projection onto the simplex. In: 2014 22nd International Conference on Pattern Recognition, ICPR, pp. 1289–1294. IEEE (2014)
Dupont, C., TobÃas, L., Luvison, B.: Crowd-11: a dataset for fine grained crowd behaviour analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 9–16 (2017)
Guy, S.J., Kim, S., Lin, M.C., Manocha, D.: Simulating heterogeneous crowd behaviors using personality trait theory. In: Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 43–52. ACM (2011)
Hassner, T., Itcher, Y., Kliper-Gross, O.: Violent flows: real-time detection of violent crowd behavior. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW, pp. 1–6. IEEE (2012)
Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)
Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 935–942. IEEE (2009)
Mousavi, H., Mohammadi, S., Perina, A., Chellali, R., Murino, V.: Analyzing tracklets for the detection of abnormal crowd behavior. In: 2015 IEEE Winter Conference on Applications of Computer Vision, WACV, pp. 148–155. IEEE (2015)
Rabiee, H., Haddadnia, J., Mousavi, H.: Crowd behavior representation: an attribute-based approach. SpringerPlus 5(1), 1179 (2016)
Rabiee, H., Haddadnia, J., Mousavi, H., Kalantarzadeh, M., Nabi, M., Murino, V.: Novel dataset for fine-grained abnormal behavior understanding in crowd. In: 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS, pp. 95–101. IEEE (2016)
Sabokrou, M., Fayyaz, M., Fathy, M., et al.: Fully convolutional neural network for fast anomaly detection in crowded scenes. arXiv preprint arXiv:1609.00866 (2016)
Saifi, L., Boubetra, A., Nouioua, F.: An approach for emotions and behavior modeling in a crowd in the presence of rare events. Adapt. Behav. 24(6), 428–445 (2016)
Shao, J., Kang, K., Change Loy, C., Wang, X.: Deeply learned attributes for crowded scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4657–4666 (2015)
Amir Sjarif, N.N., Shamsuddin, S.M., Mohd Hashim, S.Z., Yuhaniz, S.S.: Crowd analysis and its applications. In: Mohamad Zain, J., Wan Mohd, W.M., El-Qawasmeh, E. (eds.) ICSECS 2011. CCIS, vol. 179, pp. 687–697. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22170-5_59
Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: 2015 IEEE International Conference on Computer Vision, ICCV, pp. 4489–4497. IEEE (2015)
Turner, R.H., Killian, L.M., et al.: Collective Behavior. Prentice-Hall, Englewood Cliffs (1957)
Urizar, O.J., Barakova, E.I., Marcenaro, L., Regazzoni, C.S., Rauterberg, M.: Emotion estimation in crowds: a survey (2017)
Wang, H., Kläser, A., Schmid, C., Liu, C.L.: Dense trajectories and motion boundary descriptors for action recognition. Int. J. Comput. Vis. 103(1), 60–79 (2013)
Wang, J., Xu, Z.: Spatio-temporal texture modelling for real-time crowd anomaly detection. Comput. Vis. Image Underst. 144, 177–187 (2016)
Xu, M., et al.: Crowd behavior simulation with emotional contagion in unexpected multi-hazard situations. arXiv preprint arXiv:1801.10000 (2018)
Yogameena, B., Nagananthini, C.: Computer vision based crowd disaster avoidance system: a survey. Int. J. Disaster Risk Reduct. 22, 95–129 (2017)
Zhang, L., Feng, Y., Han, J., Zhen, X.: Realistic human action recognition: when deep learning meets VLAD. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, pp. 1352–1356. IEEE (2016)
Zitouni, M.S., Bhaskar, H., Dias, J., Al-Mualla, M.E.: Advances and trends in visual crowd analysis: a systematic survey and evaluation of crowd modelling techniques. Neurocomputing 186, 139–159 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Varghese, E.B., Thampi, S.M. (2018). A Deep Learning Approach to Predict Crowd Behavior Based on Emotion. In: Basu, A., Berretti, S. (eds) Smart Multimedia. ICSM 2018. Lecture Notes in Computer Science(), vol 11010. Springer, Cham. https://doi.org/10.1007/978-3-030-04375-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-04375-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04374-2
Online ISBN: 978-3-030-04375-9
eBook Packages: Computer ScienceComputer Science (R0)