Skip to main content
SpringerLink
Log in

The utility of datasets in crowd modelling and analysis: a survey

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Computer Vision-based smart surveillance systems are needed in the present era that can analyse crowd events for behaviour assessment, activity and event recognition, anomaly detection and recognition, crowd density estimation, and counting etc. Even with the human resource available for surveillance of an event, any turn of events can convert a peaceful crowd to a violent one which can cause causalities in no time. Therefore, smart systems need to be introduced which recognizes the behaviour of the crowd and inform the officials beforehand. However, datasets related to the above-mentioned problems are diversely classified. Thus, a need was felt to organize crowd datasets available on the web on their crowd definition, applications, methodologies, and metadata. This paper attempts to do this and gives a comprehensive survey of online publicly available datasets for studying crowd dynamics. It was also observed that available datasets do not cover several important natural events like gate entry and exit surveillance, exit events after religious rituals, and violent activities etc. Some of such events play a crucial role in defining abnormal behaviour. Furthermore, the number of crowd events in some of the available datasets are quite a few and are simulated. To overcome the limitations of the existing datasets, a crowd dataset, named CRUETPAK (CRowd UET PAKistan) is presented. The dataset includes video clips of group and crowd activities related to surveillance, sports, dining, education, and various human interactions (surpassing counts and realism of existing datasets).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Data availability

Available on request.

Notes

  1. http://cvml.kics.edu.pk/datasets/crowd.html

  2. http://cvml.kics.edu.pk/datasets/crowd.html

  3. http://cvml.kics.edu.pk/datasets/crowd.html

References

  1. Ali S, Shah M (2007) A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: Computer vision and pattern recognition, 2007. CVPR'07. IEEE conference on IEEE. pp. 1-6

  2. Ali S, Shah M (2008) Floor fields for tracking in high density crowd scenes. Computer Vision–ECCV 2008, 1–14

  3. Almeida JE, Rosseti RJ, Coelho AL (2013) Crowd simulation modeling applied to emergency and evacuation simulations using multi-agent systems. arXiv preprint arXiv:1303.4692

  4. Andrade EL, Fisher RB (2005) Simulation of crowd problems for computer vision. In: First international workshop on crowd simulation. Vol. 3, pp. 71-80

  5. Andrade EL, Blunsden S, Fisher RB (2006) Modelling crowd scenes for event detection. In: Pattern recognition, 2006. ICPR 2006. 18th international conference on IEEE. Vol. 1, pp. 175-178

  6. Andrade EL, Blunsden S, Fisher RB (2006) Hidden markov models for optical flow analysis in crowds. In: Pattern recognition, 2006. ICPR 2006. 18th international conference on IEEE. Vol. 1, pp. 460-463

  7. Atrey PK, Cavallaro A, Kankanhalli M (2013) Intelligent multimedia surveillance. Springer, Berlin

    Book  Google Scholar 

  8. AUTHOR (n.d.) Retrieved June 10, 2017, from http://homepages.inf.ed.ac.uk/rbf/CAVIAR/pubs.htm

  9. Bandini S, Manzoni S, Vizzari G (2006) Crowd modeling and simulation. Innovations in Design & Decision Support Systems in Architecture and Urban Planning, 105–120

  10. Bird N, Atev S, Caramelli N, Martin R, Masoud O, Papanikolopoulos N (2006) Real time, online detection of abandoned objects in public areas. In: Robotics and automation, 2006. ICRA 2006. Proceedings 2006 IEEE international conference on IEEE. pp. 3775-3780

  11. Blunsden S, Fisher RB (2010) The BEHAVE video dataset: ground truthed video for multi-person behavior classification. Annals BMVA 4(1–12):4

    Google Scholar 

  12. Brostow GJ, Cipolla R (2006) Unsupervised bayesian detection of independent motion in crowds. In: Computer vision and pattern recognition, 2006 IEEE computer society conference on IEEE. Vol. 1, pp. 594–601

  13. Change Loy C, Gong S, Xiang T (2013) From semi-supervised to transfer counting of crowds. In: Proceedings of the IEEE international conference on computer vision. pp. 2256-2263

  14. Chaquet JM, Carmona EJ, Fernández-Caballero A (2013) A survey of video datasets for human action and activity recognition. Comput Vis Image Underst 117(6):633–636

    Article  Google Scholar 

  15. Chen K, Loy CC, Gong S, Xiang T (2012) Feature Mining for Localised Crowd Counting. BMVC 1(2):3

    Google Scholar 

  16. Chen K, Gong S, Xiang T, Change Loy C (2013) Cumulative attribute space for age and crowd density estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2467-2474)

  17. Cheriyadat AM, Radke RJ (2008) Detecting dominant motions in dense crowds. IEEE J Sel Topics Signal Process 2(4):568–581

    Article  Google Scholar 

  18. Courty N, Corpetti T (2007) Crowd motion capture. Comput Animat Virtual Worlds 18(4–5):361–370

    Article  Google Scholar 

  19. Courty N, Allain P, Creusot C, Corpetti T (2014) Using the AGORASET dataset: assessing for the quality of crowd video analysis methods. Pattern Recogn Lett 44:161–170

    Article  Google Scholar 

  20. Davies AC, Yin JH, Velastin SA (1995) Crowd monitoring using image processing. Electron Commun Eng J 7(1):37–47

    Article  Google Scholar 

  21. Ess A, Leibe B, Van Gool L (2007) Depth and appearance for mobile scene analysis. In: Computer vision, 2007. ICCV 2007. IEEE 11th international conference on IEEE. pp. 1-8

  22. Ess A, Leibe B, Schindler K, Van Gool L (2008) A mobile vision system for robust multi-person tracking. In 2008 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-8). IEEE

  23. Ess A, Leibe B, Schindler K, Van Gool L (2009) Moving obstacle detection in highly dynamic scenes. In: Robotics and automation, 2009. ICRA'09. IEEE international conference on IEEE. pp. 56-63

  24. Ferraro F, Mostafazadeh N, Vanderwende L, Devlin J, Galley M, Mitchell M (2015) A survey of current datasets for vision and language research. arXiv preprint arXiv:1506.06833

  25. Ferryman J, Shahrokni A (2009) Pets2009: Dataset and challenge. In: Performance Evaluation of Tracking and Surveillance (PETS-Winter), 2009 Twelfth IEEE International Workshop on IEEE. pp. 1–6

  26. Fisher R, Santos-Victor J, Crowley J (2005) CAVIAR: context aware vision using image-based active recognition. 2013-07-01)[2012-06-10]. http://homepages.Inf.Ed.ac.Uk/rbf/CAVIAR

  27. Fruin JJ (1971) Pedestrian planning and design (no. 206 pp)

  28. Hassner T, Itcher Y, Kliper-Gross O (2012) Violent flows: real-time detection of violent crowd behavior. In: Computer vision and pattern recognition workshops (CVPRW), 2012 IEEE computer society conference on IEEE. pp. 1-6

  29. Helbing D, Johansson A, Al-Abideen HZ (2007) Dynamics of crowd disasters: an empirical study. Phys Rev E 75(4):046109

    Article  Google Scholar 

  30. http://cvml.kics.edu.pk/dataset/cvml_crowd

  31. Idrees H, Saleemi I, Seibert C, Shah M (2013) Multi-source multi-scale counting in extremely dense crowd images. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2547-2554

  32. Ihaddadene N, Djeraba C (2008) Real-time crowd motion analysis. In: Pattern recognition, 2008. ICPR 2008. 19th international conference on IEEE. pp. 1-4

  33. Jacques JCS, Braun A, Soldera J, Musse SR, Jung CR (2007) Understanding people motion in video sequences using Voronoi diagrams. Pattern Anal Appl 10(4):321–332

    Article  MathSciNet  Google Scholar 

  34. Junior JCSJ, Musse SR, Jung CR (2010) Crowd analysis using computer vision techniques. IEEE Signal Process Mag 27(5):66–77

    Google Scholar 

  35. Leal-Taixé L, Milan A, Reid I, Roth S, Schindler K (2015) MOTChallenge 2015: towards a benchmark for multi-target tracking. arXiv preprint arXiv:1504.01942

  36. Lee KH, Choi MG, Hong Q, Lee J (2007) Group behavior from video: a data-driven approach to crowd simulation. In: Proceedings of the 2007 ACM SIGGRAPH/Eurographics symposium on computer animation. Eurographics association. pp. 109-118

  37. Leibe B, Cornelis N, Cornelis K, Van Gool L (2007) Dynamic 3d scene analysis from a moving vehicle. In: Computer vision and pattern recognition, 2007. CVPR'07. IEEE conference on IEEE. pp. 1-8

  38. Lerner A, Chrysanthou Y, Lischinski D (2007) Crowds by example. Computer Graphics Forum 26(3):655–664 Blackwell publishing ltd

  39. Li, W., Mahadevan, V., & Vasconcelos, N. (2013). Anomaly detection and localization in crowded scenes. IEEE transactions on pattern analysis and machine intelligence, 36(1), 18-32.

  40. Lim MK, Kok VJ, Loy CC, Chan CS (2014) Crowd saliency detection via global similarity structure. In: Pattern recognition (ICPR), 2014 22nd international conference on IEEE. pp. 3957-3962

  41. Loy CC, Chen K, Gong S, Xiang T (2013) Crowd counting and profiling: methodology and evaluation. In: Modeling, simulation and visual analysis of crowds. Springer, New York, pp 347–382

    Chapter  Google Scholar 

  42. Ma R, Li L, Huang W, Tian Q (2004) On pixel count based crowd density estimation for visual surveillance. In: Cybernetics and intelligent systems, 2004 IEEE conference on IEEE. Vol. 1, pp. 170-173

  43. Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. In: Computer vision and pattern recognition (CVPR), 2010 IEEE conference on IEEE. pp. 1975-1981

  44. Mehran R, Oyama, A, Shah M (2009) Abnormal crowd behavior detection using social force model. In 2009 IEEE conference on computer vision and pattern recognition (pp. 935-942). IEEE

  45. Musse SR, Paravisi M, Rodrigues R, Jacques Jr JCS, Jung CR (2007) Using synthetic ground truth data to evaluate computer vision techniques. In: IEEE workshop on performance evaluation of tracking systems (in conjunction with ICCV 07). pp. 25-32

  46. Pellegrini S, Ess A, Schindler K, Van Gool L (2009) You'll never walk alone: modeling social behavior for multi-target tracking. In: computer vision, 2009 IEEE 12th international conference on IEEE. pp. 261-268

  47. Possegger H, Sternig S, Mauthner T, Roth PM, Bischof H (2013) Robust real-time tracking of multiple objects by volumetric mass densities. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2395-2402

  48. Rodriguez M, Sivic J, Laptev I, Audibert JY (2011) Data-driven crowd analysis in videos. In: Computer vision (ICCV), 2011 IEEE international conference on IEEE. pp. 1235-1242

  49. Shao J, Change Loy C, Wang X (2014) Scene-independent group profiling in crowd. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2219-2226

  50. Shao J, Kang K, Change Loy C, Wang X (2015) Deeply learned attributes for crowded scene understanding. In:Proceedings of the IEEE conference on computer vision and pattern recognition pp. 4657–4666

  51. Taylor GR, Chosa, AJ, Brewer PC (2007) Ovvv: using virtual worlds to design and evaluate surveillance systems. In: Computer vision and pattern recognition, 2007. CVPR'07. IEEE conference on IEEE. pp. 1-8

  52. Thida M, Yong YL, Climent-Pérez P, Eng HL, Remagnino P (2013) A literature review on video analytics of crowded scenes. In: Intelligent Multimedia Surveillance. Springer, Berlin Heidelberg, pp 17–36

    Chapter  Google Scholar 

  53. ViPER: The Video Performance Evaluation Resource. (n.d.). Retrieved June 06, 2017, from http://viper-toolkit.sourceforge.net/

  54. Vreugdenhil BJ, Bellomo N, Townsend PS (2015) Using crowd modelling in evacuation decision making. In: Proceedings of the ISCRAM 2015 conference, Kristiansand. pp. 24-27

  55. Wu X, Liang G, Lee KK, Xu Y (2006) Crowd density estimation using texture analysis and learning. In: Robotics and biomimetics, 2006. ROBIO'06. IEEE international conference on IEEE. pp. 214–219

  56. Zawbaa H, Aly SA (2012) Hajj and umrah event recognition datasets. arXiv preprint arXiv:1205.2345

  57. Zhan B, Monekosso DN, Remagnino P, Velastin SA, Xu LQ (2008) Crowd analysis: a survey. Mach Vis Appl 19(5–6):345–357

    Article  Google Scholar 

  58. Zhou B, Wang X, Tang X (2012) Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: Computer vision and pattern recognition (CVPR), 2012 IEEE conference on IEEE. pp. 2871-2878

Download references

Acknowledgements

The project ‘Automatic Security and Surveillance System for Video Sequences’ and this research are fully supported by the National ICT R&D Fund. The authors would also like to appreciate the efforts of the research staff at CVML Lab, KICS, UET, Lahore, Pakistan, in assisting the generation of the presented crowd dataset.

Code availability

Not Applicable.

Funding

The project ‘Automatic Security and Surveillance System for Video Sequences’ and this research are fully supported by the National ICT R&D Fund.

Author information

Authors and Affiliations

  1. Computer Engineering, University of Engineering and Technology, Lahore, Pakistan

    Sahar Waqar

  2. Computer Science, University of Engineering and Technology, Lahore, Pakistan

    Usman Ghani Khan & Samyan Qayyum

  3. Electrical Engineering, University of Engineering and Technology, Lahore, Pakistan

    M. Hamza Waseem

Authors
  1. Sahar Waqar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Usman Ghani Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Hamza Waseem
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samyan Qayyum
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Ms. Sahar conceived of the presented idea and her advisor was Dr. Usman Ghani Khan. Mr. Samyan and Mr. Hamza worked on the compilation and review of the document.

Corresponding author

Correspondence to Sahar Waqar.

Ethics declarations

Conflict of interest

Not Applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Waqar, S., Khan, U.G., Waseem, M.H. et al. The utility of datasets in crowd modelling and analysis: a survey. Multimed Tools Appl 81, 43947–43978 (2022). https://doi.org/10.1007/s11042-022-13227-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13227-x

Keywords

Search

Navigation