Skip to main content
SpringerLink
Log in
Book cover

Intelligent Multimedia Surveillance pp 17–36Cite as

A Literature Review on Video Analytics of Crowded Scenes

  • Chapter

Abstract

This chapter presents a review and systematic comparison of the state of the art on crowd video analysis. The rationale of our review is justified by a recent increase in intelligent video surveillance algorithms capable of analysing automatically visual streams of very crowded and cluttered scenes, such as those of airport concourses, railway stations, shopping malls and the like. Since the safety and security of potentially very crowded public spaces have become a priority, computer vision researchers have focused their research on intelligent solutions. The aim of this chapter is to propose a critical review of existing literature pertaining to the automatic analysis of complex and crowded scenes. The literature is divided into two broad categories: the macroscopic and the microscopic modelling approach. The effort is meant to provide a reference point for all computer vision practitioners currently working on crowd analysis. We discuss the merits and weaknesses of various approaches for each topic and provide a recommendation on how existing methods can be improved.

Keywords

  • Unstructured Crowded Scenes
  • Crowd Analysis
  • Typical Motion Patterns
  • Sink Path
  • Optical Flow Clustering

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Aggarwal, J.K., Cai, Q.: Human motion analysis: a review. Comput. Vis. Image Underst. 73(3), 428–440 (1999)

    CrossRef  Google Scholar 

  2. Ali, I., Dailey, M.N.: Multiple human tracking in high-density crowds. In: Advanced Concepts in Intelligent Vision Systems, pp. 540–549 (2009)

    CrossRef  Google Scholar 

  3. Ali, S., Shah, M.: A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Minneapolis, Florida, pp. 1–6. IEEE, New York (2007)

    Google Scholar 

  4. Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. In: Proceedings of European Conference on Computer Vision, Marseille, France, pp. 1–14. Springer, Berlin (2008)

    Google Scholar 

  5. Alper, Y., Omar, J., Mubarak, S.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13–58 (2006)

    CrossRef  Google Scholar 

  6. Andrade, E., Fisher, R.: Simulation of crowd problems for computer vision. In: Proceedings of 19th International Conference on Pattern Recognition, vol. 3, pp. 71–80 (2005)

    Google Scholar 

  7. Andrade, E., Fisher, R., Blunsden, S.: Modelling crowd scenes for event detection. In: Proceedings of 19th International Conference on Pattern Recognition, vol. 1, pp. 175–178 (2006)

    Google Scholar 

  8. Andriyenko, A., Schindler, K.: Multi-target tracking by continuous energy minimization. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, pp. 1265–1272. IEEE, New York (2011)

    Google Scholar 

  9. Antonini, G., Martinez, S.V., Bierlaire, M., Thiran, J.P.: Behavioral priors for detection and tracking of pedestrians in video sequences. Int. J. Comput. Vis. 69(2), 159–180 (1998)

    CrossRef  Google Scholar 

  10. Bai, K.: Particle filter tracking with mean shift and joint probability data association. In: 2010 International Conference on Image Analysis and Signal Processing (IASP), pp. 607–612. IEEE, New York (2010)

    Google Scholar 

  11. Barron, J., Fleet, D.J., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)

    CrossRef  Google Scholar 

  12. BenAbdelkader, C., Cutler, R., Nanda, H., Davis, L.: Eigengait: motion-based recognition of people using image self-similarity. Technical report (2001)

    Google Scholar 

  13. Boltes, M., Seyfried, A.: Collecting pedestrian trajectories. Neurocomputing 100, 127–133 (2013)

    CrossRef  Google Scholar 

  14. Cai, Y., Pietikäinen, M.: Person re-identification based on global color context. In: Asian Conference on Computer Vision 2010 Workshops (2011)

    Google Scholar 

  15. Cai, Y., de Freitas, N., Little, J.J.: Robust visual tracking for multiple targets. In: Proceedings of Eighth European Conference on Computer Vision, vol. 3954, pp. 107–118. IEEE, New York (2006)

    Google Scholar 

  16. Chen, A.h., Yang, B.q, Chen, Z.g.: A timely occlusion detection based on mean shift algorithm. In: Deng, W. (ed.) Future Control and Automation. Lecture Notes in Electrical Engineering, vol. 173, pp. 51–56. Springer, Berlin (2012)

    CrossRef  Google Scholar 

  17. Dexter, E., Pérez, P., Laptev, I.: Multi-view synchronization of human actions and dynamic scenes. In: Proceedings of the British Machine Vision Conference 2009, British Machine Vision Association, pp. 122.1–122.11 (2009)

    Google Scholar 

  18. Dockstader, S.L., Tekalp, A.M.: Multiple camera tracking of interacting and occluded human motion. Proc. IEEE 89(10), 1441–1455 (2001)

    CrossRef  Google Scholar 

  19. French, A., Naeem, A., Dryden, I., Pridmore, T.: Using social effects to guide tracking in complex scenes. In: Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 212–217 (2007)

    Google Scholar 

  20. Gilbert, A., Bowden, R.: Multi person tracking within crowded scenes. In: Proceedings of Workshop on Human Motion, pp. 166–179 (2007)

    Google Scholar 

  21. Haselhoff, A., Hoehmann, L., Nunn, C., Meuter, M., Kummert, A.: On occlusion-handling for people detection fusion in multi-camera networks. In: Dziech, A., Czyżewski, A. (eds.) Multimedia Communications, Services and Security. Communications in Computer and Information Science, vol. 149, pp. 113–119. Springer, Berlin (2011)

    CrossRef  Google Scholar 

  22. Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282–4286 (1995)

    CrossRef  Google Scholar 

  23. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    CrossRef  Google Scholar 

  24. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 34(3), 334–352 (2004)

    CrossRef  Google Scholar 

  25. Hu, W., Xiao, X., Fu, Z., Dan, X., Tan, T., Steve, M.: A system for learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1450–1464 (2006)

    CrossRef  Google Scholar 

  26. Hu, M., Ali, S., Shah, M.: Detecting global motion patterns in complex videos. In: Proceedings of International Conference on Pattern Recognition, Tempa, Florida, pp. 1–5. IEEE, New York (2008)

    Google Scholar 

  27. Hu, M., Ali, S., Shah, M.: Learning motion patterns in crowded scenes using motion flow field. In: Proceedings of International Conference on Pattern Recognition, Tempa, Florida, pp. 1–5. IEEE, New York (2008)

    Google Scholar 

  28. Hu, N., Bouma, H., Worring, M.: Tracking individuals in surveillance video of a high-density crowd. In: Proceedings of SPIE, vol. 8399, p. 839909 (2012)

    Google Scholar 

  29. Isard, M., Blake, A.: CONDENSATION conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)

    CrossRef  Google Scholar 

  30. Jacques Junior, J.C.S., Musse, S.R., Jung, C.R.: Crowd analysis using computer vision techniques: a survey. IEEE Signal Process. Mag. (September), 66–77 (2010)

    Google Scholar 

  31. Jiang, Y., Zhang, P., Wong, S., Liu, R.: A higher-order macroscopic model for pedestrian flows. Phys. A, Stat. Mech. Appl. 389(21), 4623–4635 (2010)

    CrossRef  Google Scholar 

  32. Jo, H., Chug, K., Sethi, R.J., Rey, M.: A review of physics-based methods for group and crowd analysis in computer vision. J. Postdr. Res. 1(1), 4–7 (2013)

    Google Scholar 

  33. Johnson, N., Hogg, D.: Learning the distribution of object trajectories for event recognition. Image Vis. Comput. 14(8), 583–592 (1996)

    Google Scholar 

  34. Junejo, I., Dexter, E., Laptev, I., Pérez, P.: Cross-view action recognition from temporal self-similarities. In: Proceedings of the European Conference on Computer Vision 2008 (2008)

    Google Scholar 

  35. Junejo, I.N., Dexter, E., Laptev, I., Pérez, P.: View-independent action recognition from temporal self-similarities. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 172–185 (2011)

    CrossRef  Google Scholar 

  36. Khan, S.M., Shah, M.: A multi-view approach to tracking people in dense crowded scenes using a planar homography constraint. In: Proceedings of Workshop on Human Motion, Graz, Austria, pp. 133–146 (2006)

    Google Scholar 

  37. Khan, S., Shah, M.: Tracking multiple occluding people by localizing on multiple scene planes. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 505–519 (2009)

    CrossRef  Google Scholar 

  38. Khan, S.M., Shah, M.: Tracking multiple occluding people by localizing on multiple scene planes. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 505–519 (2009)

    CrossRef  Google Scholar 

  39. Khan, Z., Balch, T., Dellaert, F.: MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE Trans. Pattern Anal. Mach. Intell. 27(11), 1805–1918 (2005)

    CrossRef  Google Scholar 

  40. Kim, J., Grauman, K.: Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2921–2928 (2009)

    Google Scholar 

  41. Kratz, L., Nishino, K.: Spatio-temporal motion pattern modelling of extremely crowded scenes. In: The 1st International Workshop on Machine Learning for Vision-Based Motion Analysis, Marseille, France (2008)

    Google Scholar 

  42. Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, Maimi Beach, Florida, pp. 1446–1453 (2009)

    Google Scholar 

  43. Kratz, L., Nishino, K.: Tracking with local spatio-temporal motion patterns in extremely crowded scenes. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, San Francisco, USA, pp. 693–700 (2010)

    Google Scholar 

  44. Kwak, S., Nam, W., Han, B., Han, J.H.: Learning occlusion with likelihoods for visual tracking. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1551–1558 (2011)

    CrossRef  Google Scholar 

  45. Lee, L., Romano, R., Stein, G.: Monitoring activities from multiple video streams: establishing a common coordinate frame. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 758–767 (2000)

    CrossRef  MATH  Google Scholar 

  46. Li, M., Zhang, Z., Huang, K., Tan, T.: Rapid and robust human detection and tracking based on omega-shape features. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2545–2548 (2009)

    CrossRef  Google Scholar 

  47. Li, J., Lu, X., Ding, L., Lu, H.: Moving target tracking via particle filter based on color and contour features. In: 2010 2nd International Conference on Information Engineering and Computer Science (ICIECS), pp. 1–4. IEEE, New York (2010)

    CrossRef  Google Scholar 

  48. Luber, M., Stork, J.a., Tipaldi, G.D., Arras, K.O.: People tracking with human motion predictions from social forces. In: 2010 IEEE International Conference on Robotics and Automation, pp. 464–469. IEEE, New York (2010)

    CrossRef  Google Scholar 

  49. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of Image Understanding Workshop, pp. 121–130 (1981)

    Google Scholar 

  50. Ma, Y., Cisar, P.: Activity representation in crowd. In: Proceedings of the 2008 Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, Florida, USA, pp. 107–116. Springer, Berlin (2008)

    Google Scholar 

  51. Ma, L., Liu, J., Wang, J., Cheng, J., Lu, H.: A improved silhouette tracking approach integrating particle filter with graph cuts. In: 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 1142–1145. IEEE, New York (2010)

    CrossRef  Google Scholar 

  52. Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 1975–1981 (2010)

    Google Scholar 

  53. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, Maimi Beach, Florida, pp. 935–942. IEEE, New York (2009)

    Google Scholar 

  54. Michel, B., Gianluca, A., Mats, W.: Behavioural dynamics for pedestrians. In: Lecture Notes in Computer Science, pp. 1–18 (2003)

    Google Scholar 

  55. Mittal, A., Davis, L.S.: M2Tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int. J. Comput. Vis. 51(3), 189–203 (2003)

    CrossRef  Google Scholar 

  56. Moore, B.E., Ali, S., Mehran, R., Shah, M.: Visual crowd surveillance through a hydrodynamics lens. Commun. ACM 54(12), 64–73 (2011)

    CrossRef  Google Scholar 

  57. Ni, Z., Sunderrajan, S., Rahimi, A., Manjunath, B.: Distributed particle filter tracking with online multiple instance learning in a camera sensor network. In: 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 37–40. IEEE, New York (2010)

    CrossRef  Google Scholar 

  58. Okuma, K., Taleghani, A., Freitas, N.D., Little, J.J., Lowe, D.G.: A boosted particle filter: multitarget detection and tracking. In: Proceedings of Eighth European Conference on Computer Vision, pp. 28–39. IEEE, New York (2004)

    Google Scholar 

  59. Qi, Z., Ting, R., Husheng, F., Jinlin, Z.: Particle filter object tracking based on Harris-SIFT feature matching. Proc. Eng. 29, 924–929 (2012)

    CrossRef  Google Scholar 

  60. Rani, M., Arumugam, G.: An efficient gait recognition system for human identification using modified ICA. Int. J. Comput. Sci. Inf. Technol. 2(1), 55–67 (2010)

    Google Scholar 

  61. Reddy, V., Sanderson, C., Lovell, B.: Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: MLvMA Workshop, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 57–63. IEEE, New York (2011)

    Google Scholar 

  62. Revathi, A., Kumar, D.: A review of human activity recognition and behaviour understanding in video surveillance. Comput. Sci. Inf. Technol. 2, 375–384 (2012)

    Google Scholar 

  63. Rodriguez, M., Ali, S., Kanade, T.: Tracking in unstructured crowded scenes. In: Proceedings of IEEE International Conference on Computer Vision, Kyoto, Japan, pp. 1389–1396. IEEE, New York (2009)

    Google Scholar 

  64. Saleemi, I., Hartung, L., Shah, M.: Scene understanding by statistical modeling of motion patterns. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 2069–2076 (2010)

    Google Scholar 

  65. Scott, S.: A system for tracking and recognizing multiple people with multiple camera. Technical report GIT-GVU-98-25, Georgia Institute of Technology (1998)

    Google Scholar 

  66. Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE, New York (2007)

    CrossRef  Google Scholar 

  67. Shu-hong, C., Chun-hai, H.: Particle filter tracking algorithm based on multi-information fusion. In: 2009 International Conference on Information Engineering and Computer Science, ICIECS 2009, pp. 1–4. IEEE, New York (2009)

    Google Scholar 

  68. Sodemann, A.A., Ross, M.P., Borghetti, B.J.: A review of anomaly detection in automated surveillance. IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 42(6), 1257–1272 (2012)

    CrossRef  Google Scholar 

  69. Sugano, H., Miyamoto, R.: Parallel implementation of pedestrian tracking using multiple cues on GPGPU. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 900–906. IEEE, New York (2009)

    CrossRef  Google Scholar 

  70. Tang, S.L., Kadim, Z., Liang, K.M., Lim, M.K.: Hybrid blob and particle filter tracking approach for robust object tracking. Proc. Comput. Sci. 1(1), 2549–2557 (2010)

    CrossRef  Google Scholar 

  71. Thida, M., Eng, H.L., Monekosso, D.N., Remagnino, P.: Learning video manifold for segmenting crowd events and abnormality detection. In: Proceedings of 10th Asian Conference on Computer Vision, pp. 439–449. Springer, Berlin (2010)

    Google Scholar 

  72. Vezzani, R., Grana, C., Cucchiara, R.: Probabilistic people tracking with appearance models and occlusion classification: the ad-hoc system. Pattern Recognit. Lett. 32(6), 867–877 (2011)

    CrossRef  Google Scholar 

  73. Vishwakarma, S., Agrawal, A.: A survey on activity recognition and behaviour understanding in video surveillance. Vis. Comput. (September) (2012)

    Google Scholar 

  74. Walk, S., Majer, N., Schindler, K., Schiele, B.: New features and insights for pedestrian detection. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1030–1037. IEEE, New York (2010)

    CrossRef  Google Scholar 

  75. Wang, L., Hu, W., Tan, T.: Recent developments in human motion analysis. Pattern Recognit. 36(3), 585–601 (2003)

    CrossRef  Google Scholar 

  76. Wang, X., Tieu, K., Grimson, E.: Learning semantic scene models by trajectory analysis. In: Proceedings of European Conference on Computer Vision, vol. 3, pp. 110–123 (2006)

    Google Scholar 

  77. Wang, P., Li, W., Zhu, W., Qiao, H.: Object tracking with serious occlusion based on occluder modeling. In: 2012 International Conference on Mechatronics and Automation (ICMA) pp. 1960–1965 (2012)

    CrossRef  Google Scholar 

  78. Wu, P., Kong, L., Zhao, F., Li, X.: Particle filter tracking based on color and SIFT features. In: 2008 International Conference on Audio, Language and Image Processing, pp. 932–937. IEEE, New York (2008)

    Google Scholar 

  79. Wu, S., Moore, B.E., Shah, M.: Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, pp. 2054–2060. IEEE, New York (2010)

    Google Scholar 

  80. Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, pp. 1345–1352. IEEE, New York (2011)

    Google Scholar 

  81. Yang, Y., Liu, J., Shah, M.: Video scene understanding using multi-scale analysis. In: Proceedings of IEEE International Conference on Computer Vision, Kyoto, Japan, pp. 1669–1676. IEEE, New York (2009)

    Google Scholar 

  82. Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4) (2006)

    Google Scholar 

  83. Zhan, B., Monekosso, D.N., Remagnino, P., Velastin, S.A., Xu, L.Q.: Crowd analysis: a survey. Mach. Vis. Appl. 19(5–6), 345–357 (2008)

    CrossRef  Google Scholar 

  84. Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via structured multi-task sparse learning. Int. J. Comput. Vis. 1–17 (2012)

    Google Scholar 

  85. Zhang, C., Xu, J., Beaugendre, A., Goto, S.: A klt-based approach for occlusion handling in human tracking. In: Picture Coding Symposium (PCS), 2012, pp. 337–340 (2012)

    Google Scholar 

  86. Zhong, Q., Qingqing, Z., Tengfei, G.: Moving object tracking based on codebook and particle filter. Proc. Eng. 29, 174–178 (2012)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ZWEEC Analytics, Singapore, Singapore

    Myo Thida & How-lung Eng

  2. Kingston University, London, UK

    Yoke Leng Yong, Pau Climent-Pérez & Paolo Remagnino

Authors
  1. Myo Thida
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoke Leng Yong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pau Climent-Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. How-lung Eng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paolo Remagnino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Myo Thida .

Editor information

Editors and Affiliations

  1. Department of Applied Computer Science, University of Winnipeg, Winnipeg, Canada

    Pradeep K. Atrey

  2. Dept. of Computer Science, National University of Singapore, Singapore, Singapore

    Mohan S. Kankanhalli

  3. Elec. Engineering and Computer Science, Queen Mary University of London, London, United Kingdom

    Andrea Cavallaro

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Thida, M., Yong, Y.L., Climent-Pérez, P., Eng, Hl., Remagnino, P. (2013). A Literature Review on Video Analytics of Crowded Scenes. In: Atrey, P., Kankanhalli, M., Cavallaro, A. (eds) Intelligent Multimedia Surveillance. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41512-8_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-41512-8_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-41511-1

  • Online ISBN: 978-3-642-41512-8

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation