Advertisement

A Literature Review on Video Analytics of Crowded Scenes

  • Myo ThidaEmail author
  • Yoke Leng Yong
  • Pau Climent-Pérez
  • How-lung Eng
  • Paolo Remagnino

Abstract

This chapter presents a review and systematic comparison of the state of the art on crowd video analysis. The rationale of our review is justified by a recent increase in intelligent video surveillance algorithms capable of analysing automatically visual streams of very crowded and cluttered scenes, such as those of airport concourses, railway stations, shopping malls and the like. Since the safety and security of potentially very crowded public spaces have become a priority, computer vision researchers have focused their research on intelligent solutions. The aim of this chapter is to propose a critical review of existing literature pertaining to the automatic analysis of complex and crowded scenes. The literature is divided into two broad categories: the macroscopic and the microscopic modelling approach. The effort is meant to provide a reference point for all computer vision practitioners currently working on crowd analysis. We discuss the merits and weaknesses of various approaches for each topic and provide a recommendation on how existing methods can be improved.

References

  1. 1.
    Aggarwal, J.K., Cai, Q.: Human motion analysis: a review. Comput. Vis. Image Underst. 73(3), 428–440 (1999) CrossRefGoogle Scholar
  2. 2.
    Ali, I., Dailey, M.N.: Multiple human tracking in high-density crowds. In: Advanced Concepts in Intelligent Vision Systems, pp. 540–549 (2009) CrossRefGoogle Scholar
  3. 3.
    Ali, S., Shah, M.: A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Minneapolis, Florida, pp. 1–6. IEEE, New York (2007) Google Scholar
  4. 4.
    Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. In: Proceedings of European Conference on Computer Vision, Marseille, France, pp. 1–14. Springer, Berlin (2008) Google Scholar
  5. 5.
    Alper, Y., Omar, J., Mubarak, S.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13–58 (2006) CrossRefGoogle Scholar
  6. 6.
    Andrade, E., Fisher, R.: Simulation of crowd problems for computer vision. In: Proceedings of 19th International Conference on Pattern Recognition, vol. 3, pp. 71–80 (2005) Google Scholar
  7. 7.
    Andrade, E., Fisher, R., Blunsden, S.: Modelling crowd scenes for event detection. In: Proceedings of 19th International Conference on Pattern Recognition, vol. 1, pp. 175–178 (2006) Google Scholar
  8. 8.
    Andriyenko, A., Schindler, K.: Multi-target tracking by continuous energy minimization. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, pp. 1265–1272. IEEE, New York (2011) Google Scholar
  9. 9.
    Antonini, G., Martinez, S.V., Bierlaire, M., Thiran, J.P.: Behavioral priors for detection and tracking of pedestrians in video sequences. Int. J. Comput. Vis. 69(2), 159–180 (1998) CrossRefGoogle Scholar
  10. 10.
    Bai, K.: Particle filter tracking with mean shift and joint probability data association. In: 2010 International Conference on Image Analysis and Signal Processing (IASP), pp. 607–612. IEEE, New York (2010) Google Scholar
  11. 11.
    Barron, J., Fleet, D.J., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994) CrossRefGoogle Scholar
  12. 12.
    BenAbdelkader, C., Cutler, R., Nanda, H., Davis, L.: Eigengait: motion-based recognition of people using image self-similarity. Technical report (2001) Google Scholar
  13. 13.
    Boltes, M., Seyfried, A.: Collecting pedestrian trajectories. Neurocomputing 100, 127–133 (2013) CrossRefGoogle Scholar
  14. 14.
    Cai, Y., Pietikäinen, M.: Person re-identification based on global color context. In: Asian Conference on Computer Vision 2010 Workshops (2011) Google Scholar
  15. 15.
    Cai, Y., de Freitas, N., Little, J.J.: Robust visual tracking for multiple targets. In: Proceedings of Eighth European Conference on Computer Vision, vol. 3954, pp. 107–118. IEEE, New York (2006) Google Scholar
  16. 16.
    Chen, A.h., Yang, B.q, Chen, Z.g.: A timely occlusion detection based on mean shift algorithm. In: Deng, W. (ed.) Future Control and Automation. Lecture Notes in Electrical Engineering, vol. 173, pp. 51–56. Springer, Berlin (2012) CrossRefGoogle Scholar
  17. 17.
    Dexter, E., Pérez, P., Laptev, I.: Multi-view synchronization of human actions and dynamic scenes. In: Proceedings of the British Machine Vision Conference 2009, British Machine Vision Association, pp. 122.1–122.11 (2009) Google Scholar
  18. 18.
    Dockstader, S.L., Tekalp, A.M.: Multiple camera tracking of interacting and occluded human motion. Proc. IEEE 89(10), 1441–1455 (2001) CrossRefGoogle Scholar
  19. 19.
    French, A., Naeem, A., Dryden, I., Pridmore, T.: Using social effects to guide tracking in complex scenes. In: Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 212–217 (2007) Google Scholar
  20. 20.
    Gilbert, A., Bowden, R.: Multi person tracking within crowded scenes. In: Proceedings of Workshop on Human Motion, pp. 166–179 (2007) Google Scholar
  21. 21.
    Haselhoff, A., Hoehmann, L., Nunn, C., Meuter, M., Kummert, A.: On occlusion-handling for people detection fusion in multi-camera networks. In: Dziech, A., Czyżewski, A. (eds.) Multimedia Communications, Services and Security. Communications in Computer and Information Science, vol. 149, pp. 113–119. Springer, Berlin (2011) CrossRefGoogle Scholar
  22. 22.
    Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282–4286 (1995) CrossRefGoogle Scholar
  23. 23.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981) CrossRefGoogle Scholar
  24. 24.
    Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 34(3), 334–352 (2004) CrossRefGoogle Scholar
  25. 25.
    Hu, W., Xiao, X., Fu, Z., Dan, X., Tan, T., Steve, M.: A system for learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1450–1464 (2006) CrossRefGoogle Scholar
  26. 26.
    Hu, M., Ali, S., Shah, M.: Detecting global motion patterns in complex videos. In: Proceedings of International Conference on Pattern Recognition, Tempa, Florida, pp. 1–5. IEEE, New York (2008) Google Scholar
  27. 27.
    Hu, M., Ali, S., Shah, M.: Learning motion patterns in crowded scenes using motion flow field. In: Proceedings of International Conference on Pattern Recognition, Tempa, Florida, pp. 1–5. IEEE, New York (2008) Google Scholar
  28. 28.
    Hu, N., Bouma, H., Worring, M.: Tracking individuals in surveillance video of a high-density crowd. In: Proceedings of SPIE, vol. 8399, p. 839909 (2012) Google Scholar
  29. 29.
    Isard, M., Blake, A.: CONDENSATION conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998) CrossRefGoogle Scholar
  30. 30.
    Jacques Junior, J.C.S., Musse, S.R., Jung, C.R.: Crowd analysis using computer vision techniques: a survey. IEEE Signal Process. Mag. (September), 66–77 (2010) Google Scholar
  31. 31.
    Jiang, Y., Zhang, P., Wong, S., Liu, R.: A higher-order macroscopic model for pedestrian flows. Phys. A, Stat. Mech. Appl. 389(21), 4623–4635 (2010) CrossRefGoogle Scholar
  32. 32.
    Jo, H., Chug, K., Sethi, R.J., Rey, M.: A review of physics-based methods for group and crowd analysis in computer vision. J. Postdr. Res. 1(1), 4–7 (2013) Google Scholar
  33. 33.
    Johnson, N., Hogg, D.: Learning the distribution of object trajectories for event recognition. Image Vis. Comput. 14(8), 583–592 (1996) Google Scholar
  34. 34.
    Junejo, I., Dexter, E., Laptev, I., Pérez, P.: Cross-view action recognition from temporal self-similarities. In: Proceedings of the European Conference on Computer Vision 2008 (2008) Google Scholar
  35. 35.
    Junejo, I.N., Dexter, E., Laptev, I., Pérez, P.: View-independent action recognition from temporal self-similarities. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 172–185 (2011) CrossRefGoogle Scholar
  36. 36.
    Khan, S.M., Shah, M.: A multi-view approach to tracking people in dense crowded scenes using a planar homography constraint. In: Proceedings of Workshop on Human Motion, Graz, Austria, pp. 133–146 (2006) Google Scholar
  37. 37.
    Khan, S., Shah, M.: Tracking multiple occluding people by localizing on multiple scene planes. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 505–519 (2009) CrossRefGoogle Scholar
  38. 38.
    Khan, S.M., Shah, M.: Tracking multiple occluding people by localizing on multiple scene planes. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 505–519 (2009) CrossRefGoogle Scholar
  39. 39.
    Khan, Z., Balch, T., Dellaert, F.: MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE Trans. Pattern Anal. Mach. Intell. 27(11), 1805–1918 (2005) CrossRefGoogle Scholar
  40. 40.
    Kim, J., Grauman, K.: Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2921–2928 (2009) Google Scholar
  41. 41.
    Kratz, L., Nishino, K.: Spatio-temporal motion pattern modelling of extremely crowded scenes. In: The 1st International Workshop on Machine Learning for Vision-Based Motion Analysis, Marseille, France (2008) Google Scholar
  42. 42.
    Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, Maimi Beach, Florida, pp. 1446–1453 (2009) Google Scholar
  43. 43.
    Kratz, L., Nishino, K.: Tracking with local spatio-temporal motion patterns in extremely crowded scenes. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, San Francisco, USA, pp. 693–700 (2010) Google Scholar
  44. 44.
    Kwak, S., Nam, W., Han, B., Han, J.H.: Learning occlusion with likelihoods for visual tracking. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1551–1558 (2011) CrossRefGoogle Scholar
  45. 45.
    Lee, L., Romano, R., Stein, G.: Monitoring activities from multiple video streams: establishing a common coordinate frame. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 758–767 (2000) CrossRefzbMATHGoogle Scholar
  46. 46.
    Li, M., Zhang, Z., Huang, K., Tan, T.: Rapid and robust human detection and tracking based on omega-shape features. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2545–2548 (2009) CrossRefGoogle Scholar
  47. 47.
    Li, J., Lu, X., Ding, L., Lu, H.: Moving target tracking via particle filter based on color and contour features. In: 2010 2nd International Conference on Information Engineering and Computer Science (ICIECS), pp. 1–4. IEEE, New York (2010) CrossRefGoogle Scholar
  48. 48.
    Luber, M., Stork, J.a., Tipaldi, G.D., Arras, K.O.: People tracking with human motion predictions from social forces. In: 2010 IEEE International Conference on Robotics and Automation, pp. 464–469. IEEE, New York (2010) CrossRefGoogle Scholar
  49. 49.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of Image Understanding Workshop, pp. 121–130 (1981) Google Scholar
  50. 50.
    Ma, Y., Cisar, P.: Activity representation in crowd. In: Proceedings of the 2008 Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, Florida, USA, pp. 107–116. Springer, Berlin (2008) Google Scholar
  51. 51.
    Ma, L., Liu, J., Wang, J., Cheng, J., Lu, H.: A improved silhouette tracking approach integrating particle filter with graph cuts. In: 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 1142–1145. IEEE, New York (2010) CrossRefGoogle Scholar
  52. 52.
    Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 1975–1981 (2010) Google Scholar
  53. 53.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, Maimi Beach, Florida, pp. 935–942. IEEE, New York (2009) Google Scholar
  54. 54.
    Michel, B., Gianluca, A., Mats, W.: Behavioural dynamics for pedestrians. In: Lecture Notes in Computer Science, pp. 1–18 (2003) Google Scholar
  55. 55.
    Mittal, A., Davis, L.S.: M2Tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int. J. Comput. Vis. 51(3), 189–203 (2003) CrossRefGoogle Scholar
  56. 56.
    Moore, B.E., Ali, S., Mehran, R., Shah, M.: Visual crowd surveillance through a hydrodynamics lens. Commun. ACM 54(12), 64–73 (2011) CrossRefGoogle Scholar
  57. 57.
    Ni, Z., Sunderrajan, S., Rahimi, A., Manjunath, B.: Distributed particle filter tracking with online multiple instance learning in a camera sensor network. In: 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 37–40. IEEE, New York (2010) CrossRefGoogle Scholar
  58. 58.
    Okuma, K., Taleghani, A., Freitas, N.D., Little, J.J., Lowe, D.G.: A boosted particle filter: multitarget detection and tracking. In: Proceedings of Eighth European Conference on Computer Vision, pp. 28–39. IEEE, New York (2004) Google Scholar
  59. 59.
    Qi, Z., Ting, R., Husheng, F., Jinlin, Z.: Particle filter object tracking based on Harris-SIFT feature matching. Proc. Eng. 29, 924–929 (2012) CrossRefGoogle Scholar
  60. 60.
    Rani, M., Arumugam, G.: An efficient gait recognition system for human identification using modified ICA. Int. J. Comput. Sci. Inf. Technol. 2(1), 55–67 (2010) Google Scholar
  61. 61.
    Reddy, V., Sanderson, C., Lovell, B.: Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture. In: MLvMA Workshop, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 57–63. IEEE, New York (2011) Google Scholar
  62. 62.
    Revathi, A., Kumar, D.: A review of human activity recognition and behaviour understanding in video surveillance. Comput. Sci. Inf. Technol. 2, 375–384 (2012) Google Scholar
  63. 63.
    Rodriguez, M., Ali, S., Kanade, T.: Tracking in unstructured crowded scenes. In: Proceedings of IEEE International Conference on Computer Vision, Kyoto, Japan, pp. 1389–1396. IEEE, New York (2009) Google Scholar
  64. 64.
    Saleemi, I., Hartung, L., Shah, M.: Scene understanding by statistical modeling of motion patterns. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 2069–2076 (2010) Google Scholar
  65. 65.
    Scott, S.: A system for tracking and recognizing multiple people with multiple camera. Technical report GIT-GVU-98-25, Georgia Institute of Technology (1998) Google Scholar
  66. 66.
    Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE, New York (2007) CrossRefGoogle Scholar
  67. 67.
    Shu-hong, C., Chun-hai, H.: Particle filter tracking algorithm based on multi-information fusion. In: 2009 International Conference on Information Engineering and Computer Science, ICIECS 2009, pp. 1–4. IEEE, New York (2009) Google Scholar
  68. 68.
    Sodemann, A.A., Ross, M.P., Borghetti, B.J.: A review of anomaly detection in automated surveillance. IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 42(6), 1257–1272 (2012) CrossRefGoogle Scholar
  69. 69.
    Sugano, H., Miyamoto, R.: Parallel implementation of pedestrian tracking using multiple cues on GPGPU. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 900–906. IEEE, New York (2009) CrossRefGoogle Scholar
  70. 70.
    Tang, S.L., Kadim, Z., Liang, K.M., Lim, M.K.: Hybrid blob and particle filter tracking approach for robust object tracking. Proc. Comput. Sci. 1(1), 2549–2557 (2010) CrossRefGoogle Scholar
  71. 71.
    Thida, M., Eng, H.L., Monekosso, D.N., Remagnino, P.: Learning video manifold for segmenting crowd events and abnormality detection. In: Proceedings of 10th Asian Conference on Computer Vision, pp. 439–449. Springer, Berlin (2010) Google Scholar
  72. 72.
    Vezzani, R., Grana, C., Cucchiara, R.: Probabilistic people tracking with appearance models and occlusion classification: the ad-hoc system. Pattern Recognit. Lett. 32(6), 867–877 (2011) CrossRefGoogle Scholar
  73. 73.
    Vishwakarma, S., Agrawal, A.: A survey on activity recognition and behaviour understanding in video surveillance. Vis. Comput. (September) (2012) Google Scholar
  74. 74.
    Walk, S., Majer, N., Schindler, K., Schiele, B.: New features and insights for pedestrian detection. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1030–1037. IEEE, New York (2010) CrossRefGoogle Scholar
  75. 75.
    Wang, L., Hu, W., Tan, T.: Recent developments in human motion analysis. Pattern Recognit. 36(3), 585–601 (2003) CrossRefGoogle Scholar
  76. 76.
    Wang, X., Tieu, K., Grimson, E.: Learning semantic scene models by trajectory analysis. In: Proceedings of European Conference on Computer Vision, vol. 3, pp. 110–123 (2006) Google Scholar
  77. 77.
    Wang, P., Li, W., Zhu, W., Qiao, H.: Object tracking with serious occlusion based on occluder modeling. In: 2012 International Conference on Mechatronics and Automation (ICMA) pp. 1960–1965 (2012) CrossRefGoogle Scholar
  78. 78.
    Wu, P., Kong, L., Zhao, F., Li, X.: Particle filter tracking based on color and SIFT features. In: 2008 International Conference on Audio, Language and Image Processing, pp. 932–937. IEEE, New York (2008) Google Scholar
  79. 79.
    Wu, S., Moore, B.E., Shah, M.: Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, pp. 2054–2060. IEEE, New York (2010) Google Scholar
  80. 80.
    Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, pp. 1345–1352. IEEE, New York (2011) Google Scholar
  81. 81.
    Yang, Y., Liu, J., Shah, M.: Video scene understanding using multi-scale analysis. In: Proceedings of IEEE International Conference on Computer Vision, Kyoto, Japan, pp. 1669–1676. IEEE, New York (2009) Google Scholar
  82. 82.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4) (2006) Google Scholar
  83. 83.
    Zhan, B., Monekosso, D.N., Remagnino, P., Velastin, S.A., Xu, L.Q.: Crowd analysis: a survey. Mach. Vis. Appl. 19(5–6), 345–357 (2008) CrossRefGoogle Scholar
  84. 84.
    Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via structured multi-task sparse learning. Int. J. Comput. Vis. 1–17 (2012) Google Scholar
  85. 85.
    Zhang, C., Xu, J., Beaugendre, A., Goto, S.: A klt-based approach for occlusion handling in human tracking. In: Picture Coding Symposium (PCS), 2012, pp. 337–340 (2012) Google Scholar
  86. 86.
    Zhong, Q., Qingqing, Z., Tengfei, G.: Moving object tracking based on codebook and particle filter. Proc. Eng. 29, 174–178 (2012) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Myo Thida
    • 1
    Email author
  • Yoke Leng Yong
    • 2
  • Pau Climent-Pérez
    • 2
  • How-lung Eng
    • 1
  • Paolo Remagnino
    • 2
  1. 1.ZWEEC AnalyticsSingaporeSingapore
  2. 2.Kingston UniversityLondonUK

Personalised recommendations