Skip to main content
SpringerLink
Log in

Representing dense crowd patterns using bag of trajectory graphs

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The aim of this paper was to address the problem of dense crowd event recognition in the surveillance video. Previous particle flow-based methods efficiently capture the convolutional motion in the crowded scene. However, the group-level description was rarely studied due to huge loss of group structure and intra-class variability. To address these issues, we present a novel crowd behavior representation called bag of trajectory graphs (BoTG). Firstly, we design a group-level representation beyond particle flow. From the observation that crowd particles are composed of atomic subgroups corresponding to informative behavior patterns, particle trajectories that simulate motion of individuals will be clustered to form groups. Secondly, we connect nodes in each group as a trajectory graph and propose 3 informative features to encode the graphs, namely, graph structure, group attribute, and dynamic motion, which characterize the structure, the motion within, and among the trajectory graphs. Finally, each clip of crowd event can be further described by BoTG as the occurrences of behavior patterns, which provides critical clues for categorizing specific crowd event. We conduct extensive experiments on public datasets for abnormality detection and event recognition. The results demonstrate the effectiveness of our BoTG on characterizing the group behaviors in dense crowd.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://mha.cs.umn.edu/movies/crowdactivity-all.avi.

  2. http://ftp.cs.rdg.ac.uk/PETS2009.

References

  1. Adam, A., Rivlin, E., Shimshoni, I., Reinitz, D.: Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 555–560 (2008)

  2. Ali, S., Shah, M.: A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: CVPR 2007, IEEE, pp. 1–6 (2007)

  3. Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. ECCV 2008, 1–14 (2008)

    MathSciNet  Google Scholar 

  4. Basharat, A., Gritai, A., Shah, M.: Learning object motion patterns for anomaly detection and improved object detection. In: CVPR 2008, IEEE, pp. 1–8 (2008)

  5. Cheng, Z., Qin, L., Huang, Q., Jiang, S., Tian, Q.: Group activity recognition by gaussian processes estimation. In: ICPR 2010, IEEE, pp. 3228–3231 (2010)

  6. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  7. Cong, Y., Yuan, J., Liu, J.: Sparse reconstruction cost for abnormal event detection. In: CVPR 2011, IEEE, pp. 3449–3456 (2011)

  8. Cristani, M., Bazzani, L., Paggetti, G., Fossati, A., Tosato, D., Del Bue, A., Menegaz, G., Murino, V.: Social interaction discovery by statistical analysis of f-formations. In: BMVC, pp. 1–12 (2011)

  9. Cui, X., Liu, Q., Gao, M., Metaxas, D.: Abnormal detection using interaction energy potentials. In: CVPR 2011, IEEE, pp. 3161–3167 (2011)

  10. Ge, W., Collins, R., Ruback, B.: Automatically detecting the small group structure of a crowd. In: WACV 2009, IEEE, pp. 1–8 (2009)

  11. Hospedales, T., Gong, S., Xiang, T.: A markov clustering topic model for mining behaviour in video. In: ICCV 2009, IEEE, pp. 1165–1172 (2009)

  12. Hu, M., Ali, S., Shah, M.: Learning motion patterns in crowded scenes using motion flow field In: ICPR 2008, pp. 1–5 (2008)

  13. Ihaddadene, N., Djeraba, C.: Real-time crowd motion analysis. In: ICPR 2008, IEEE, pp. 1–4 (2008)

  14. Johnson, N., Hogg, D.: Learning the distribution of object trajectories for event recognition. Image Vis. Comput. 14(8), 609–615 (1996)

    Article  Google Scholar 

  15. Kim, J., Grauman, K.: Observe locally, infer globally: a space-time mrf for detecting abnormal activities with incremental updates. In: CVPR 2009, IEEE, pp. 2921–2928 (2009)

  16. Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: CVPR 2009, IEEE, pp. 1446–1453 (2009)

  17. Kuettel, D., Breitenstein, M., Van Gool, L., Ferrari, V.: What’s going on? Discovering spatio-temporal dependencies in dynamic scenes. In: CVPR 2010, IEEE, pp. 1951–1958 (2010)

  18. Lan, T., Wang, Y., Yang, W., Mori, G.: Beyond actions: discriminative models for contextual group activities. NIPS 4321, 4322–4325 (2010)

    Google Scholar 

  19. Lin, W., Sun, M.T., Poovendran, R., Zhang, Z.: Group event detection with a varying number of group members for video surveillance. IEEE Trans. Circuits Syst. Video Technol. 20(8), 1057–1067 (2010)

    Article  Google Scholar 

  20. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: CVPR 2009, IEEE, pp. 935–942 (2009)

  21. Mehran, R., Moore, B., Shah, M.: A streakline representation of flow in crowded scenes. ECCV 2010, 439–452 (2010)

    Google Scholar 

  22. Ni, B., Yan, S., Kassim, A.: Recognizing human group activities with localized causalities. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, pp. 1470–1477 (2009)

  23. Oliver, N., Rosario, B., Pentland, A.: Graphical models for recognizing human interactions. In: NIPS, Citeseer, pp. 924–930 (1998)

  24. Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: ICCV 2009, IEEE, pp. 261–268 (2009)

  25. Pellegrini, S., Ess, A., Van Gool, L.: Improving data association by joint modeling of pedestrian trajectories and groupings. In: ECCV (2010)

  26. Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757 (2000)

    Article  Google Scholar 

  27. Tang, J., Zhang, C., Luo, B.: A graph and pnn-based approach to image classification. In: Proceedings of 2005 International Conference on Machine Learning and Cybernetics, 2005. IEEE, vol 8, pp. 5122–5126 (2005)

  28. Tran, D., Yuan, J.: Optimal spatio-temporal path discovery for video event detection. In: CVPR 2011, IEEE, pp. 3321–3328 (2011)

  29. Wang, X., Ma, K.T., Ng, G.W., Grimson, W.E.L.: Trajectory analysis and semantic region modeling using nonparametric hierarchical bayesian models. Int. J. Comput. Vis. 95(3), 287–312 (2011)

    Article  Google Scholar 

  30. Wu, S.W.S., Moore, B., Shah, M.: Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. CVPR (2010)

  31. Zhang, Y., Qin, L., Yao, H., Huang, Q.: Abnormal crowd behavior detection based on social attribute-aware force model. In: ICIP 2012, IEEE, pp. 2689–2692 (2012)

  32. Zhang, Y., Qin, L., Yao, H., Xu, P., Huang, Q.: Beyond particle flow: bag of trajectory graphs for dense crowd event recognition. In: ICIP (2013)

Download references

Acknowledgments

This work was supported in part by National Basic Research Program of China (973 Program): 2012CB316400, in part by National Natural Science Foundation of China: 61025011, 61133003, 61332016 and 61035001.

Author information

Authors and Affiliations

  1. School of Computer Science, Harbin Institute of Technology, Harbin, 150001, Heilongjiang, China

    Yanhao Zhang, Qingming Huang, Sicheng Zhao, Hongxun Yao & Pengfei Xu

  2. Institue of Computer Techology, Chinese Academy of Science, Beijing, 100010, China

    Lei Qin

Authors
  1. Yanhao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingming Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sicheng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongxun Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pengfei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanhao Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Huang, Q., Qin, L. et al. Representing dense crowd patterns using bag of trajectory graphs. SIViP 8 (Suppl 1), 173–181 (2014). https://doi.org/10.1007/s11760-014-0669-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-014-0669-9

Keywords

Search

Navigation