Finding Coherent Motions and Semantic Regions in Crowd Scenes: A Diffusion and Clustering Approach

  • Weiyue Wang
  • Weiyao Lin
  • Yuanzhe Chen
  • Jianxin Wu
  • Jingdong Wang
  • Bin Sheng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8689)


This paper addresses the problem of detecting coherent motions in crowd scenes and subsequently constructing semantic regions for activity recognition. We first introduce a coarse-to-fine thermal-diffusion-based approach. It processes input motion fields (e.g., optical flow fields) and produces a coherent motion filed, named as thermal energy field. The thermal energy field is able to capture both motion correlation among particles and the motion trends of individual particles which are helpful to discover coherency among them. We further introduce a two-step clustering process to construct stable semantic regions from the extracted time-varying coherent motions. Finally, these semantic regions are used to recognize activities in crowded scenes. Experiments on various videos demonstrate the effectiveness of our approach.


Thermal Energy Motion Vector Input Motion Coherent Motion Cluster Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
  2. 2.
    Ali, S., Shah, M.: A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: CVPR (2007)Google Scholar
  3. 3.
    Beucher, S., Meyer, F.: The morphological approach to segmentation: the watershed transformation. Optical Engineering (1992)Google Scholar
  4. 4.
    Brox, T., Malik, J.: Object segmentation by long term analysis of point trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Brox, T., Rousson, M., Deriche, R., Weickert, J.: Colour, texture, and motion in level set based segmentation and tracking. Image Vis. Comput. (2010)Google Scholar
  6. 6.
    Bruh, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int’l J. Computer Vision (2005)Google Scholar
  7. 7.
    Carslaw, H., Jaeger, J.: Conduction of Heat in Solids. IEEE Trans. Pattern Analysis and Machine Intelligence (1986)Google Scholar
  8. 8.
    Chang, C., Lin, C.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–27 (2011)CrossRefGoogle Scholar
  9. 9.
    Cremers, D., Soatto, S.: Motion competition: A variational approach to piecewise parametric motion segmentation. Int. J. Comput. Vis. (2005)Google Scholar
  10. 10.
    Cui, X., Liu, Q., Gao, M., Metaxas, D.N.: Abnormal detection using interaction energy potentials. In: CVPR (2011)Google Scholar
  11. 11.
    Edelsbrunner, H., Shah, N.: Incremental topological flipping works for regular triangulations. Algorithmica (1996)Google Scholar
  12. 12.
    Hu, M., Ali, S., Shah, M.: Learning motion patterns in crowded scenes using motion flow field. In: ICPR (2008)Google Scholar
  13. 13.
    Li, J., Gong, S., Xiang, T.: Scene segmentation for behaviour correlation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 383–395. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Lin, D., Grimson, E., Fisher, J.: Learning visual flows: a lie algebraic approach. In: CVPR (2009)Google Scholar
  15. 15.
    Loy, C.C., Xiang, T., Gong, S.: Multi-camera activity correlation analysis. In: CVPR (2009)Google Scholar
  16. 16.
    Lu, Z., Yang, X., Lin, W., Zha, H., Chen, X.: Inferring user image search goals under the implicit guidance of users. IEEE Trans. Circuits and Systems for Video Technology (2014)Google Scholar
  17. 17.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: CVPR (2009)Google Scholar
  18. 18.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Analysis and Machine Intelligence 12(7), 629–639 (1990)CrossRefGoogle Scholar
  19. 19.
    Wang, H., Klaser, A., Schmid, C., Liu, C.: Action recognition by dense trajectories. In: CVPR (2011)Google Scholar
  20. 20.
    Weickert, J.: Anisotropic diffusion in image processing. Teubner, Stuttgart (1998)zbMATHGoogle Scholar
  21. 21.
    Wu, S., Wong, H.: Crowd motion partitioning in a scattered motion field. IEEE Trans. Systems, Man, and Cybernetics (2012)Google Scholar
  22. 22.
    Wu, Y., Wang, Y., Jia, Y.: Adaptive diffusion flow active contours for image segmentation. Computer Vision and Image Understanding, 1421–1435 (2013)Google Scholar
  23. 23.
    Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. IEEE Trans. Pattern Analysis and Machine Intelligence 34(9), 1744–1757 (2012)CrossRefGoogle Scholar
  24. 24.
    Xu, T., Peng, P., Fang, X., Su, C., Wang, Y., Tian, Y., Zeng, W., Huang, T.: Single and multiple view detection, tracking and video analysis in crowded environments. In: AVSS (2012)Google Scholar
  25. 25.
    Zhan, B., Monekosso, D., Remagnino, P., Velastin, S., Xu, L.: Crowd analysis: a survey. Machine Vision and Applications (2008)Google Scholar
  26. 26.
    Zhou, B., Tang, X., Wang, X.: Coherent filtering: Detecting coherent motions from crowd clutters. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 857–871. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  27. 27.
    Zhou, B., Tang, X., Wang, X.: Measuring crowd collectiveness. In: CVPR (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Weiyue Wang
    • 1
  • Weiyao Lin
    • 1
  • Yuanzhe Chen
    • 1
  • Jianxin Wu
    • 2
  • Jingdong Wang
    • 3
  • Bin Sheng
    • 4
  1. 1.Dept. Electronic Engr.Shanghai Jiao Tong Univ.China
  2. 2.National Key Laboratory for Novel Software TechnologyNanjing Univ.China
  3. 3.Microsoft ResearchBeijingChina
  4. 4.Dept. Computer Science & Engr.Shanghai Jiao Tong Univ.China

Personalised recommendations