Unsupervised Group Activity Detection by Hierarchical Dirichlet Processes

  • Ali Al-RaziqiEmail author
  • Joachim Denzler
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10317)


Detecting groups plays an important role for group activity detection. In this paper, we propose an automatic group activity detection by segmenting the video sequences automatically into dynamic clips. As the first step, groups are detected by adopting a bottom-up hierarchical clustering, where the number of groups is not provided beforehand. Then, groups are tracked over time to generate consistent trajectories. Furthermore, the Granger causality is used to compute the mutual effect between objects based on motion and appearances features. Finally, the Hierarchical Dirichlet Process is used to cluster the groups. Our approach not only detects the activity among the objects of a particular group (intra-group) but also extracts the activities among multiple groups (inter-group). The experiments on public datasets demonstrate the effectiveness of the proposed method. Although our approach is completely unsupervised, we achieved results with a clustering accuracy of up to 79.35\(\%\) and up to 81.94\(\%\) on the Behave and the NUS-HGA datasets.


Granger Causality Dynamic Time Warping Dirichlet Process Deep Neural Network Sift Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors are thankful to Mahesh Krishna and Manuel Amthor for useful discussions and suggestions.


  1. 1.
    Aggarwal, J.K., Xia, L.: Human activity recognition from 3D data: a review. Pattern Recogn. Lett. 48, 70–80 (2014)CrossRefGoogle Scholar
  2. 2.
    Al-Raziqi, A., Denzler, J.: Unsupervised framework for interactions modeling between multiple objects. In: VISAPP, vol. 4, pp. 509–516 (2016)Google Scholar
  3. 3.
    Blunsden, S., Fisher, R.B.: The BEHAVE video dataset: ground truthed video for multi-person behavior classification. BMVA 4, 1–12 (2010)Google Scholar
  4. 4.
    Cheng, Z., Qin, L., Huang, Q., Yan, S., Tian, Q.: Recognizing human group action by layered model with multiple cues. Neurocomputing 136, 124–135 (2014)CrossRefGoogle Scholar
  5. 5.
    Dehghan, A., Assari, S., Shah, M.: GMMCP tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: CVPR, pp. 4091–4099 (2015)Google Scholar
  6. 6.
    Deng, Z., Vahdat, A., Hu, H., Mori, G.: Structure inference machines: recurrent neural networks for analyzing relations in group activity recognition. In: CVPR (2016)Google Scholar
  7. 7.
    Deng, Z., Zhai, M., Chen, Y., Muralidharan, L.S., Roshtkhari, M., Mori, G.: Deep structured models for group activity recognition. arXiv preprint arXiv:1506.04191 (2015)
  8. 8.
    Dong, Z., Kong, Y., Liu, C., Li, H., Jia, Y.: Recognizing human interaction by multiple features. In: ACPR, pp. 77–81 (2011)Google Scholar
  9. 9.
    Granger, C.W.J.: Investigating causal relations by econometric models and cross-spectral methods. Econom.: J. Econom. Soc. 37, 424–438 (1969)Google Scholar
  10. 10.
    Ibrahim, M.S., Muralidharan, S., Deng, Z., Vahdat, A., Mori, G.: A hierarchical deep temporal model for group activity recognition. In: CVPR, June 2016Google Scholar
  11. 11.
    Li, B., Ayazoglu, M., Mao, T., Camps, O.I., Sznaier, M.: Activity recognition using dynamic subspace angles. In: CVPR, pp. 3193–3200 (2011)Google Scholar
  12. 12.
    Münch, D., Michaelsen, E., Arens, M.: Supporting fuzzy metric temporal logic based situation recognition by mean shift clustering. In: Glimm, B., Krüger, A. (eds.) KI 2012. LNCS, vol. 7526, pp. 233–236. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33347-7_21 CrossRefGoogle Scholar
  13. 13.
    Park, U., Park, J.-S., Cho, N.-G., Kim, Y.-J., Lee, S.-W.: Group activity recognition with group interaction zone based on relative distance between human objects. Pattern Recogn. Artif. Intell. 29(05), 1555007 (2015)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Cho, N.-G., Kim, Y.-J., Park, U., Park, J.-S., Lee, S.-W.: Recognizing human group activities with localized causalities. In: CVPR, pp. 1470–1477 (2009)Google Scholar
  15. 15.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. arXiv preprint arXiv:1506.02640 (2015)
  16. 16.
    Stephens, K., Bors, A.G.: Group activity recognition on outdoor scenes. In: Advanced Video and Signal Based Surveillance (AVSS), pp. 59–65 (2016)Google Scholar
  17. 17.
    Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical Dirichlet processes. JASA 101(476), 1566–1581 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Yin, Y., Yang, G., Xu, J., Man, H.: Small group human activity recognition. In: Image Processing (ICIP), pp. 2709–2712. IEEE (2012)Google Scholar
  19. 19.
    Zhang, C., Yang, X., Lin, W., Zhu, J.: Recognizing human group behaviors with multi-group causalities. In: WI-IAT, vol. 3, pp. 44–48 (2012)Google Scholar
  20. 20.
    Zhou, Y., Ni, B., Yan, S., Huang, T.S.: Recognizing pair-activities by causality analysis. ACM - TIST 2(1), 5 (2011)Google Scholar
  21. 21.
    Zhu, G., Yan, S., Han, T.X., Xu, C.: Generative group activity analysis with quaternion descriptor. In: Lee, K.-T., Tsai, W.-H., Liao, H.-Y.M., Chen, T., Hsieh, J.-W., Tseng, C.-C. (eds.) MMM 2011. LNCS, vol. 6524, pp. 1–11. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-17829-0_1 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Computer Vision GroupFriedrich-Schiller-Universität JenaJenaGermany

Personalised recommendations