Multimedia Tools and Applications

, Volume 75, Issue 22, pp 15079–15100 | Cite as

Complex video event detection via pairwise fusion of trajectory and multi-label hypergraphs

  • Xiao-jun Chen
  • Yong-zhao Zhan
  • Jia Ke
  • Xiao-bo Chen


The video data is rich in motion event information. Detecting complex events and analyzing the inherent high-level semantics information have been a hot topic in video analysis and understanding. Detecting complex events in the video involves detecting multiple semantic concepts, describing features of multiple moving targets and discovering the relationship between low-level features and high-level semantic concepts. It can extract semantic concept patterns from various video features and original video data, thus bridging the semantic gap. Based on the hypergraph theory, this paper proposes to construct trajectory and multi-label hypergraphs considering the features of moving targets. The two hypergraphs are fused to detect complex events. The experimental results show that in comparison with other methods including ordinary graph based method and hypergraph based multi-label semi-supervised learning method, our method achieves better average precision and average recall when detecting complex events.


Hypergraph Trajectory Multi-label Pairwise fusion Spectral segmentation 



This research has partially been supported by National Natural Science Foundation of China under Grant No. 41374129, 60673190 and 61203244, College Natural Science Research of Jiangsu Province under Grant No. 14KJB520008, Senior Technical Personnel of Scientific Research Fund of Jiangsu University under Grant No. 13JDG126.


  1. 1.
    Assfalg J, Bertini M, Colombo C et al (2003) Semantic annotation of soccer videos: automatic highlights identification. Comput Vis Image Underst 92(2):285–305. doi: 10.1016/j.cviu.2003.06.004 CrossRefGoogle Scholar
  2. 2.
    Chen ZJ, Chen XJ, He H (2007) Moving object detection based on improved mixture gaussian models. J Image Graph 12(9):1585–1589Google Scholar
  3. 3.
    Chen DW, Liu R, Yuan ZM et al (2011) Research on multi-concept learning based on inter-concept relation. Comput Sci 38(4):244–248Google Scholar
  4. 4.
    Doulamis ND (2010) Coupled multi-object tracking and labeling for vehicle trajectory estimation and matching. Multimed Tools Appl 50:173–198CrossRefGoogle Scholar
  5. 5.
    Gao Y, Wang WB, Yong JH et al (2009) Dynamic video summarization using two-level redundancy detection. Multimed Tools Appl 42:233–250. doi: 10.1007/s11042-008-0236-x CrossRefGoogle Scholar
  6. 6.
    Hakeem A, Shah M (2007) Learning, detection and representation of multi-agent events in videos. Artif Intell 171(8–9):586–605. doi: 10.1016/j.artint.2007.04.002 CrossRefGoogle Scholar
  7. 7.
    Han YH, Shao J, Wu F et al (2010) Multiple hypergraph ranking for video concept detection. J Zhejiang Univ-Sci C (Comput Electron) 11(7):525–537. doi: 10.1631/jzus.C0910453 CrossRefGoogle Scholar
  8. 8.
    Hongeng S, Navatia R, Bremond F (2004) Video-based event recognition-activity representation and probabilistic recognition methods. Comput Vis Image Underst 96:129–162. doi: 10.1016/j.cviu.2004.02.005 CrossRefGoogle Scholar
  9. 9.
    Huang YC, Liu QS, Dimitris M (2009) Video object segmentation by hypergraph Cut[C]// Proceedings of Int’l Conf. Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Miami, pp 1738–1735Google Scholar
  10. 10.
    Huang CL, Shih HC, Chao CY (2006) Semantic analysis of soccer video using dynamic Bayesian network. IEEE Trans Multimed 8(4):749–760. doi: 10.1109/TMM.2006.876289 CrossRefGoogle Scholar
  11. 11.
    Jiang Y, She QQ, Li M et al (2008) A transductive multilabel text categorization approach. J Comput Res Dev 45(11):1817–1822Google Scholar
  12. 12.
    Kalman RE, Bucy RS (1961) New results in linear filtering and prediction theory. Trans ASME-J Basic Eng 83:95–107MathSciNetCrossRefGoogle Scholar
  13. 13.
    Ke J, Zhan YZ, Chen XJ et al (2013) The retrieval of motion event by associations of temporal frequent pattern growth. Futur Gener Comput Syst 29:442–450. doi: 10.1016/j.future.2011.06.004 CrossRefGoogle Scholar
  14. 14.
    Ke J, Zhan YZ, Chen XJ (2009) The research of detect moving target algorithm pseudo invariant line moment-based[C]// Proceedings of ICIC2009,LNCS 5754. Springer, Ulsan, pp 615–624Google Scholar
  15. 15.
    Lan Z-Z, Bao L, Yu S-I, Liu W, Hauptmann AG (2014) Multimedia classification and event detection using double fusion. Multimed Tools Appl 71:333–347CrossRefGoogle Scholar
  16. 16.
    Lu HQ, Liu J (2008) Image annotation based on graph learning. Chin J Comput 31(9):1629–1639CrossRefGoogle Scholar
  17. 17.
    Lu H, Tan YP (2001) Sports video analysis and structuring [C]// Proceedings of the IEEE 4th Workshop on Multimedia Signal Processing. Institute of Electrical and Electronics Engineers Inc, Cannes, pp 45–50Google Scholar
  18. 18.
    Maggio E, Cavallaro A (2005) Hybrid particle filter and mean shift trajectoryer with adaptive transition model[C]//Proceedings of IEEE Signal Processing Society International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers Inc, Philadelphia, pp 221–224. doi: 10.1109/ICASSP.2005.1415381 Google Scholar
  19. 19.
    Mezaris V, Scherp A, Jain R, Kankanhalli MS (2014) Real-life events in multimedia detection, representation, retrieval, and applications. Multimed Tools Appl 70:1–6CrossRefGoogle Scholar
  20. 20.
    Piciarelli C, Foresti GL (2006) On-line trajectory clustering for anomalous events detection. Pattern Recogn Lett 27(15):1835–1842. doi: 10.1016/j.patrec.2006.02.004 CrossRefGoogle Scholar
  21. 21.
    Qian X, Wang H, Liu G, Hou X (2012) HMM based soccer video event detection using enhanced mid-level semantic. Multimed Tools Appl 60:233–255CrossRefGoogle Scholar
  22. 22.
    Ruocco M, Ramampiaro H (2014) A scalable algorithm for extraction and clustering of event-related pictures. Multimed Tools Appl 70:55–88CrossRefGoogle Scholar
  23. 23.
    Suk HI, Sin BK, Lee SW (2010) Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recogn 43(9):3059–3072. doi: 10.1016/j.patcog.2010.03.016 CrossRefMATHGoogle Scholar
  24. 24.
    Talukder A, Panangadan A (2014) Extreme event detection and assimilation from multimedia sources. Multimed Tools Appl 70:237–261CrossRefGoogle Scholar
  25. 25.
    Voulodimos AS, Kosmopoulos DI, Doulamis ND, Varvarigou TA (2014) A top-down event-driven approach for concurrent activity recognition. Multimed Tools Appl 69:293–311CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Xiao-jun Chen
    • 1
    • 2
  • Yong-zhao Zhan
    • 2
  • Jia Ke
    • 2
    • 3
  • Xiao-bo Chen
    • 4
  1. 1.Affiliated Hospital of Jiangsu University, Information DepartmentJiangsu UniversityZhenjiangChina
  2. 2.School of Computer Science and Telecommunication Engineering, Engineering DepartmentJiangsu UniversityZhenjiangChina
  3. 3.School of management, Information management DepartmentJiangsu UniversityZhenjiangChina
  4. 4.Automotive Engineering Research Institute, Information DepartmentJiangsu UniversityZhenjiangChina

Personalised recommendations