Soccer Video Event Detection by Fusing Middle Level Visual Semantics of an Event Clip

  • Xueming Qian
  • Guizhong Liu
  • Huan Wang
  • Zhi Li
  • Zhe Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6298)


Highlight event detection is a fundamental step of semantic based video retrieval and personalized sports video browsing. In this paper, an enhanced hidden Markov models (EHMM) based soccer video event detection method is proposed. Firstly, each soccer video shot is classified into one of the thirteen middle level semantics. Then the sequential soccer video sequence is segmented into event clips. Finally, HMMs are utilized to model the defined four highlights (goal, shoot, foul, and placed kick) and a normal kick. Not only the transitions of the middle level semantics and but also the overall features of an event clip are fused by HMMs to determine the event type. Comparisons are made with some existing soccer video event detection approaches. Experimental results show the effectiveness of the proposed EHMM based soccer video event detection approach. The influences of hidden state number and overall feature types to the event detection performances are discussed.


Hidden Markov Models event detection soccer video 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Li, B., Errico, J., Pan, H., Sezan, M.: Bridging the semantic gap in sports video retrieval and summarization. J. Vis. Commun. Image R. 17, 393–424 (2004)CrossRefGoogle Scholar
  2. 2.
    Rabiner, L.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–285 (1989)CrossRefGoogle Scholar
  3. 3.
    Pan, H., Li, B., Sezan, M.: Automatic detection of replay segments in broadcast sports programs by detecting of logos in scene transitions. In: Proc. Int. Conf. Acoustics, Speech, and Signal Processing, May 2002, vol. 4, pp. 3385–3388 (2002)Google Scholar
  4. 4.
    Zhao, Z., Jiang, S., Huang, Q., Zhu, G.: Highlight summarization in sports video based on replay detection. In: Proc. Int. Conf. Mulmedia and Expo., Toronto, Ontario, Canada, July 2006, pp. 1613–1616 (2006)Google Scholar
  5. 5.
    Cheng, C., Hsu, C.: Fusion of audio and motion information on HMM-based highlight extraction for baseball games. IEEE Trans. Multimedia 8(3), 585–599 (2006)CrossRefGoogle Scholar
  6. 6.
    Xie, L., Chang, S., Divakaran, A., Sun, H.: Structure analysis of soccer video with hidden Markov models. In: Proc. Int. Conf. Acoustics, Speech, and Signal Processing, pp. 4096–4099 (2002)Google Scholar
  7. 7.
    Ekin, Tekalp, A.: Generic play-break event detection for summarization and hierarchical sports video analysis. In: Proc. Int. Conf. Mulmedia and Expo., vol. 1, pp. 169–172 (2003)Google Scholar
  8. 8.
    Snoek, Worring, M.: Multimedia event-based video indexing using time intervals. IEEE Trans. Multimedia 7(4), 638–647 (2005)CrossRefGoogle Scholar
  9. 9.
    Zhu, G., Xu, C., Huang, Q., Rui, Y., Jiang, S., Gao, W., Yao, H.: Event Tactic Analysis Based on Broadcast Sport Video. IEEE Trans. Multimedia 11(1), 49–67 (2009)CrossRefGoogle Scholar
  10. 10.
    Chen, S., Chen, M., Zhang, C., Shyu, M.: Exciting event detection using multi-level multimodal descriptors and data classification. In: Proc. ISM (2006)Google Scholar
  11. 11.
    Wang, T., Li, J., Diao, Q., Hu, W., Zhang, Y., Dulong, C.: Semantic event detection using conditional random fields. In: Proc. Computer Vision and Pattern Recognition Workshop, pp. 109–115 (2006)Google Scholar
  12. 12.
    Nan, N., Liu, G., Qian, X., Wang, C.: An SVM-based soccer video shot classification scheme using projection histograms. In: Huang, Y.-M.R., Xu, C., Cheng, K.-S., Yang, J.-F.K., Swamy, M.N.S., Li, S., Ding, J.-W. (eds.) PCM 2008. LNCS, vol. 5353, pp. 883–886. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Wickramaratna, K., Chen, M., Chen, S., Shyu, M.: Neural network based framework for goal event detection in soccer videos. In: Proc. Int. Symposium on Multimedia, December 2005, pp. 21–28 (2005)Google Scholar
  14. 14.
    Duan, L., Xu, M., Chua, T., Tian, Q., Xu, C.: A mid-level representation framework for semantic sports video analysis. In: Proc. ACM Multimedia, pp. 29–32 (2003)Google Scholar
  15. 15.
    Sadlier, D., O’Connor, N.: Event detection in field sports video using audio-visual features and a support vector Machine. IEEE Trans. Circuits Syst. Video Technol. 15(10), 602–615 (2005)CrossRefGoogle Scholar
  16. 16.
    Xu, P., Xie, L., Chang, S.: Algorithms and systems for segmentation and structure analysis in soccer video. In: Proc. Int. Conf. Multimedia & Expo., pp. 184–187 (2001)Google Scholar
  17. 17.
    Xu, C., Wang, J., Lu, H., Zhang, Y.: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video. IEEE Transactions on Multimedia 10(3), 421–436 (2008)CrossRefGoogle Scholar
  18. 18.
    Duan, L., Xu, M., Tian, Q., Xu, C., Jin, J.S.: A unified framework for semantic shot classification in sports video. IEEE Trans. Multimedia 7(6), 1066–1083 (2005)CrossRefGoogle Scholar
  19. 19.
    Ding, Y., Fan, G., Bryan, W.: Two-layer generative models for sport video mining. In: Proc. Int. Conf. Multimedia & Expo., pp. 1731–1734 (2007)Google Scholar
  20. 20.
    Ekin, Tekalp, A., Mehrotra, R.: Automatic soccer video analysis and summarization. IEEE Trans. Image Processing 12(7), 796–807 (2003)CrossRefGoogle Scholar
  21. 21.
    Dao, M., Babaguchi, N.: Sports event detection using temporal patterns mining and web-casting text. In: Proc. ACM AREA, pp. 33–40 (2008)Google Scholar
  22. 22.
    Zhu, X., Wu, X., Elmagarmid, A., Feng, Z., Wu, L.: Video data mining semantic indexing and event detection from the association perspective. IEEE Trans. Knowledge and Data Engineering 17(5), 665–677 (2005)CrossRefGoogle Scholar
  23. 23.
    Xiong, Z., Radhakrishnan, R., Divakaran, A., Huang, T.: Highlights extraction from sports video based on an audio-visual marker detection framework. In: Proc. Int. Conf. Multimedia & Expo., pp. 29–32 (2005)Google Scholar
  24. 24.
    Xu, C., Zhang, Y., Zhu, G., Rui, Y., Lu, H., Huang, Q.: Using Webcast Text for Semantic Event Detection in Broadcast Sports Video. IEEE Trans. Multimedia 10(7), 1342–1345 (2008)CrossRefGoogle Scholar
  25. 25.
    Wang, Y., Liu, Z., Huang, J.: Multimedia content analysis using both audio and video clues. IEEE Signal Processing Magazine (2000)Google Scholar
  26. 26.
    Huang, C., Shih, H., Chao, C.: Semantic analysis of soccer video using dynamic Bayesian network. IEEE Trans. Multimedia 8(4), 749–760 (2006)CrossRefGoogle Scholar
  27. 27.
    Zhang, D., Chang, S.: Event detection in baseball video using superimposed caption recognition. In: Proc. ACM Multimedia, Juan-les-Pins, France, November 1, pp. 315–318 (2002)Google Scholar
  28. 28.
    Su, Y., Sun, M., Hsu, V.: Global motion estimation from coarsely sampled motion vector field and the applications. IEEE Trans. Circuits Syst. Video Technol. 15(2), 232–242 (2005)CrossRefGoogle Scholar
  29. 29.
    Lyu, M., Song, J., Cai, M.: A comprehensive method for text detection, localization, and extraction. IEEE Trans. Circuits and Systems for Video Technology 15(2), 243–255 (2005)CrossRefGoogle Scholar
  30. 30.
    Wang, J., Xu, C., Chng, E., Tian, Q.: Sports highlight detection from keyword sequences using HMM. In: ICME 2004 (2004)Google Scholar
  31. 31.
    Lienhart, R., Wernicke, A.: Localizing and segmenting text in images and videos. IEEE Trans. Circuits Syst. Video Technol. 12(4), 256–267 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Xueming Qian
    • 1
  • Guizhong Liu
    • 1
  • Huan Wang
    • 1
  • Zhi Li
    • 1
  • Zhe Wang
    • 1
  1. 1.Department of Information and Communication EngineeringXi’an Jiaotong UniversityXi’anChina

Personalised recommendations