Pattern Analysis and Applications

, Volume 7, Issue 4, pp 411–421 | Cite as

Highlights modeling and detection in sports videos

  • M. BertiniEmail author
  • A. Del Bimbo
  • W. Nunziati
Theoretical Advances


Automatic annotation of semantic events allows effective retrieval of video content. In this work, we present solutions for highlights detection in sports videos. This application is particularly interesting for broadcasters, since they extensively use manual annotation to select interesting highlights that are edited to create new programmes. The proposed approach exploits the typical structure of a wide class of sports videos, namely, those related to sports which are played in delimited venues with playfields of well known geometry, like soccer, basketball, swimming, track and field disciplines, and so on. For this class of sports, a modeling scheme based on a limited set of visual cues and on finite state machines (FSM) that encode the temporal evolution of highlights is presented. Algorithms for model checking and for visual cues estimation are discussed, as well as applications of the representation to different sport domains.


Semantic annotation Highlights detection Sports videos Visual cues 


  1. 1.
    Assfalg J, Bertini M, Del Bimbo A, Nunziati W, Pala P (2002) Soccer highlights detection and recognition using HMMs. In: Proceedings of the IEEE international conference on multimedia and expo (ICME 2002), Lausanne, Switzerland, August 2002Google Scholar
  2. 2.
    Baldi G, Colombo C, Del Bimbo A (1999) A compact and retrieval-oriented video representation using mosaics. In: Proceedings of the 3rd international conference on visual information systems (VISUAL’99), Amsterdam, The Netherlands, June 1999, pp 171–178Google Scholar
  3. 3.
    Bengio Y (1998) Markovian models for sequential data. Neural Comp Surv 2:129–162Google Scholar
  4. 4.
    Bertini M, Del Bimbo A, Pala P (2001) Content-based indexing and retrieval of TV news. Pattern Recogn Lett 22(5):503–516CrossRefzbMATHGoogle Scholar
  5. 5.
    Brand M, Oliver N, Pentland A (1997) Coupled hidden Markov models for complex action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR’97), San Juan, Puerto Rico, June 1997Google Scholar
  6. 6.
    Hampapur A (1999) Semantic video indexing: approach and issues. SIGMOD Record 28(1):32–39Google Scholar
  7. 7.
    Hartley R, Zisserman A (2000) Multiple view geometry in computer vision. Cambridge University Press, CambridgeGoogle Scholar
  8. 8.
    Ekin A, Murat Tekalp A, Mehrotra R (2003) Automatic soccer video analysis and summarization. IEEE Trans Image Process 12(7):796–807Google Scholar
  9. 9.
    Intille SS, Bobick AF (2001) Recognizing planned, multi-person action. Comput Vis Image Und 81(3):414–445CrossRefzbMATHGoogle Scholar
  10. 10.
    Jordan MI (1999) Learning in graphical models. MIT Press, Cambridge, MassachusettsGoogle Scholar
  11. 11.
    Kittler JV, Messer K, Christmas W, Levienaise-Obadia B, Koubaroulis D (2001) Generation of semantic cues for sports video annotation. In: Proceedings of the IEEE international conference on image processing (ICIP 2001), Thessaloniki, Greece, October 2001, vol 3, pp 26–29Google Scholar
  12. 12.
    Leonardi R, Migliorati P (2002) Semantic indexing of multimedia documents. IEEE Multimedia 9(2):44–51CrossRefGoogle Scholar
  13. 13.
    Mottaleb M, Ravitz G (2003) Detection of plays and breaks in football games using audiovisual features and HMM. In: Proceedings of the 9th international conference on distributed multimedia systems (DMS 2003), Miami, Florida, September 2003, pp 154–160Google Scholar
  14. 14.
    Pavlovic V, Sharma R, Huang T (1997) Visual interpretation of hand gestures for human–computer interaction: a review. IEEE Trans Pattern Anal Mach Intell 19(7):677–695Google Scholar
  15. 15.
    Rabiner LR (1989) A tutorial on HMM and selected applications in speech recognition. Proc IEEE 77(2):257–286CrossRefGoogle Scholar
  16. 16.
    Russell S, Norvig P (1995) Artificial intelligence: a modern approach. Prentice Hall, Englewood Cliffs, New JerseyGoogle Scholar
  17. 17.
    Sudhir G, Lee JCM, Jain AK (1998) Automatic classification of tennis video for high-level content-based retrieval. In: Proceedings of the international workshop on content-based access of image and video databases (CAIVD’98), Bombay, India, January 1998, pp 81–90Google Scholar
  18. 18.
    Tovinkere V, Qian RJ (2001) Detecting semantic events in soccer games: towards a complete solution. In: Proceedings of the international conference on multimedia and expo (ICME 2001), Tokyo, Japan, August 2001, pp 1040–1043Google Scholar
  19. 19.
    Xie L, Xu P, Chang S-F, Divakaran A, Sun H (2002) Structure analysis of soccer video with domain knowledge and hidden Markov models. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing (ICASSP 2002), Orlando, Florida, May 2002, pp 4096–4099Google Scholar
  20. 20.
    Xiong Z, Radhakrishnan R, Divakaran A, Huang TS (2003) Audio events detection based highlights extraction from baseball, golf and soccer games in a unified framework. In: Proceedings of the IEEE international conference on multimedia and expo (ICME 2003), Baltimore, Maryland, July 2003, pp 401–404Google Scholar
  21. 21.
    Yu X, Xu C, Leong HW, Tian Q, Tang Q, Wan KW (2003) Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In: Proceedings of the 11th ACM international conference on multimedia (MM 2003), Berkeley, California, November 2003Google Scholar
  22. 22.
    Zhou W, Vellaikal A, Kuo CCJ (2000) Rule-based video classification system for basketball video indexing. In: Proceedings of the 8th ACM international conference on multimedia (MM 2000), Los Angeles, California, October/November 2000pp 213–216Google Scholar

Copyright information

© Springer-Verlag London Limited 2005

Authors and Affiliations

  1. 1.Dipartimento Sistemi e InformaticaUniversità di FirenzeFirenzeItaly

Personalised recommendations