Automatic Annotation of Sport Video Content

  • Marco Bertini
  • Alberto Del Bimbo
  • Walter Nunziati
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3773)


Automatic semantic annotation of video streams allows to extract significant clips for archiving and retrieval of video content. In this paper, we present a system that performs automatic annotation of soccer videos, detecting principal highlights, and recognizing identity of players. Highlight detection is carried out by means of finite state machines that encode domain knowledge, while player identification is based on face detection, and on the analysis of contextual information such as jersey’s numbers and superimposed text captions. Results obtained on actual soccer videos shows overall highlight detection rates of about 90%. Lower, but still promising, accuracy is achieved on the very difficult player identification task.


Camera Motion Automatic Annotation Sift Descriptor Sport Video Text Detector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Assfalg, J., Bertini, M., Colombo, C., Del Bimbo, A.,, A., Nunziati, W.: Semantic annotation of soccer videos: automatic highlights identification. Computer Vision and Image Understanding 92 (November–December 2003)Google Scholar
  2. 2.
    Baldi, G., Colombo, C., Del Bimbo, A.: A compact and retrieval-oriented video representation using mosaics. In: Proc. 3rd ICVS, Amsterdam (1999)Google Scholar
  3. 3.
    Berg, T.L., Berg, A.C., Edwards, J., Maire, M., White, R., Teh, Y.-W., Learned-Miller, E., Forsyth, D.A.: Names and Faces in the News. In: Proc. of CVPR (2004)Google Scholar
  4. 4.
    Bertini, M., Del Bimbo, A., Pala, P.: Content-based indexing and retrieval of TV news. Pattern Recognition Letters 22(5) (2001)Google Scholar
  5. 5.
    Chen, D., Odobez, J.-M., Bourlard, H.: Text detection and recognition in images and video frames. Pattern Recognition 37 (March 2004)Google Scholar
  6. 6.
    Ekin, A., Tekalp, A.M., Mehrotra, R.: Automatic soccer video analysis and summarization. IEEE Transcations on Image Processing (July 2003)Google Scholar
  7. 7.
    Everingham, M., Zisserman, A.: Automated Person Identification in Video. In: Enser, P.G.B., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A., Smeulders, A.W.M. (eds.) CIVR 2004, vol. 3115, pp. 289–298. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Proc. of Eurocolt 1995. Springer, Heidelberg (1995)Google Scholar
  9. 9.
    GOCR: Open Source Character Recognition,
  10. 10.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  11. 11.
    Intille, S.S., Bobick, A.F.: Recognizing planned, multi-person action. Computer Vision and Image Understanding (1077-3142)81(3), 414–445 (2001)zbMATHCrossRefGoogle Scholar
  12. 12.
    Lowe, D.: Distinctive image features from scale–invariant keypoints. International Journal of Computer Vision 60(2) (2004)Google Scholar
  13. 13.
    Mottaleb, M., Ravitz, G.: Detection of Plays and Breaks in Football Games Using Audiovisual Features and HMM. In: Proc. of Ninth Int’l Conf. on Distributed Multimedia Systems, pp. 154–160 (2003)Google Scholar
  14. 14.
    Satoh, S., Nakamura, Y., Kanade, T.: Name-It: Naming and Detecting Faces in News Videos. IEEE MultiMedia 6(1) (January-March 1999)Google Scholar
  15. 15.
    Sivic, J., Everingham, M., Zissermann, A.: ‘Person spotting: video shot retrieval for face sets. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005, vol. 3568, pp. 226–236. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  16. 16.
    Snoek, C.G.M., Worring, M.: Multimodal video indexing: a review of the state-of-the-art. Multimedia, Tools and Applications 25 (January 2005)Google Scholar
  17. 17.
    Sudhir, G., Lee, J.C.M., Jain, A.K.: Automatic Classification of Tennis Video for High-level Content-based Retrieval. In: Proc. of CAIVD 1998, pp. 81–90 (1998)Google Scholar
  18. 18.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. CVPR, pp. 511–518 (2001)Google Scholar
  19. 19.
    Yu, X., Xu, C., Leong, H.W., Tian, Q., Tang, Q., Wan, K.W.: Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In: Proc. of ACM Multimedia, November 02-08 (2003)Google Scholar
  20. 20.
    Yu, X., Farin, D.: Current and Emerging Topics in Sports Video processing. In: Proc. of IEEE ICME (2005)Google Scholar
  21. 21.
    Zhou, W., Vellaikal, A., Kuo, C.C.J.: Rule-based video classification system for basketball video indexing. In: Proc.  of ACM Multimedia 2000 workshop (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Marco Bertini
    • 1
  • Alberto Del Bimbo
    • 1
  • Walter Nunziati
    • 1
  1. 1.Dipartimento di Sistemi e InformaticaUniversitá degli Studi di Firenze 

Personalised recommendations