Enhanced Sports Image Annotation and Retrieval Based Upon Semantic Analysis of Multimodal Cues

  • Kraisak Kesorn
  • Stefan Poslad
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5414)


This paper presents a framework for semi-automatic annotation and semantic image retrieval, applied to the sports domain, based upon semantic analysis of both image text captions and visual features of the image. Unstructured text captions of images are analysed in order to extract the concepts and restructure them into a semantic model. SVM classification of the multi-dominant colours and edge ratio information of the images are used to classify the sport genre. The novelty of the proposed semantic framework is that it can find both the indirectly relevant concepts (concepts not directly referred to) in the visual information and can represent the semantic of images at a higher level by combining image captions and visual feature information. In addition, integrating LSI into the semantic framework enables the proposed system to tolerate ontology imperfections. Experimental results show that the use of the semantic approach significantly enhances image retrieval. Semantic visual information classification and retrieval based upon multimodal cues.


Ontology Semantic Model Image Classification Knowledge base Image Retrieval 


  1. 1.
    Dasiopoulou, S., Spyrou, E., Avrithis, Y., Kompatsiaris, Y., Strintzis, M.G.: Color Image Processing: Methods and Applications. CRC Press / Taylor & Francis (October 2006)Google Scholar
  2. 2.
    Smeulder, A.W.M., Worring, M., Anntini, S., Gupta, A., Jain, R.: Content-based Image Retrieval at the End of the Early Years. IEEE Trans. Pattern Analysis and Machine Intelligence 22, 1349–1380 (2000)CrossRefGoogle Scholar
  3. 3.
    Assfalg, J., Bertini, M., Colombo, C., Bimbo, A.D.: Semantic Annotation of Sports Video. IEEE Trans. Multimedia 9, 52–60 (2002)CrossRefzbMATHGoogle Scholar
  4. 4.
    Messer, K., Christmas, W., Kittler, J.: Automatic Sports Classification. In: 16th International Conference on Pattern Recognition, vol. 2, pp. 1005–1008 (August 2002)Google Scholar
  5. 5.
    Wang, L., Zeng, B., Lin, S., Xu, G., Shun, H.-Y.: Automatic Extraction of Semantic Colours in Sport Video. In: The International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 617–620 (May 2004)Google Scholar
  6. 6.
    Jang, S., Song, M., Cho, H.: Semantic Classification of Sports News Video Using Colour and Motion Features. In: The 2006 International Conference on Hybrid Information Technology, vol. 2, pp. 745–750 (November 2006)Google Scholar
  7. 7.
    Yuan, Y., Wan, C.: The Application of Edge Feature in Automatic Sport Genre Classification. In: The International Conference on Cybernetic and Intelligent System, vol. 2, pp. 1133–1136 (December 2004)Google Scholar
  8. 8.
    Frankel, C., Swain, M.J., Athitsos, V.: WebSeer: An Image Search Engine for the World Wide Web, Technical Report, The University of Chicago, Illinois (August 1996)Google Scholar
  9. 9.
    Hu, J., Bagga, A.: Categorizing Images in Web documents. IEEE Multimedia 11, 22–30 (2004)Google Scholar
  10. 10.
    Song, X., Ching-Yung, L., Ming-Ting, S.: Autonomous Visual Model Building based on Image Crawling through Internet Search Engines. In: 6th ACM SIGMM international workshop on Multimedia information retrieval (MIR 2004), pp. 315–322 (October 2004)Google Scholar
  11. 11.
    Wang, X., Ma, W., Li, X.: Data-Driven Approach for Bridging the Cognitive Gap in Image Retrieval. In: IEEE International Conference on Multimedia and Expo. (ICME 2004), vol. 3, pp. 2231–2234 (June 2004)Google Scholar
  12. 12.
    Hacid, H., Zighed, A.D.: Semantic-Based Visual Information Retrieval, ch. X. IRM Press, London (2007)Google Scholar
  13. 13.
    Schreiber, A., Dubbeldam, B., Wielemaker, J., Wielinga, B.J.: Ontology-based photo annotation. IEEE Intelligent Systems 16, 66–74 (2001)CrossRefGoogle Scholar
  14. 14.
    Zhu, J., ESpotter- Adaptive Named Entity Recognition for Web Browsing,
  15. 15.
    Chisholm, E., Kolda, T.G.: New Term Weighting Formulas for The Space Method in Information Retrieval. Computer Science and Mathematics Division (March 1999)Google Scholar
  16. 16.
    Histogram-Based Color Image Retrieval,
  17. 17.
    LIBSVM: A library for support vector machines,
  18. 18.
  19. 19.
    Jena Semantic Web Framework,
  20. 20.
  21. 21.
    Olympic organization,
  22. 22.
    Apache Lucene,
  23. 23.
  24. 24.

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Kraisak Kesorn
    • 1
  • Stefan Poslad
    • 1
  1. 1.School of Electronic Engineering and Computer ScienceQueen Mary University of LondonLondonUnited Kingdom

Personalised recommendations