Improving Automatic Video Retrieval with Semantic Concept Detection

  • Markus Koskela
  • Mats Sjöberg
  • Jorma Laaksonen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5575)


We study the usefulness of intermediate semantic concepts in bridging the semantic gap in automatic video retrieval. The results of a series of large-scale retrieval experiments, which combine text-based search, content-based retrieval, and concept-based retrieval, is presented. The experiments use the common video data and sets of queries from three successive TRECVID evaluations. By including concept detectors, we observe a consistent improvement on the search performance, despite the fact that the performance of the individual detectors is still often quite modest.


Automatic Speech Recognition Semantic Concept Mean Average Precision Video Retrieval Concept Ontology 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Hauptmann, A.G., Christel, M.G., Yan, R.: Video retrieval based on semantic concepts. Proceedings of the IEEE 96(4), 602–622 (2008)CrossRefGoogle Scholar
  2. 2.
    Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and TRECVid. In: MIR 2006: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pp. 321–330. ACM Press, New York (2006)Google Scholar
  3. 3.
    Naphade, M., Smith, J.R., Tešić, J., Chang, S.F., Hsu, W., Kennedy, L., Hauptmann, A., Curtis, J.: Large-scale concept ontology for multimedia. IEEE MultiMedia 13(3), 86–91 (2006)CrossRefGoogle Scholar
  4. 4.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  5. 5.
    Koskela, M., Laaksonen, J.: Semantic concept detection from news videos with self-organizing maps. In: Proceedings of 3rd IFIP Conference on Artificial Intelligence Applications and Innovations, Athens, Greece, June 2006, pp. 591–599 (2006)Google Scholar
  6. 6.
    Snoek, C.G.M., Worring, M.: Are concept detector lexicons effective for video search? In: Proceedings of the IEEE International Conference on Multimedia & Expo. (ICME 2007), Beijing, China, July 2007, pp. 1966–1969 (2007)Google Scholar
  7. 7.
    Natsev, A.P., Haubold, A., Tešić, J., Xie, L., Yan, R.: Semantic concept-based query expansion and re-ranking for multimedia retrieval. In: Proceedings of ACM Multimedia (ACM MM 2007), Augsburg, Germany, September 2007, pp. 991–1000 (2007)Google Scholar
  8. 8.
    Fellbaum, C. (ed.): WordNet: An Electronic Lexical DatabaseGoogle Scholar
  9. 9.
    Kennedy, L.S., Natsev, A.P., Chang, S.F.: Automatic discovery of query-class-dependent models for multimodal search. In: Proceedings of ACM Multimedia (ACM MM 2005), Singapore, November 2005, pp. 882–891 (2005)Google Scholar
  10. 10.
    de Rooij, O., Snoek, C.G.M., Worring, M.: Balancing thread based navigation for targeted video search. In: Proceedings of the International Conference on Image and Video Retrieval (CIVR 2008), Niagara Falls, Canada, pp. 485–494 (2008)Google Scholar
  11. 11.
    Yilmaz, E., Aslam, J.A.: Estimating average precision with incomplete and imperfect judgments. In: Proceedings of 15th International Conference on Information and Knowledge Management (CIKM 2006), Arlington, VA, USA (November 2006)Google Scholar
  12. 12.
    Laaksonen, J., Koskela, M., Oja, E.: PicSOM—Self-organizing image retrieval with MPEG-7 content descriptions. IEEE Transactions on Neural Networks, Special Issue on Intelligent Multimedia Processing 13(4), 841–853 (2002)CrossRefzbMATHGoogle Scholar
  13. 13.
    Sjöberg, M., Muurinen, H., Laaksonen, J., Koskela, M.: PicSOM experiments in TRECVID 2006. In: Proceedings of the TRECVID 2006 Workshop, Gaithersburg, MD, USA (November 2006)Google Scholar
  14. 14.
    Koskela, M., Sjöberg, M., Viitaniemi, V., Laaksonen, J., Prentis, P.: PicSOM experiments in TRECVID 2007. In: Proceedings of the TRECVID 2007 Workshop, Gaithersburg, MD, USA (November 2007)Google Scholar
  15. 15.
    Koskela, M., Sjöberg, M., Viitaniemi, V., Laaksonen, J.: PicSOM experiments in TRECVID 2008. In: Proceedings of the TRECVID 2008 Workshop, Gaithersburg, MD, USA (November 2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Markus Koskela
    • 1
  • Mats Sjöberg
    • 1
  • Jorma Laaksonen
    • 1
  1. 1.Department of Information and Computer ScienceHelsinki University of Technology (TKK)EspooFinland

Personalised recommendations