Exploiting Visual Concepts to Improve Text-Based Image Retrieval

  • Sabrina Tollari
  • Marcin Detyniecki
  • Christophe Marsala
  • Ali Fakeri-Tabrizi
  • Massih-Reza Amini
  • Patrick Gallinari
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5478)

Abstract

In this paper, we study how to automatically exploit visual concepts in a text-based image retrieval task. First, we use Forest of Fuzzy Decision Trees (FFDTs) to automatically annotate images with visual concepts. Second, using optionally WordNet, we match visual concepts and textual query. Finally, we filter the text-based image retrieval result list using the FFDTs. This study is performed in the context of two tasks of the CLEF2008 international campaign: the Visual Concept Detection Task (VCDT) (17 visual concepts) and the photographic retrieval task (ImageCLEFphoto) (39 queries and 20k images). Our best VCDT run is the 4th best of the 53 submitted runs. The ImageCLEFphoto results show that there is a clear improvement, in terms of precision at 20, when using the visual concepts explicitly appearing in the query.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arni, T., Clough, P., Sanderson, M., Grubinger, M.: Overview of the ImageCLEFphoto 2008 photographic retrieval task. In: Evaluating Systems for Multilingual and Multimodal Information Access – 9th Workshop of the Cross-Language Evaluation Forum. LNCS. Springer, Heidelberg (2009)Google Scholar
  2. 2.
    Barnard, K., Duygulu, P., de Freitas, N., Forsyth, D., Blei, D., Jordan, M.I.: Matching words and pictures. Machine Learning Research 3, 1107–1135 (2003)MATHGoogle Scholar
  3. 3.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40(2) (2008)Google Scholar
  4. 4.
    Deselaers, T., Deserno, T.M.: The visual concept detection task in ImageCLEF 2008. In: Evaluating Systems for Multilingual and Multimodal Information Access – 9th Workshop of the Cross-Language Evaluation Forum. LNCS. Springer, Heidelberg (2009)Google Scholar
  5. 5.
    Fellbaum, C.: WordNet - An Electronic Lexical Database. Bradford books (1998)Google Scholar
  6. 6.
    Marsala, C., Bouchon-Meunier, B.: Forest of fuzzy decision trees. In: International Fuzzy Systems Association World Congress, vol. 1, pp. 369–374 (1997)Google Scholar
  7. 7.
    Marsala, C., Detyniecki, M.: Trecvid 2006: Forests of fuzzy decision trees for high-level feature extraction. In: TREC Video Retrieval Evaluation Online Proceedings (2006)Google Scholar
  8. 8.
    Tollari, S., Glotin, H.: Web image retrieval on ImagEVAL: Evidences on visualness and textualness concept dependency in fusion model. In: ACM CIVR (2007)Google Scholar
  9. 9.
    Yavlinsky, A., Heesch, D., Rüger, S.M.: A large scale system for searching and browsing images from the world wide web. In: Sundaram, H., Naphade, M., Smith, J.R., Rui, Y. (eds.) CIVR 2006. LNCS, vol. 4071, pp. 537–540. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Sabrina Tollari
    • 1
  • Marcin Detyniecki
    • 1
  • Christophe Marsala
    • 1
  • Ali Fakeri-Tabrizi
    • 1
  • Massih-Reza Amini
    • 1
  • Patrick Gallinari
    • 1
  1. 1.Laboratoire d’Informatique de Paris 6 - UMR CNRS 7606Université Pierre et Marie Curie - Paris 6ParisFrance

Personalised recommendations