Multimodal Image Retrieval over a Large Database

  • Débora Myoupo
  • Adrian Popescu
  • Hervé Le Borgne
  • Pierre-Alain Moëllic
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6242)


We introduce a new multimodal retrieval technique which combines query reformulation and visual image reranking in order to deal with results sparsity and imprecision, respectively. Textual queries are reformulated using Wikipedia knowledge and results are then reordered using a k-NN based reranking method. We compare textual and multimodal retrieval and show that introducing visual reranking results in a significant improvement of performance.


Query Expansion Visual Model Textual Retrieval Late Fusion Initial Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arni, T., Clough, P., Sanderson, M., Grubinger, M.: Overview of the ImageCLEFphoto 2008 photographic retrieval task. In: Peters, C., Deselaers, T., Ferro, N., Gonzalo, J., Jones, G.J.F., Kurimo, M., Mandl, T., Peñas, A., Petras, V. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 500–511. Springer, Heidelberg (2009)Google Scholar
  2. 2.
    Cheng, Y.-C., Chen, S.-Y.: Image classification using color, texture and regions. In: Image Vision Computing (2003)Google Scholar
  3. 3.
    Deselaers, T., Gass, T., Dreuw, P., Ney, H.: Jointly Optimising Relevance and Diversity in Image Retrieval. In: Proc. of CIVR (2009)Google Scholar
  4. 4.
    Gabrilovich, E., Markovich, S.: Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis. In: Proc. of IJCAI (2007)Google Scholar
  5. 5.
    Jing, Y., Baluja, S.: VisualRank: Applying PageRank to Large-Scale Image Search. PAMI (2008)Google Scholar
  6. 6.
    Milne, D., Witten, I.H.: An effective, low-cost measure of semantic relatedness obtained from Wikipedia links. In: Proc. of WIKIAI (2008)Google Scholar
  7. 7.
    Moëllic, P.-A., Haugeard, J.-E., Pitel, G.: Image clustering based on a shared nearest neighbors approach for tagged collections. In: Proc. of CIVR (2008)Google Scholar
  8. 8.
    Popescu, A., Le Borgne, H., Moëllic, P.-A.: Conceptual Image retrieval over a Large Scale Database. In: Peters, C., Deselaers, T., Ferro, N., Gonzalo, J., Jones, G.J.F., Kurimo, M., Mandl, T., Peñas, A., Petras, V. (eds.) CLEF 2008. LNCS, vol. 5706. Springer, Heidelberg (2009)Google Scholar
  9. 9.
    Strube, M., Ponzetto, S.P.: WikiRelate! Computing Semantic Relatedness Using Wikipedia. In: Proc. of AAAI (2006)Google Scholar
  10. 10.
    Tsikrika, T., Kludas, J.: Overview of the wikipediaMM task at ImageCLEF 2009. In: CLEF 2009 Workshop, Part II. LNCS, vol. 6242, pp. 60–71. Springer, Heidelberg (2010)Google Scholar
  11. 11.
    Van Leuken, R.H., Garcia, L., Olivares, X., van Zwol, R.: Visual Diversification of Image Search Results. In: Proc. of WWW (2009)Google Scholar
  12. 12.
    Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classifcation of texture and object categories: An in-depth study. Technical Report RR-5737, INRIA (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Débora Myoupo
    • 1
  • Adrian Popescu
    • 2
  • Hervé Le Borgne
    • 1
  • Pierre-Alain Moëllic
    • 1
  1. 1.CEA, LISTLaboratoire d’ingénierie de la connaissance multimédia et multilingueFontenay-aux-RosesFrance
  2. 2.Computer Science Dept.Télécom Bretagne 

Personalised recommendations