A Discriminative Approach for the Retrieval of Images from Text Queries

  • David Grangier
  • Florent Monay
  • Samy Bengio
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4212)


This work proposes a new approach to the retrieval of images from text queries. Contrasting with previous work, this method relies on a discriminative model: the parameters are selected in order to minimize a loss related to the ranking performance of the model, i.e. its ability to rank the relevant pictures above the non-relevant ones when given a text query. In order to minimize this loss, we introduce an adaptation of the recently proposed Passive-Aggressive algorithm. The generalization performance of this approach is then compared with alternative models over the Corel dataset. These experiments show that our method outperforms the current state-of-the-art approaches, e.g. the average precision over Corel test data is 21.6% for our model versus 16.7% for the best alternative, Probabilistic Latent Semantic Analysis.


  1. 1.
    Crammer, K., Dekel, O., Shalev-Shwartz, S., Singer, Y.: Online passive-aggressive algorithms. In: Neural Information Processing Systems (NIPS) (2003)Google Scholar
  2. 2.
    Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. Journal of Machine Learning Research (JMLR) 3, 1107–1135 (2003)zbMATHCrossRefGoogle Scholar
  3. 3.
    Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: ACM Special Interest Group on Information Retrieval (SIGIR) (2003)Google Scholar
  4. 4.
    Monay, F., Gatica-Perez, D.: PLSA-based image auto-annotation: constraining the latent space. In: ACM Multimedia, pp. 348–351 (2004)Google Scholar
  5. 5.
    Pan, J.Y., Yang, H.J., Duygulu, P., Faloutsos, C.: Automatic image captioning. In: International Conference on Multimedia and Expo (ICME), pp. 1987–1990 (2004)Google Scholar
  6. 6.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley, Harlow (1999)Google Scholar
  7. 7.
    Joachims, T.: Optimizing search engines using clickthrough data. In: International Conference on Knowledge Discovery and Data Mining (KDD) (2002)Google Scholar
  8. 8.
    Duygulu, P., Barnard, K., de Freitas, N., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 97–112. Springer, Heidelberg (2002)Google Scholar
  9. 9.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision (IJCV) 60(2), 91–110 (2004)CrossRefGoogle Scholar
  10. 10.
    Tieu, K., Viola, P.: Boosting image retrieval. International Journal of Computer Vision (IJCV) 56(1), 17–36 (2004)CrossRefGoogle Scholar
  11. 11.
    Wu, H., LuE, H., Ma, S.: A practical SVM-based algorithm for ordinal regression in image retrieval. In: ACM Multimedia (2003)Google Scholar
  12. 12.
    Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Machine Learning 42, 177–196 (2001)zbMATHCrossRefGoogle Scholar
  13. 13.
    Rice, J.: Rice, Mathematical Statistics and Data Analysis. Duxbury Press (1995)Google Scholar
  14. 14.
    Wallraven, C., Caputo, B.: Recognition with local features: the kernel recipe. In: International Conference on Computer Vision (ICCV) (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • David Grangier
    • 1
    • 2
  • Florent Monay
    • 1
    • 2
  • Samy Bengio
    • 1
  1. 1.IDIAP Research InstituteMartignySwitzerland
  2. 2.Ecole Polytechnique Fédérale de Lausanne (EPFL)Switzerland

Personalised recommendations