A Latent Variable Ranking Model for Content-Based Retrieval

  • Ariadna Quattoni
  • Xavier Carreras
  • Antonio Torralba
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7224)


Since their introduction, ranking SVM models [11] have become a powerful tool for training content-based retrieval systems. All we need for training a model are retrieval examples in the form of triplet constraints, i.e. examples specifying that relative to some query, a database item a should be ranked higher than database item b. These types of constraints could be obtained from feedback of users of the retrieval system. Most previous ranking models learn either a global combination of elementary similarity functions or a combination defined with respect to a single database item. Instead, we propose a “coarse to fine” ranking model where given a query we first compute a distribution over “coarse” classes and then use the linear combination that has been optimized for queries of that class. These coarse classes are hidden and need to be induced by the training algorithm. We propose a latent variable ranking model that induces both the latent classes and the weights of the linear combination for each class from ranking triplets. Our experiments over two large image datasets and a text retrieval dataset show the advantages of our model over learning a global combination as well as a combination for each test point (i.e. transductive setting). Furthermore, compared to the transductive approach our model has a clear computational advantages since it does not need to be retrained for each test query.


Latent Classis Query Image Ranking Model Local Learning Hinge Loss 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bar-Hillel, A., Hertz, T., Shental, N., Weinshall, D.: Learning a mahalanobis metric from equivalence constraints. J. Mach. Learn. Res. 6, 937–965 (2005)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Basu, S., Banerjee, S., Mooney, R.: Semi-supervised clustering by seeding. In: ICML (2002)Google Scholar
  3. 3.
    Bilenko, M., Basu, S., Mooney, R.J.: Integrating constraints and metric learning in semi-supervised clustering. In: Proceedings of the Twenty-First International Conference on Machine Learning, ICML 2004, p. 11. ACM, New York (2004)CrossRefGoogle Scholar
  4. 4.
    Bottou, E., Vapnik, V.: Local learning algorithms. Neural Computation 4, 888–900 (1992)CrossRefGoogle Scholar
  5. 5.
    Chechik, G., Sharma, V., Shalit, U., Bengio, S.: Large scale online learning of image similarity through ranking. Journal of Machine Learning Research 11, 1109–1135 (2010)MathSciNetGoogle Scholar
  6. 6.
    Choi, M.J., Lim, J.J., Torralba, A., Willsky, A.S.: Exploiting hierarchical context on a large database of object categories. In: Proceedings of CVPR (2010)Google Scholar
  7. 7.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)Google Scholar
  8. 8.
    Frome, A., Sha, F., Singer, Y., Malik, J.: Learning globally-consistent local distance functions for shape-based image retrieval and classification. In: ICCV (2007)Google Scholar
  9. 9.
    Frome, A., Singer, Y., Malik, J.: Image Retrieval and Classification Using Local Distance Functions. In: Schölkopf, B., Platt, J.C., Hoffman, T. (eds.) NIPS, pp. 417–424. MIT Press (2006)Google Scholar
  10. 10.
    Hertz, T., Bar-hillel, A., Weinshall, D.: Learning distance functions for image retrieval. In: Proceedings of CVPR, pp. 570–577 (2004)Google Scholar
  11. 11.
    Joachims, T.: Optimizing search engines using clickthrough data. In: ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 133–142 (2002)Google Scholar
  12. 12.
    Klein, D., Kamvar, S., Manning, C.: From Instance Level Constraints to Space Level Constraints: Making the most of prior knowledge in data-clustering. In: CVPR (2005)Google Scholar
  13. 13.
    Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.: Learning the Kernel Matrix with Semidefinite Programming. Journal of Machine Learning Research (2004)Google Scholar
  14. 14.
    Lewis, D.D., Yang, Y., Rose, T.G., Li, F., Dietterich, G., Li, F.: Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research 5, 361–397 (2004)Google Scholar
  15. 15.
    Quattoni, A., Carreras, X., Collins, M., Darrell, T.: An efficient projection for l1,infinity regularization. In: Proceedings of ICML (2009)Google Scholar
  16. 16.
    Quattoni, A., Collins, M., Darrell, T.: Learning visual representations using images with captions. In: Proceedings of CVPR (2007)Google Scholar
  17. 17.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: A database and web-based tool for image annotation. Tech. Rep. MIT-CSAIL-TR-2005-056, Massachusetts Institute of Technology (2005)Google Scholar
  18. 18.
    Schultz, M., Joachims, T.: Learning a Distance Metric from Relative Comparisons. In: NIPS (2004)Google Scholar
  19. 19.
    Shalev-Shwartz, S., Singer, Y., Srebro, N.: Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In: Proc. of Intl. Conf. on Machine Learning, pp. 807–814 (2007)Google Scholar
  20. 20.
    Urtasun, R., Darrell, T.: Sparse probabilistic regression for activity-independent human pose inference. In: Proceedings of CVPR, pp. 1–8 (2008)Google Scholar
  21. 21.
    Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained k-means clustering with background knowledge. In: ICML (2001)Google Scholar
  22. 22.
    Yan, R., Hauptmann, A.G.: Probabilistic latent query analysis for combining multiple retrieval sources. In: Proceedings of the 29th International ACM SIGIR Conference, pp. 324–331. ACM Press (2006)Google Scholar
  23. 23.
    Zhang, H., Berg, A.C., Maire, M., Malik, J.: Svm-knn: Discriminative nearest neighbor classification for visual category recognition. In: Proceedings of CVPR, pp. 2126–2136 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ariadna Quattoni
    • 1
  • Xavier Carreras
    • 1
  • Antonio Torralba
    • 2
  1. 1.Dept. LSIUniversitat Politècnica de CatalunyaBarcelonaSpain
  2. 2.Massachusetts Institute of Technology, CSAILCambridgeUSA

Personalised recommendations