Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search

  • Herve Jegou
  • Matthijs Douze
  • Cordelia Schmid
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5302)

Abstract

This paper improves recent methods for large scale image search. State-of-the-art methods build on the bag-of-features image representation. We, first, analyze bag-of-features in the framework of approximate nearest neighbor search. This shows the sub-optimality of such a representation for matching descriptors and leads us to derive a more precise representation based on 1) Hamming embedding (HE) and 2) weak geometric consistency constraints (WGC). HE provides binary signatures that refine the matching based on visual words. WGC filters matching descriptors that are not consistent in terms of angle and scale. HE and WGC are integrated within the inverted file and are efficiently exploited for all images, even in the case of very large datasets. Experiments performed on a dataset of one million of images show a significant improvement due to the binary signature and the weak geometric consistency constraints, as well as their efficiency. Estimation of the full geometric transformation, i.e., a re-ranking step on a short list of images, is complementary to our weak geometric consistency constraints and allows to further improve the accuracy.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60, 91–110 (2004)CrossRefGoogle Scholar
  2. 2.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. IJCV 60(1), 63–86 (2004)CrossRefGoogle Scholar
  3. 3.
    Matas, J., Chum, O., Martin, U., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: BMVC, pp. 384–393 (2002)Google Scholar
  4. 4.
    Sivic, J., Zisserman, A.: Video Google: A text retrieval approach to object matching in videos. In: ICCV, pp. 1470–1477 (2003)Google Scholar
  5. 5.
    Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: CVPR, pp. 2161–2168 (2006)Google Scholar
  6. 6.
    Omercevic, D., Drbohlav, O., Leonardis, A.: High-dimensional feature matching: employing the concept of meaningful nearest neighbors. In: ICCV (2007)Google Scholar
  7. 7.
    Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.: Locality-sensitive hashing scheme based on p-stable distributions, pp. 253–262 (2004)Google Scholar
  8. 8.
    Shakhnarovich, G., Darrell, T., Indyk, P.: Nearest-Neighbor Methods in Learning and Vision: Theory and Practice. MIT Press, Cambridge (2006)Google Scholar
  9. 9.
    Philbin, J., Chum, O., Isard, M.A., Zisserman, J.S.: Object retrieval with large vocabularies and fast spatial matching. In: CVPR (2007)Google Scholar
  10. 10.
    Jegou, H., Harzallah, H., Schmid, C.: A contextual dissimilarity measure for accurate and efficient image search. In: CVPR (2007)Google Scholar
  11. 11.
    Fraundorfer, F., Stewenius, H., Nister, D.: A binning scheme for fast hard drive based image search. In: CVPR (2007)Google Scholar
  12. 12.
    Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In: CVPR (2007)Google Scholar
  13. 13.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: Improving particular object retrieval in large scale image databases. In: CVPR (2008)Google Scholar
  14. 14.
    Jegou, H., Douze, M.: INRIA Holidays dataset (2008), http://lear.inrialpes.fr/people/jegou/data.php
  15. 15.
    Mikolajczyk, K.: Binaries for affine covariant region descriptors (2007), http://www.robots.ox.ac.uk/~vgg/research/affine/

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Herve Jegou
    • 1
  • Matthijs Douze
    • 1
  • Cordelia Schmid
    • 1
  1. 1.INRIA Grenoble, LEAR, LJKFrance

Personalised recommendations