Hybrid-Indexing Multi-type Features for Large-Scale Image Search

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9003)


Indexing local features with a vocabulary tree and indexing holistic features by compact hashing codes are two successful but separated lines of research. Both of the two indexing models are suited for specific features and are limited to certain scenarios like partial-duplicate search and similar image search, respectively. To conquer such limitations, we propose a novel hybrid-indexing strategy, which incorporates multiple similarity metrics into one inverted index file during off-line indexing. Hybrid-Indexing only requires the Bag-of-visual Words (BoWs) model as input for online query, but could obtain more satisfying retrieval results because the index file conveys hybrid similarities among images. Moreover, hybrid-indexing does not degrade the efficiency of classic BoWs based image search. Experiments on several public datasets manifest the effectiveness and efficiency of our proposed method.


Image Retrieval Visual Word Image Search Inverted Index Vocabulary Tree 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported in part to Dr. Qi Tian by ARO grant W911NF-12-1-0057 and Faculty Research Awards by NEC Laboratories of America. This work was supported in part by National Science Foundation of China (NSFC) 61429201.


  1. 1.
    Smeulders, A.W., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. TPAMI 22, 1349–1380 (2000)CrossRefGoogle Scholar
  2. 2.
    Gehler, P., Nowozin, S.: On feature combination for multiclass object classification. In: ICCV (2009)Google Scholar
  3. 3.
    Zhang, S., Huang, J., Huang, Y., Yu, Y., Li, H., Metaxas, D.N.: Automatic image annotation using group sparsity. In: CVPR, pp. 3312–3319. IEEE (2010)Google Scholar
  4. 4.
    Fagin, R., Kumar, R., Sivakumar, D.: Efficient similarity search and classification via rank aggregation. In: ACM SIGMOD (2003)Google Scholar
  5. 5.
    Jégou, H., Schmid, C., Harzallah, H., Verbeek, J.: Accurate image search using the contextual dissimilarity measure. TPAMI 32, 2–11 (2010)CrossRefGoogle Scholar
  6. 6.
    Zhang, S., Yang, M., Cour, T., Yu, K., Metaxas, D.N.: Query specific fusion for image retrieval. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 660–673. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  7. 7.
    Ye, G., Liu, D., Jhuo, I.H., Chang, S.F.: Robust late fusion with rank minimization. In: CVPR (2012)Google Scholar
  8. 8.
    Zhang, S., Yang, M., Wang, X., Lin, Y., Tian, Q.: Semantic-aware co-indexing for image retrieval. In: ICCV (2013)Google Scholar
  9. 9.
    Luo, Q., Zhang, S., Huang, T., Gao, W., Tian, Q.: Superimage: packing semantic-relevant images for indexing and retrieval. In: ICMR (2014)Google Scholar
  10. 10.
    Zhou, W., Lu, Y., Li, H., Tian, Q.: Scalar quantization for large scale image search. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 169–178. ACM (2012)Google Scholar
  11. 11.
    Lowe, D.G.: Distinctive image features from scale invariant keypoints. IJCV 60, 91–110 (2004)CrossRefGoogle Scholar
  12. 12.
    Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In: ICCV (2011)Google Scholar
  13. 13.
    Zhang, S., Tian, Q., Huang, Q., Gao, W., Rui, Y.: USB: ultra short binary descriptor for fast visual matching and retrieval. IEEE Trans. Image Process. 23(8), 3671–3683 (2014)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Tian, Q., Sebe, N., Loupias, E., Huang, T., Lew, M.: Image retrieval using wavelet-based salient points. J. Electron. Imaging 10, 835–849 (2001)CrossRefGoogle Scholar
  15. 15.
    Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: CVPR (2006)Google Scholar
  16. 16.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: CVPR (2007)Google Scholar
  17. 17.
    Zhang, S., Tian, Q., Huang, Q., Rui, Y.: Embedding multi-order spatial clues for scalable visual matching and retrieval. IEEE J. Emerg. Sel. Top. Circuits Syst. 4, 130–141 (2014)CrossRefGoogle Scholar
  18. 18.
    Chum, O., Philbin, J., Sivic, J., Isard, M., Zisserman, A.: Total recall: automatic query expansion with a generative feature model for object retrieval. In: ICCV (2007)Google Scholar
  19. 19.
    Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 304–317. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  20. 20.
    Zhang, S., Tian, Q., Hua, G., Huang, Q., Gao, W.: Descriptive visual words and visual phrases for image applications. In: ACM Multimedia (2009)Google Scholar
  21. 21.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. IJCV 42, 145–175 (2001)CrossRefGoogle Scholar
  22. 22.
    Deng, J., Berg, A.C., Fei-Fei, L.: Hierarchical semantic indexing for large scale image retrieval. In: CVPR (2011)Google Scholar
  23. 23.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: CVPR (2009)Google Scholar
  24. 24.
    Douze, M., Ramisa, A., Schmid, C.: Combining attributes and fisher vectors for effcient image retrieval. In: CVPR (2011)Google Scholar
  25. 25.
    Andoni, A., Indyk, P.: Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: FOCS (2006)Google Scholar
  26. 26.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: Imagenet large scale visual recognition challenge (2014)Google Scholar
  27. 27.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: Bringing order to the web (1999)Google Scholar
  28. 28.
    Huiskes, M.J., Lew, M.S.: The mir flickr retrieval evaluation. In: MIR ’08: Proceedings of the 2008 ACM ICMIR, ACM, New York (2008)Google Scholar
  29. 29.
    Wang, X., Yang, M., Cour, T., Zhu, S., Yu, K., Han, T.X.: Contextual weighting for vocabulary tree based image retrieval. In: ICCV (2011)Google Scholar
  30. 30.
    Shen, X., Lin, Z., Brandt, J., Avidan, S., Wu, Y.: Object retrieval and localization with spatially-constrained similarity measure and k-NN reranking. In: CVPR (2012)Google Scholar
  31. 31.
    Torresani, L., Szummer, M., Fitzgibbon, A.: Efficient object category recognition using classemes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 776–789. Springer, Heidelberg (2010) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.School of Electronics Engineering and Computer SciencePeking UniversityBeijingPeople’s Republic of China
  2. 2.Department of Computer ScienceUniversity of Texas at San AntonioSan AntonioUSA

Personalised recommendations