Improving Image Retrieval by Local Feature Reselection with Query Expansion

  • Hanli WangEmail author
  • Tianyao Sun
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9916)


A novel approach related to query expansion is proposed to improve image retrieval performance. The proposed approach investigates the problem that not all of the visual features extracted from images are appropriate to be employed for similarity matching. To address this issue, we distinguish image features as effective features from noisy features. The former is benefit for image retrieval while the latter causes deterioration, since the matching of noisy features may rise the similarity score of irrelevant images. In this work, a detailed illustration of effective and noisy features is given and the aforementioned problem is solved by selecting effective features to enhance query feature set while removing noisy features via spatial verification. Experimental results demonstrate that the proposed approach outperforms a number of state-of-the-art query expansion approaches.


Image retrieval Query expansion Feature reselection 


  1. 1.
    Sivic, J., Zisserman, A.: Video google: a text retrieval approach to object matching in videos. In: Proceedings of the ICCV 2003, pp. 1470–1477, October 2003Google Scholar
  2. 2.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)CrossRefGoogle Scholar
  3. 3.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  4. 4.
    Chum, O., Philbin, J., Sivic, J., Isard, M., Zisserman, A.: Total recall: automatic query expansion with a generative feature model for object retrieval. In: Proceedings of the ICCV 2007, pp. 1–8, October 2007Google Scholar
  5. 5.
    Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 304–317. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88682-2_24 CrossRefGoogle Scholar
  6. 6.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the CVPR 2007, pp. 1–8, June 2007Google Scholar
  7. 7.
    Wang, W., Zhang, D., Zhang, Y., Li, J.: Fast and robust spatial matching for object retrieval. In: Proceedings of the ICASSP 2010, pp. 1238–1241, March 2010Google Scholar
  8. 8.
    Chum, O., Mikulik, A., Perdoch, M., Matas, J.: Total recall II: query expansion revisited. In: Proceedings of the CVPR 2011, pp. 889–896, June 2011Google Scholar
  9. 9.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Tolias, G., Avrithis, Y., Jégou, H.: To aggregate or not to aggregate: Selective match kernels for image search. In: Proceedings of the ICCV 2013, pp. 1401–1408, December 2013Google Scholar
  11. 11.
    Jégou, H., Douze, M., Schmid, C.: On the burstiness of visual elements. In: Proceedings of the CVPR 2009, pp. 1169–1176, June 2009Google Scholar
  12. 12.
    Perdoch, M., Chum, O., Matas, J.: Efficient representation of local geometry for large scale object retrieval. In: Proceedings of the CVPR 2009, pp. 9–16, June 2009Google Scholar
  13. 13.
    Verbeek, J., Harzallah, H., Schmid, C., Jégou, H.: Accurate image search using the contextual dissimilarity measure. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 2–11 (2010)CrossRefGoogle Scholar
  14. 14.
    Mikulík, A., Perdoch, M., Chum, O., Matas, J.: Learning a fine vocabulary. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 1–14. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15558-1_1 CrossRefGoogle Scholar
  15. 15.
    Arandjelovic, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: Proceedings of the CVPR 2012, pp. 2911–2918, June 2012Google Scholar
  16. 16.
    Tolias, G., Jégou, H.: Visual query expansion with or without geometry: refining local descriptors by feature aggregation. Pattern Recog. 47(10), 3466–3476 (2014)CrossRefGoogle Scholar
  17. 17.
    Qin, D., Wengert, C., Gool, L.V.: Query adaptive similarity for large scale object retrieval. In: Proceedings of the CVPR 2013, pp. 1610–1617, June 2013Google Scholar
  18. 18.
    Mikulík, A., Perdoch, M., Chum, O., Matas, J.: Learning vocabularies over a fine quantization. Int. J. Comput. Vis. 103(1), 163–175 (2013)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Tolias, G., Avrithis, Y., Jégou, H.: Image search with selective match kernels: aggregation across single and multiple images. Int. J. Comput. Vis. 116(3), 247–261 (2016)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer Science and TechnologyTongji UniversityShanghaiChina
  2. 2.Key Laboratory of Embedded System and Service Computing, Ministry of EducationTongji UniversityShanghaiChina

Personalised recommendations