Multimedia Systems

, Volume 23, Issue 2, pp 169–181 | Cite as

Optimized residual vector quantization for efficient approximate nearest neighbor search

  • Liefu Ai
  • Junqing Yu
  • Zebin Wu
  • Yunfeng He
  • Tao Guan
Regular Paper


In this paper, an optimized residual vector quantization-based approach is presented for improving the quality of vector quantization and approximate nearest neighbor search. The main contributions are as follows. Based on residual vector quantization (RVQ), a joint optimization process called enhanced RVQ (ERVQ) is introduced. Each stage codebook is iteratively optimized by the others aiming at minimizing the overall quantization errors. Thus, an input vector is approximated by its quantization outputs more accurately. Consequently, the precision of approximate nearest neighbor search is improved. To efficiently find nearest centroids when quantizing vectors, a non-linear vector quantization method is proposed. The vectors are embedded into 2-dimensional space where the lower bounds of Euclidean distances between the vectors and centroids are calculated. The lower bound is used to filter non-nearest centroids for the purpose of reducing computational costs. ERVQ is noticeably optimized in terms of time efficiency on quantizing vectors when combining with this method. To evaluate the accuracy that vectors are approximated by their quantization outputs, an ERVQ-based exhaustive method for approximate nearest neighbor search is implemented. Experimental results on three datasets demonstrate that our approaches outperform the state-of-the-art methods over vector quantization and approximate nearest neighbor search.


Approximate nearest neighbor search Vector quantization Codebook optimization Filtration 



This paper is financially supported by the National Natural Science Foundation of China (NSFC) under Grant No. 61173114, 61202300, and 61272202.


  1. 1.
    Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in video. In: ICCV, pp. 1470–1477 (2003)Google Scholar
  2. 2.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–100 (2004)CrossRefGoogle Scholar
  3. 3.
    Bohm, C., Berchtold, S., Keim, D.A.: Searching in high-dimensional spaces: index structures for improving the performance of multimedia databases. ACM Comput. Surv. 33(3), 322–373 (2001)CrossRefGoogle Scholar
  4. 4.
    Jegou, H., Matthijs, D., Cordelia, S.: Product quantization for nearest neighbor search. IEEE Trans. PAMI 33(1), 117–128 (2011)CrossRefGoogle Scholar
  5. 5.
    Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)CrossRefzbMATHGoogle Scholar
  6. 6.
    Silpa-Anan C., Hartley R.: Optimised kd-trees for fast image descriptor matching. In: CVPR, pp. 1–8 (2008)Google Scholar
  7. 7.
    Jia Y., Wang J., Zeng G., Zha H., Hua X. S.: Optimizing kd-trees for scalable visual descriptor indexing. In: CVPR, pp. 3392–3399 (2010)Google Scholar
  8. 8.
    Wang, J., Wang, N., Jia, Y., Li, J., Zeng, G., Zha, H., Hua, X.S.: Triary-projection trees for approximate nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 388–403 (2014)CrossRefGoogle Scholar
  9. 9.
    Philbin J., Chum O., Isard M., Sivic J., Zisserman A.: Object retrieval with large vocabularies and fast spatial matching. In: CVPR, pp. 1–8 (2007)Google Scholar
  10. 10.
    Nister D., Stewenius H.: Scalable recognition with a vocabulary tree. In: CVPR, pp. 2161–2168 (2006)Google Scholar
  11. 11.
    Muja M., Lowe D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: VISSSAPP, pp. 331–340 (2009)Google Scholar
  12. 12.
    Data, M., Immorlica, N., Indyk, P., Mirrokni, D.V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: Symposium on Computational geometry, pp. 253–262 (2004)Google Scholar
  13. 13.
    Panigrahy, R.: Entropy based nearest neighbor search in high dimensions. In: ACM-SIAM SODA, pp. 1186–1195 (2006)Google Scholar
  14. 14.
    Lv, Q., Josephson, W., Wang, Z., Charikar, M., Li, K.: Multi-Probe LSH: Efficient indexing for high-dimensional similarity search. In: VLDB, pp. 950–961 (2007)Google Scholar
  15. 15.
    Kuo, Y. H., Chen, K.T.C., Chiang, H., Hsu, W.H.: Query expansion for hash-based image object retrieval. In: ACM Conference on Multimedia, pp. 65–74 (2009)Google Scholar
  16. 16.
    Jegou, H., Amsaleg, L., Schmid, C., Gros, P.: Query-adaptative locality sensitive hashing. In: Conference on ICASSP, pp. 825–828 (2008)Google Scholar
  17. 17.
    Torralba, A., Fergus, R., Weiss, Y.: Small codes and large image databases for recognition. In: International conference on CVPR, pp. 1–8 (2008)Google Scholar
  18. 18.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: NIPS, pp. 1753–1760 (2009)Google Scholar
  19. 19.
    Heo, J.P., Lee, Y., He, J., Chang, S.F., Yoon, S.E.: Spherical hashing. In: International conference on CVPR, pp. 2957–2964 (2012)Google Scholar
  20. 20.
    Jegou, H., Douze, M., Schmid, C.: Improving Bag-of-Features for Large Scale Image Search. Int J Comput Vision 87(3), 316–336 (2010)CrossRefGoogle Scholar
  21. 21.
    Jegou, H., Douze, M., Schmid, C.: Packing bag-of-features. In: International Conference on Computer Vision (ICCV), pp. 2357–2364 (2009)Google Scholar
  22. 22.
    He, K., Wen, F., Sun, J.: K-means Hashing: an affinity-preserving quantization method for learning Binary Compact Codes. In: International Conference on CVPR, pp. 2938–2945 (2013)Google Scholar
  23. 23.
    Hajebi, K., Yadkori, Y.A., Shahbazi H., Zhang H.: Fast approximate nearest-neighbor search with k-nearest neighbor graph. In: IJCAI, pp. 1312–1317 (2011)Google Scholar
  24. 24.
    Wang, J., Wang, J., Zeng, G., Tu, Z., Gan, R., Li, S.: Scalable k-nn graph construction for visual search. In: CVPR, pp. 1106–1113 (2012)Google Scholar
  25. 25.
    Wang, J., Li, S.: Query-Driven Iterated Neighborhood Graph Search for Large Scale Indexing. In: ACM Multimedia, pp. 179–188 (2012)Google Scholar
  26. 26.
    Wang, J., Wang, J., Zeng, G., Gan, R., Li, S., Guo, B.: Fast Neighborhood Graph Search using Cartesian Concatenation., In: ICCV, pp. 2128–2135 (2013)Google Scholar
  27. 27.
    Brandt, J.: Transform coding for fast approximate nearest neighbor search in high dimensions. In: International conference on CVPR, pp. 1815–1822 (2010)Google Scholar
  28. 28.
    Chen, Y., Guan, T., Wang, C.: Approximate nearest neighbor search by residual vector quantization. Sensors 10, 11259–11273 (2010)CrossRefGoogle Scholar
  29. 29.
    Babenko, A., Lempitsky, V.: The Inverted multi-index. In: International Conference on CVPR, pp. 3069–3076 (2012)Google Scholar
  30. 30.
    Jegou, H., Tavenard, R., Douze, M., Amsaleg, L.: Search in one Billion Vectors: re-rank with Source Coding. In: IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), pp. 861–864 (2011)Google Scholar
  31. 31.
    Ge, T., He, K., Ke, Q., Sun, J.: Optimized product quantization for approximate nearest neighbor search. In: International Conference on CVPR, pp. 2946–2953 (2013)Google Scholar
  32. 32.
    Gray, R., Neuhoff, D.: Quantization. IEEE Trans. Inf. Theory 44(6), 2325–2383 (1998)CrossRefzbMATHGoogle Scholar
  33. 33.
    Norouzi, M., Fleet, D.J.: Cartesian k-means. In: CVPR, pp. 3017–3024 (2013)Google Scholar
  34. 34.
    Gong, Y., Lazebnik, S.: Iterative quantization: a procrustean approach to learning binary codes. In: CVPR, pp. 817–824 (2011)Google Scholar
  35. 35.
    Chan, W., Gupta, S., Gersho, A.: Enhanced multistage vector quantization by joint codebook design. IEEE Trans Commun 40(11), 1693–1697 (1992)CrossRefGoogle Scholar
  36. 36.
    Hwang, Y., Han, B., Ahn, H.: A fast nearest neighbor search algorithm by nonlinear embedding. In: International Conference on CVPR, pp. 2053–306 (2012)Google Scholar
  37. 37.
  38. 38.
    Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: ECCV, pp. 304–317 (2008)Google Scholar
  39. 39.
  40. 40.
    Torralba, A., Fergus, F., Freeman, W.T.: 80 million tiny images: a large database for non-parametric object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 1958–1970 (2008)CrossRefGoogle Scholar
  41. 41.
    Babenko, A., Lempitsky V.: Additive quantization for extreme vector compression. In: CVPR, pp. 931–938 (2014)Google Scholar
  42. 42.
    Zhang T., Du C., Wang J.: Composite quantization for approximate nearest neighbor search. In: ICML, pp. 1–9 (2014)Google Scholar
  43. 43.
    Ai L., Yu J., Guan T., He Y.: Efficient approximate nearest neighbor search by optimized residual vector quantization. In: CBMI, pp. 1–4 (2014)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  2. 2.School of Computer and InformationAnqing Normal UniversityAnqingChina
  3. 3.Center of Network and ComputationHuazhong University of Science and TechnologyWuhanChina

Personalised recommendations