Solving Multi-codebook Quantization in the GPU

  • Julieta MartinezEmail author
  • Holger H. Hoos
  • James J. Little
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9913)


We focus on the problem of vector compression using multi-codebook quantization (MCQ). MCQ is a generalization of k-means where the centroids arise from the combinatorial sums of entries in multiple codebooks, and has become a critical component of large-scale, state-of-the-art approximate nearest neighbour search systems. MCQ is often addressed in an iterative manner, where learning the codebooks can be solved exactly via least-squares, but finding the optimal codes results in a large number of combinatorial NP-Hard problems. Recently, we have demonstrated that an algorithm based on stochastic local search for this problem outperforms all previous approaches. In this paper we introduce a GPU implementation of our method, which achieves a \(30{\times }\) speedup over a single-threaded CPU implementation. Our code is publicly available (


Unary Term Stochastic Local Search Product Quantization Additive Quantization Iterate Conditional Mode 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Crandall, D.J., Owens, A., Snavely, N., Huttenlocher, D.P.: Sfm with mrfs: discrete-continuous optimization for large-scale structure from motion. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2841–2853 (2013)CrossRefGoogle Scholar
  2. 2.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRefGoogle Scholar
  3. 3.
    Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: VISApp, vol. 1 (2009)Google Scholar
  4. 4.
    Gong, Y., Lazebnik, S.: Iterative quantization: a procrustean approach to learning binary codes. In: CVPR (2011)Google Scholar
  5. 5.
    Norouzi, M., Fleet, D.J.: Minimal loss hashing for compact binary codes. In: ICML (2011)Google Scholar
  6. 6.
    Babenko, A., Lempitsky, V.: Additive quantization for extreme vector compression. In: CVPR (2014)Google Scholar
  7. 7.
    Martinez, J., Clement, J., Hoos, H., Little, J.: Revisiting additive quantization. In: ECCV (2016)Google Scholar
  8. 8.
    Hoos, H.H., Stützle, T.: Stochastic local search: Foundations & applications. Elsevier, Amsterdam (2004)zbMATHGoogle Scholar
  9. 9.
    Jégou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. TPAMI 33(1), 117–128 (2011)CrossRefGoogle Scholar
  10. 10.
    Zhang, T., Du, C., Wang, J.: Composite quantization for approximate nearest neighbor search. In: ICML (2014)Google Scholar
  11. 11.
    Tung, F., Little, J.J.: CollageParsing: nonparametric scene parsing by adaptive overlapping windows. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 511–525. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10599-4_33 Google Scholar
  12. 12.
    Krähenbühl, P., Koltun, V.: Efficient inference in fully connected crfs with gaussian edge potentials. arXiv preprint arXiv:1210.5644 (2012)
  13. 13.
    Vineet, V., Narayanan, P.: Cuda cuts: fast graph cuts on the gpu. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2008, pp. 1–8. IEEE (2008)Google Scholar
  14. 14.
    Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: VMV, pp. 243–252 (2008)Google Scholar
  15. 15.
    Norouzi, M., Fleet, D.J.: Cartesian k-means. In: CVPR (2013)Google Scholar
  16. 16.
    Bezanson, J., Edelman, A., Karpinski, S., Shah, V.B.: Julia: a fresh approach to numerical computing. arXiv preprint arXiv:1411.1607 (2014)

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Julieta Martinez
    • 1
    Email author
  • Holger H. Hoos
    • 1
  • James J. Little
    • 1
  1. 1.University of British ColumbiaVancouverCanada

Personalised recommendations