Is Simhash Achilles?

  • Qixia Jiang
  • Yan Zhang
  • Liner Yang
  • Maosong Sun
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7097)

Abstract

Simhash generates compact binary codes for the input data thus improves the search efficiency. Most recent works on Simhash are designed to speed-up the search, generate high-quality descriptors, etc. However, few works discuss in what situations Simhash can be directly applied. This paper proposes a novel method to quantitatively analyze this question. Our method is based on Support Vector Data Description (SVDD), which tries to find a tighten sphere to cover most points. Using the geometry relation between the unit sphere and the SVDD sphere, we give a quantitative analysis on in what situations Simhash is feasible. We also extend the basic Simhash to handle those unfeasible cases. To reduce the complexity, an approximation algorithm is proposed, which is easy for implementation. We evaluate our method on synthetic data and a real-world image dataset. Most results show that our method outperforms the basic Simhash significantly.

Keywords

Information Retrieval Simhash Support Vector Data Description 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Charikar, M.S.: Similarity estimation techniques from rounding algorithms. In: Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing, p. 388 (2002)Google Scholar
  2. 2.
    Stein, B.: Principles of hash-based text retrieval. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 534 (2007)Google Scholar
  3. 3.
    Manku, G.S., Jain, A., Das Sarma, A.: Detecting near-duplicates for web crawling. In: Proceedings of the 16th International Conference on World Wide Web, p. 150 (2007)Google Scholar
  4. 4.
    Tax, D.M.J., Duin, R.P.W.: Support vector data description. Machine Learning 54, 45–66 (2004)CrossRefMATHGoogle Scholar
  5. 5.
    Poland, J.: Three different algorithms for generating uniformly distributed random points on the n-sphere (2000)Google Scholar
  6. 6.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 404–420. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Bentley, J.L.: K-d trees for semi-dynamic point sets. In: Proc. of the 6th ACM Symposium on Computational Geometry (SCG), pp. 187–197 (1990)Google Scholar
  8. 8.
    Weber, R., Schek, H.J., Blott, S.: A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In: Proceedings of the International Conference on Very Large Data Bases, pp. 194–205 (1998)Google Scholar
  9. 9.
    Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing, pp. 604–613 (1998)Google Scholar
  10. 10.
    Salakhutdinov, R., Hinton, G.: Semantic hashing. International Journal of Approximate Reasoning 50, 969–978 (2009)CrossRefGoogle Scholar
  11. 11.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Advances in Neural Information Processing Systems, vol. 21, pp. 1753–1760 (2009)Google Scholar
  12. 12.
    Lv, Q., Josephson, W., Wang, Z., Charikar, M., Li, K.: Multi-probe lsh: Efficient indexing for high-dimensional similarity search. In: Proceedings of the 33rd International Conference on Very Large Data Bases, pp. 950–961 (2007)Google Scholar
  13. 13.
    Bawa, M., Condie, T., Ganesan, P.: Lsh forest: self-tuning indexes for similarity search. In: Proceedings of the 14th International Conference on World Wide Web, p. 660 (2005)Google Scholar
  14. 14.
    Kulis, B., Grauman, K.: Kernelized locality-sensitive hashing for scalable image search. In: Proc. 12th International Conference on Computer Vision, ICCV (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Qixia Jiang
    • 1
    • 2
    • 3
  • Yan Zhang
    • 1
    • 2
    • 3
  • Liner Yang
    • 1
    • 2
    • 3
  • Maosong Sun
    • 1
    • 2
    • 3
  1. 1.State Key Laboratory on Intelligent Technology and SystemsTsinghua UniversityBeijingChina
  2. 2.Tsinghua National Laboratory for Information Science and TechnologyTsinghua UniversityBeijingChina
  3. 3.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations