Data Mining and Knowledge Discovery

, Volume 17, Issue 3, pp 402–430

Learning to hash: forgiving hash functions and applications

Article

DOI: 10.1007/s10618-008-0096-z

Cite this article as:
Baluja, S. & Covell, M. Data Min Knowl Disc (2008) 17: 402. doi:10.1007/s10618-008-0096-z

Abstract

The problem of efficiently finding similar items in a large corpus of high-dimensional data points arises in many real-world tasks, such as music, image, and video retrieval. Beyond the scaling difficulties that arise with lookups in large data sets, the complexity in these domains is exacerbated by an imprecise definition of similarity. In this paper, we describe a method to learn a similarity function from only weakly labeled positive examples. Once learned, this similarity function is used as the basis of a hash function to severely constrain the number of points considered for each lookup. Tested on a large real-world audio dataset, only a tiny fraction of the points (~0.27%) are ever considered for each lookup. To increase efficiency, no comparisons in the original high-dimensional space of points are required. The performance far surpasses, in terms of both efficiency and accuracy, a state-of-the-art Locality-Sensitive-Hashing-based (LSH) technique for the same problem and data set.

Keywords

Hashing Audio matching Machine learning Locality sensitive hashing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Google, Inc.Mountain ViewUSA

Personalised recommendations