Improved maximum inner product search with better theoretical guarantee using randomized partition trees
- 200 Downloads
Recent interest in the problem of maximum inner product search (MIPS) has sparked the development of new solutions. The solutions (usually) reduce MIPS to the well-studied problem of nearest-neighbour search (NNS). To escape the curse of dimensionality, the problem is relaxed to accept approximate solutions (that is, accept anything that is approximately maximum), and locality sensitive hashing is the approximate NNS algorithm of choice. While being extremely resourceful, these existing solutions have a couple of aspects that can be improved upon—(i) MIPS can be reduced to NNS in multiple ways and there is lack of understanding (mostly theoretical but also empirical) when to choose which reduction for best accuracy or efficiency, and (ii) when MIPS is solved via approximate NNS, translating this approximation to the MIPS solution is not straightforward. To overcome these usability issues, we propose the use of randomized partition trees (RPTs) for solving MIPS. We still reduce MIPS to NNS but utilize RPTs to solve the NNS problem. RPTs find the exact NNS solution, hence the exact MIPS solution (with high probability), avoiding the need for any translation of approximation. The theoretical properties of RPTs allow us to definitively choose the best MIPS-to-NNS reduction. The empirical properties of RPTs allow us to significantly outperform the state-of-the-art while providing unique fine-grained control over the accuracy–efficiency trade-off. For example, at 80% accuracy, RPTs are 2–5\(\times \) more efficient than the state-of-the-art. Superiority of RPT comes at the cost of high space complexity overhead which can be a severe limitation of our proposed method. To address this limitation we introduce two space efficient versions of RPTs which enjoy the same superior performance of RPT while requiring a significantly reduced space complexity overhead.
KeywordsMaximum inner product search Nearest neighbor search Random projection trees Locality sensitive hashing
- Bache, K., & Lichman, M. (2013). UCI machine learning repository. http://archive.ics.uci.edu/ml.
- Bachrach, Y., Finkelstein, Y., Gilad-Bachrach, R., Katzir, L., Koeningstein, N., Nice, N., & Paquet, U. (2014). Speeding up the xbox recommender system using a euclidean transformation for inner-product spaces. In 8th ACM Conference on Recommender Systems.Google Scholar
- Cai, D. (2009). Text datasets in Matlab format. http://www.cad.zju.edu.cn/home/dengcai/Data/TextData.html.
- Cremonesi, P., Koren, Y., & Turrin, R. (2010). Performance of recommender algorithms on top-n recommendation tasks. In 4th ACM Conference on Recommender Systems.Google Scholar
- Curtin, R. R., Ram, P., & Gray, A. G. (2013). Fast exact max-kernel search. In Proceedings of SIAM data mining.Google Scholar
- Dasgupta, S., & Sinha, K. (2013). Randomized partition trees for exact nearest neighbor search. In The 26th Annual Conference on Learning Theory.Google Scholar
- Datar, M., Immorlica, N., Indyk, P., & Mirrokni, C. S. (2004). Locality-sensitive hashing based on p-stable distributions. In The 20th ACM symposium on computational geometry.Google Scholar
- Dean, T., Ruzon, M., Segal, M., Shlens, J., Vijayanarasimhan, S., & Yagnik, J. (2013). Fast accurate detection of 100,000 object classes on a single machine. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Gionis, A., Indyk, P., & Motwan, R. (1999). Similarity search in high dimensions via hashing. In 25th International Conference on Very Large Databases.Google Scholar
- Hyvönen, V., Pitkänen, T., Tasoulis, S. K., Jaasaari, E., Tuomainen, R., Wang, L., Corander, J., & Roos, T. (2016). Fast nearest neighbor search through sparse random projections and voting. In IEEE International Conference on BigData.Google Scholar
- Indyk, P., & Motwani, R. (1998). Approximate nearest neighbors: Towards removing the curse of dimensionality. In ACM symposium on theory of computing.Google Scholar
- Jain, P., & Kapoor, A. (2009). Active learning for large multi-class problem. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Joachims, T. (2006). Training linear SVM in linear time. In 12th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.Google Scholar
- Keivani, O., Sinha, K., & Ram, P. (2017). Improved inner product search with better theoretical guarantees. In International Joint Conference on Neural Network.Google Scholar
- Manjunath, B. S., & Ma, W. Y. (1996). Texture features for browsing and retrieving of large image data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 117–128.Google Scholar
- Neyshabur, B., & Srebro, N. (2015). On symmetric and asymmetric LSHs for inner product search. In 32nd International Conference on Machine Learning.Google Scholar
- Ram, P., & Gray, A. G. (2012). Maximum inner-product search using cone trees. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.Google Scholar
- Shrivasatava, A., & Li, P. (2014). Asymmetric LSH (ALSH) for sublinear time maximum inner product search (MIPS). In 28th Annual Conference on Neural Information Processing Systems.Google Scholar
- Shrivasatava, A., & Li, P. (2015). Improved asymmetric locality sensitive hashing (ALSH) for maximum inner product search (MIPS). In 31st Conference on Uncertainty in Artificial Intelligence.Google Scholar
- Sinha, K. (2014). LSH versus randomized partition trees: Which one to use for nearest neighbor search? In 13th International Conference on Machine Learning and Applications.Google Scholar
- Srebro, N., Rennie, J., & Jaaakkola, T. (2005). Maximum margin matrix factorization. In 19th Annual Conference on Neural Information Processing Systems.Google Scholar
- Trec interpolation. http://trec.nist.gov/pubs/trec16/appendices/measures.pdf. Accessed 14 Sept 2016.