Advertisement

Transactional Support for Visual Instance Search

  • Herwig Lejsek
  • Friðrik Heiðar Ásmundsson
  • Björn Þór Jónsson
  • Laurent AmsalegEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11223)

Abstract

This article addresses the issue of dynamicity and durability for scalable indexing of very large and rapidly growing collections of local features for visual instance retrieval. By extending the NV-tree, a scalable disk-based high-dimensional index, we show how to implement the ACID properties of transactions which ensure both dynamicity and durability. We present a detailed performance evaluation of the transactional NV-tree, showing that the insertion throughput is excellent despite the effort to enforce the ACID properties.

References

  1. 1.
    Babenko, A., Lempitsky, V.S.: The inverted multi-index. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1247–1260 (2015)CrossRefGoogle Scholar
  2. 2.
    Babenko, A., Lempitsky, V.S.: Efficient indexing of billion-scale datasets of deep descriptors. In: Proceedings of the CVPR, Las Vegas, NV, USA (2016)Google Scholar
  3. 3.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  4. 4.
    Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U.: When is “Nearest Neighbor” meaningful? In: Beeri, C., Buneman, P. (eds.) ICDT 1999. LNCS, vol. 1540, pp. 217–235. Springer, Heidelberg (1999).  https://doi.org/10.1007/3-540-49257-7_15CrossRefGoogle Scholar
  5. 5.
    Datar, M., Indyk, P., Immorlica, N., Mirrokni, V.: Locality-Sensitive Hashing Using Stable Distributions. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  6. 6.
    Fagin, R., Kumar, R., Sivakumar, D.: Efficient similarity search and classification via rank aggregation. In: Proceedings of the ACM SIGMOD, San Diego, CA, USA (2003)Google Scholar
  7. 7.
    Fukunaga, K., Narendra, P.M.: A branch and bound algorithms for computing k-nearest neighbors. IEEE Trans. Comput. 24(7), 750–753 (1975)CrossRefGoogle Scholar
  8. 8.
    Gray, J., Reuter, A.: Transaction Processing: Concepts and Techniques. Morgan Kaufmann, San Francisco (1993)zbMATHGoogle Scholar
  9. 9.
    Guðmundsson, G.Þ., Amsaleg, L., Jónsson, B.Þ., Franklin, M.J.: Towards engineering a web-scale multimedia service: a case study using Spark. In: Proceedings of the MMSys, Taipei, Taiwan (2017)Google Scholar
  10. 10.
    Jégou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)CrossRefGoogle Scholar
  11. 11.
    Jégou, H., Tavenard, R., Douze, M., Amsaleg, L.: Searching in one billion vectors: re-rank with source coding. In: Proceedings of the ICASSP, Prague, Czech Republic (2011)Google Scholar
  12. 12.
    Jin, Z., et al.: Complementary projection hashing. In: Proceedings of the ACM ICCV, Barcelona, Spain (2013)Google Scholar
  13. 13.
    Jónsson, B.Þ., Amsaleg, L., Lejsek, H.: SSD technology enables dynamic maintenance of persistent high-dimensional indexes. In: Proceedings of the ACM ICMR, New York, NY, USA (2016)Google Scholar
  14. 14.
    Lejsek, H., Ásmundsson, F.H., Jónsson, B.Þ., Amsaleg, L.: NV-Tree: an efficient disk-based index for approximate search in very large high-dimensional collections. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 869–883 (2009)CrossRefGoogle Scholar
  15. 15.
    Lejsek, H., Jónsson, B.Þ., Amsaleg, L.: NV-Tree: nearest neighbours at the billion scale. In: Proceedings of the ACM ICMR, Trento, Italy (2011)Google Scholar
  16. 16.
    Lejsek, H., Jónsson, B.Þ., Amsaleg, L., Ásmundsson, F.H.: Dynamicity and durability in scalable visual instance search. arXiv abs/1805.10942 (2018). https://arxiv.org/abs/1805.10942
  17. 17.
    Li, C., Chang, E., Garcia-Molina, H., Wiederhold, G.: Clustering for approximate similarity search in high-dimensional spaces. IEEE Trans. Knowl. Data Eng. 14(4), 792–808 (2002)CrossRefGoogle Scholar
  18. 18.
    Liu, T., Moore, A., Gray, A., Yang, K.: An investigation of practical approximate nearest neighbor algorithms. In: Proceedings of the NIPS, Vancouver, BC, Canada (2004)Google Scholar
  19. 19.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Marz, N., Warren, J.: Big Data: Principles and Best Practices of Scalable Real-Time Data Systems. Manning Publication co., Shelter Island (2015)Google Scholar
  21. 21.
    Mikolajczyk, K., et al.: A comparison of affine region detectors. Int. J. Comput. Vis. 65(1), 43–72 (2005)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  23. 23.
    Mohan, C., Haderle, D., Lindsay, B., Pirahesh, H., Schwarz, P.: ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. ACM Trans. Database Syst. 17(1), 94–162 (1992)CrossRefGoogle Scholar
  24. 24.
    Moise, D., Shestakov, D., Guðmundsson, G.Þ., Amsaleg, L.: Indexing and searching 100M images with map-reduce. In: Proceedings of the ACM ICMR, Dallas, TX, USA (2013)Google Scholar
  25. 25.
    Muja, M., Lowe, D.G.: Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2227–2240 (2014)CrossRefGoogle Scholar
  26. 26.
    Nowak, E., Jurie, F., Triggs, B.: Sampling strategies for bag-of-features image classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 490–503. Springer, Heidelberg (2006).  https://doi.org/10.1007/11744085_38CrossRefGoogle Scholar
  27. 27.
    Ólafsson, A., Jónsson, B.Þ., Amsaleg, L., Lejsek, H.: Dynamic behavior of balanced NV-trees. Multimed. Syst. 17(2), 83–100 (2011)CrossRefGoogle Scholar
  28. 28.
    Paulevé, L., Jégou, H., Amsaleg, L.: Locality sensitive hashing: a comparison of hash function types and querying mechanisms. Pattern Recogn. Lett. 31(11), 1348–1358 (2010)CrossRefGoogle Scholar
  29. 29.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the CVPR, Minneapolis, MN, USA (2007)Google Scholar
  30. 30.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: improving particular object retrieval in large scale image databases. In: Proceedings of the CVPR, Anchorage, AK, USA (2008)Google Scholar
  31. 31.
    Srinivasan, V., Carey, M.J.: Performance of B-tree concurrency control algorithms. In: Proceedings of the ACM SIGMOD, Denver, Colorado, USA (1991)Google Scholar
  32. 32.
    Sun, X., Wang, C., Xu, C., Zhang, L.: Indexing billions of images for sketch-based retrieval. In: Proceedings of the ACM Multimedia, Barcelona, Spain (2013)Google Scholar
  33. 33.
    Tao, Y., Yi, K., Sheng, C., Kalnis, P.: Efficient and accurate nearest neighbor and closest pair search in high-dimensional space. ACM Trans. Database Syst. 35(3), 20:1–20:46 (2010)CrossRefGoogle Scholar
  34. 34.
    Uhlmann, J.: Satisfying general proximity/similarity queries with metric trees. Inf. Process. Lett. 40(4), 175–179 (1991)CrossRefGoogle Scholar
  35. 35.
    Zhang, D., Agrawal, D., Chen, G., Tung, A.: HashFile: an efficient index structure for multimedia data. In: Proceedings of the ICDE, Hannover, Germany (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Herwig Lejsek
    • 1
  • Friðrik Heiðar Ásmundsson
    • 1
  • Björn Þór Jónsson
    • 2
    • 3
  • Laurent Amsaleg
    • 4
    Email author
  1. 1.Videntifier TechnologiesReykjavikIceland
  2. 2.Reykjavík UniversityReykjavikIceland
  3. 3.IT University of CopenhagenCopenhagenDenmark
  4. 4.CNRS–IRISARennesFrance

Personalised recommendations