An Efficient Parallel Strategy for Matching Visual Self-similarities in Large Image Databases

  • Katharina Schwarz
  • Tobias Häußler
  • Hendrik P. A. Lensch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7583)


Due to high interest of social online systems, there exists a huge and still increasing amount of image data in the web. In order to handle this massive amount of visual information, algorithms often need to be redesigned. In this work, we developed an efficient approach to find visual similarities between images that runs completely on GPU and is applicable to large image databases. Based on local self-similarity descriptors, the approach finds similarities even across modalities. Given a set of images, a database is created by storing all descriptors in an arrangement suitable for parallel GPU-based comparison. A novel voting-scheme further considers the spatial layout of descriptors with hardly any overhead. Thousands of images are searched in only a few seconds. We apply our algorithm to cluster a set of image responses to identify various senses of ambiguous words and re-tag similar images with missing tags.


Similar Image Ambiguous Word Template Image Photo Collection Large Image Database 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Shechtman, E., Irani, M.: Matching Local Self-Similarities across Images and Videos. In: IEEE Conf. on Comp. Vis. and Pat. Recogn, CVPR (2007)Google Scholar
  2. 2.
    Lowe, D.G.: Object Recognition from Local Scale-Invariant Features. In: Proc. of Int. Conf. on Comp. Vis., ICCV (1999)Google Scholar
  3. 3.
    Sivic, J., Zisserman, A.: Video Google: A Text Retrieval Approach to Object Matching in Videos. In: Proc. of Int. Conf. on Comp. Vis (ICCV), vol. 2, pp. 1470–1477 (2003)Google Scholar
  4. 4.
    Zhang, Y., Jia, Z., Chen, T.: Image Retrieval with Geometry-Preserving Visual Phrases. In: IEEE Conf. on Comp. Vis. and Pat. Recogn (CVPR), pp. 809–816 (2011)Google Scholar
  5. 5.
    Oliva, A., Torralba, A.: Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. Int. Journal of Comp. Vis. 42, 145–175 (2001)zbMATHCrossRefGoogle Scholar
  6. 6.
    Douze, M., Jégou, H., Sandhawalia, H., Amsaleg, L., Schmid, C.: Evaluation of GIST descriptors for web-scale image search. In: Proc. of ACM Int. Conf. on Image and Video Retrieval, CIVR (2009)Google Scholar
  7. 7.
    Johnson, T., Georgel, P., Raguram, R., Frahm, J.M.: Fast Organization of Large Photo Collections using CUDA. In: Wksp. on Comp. Vis. on GPUs, ECCV (2010)Google Scholar
  8. 8.
    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: SUN database: Large-scale scene recognition from abbey to zoo. In: IEEE Conf. on Comp. Vis. and Pat. Recogn, CVPR (2010)Google Scholar
  9. 9.
    Shrivastava, A., Malisiewicz, T., Gupta, A., Efros, A.A.: Data-driven Visual Similarity for Cross-domain Image Matching. ACM Trans. Graph. 30 (2011)Google Scholar
  10. 10.
    Boiman, O., Irani, M.: Detecting Irregularities in Images and in Video. Int. Journal of Comp. Vis. 74, 17–31 (2007)CrossRefGoogle Scholar
  11. 11.
    Chatfield, K., Philbin, J., Zisserman, A.: Efficient Retrieval of Deformable Shape Classes using Local Self-Similarities. In: Wksp. on Non-rigid Shape Analysis and Deformable Image Alignment, ICCV, pp. 264–271 (2009)Google Scholar
  12. 12.
    Schindler, K., Suter, D.: Object Detection by Global Contour Shape. Pattern Recogn. 41, 3736–3748 (2008)zbMATHCrossRefGoogle Scholar
  13. 13.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories, vol. 12, p. 178. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  14. 14.
    Mark, J., Huiskes, B.T., Lew, M.S.: New Trends and Ideas in Visual Concept Detection: The MIR Flickr Retrieval Evaluation Initiative. In: MIR 2010: Proc. of the 2010 ACM Int. Conf. on Multimedia Information Retrieval, pp. 527–536. ACM, New York (2010)Google Scholar
  15. 15.
    Aly, M., Munich, M., Perona, P.: Indexing in Large Scale Image Collections: Scaling Properties and Benchmark. In: IEEE Wksp. on Applications of Comp. Vis., WACV (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Katharina Schwarz
    • 1
  • Tobias Häußler
    • 1
  • Hendrik P. A. Lensch
    • 1
  1. 1.Computer GraphicsTübingen UniversityTübingenGermany

Personalised recommendations