The Visual Computer

, Volume 32, Issue 6, pp 1045–1055

Toward semantic image similarity from crowdsourced clustering

  • Yanir Kleiman
  • George Goldberg
  • Yael Amsterdamer
  • Daniel Cohen-Or
Original Article

DOI: 10.1007/s00371-016-1266-4

Cite this article as:
Kleiman, Y., Goldberg, G., Amsterdamer, Y. et al. Vis Comput (2016) 32: 1045. doi:10.1007/s00371-016-1266-4
  • 158 Downloads

Abstract

Determining the similarity between images is a fundamental step in many applications, such as image categorization, image labeling and image retrieval. Automatic methods for similarity estimation often fall short when semantic context is required for the task, raising the need for human judgment. Such judgments can be collected via crowdsourcing techniques, based on tasks posed to web users. However, to allow the estimation of image similarities in reasonable time and cost, the generation of tasks to the crowd must be done in a careful manner. We observe that distances within local neighborhoods provide valuable information that allows a quick and accurate construction of the global similarity metric. This key observation leads to a solution based on clustering tasks, comparing relatively similar images. In each query, crowd members cluster a small set of images into bins. The results yield many relative similarities between images, which are used to construct a global image similarity metric. This metric is progressively refined, and serves to generate finer, more local queries in subsequent iterations. We demonstrate the effectiveness of our method on datasets where ground truth is available, and on a collection of images where semantic similarities cannot be quantified. In particular, we show that our method outperforms alternative baseline approaches, and prove the usefulness of clustering queries, and of our progressive refinement process.

Keywords

Crowdsourcing Image similarity Image distance metric 

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Yanir Kleiman
    • 1
  • George Goldberg
    • 1
  • Yael Amsterdamer
    • 2
  • Daniel Cohen-Or
    • 1
  1. 1.Tel Aviv UniversityTel AvivIsrael
  2. 2.Bar Ilan UniversityRamat GanIsrael