Chapter

Image and Video Retrieval

Volume 4071 of the series Lecture Notes in Computer Science pp 51-60

Query by Semantic Example

  • Nikhil RasiwasiaAffiliated withStatistical Visual Computing Lab, University of California
  • , Nuno VasconcelosAffiliated withStatistical Visual Computing Lab, University of California
  • , Pedro J. MorenoAffiliated withGoogle, Inc.

* Final gross prices may vary according to local VAT.

Get Access

Abstract

A solution to the problem of image retrieval based on query-by-semantic-example (QBSE) is presented. QBSE extends the idea of query-by-example to the domain of semantic image representations. A semantic vocabulary is first defined, and a semantic retrieval system is trained to label each image with the posterior probability of appearance of each concept in the vocabulary. The resulting vector is interpreted as the projection of the image onto a semantic probability simplex, where a suitable similarity function is defined. Queries are specified by example images, which are projected onto the probability simplex. The database images whose projections on the simplex are closer to that of the query are declared its closest neighbors. Experimental evaluation indicates that 1) QBSE significantly outperforms the traditional query-by-visual-example paradigm when the concepts in the query image are known to the retrieval system, and 2) has equivalent performance even in the worst case scenario of queries composed by unknown concepts.