Spatially Aware Enhancement of BoVW-Based Image Retrieval Exploiting a Saliency Map

Abstract

The Bag-of-Visual-Words (BoVW) scheme is the most popular approach to similar image retrieval. The conventional BoVW models an image as a histogram of local features in which all of the features are uniformly weighed. Recently, researchers have focused on the similarity between the foregrounds of two images, because the foreground properly expresses the semantics of the image. Given an image, these methods approximate the likelihood that a local feature belongs to the foreground with its saliency value derived from the saliency map. The foreground histogram is then constructed in such a way that each local feature is accumulated to the histogram in proportion to its saliency value. Finally, the similarity between images is measured by comparing the foreground histograms of the images. However, the above strategy discounts local features with small saliency values, even if they lie on the genuine foreground. This paper proposes a new technique that does not disregard such local features and examines their spatial surroundings. In particular, a high weight is assigned to a local feature with a low saliency value when the spatial surrounding has a high saliency value, because this local feature is also likely to be a part of the foreground.