Towards Annotation-Based Query and Document Expansion for Image Retrieval

Abstract

In this paper we report results of experiments conducted with strategies for improving text-based image retrieval. The adopted strategies were evaluated in the photographic retrieval task at ImageCLEF2007. We propose a Web-based method for expanding textual queries with related terms. This technique was the top-ranked query expansion method among those proposed by other ImageCLEF2007 participants. We also consider two methods for combining visual and textual information in the retrieval process: late-fusion and intermedia-feedback. The best results were obtained by combining intermedia-feedback and our expansion technique. The main contribution of this paper, however, is the proposal of ”annotation-based expansion”; a novel approach that consists of using labels assigned to images (with image annotation methods) for expanding textual queries and documents. We introduce this idea and report results of initial experiments towards enhancing text-based image retrieval via image annotation. Preliminary results show that this expansion strategy could be useful for image retrieval in the near future.