Abstract
Most approaches to image retrieval on the web have their basis in document search techniques. Images are indexed based on the text that is related to the images. Queries are matched to this text to produce a set of search results, which are organized in paged grids that are reminiscent of lists of documents. Due to ambiguity both with user-supplied queries and with the text used to describe the images within the search index, most image searches contain many irrelevant images distributed throughout the search results, and are often focused on the most common interpretation of the query. We propose a method for addressing these problems in which conceptual query expansion is used to generate a diverse range of images, and a multi-resolution extension of a self-organizing map is used to group visually similar images. The resulting interface acts as an intelligent search assistant, automatically diversifying the search results and then allowing the searcher to interactively highlight and filter images based on the concepts, and zoom into an area within the image space to show additional images that are visually similar. Evaluations show that the precision of the image search results increase as a result of concept-based focusing and filtering, as well as visual zooming operations, even for uncommon interpretations of ambiguous queries.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
André P, Cutrell E, Tan DS, Smith G (2009) Designing novel image search interfaces by understanding unique characteristics and usage. In: Proceedings of the IFIP conference on human–computer interaction, pp 340–353
Bentley JL (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517
Bhogal J, Macfarlane A, Smith P (2007) A review of ontology based query expansion. Inf Process Manag 43(4):866–886. ISSN 0306-4573
Bizer C, Lehmann J, Kobilarov G, Auer S, Becker C, Cyganiak R, Hellmann S (2009) DBpedia—a crystallization point for the web of data. J Web Semant Sci Serv Agents World Wide Web 7(3):154–165
Cai D, He X, Li Z, Ma W-Y, Wen J-R (2004) Hierarchical clustering of www image search results using visual, textual and link information. In: Proceedings of the annual ACM international conference on multimedia, pp 952–959. ISBN 1-58113-893-8
Chen C, Gagaudakis G, Rosin P (2000) Similarity-based image browsing. In: Proceedings of the IFIP international conference on intelligent information processing, pp 206–213
Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):1–60
Deselaers T, Gass T, Dreuw P, Ney H (2009) Jointy optimising relevance and diversity in image retrieval. In: Proceedings of the ACM international conference on image and video retrieval, pp 1–8
Efthimiadis EN (1996) Query expansion. Annu Rev Inf Syst Technol (ARIST) 31:121–187
Gabrilovich E, Markovitch S (2007) Computing semantic relatedness using Wikipedia-based explicit semantic analysis. In: Proceedings of the international joint conference on artificial intelligence, pp 1606–1611
Heesch D (2008) A survey of browsing models for content based image retrieval. Multimed Tools Appl 42(2): 261–284
Hoeber O (2008) Web information retrieval support systems: the future of web search. In: Proceedings of the IEEE/WIC/ACM international conference on web intelligence, workshops (international workshop on web information retrieval support systems), pp 29–32
Jansen BJ, Spink A, Pedersen J (2003) An analysis of multimedia searching on AltaVista. In: Proceedings of the ACM SIGMM international workshop on multimedia information retrieval, pp 186–192
Jing Y, Baluja S (2008) VisualRank: Applying PageRank to large-scale image search. IEEE Trans Pattern Anal Mach Intell 30(11):1877–1890
Joshi D, Datta R, Zhuang Z, Weiss WP, Friedenberg M, Li J, Wang JZ (2006) PARAgrab: a comprehensive architecture for web image management and multimodal querying. In: Proceedings of the international conference on very large databases, pp 1163–1166
Kherfi ML, Ziou D, Bernardi A (2004) Image retrieval from the world wide web: issues, techniques, and systems. ACM Comput Surv 36(1):35–67
Miller GA (1995) Wordnet: a lexical database for english. Commun ACM 38(11):39–41. ISSN 0001-0782
Milne D, Witten IH (2008a) An effective, low-cost measure of semantic relatedness obtained from Wikipedia links. In: Proceedings of the AAAI workshop on Wikipedia and artificial intelligence, pp 25–30
Milne D, Witten IH (2008b) Learning to link with Wikipedia. In: Proceedings of the ACM conference on information and knowledge management, pp 509–518
Milne D, Witten IH, Nichols DM (2007) A knowledge-based search engine powered by Wikipedia. In: Proceedings of the ACM conference on information and knowledge management, pp 445–454
Moëllic P-A, Haugeard J-E, Pitel G (2008) Image clustering based on a shared nearest neighbors approach for tagged collections. In: Proceedings of the international conference on content-based image and video retrieval, pp 269–278
Myoupo D, Popescu A, Borgne HL, Moëllic P-A (2009) Multimodal image retrieval over a large database. In: Proceedings of the international conference on cross-language evaluation forum: multimedia experiments
Snavely N, Seitz SM, Szeliski R (2006) Photo tourism: exploring photo collections in 3d. In: Proceedings of the ACM international conference on computer graphics and interactive techniques, pp 835–846
Strong G, Gong M (2008) Browsing a large collection of community photos based on similarity on GPU. In: Proceedings of the international symposium on advances in visual computing, pp 390–399
Strong G, Gong M (2009) Organizing and browsing photos using different feature vectors and their evaluations. In: Proceedings of the international conference on image and video retrieval, pp 1–8
Strong G, Hoeber O, Gong M (2010) Visual image browsing and exploration (Vibe): user evaluations of image search tasks. In: Proceedings of the international conference on active media technology, pp 424–435
Strube M, Ponzetto SP (2006) WikiRelate! computing semantic relatedness using Wikipedia. In: Proceedings of the AAAI conference on artificial intelligence, pp 1419–1424
Torres RS, Silva CG, Medeiros CB, Rocha HV (2003) Visual structures for image browsing. In: Proceedings of the international conference on information and knowledge management, pp 49–55
van Leuken RH, Garcia L, Olivares X, van Zwol R (2009) Visual diversification of image search results. In: Proceedings of the international conference on world wide web, pp 341–350
Wang S, Jing F, He J, Du Q, Zhang L (2007) Igroup: presenting web image search results in semantic clusters. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 587–596. ISBN 978-1-59593-593-9
Yao Y (2002) Information retrieval support systems. In: Proceedings of the 2002 IEEE world congress on computational intelligence, pp 1092–1097
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hoque, E., Hoeber, O., Strong, G. et al. Combining conceptual query expansion and visual search results exploration for web image retrieval. J Ambient Intell Human Comput 4, 389–400 (2013). https://doi.org/10.1007/s12652-011-0094-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-011-0094-7