Improving Image Retrieval Using Semantic Resources

  • Adrian Popescu
  • Gregory Grefenstette
  • Pierre-Alain Moellic
Part of the Studies in Computational Intelligence book series (SCI, volume 93)


Many people use the Internet to find pictures of things. When extraneous images appear in response to simple queries on a search engine, the user has a hard time understanding why his seemingly clear request was not properly satisfied. If the computer could only understand what he wanted better, then maybe the results would be more precise. The introduction of an ontology, though hidden from the user, into current image retrieval engines may provide more accurate image responses to his query. The improvement of the results translates into the possibility of offering structured results, to disambiguate queries and to provide more interactivity options to the user, transforming the current string of character based retrieval into a concept based process. Each one of these aspects is presented and examples are used to support our proposals. We equally discuss the notion of picturability and justify our choice to work exclusively with entities that can be directly represented in a picture. Coordinating the use of a lexical ontology (an OWL representation of WordNet) with image processing techniques, we have developed a system that, given an initial query, automatically returns images associated with the query using automatic reformulation (each concepts is represented by its deepest hyponyms from the ontology). We show that picking randomly from this new set of pictures provides an improved representation for the initial, more general query. We also treat the visual aspects of the images for these deepest hyponyms (the leaves of WordNet). The depictions associated to leaf categories are clustered into coherent sets using low-level image features like color and texture. Some limitations (e.g. the quality and coverage of the semantic structure, the impossibility to answer complex queries) of the ontology based retrieval are equally discussed.


Search Engine Image Retrieval Physical Entity Image Cluster Image Retrieval System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barnard K and Forsyth D (2001) Learning the Semantic of Words and Pictures. In: Proc. of ICCV 2001, Vancouver, Canada: 408–415Google Scholar
  2. 2.
    Cimiano P, Handschuh S and Staab S (2004) Towards the Self-Annotating Web. In: Proc. of WWW 2004, Manhattan, NY: 462–471Google Scholar
  3. 3.
    Doulaverakis C, Nidekou E, Gounaris A and Kompatsiaris Y (2006) A Hybrid Ontology and Content-Based Search Engine for Multimedia Retrieval. In: Proc. of the 10th East - European Conference on Advances in Databases and Information Systems, ADBIS 2006, Thessalonki, HellasGoogle Scholar
  4. 4.
    Eco U (1997) The Search for the Perfect Language. Blackwell PublishersGoogle Scholar
  5. 5.
    Ertoz L, Steibach M and Kumar V (2003) Finding Topics in Collections of Documents. A Shared Nearest Neighbor Approach. In: Wu W, Xiong H and Shekar S (eds) Clustering and Information Retrieval, KluwerGoogle Scholar
  6. 6.
    Gangemi A, Navigli R and Velardi P (2003) The OntoWordNet Project: Extension and Axiomatisation of Conceptual Relations in WordNet. In: Proc. of CoopIS/DOA/ODBASE, Catania, Sicily, Italy: 689–706Google Scholar
  7. 7.
    Goodrum A and Spink A (2001) Image Searching on the Excite Web Search Engine. International Journal of Information Processing and Management 37, 2: 295–311zbMATHCrossRefGoogle Scholar
  8. 8.
    Guha R V and Lenat D B (1990) Cyc: A Midterm Report, AI Magazine 11, 3:32–59Google Scholar
  9. 9.
    Hollink L (2006) Semantic Annotation for Retrieval of Visual Resources. Vrije Universiteit AmsterdamGoogle Scholar
  10. 10.
    Keil F C (1992) Concepts, Kinds, and Conceptual Development. Bradford BooksGoogle Scholar
  11. 11.
    Kuo C H, Huang Y T, Lan Y H and Chou T C (2004) Building Semantic Indexing for Image Retrieval Systems. In: Proc. of International Computer Symposium, Taipein, Taiwan: 208–213Google Scholar
  12. 12.
    Liao S P, Cheng P J, Chen R C and Chien L F (2005) LiveImage: Organizing Web Images by Relevant Concept. In: Proc. of the Workshop on the Science of the Artificial Hualien, Taiwan: 210–220Google Scholar
  13. 13.
    Liu H and Singh P ConceptNet (2004) A Practical Commonsense Reasoning Toolkit. BT Technology Journal, Kluwer Academic 22, 4: 211–226Google Scholar
  14. 14.
    Miller G A (1990) Nouns in WordNet: A Lexical Inheritance System. International Journal of Lexicography 3, 4: 245–264CrossRefGoogle Scholar
  15. 15.
    Missikoff M, Navigli R and Velardi P (2002) Integrated Approach to Web Ontology Learning and Engineering. IEEE Computer, 35(11): 60–63Google Scholar
  16. 16.
    Pastra K (2006) Image - Language Association: Are We Looking at the Right Features?. In: Proc. of the Workshop on Language Resources for Content-based Image Retrieval, LREC 2006, Genoa, Italy: 40–44Google Scholar
  17. 17.
    Petridis K, Bloehdorn S, Saathoff C, Simou N, Dasiopoulou S, Tzouvaras V, Handschuh S, Avrithis Y, Kompatsiaris Y and Staab S (2006) Knowledge Representation and Semantic Annotation of Multimedia Content. IEEE Proceedings on Vision, Image and Signal Processing, 153/32: 55–262Google Scholar
  18. 18.
    Pianta E, Bentivogli L and Girardi C (2002) MultiWordNet: Developing an Aligned Multilingual Database. In: Proc. of the 1st International Conference on Global WordNet, Mysore, India: 293–302Google Scholar
  19. 19.
    Popescu A, Grefenstette G and Moellic P A (2007) Image Retrieval Using a Multilingual Ontology. accepted for RIAO2007, Pittsburgh, USAGoogle Scholar
  20. 20.
    Rosch E, Mervis C B, Gray W D, Johnson D M and Boyes-Braem P (1976) Basic Objects in Natural Categories. Cognitive Psychology, 8: 382–439CrossRefGoogle Scholar
  21. 21.
    Stehling R O, Nascimento M A and Falcao A X (2002) Compact and Efficient Image Retrieval Approach Based on Border/Interior Pixel Classification. In: Proc. of CKIM 2002, Mc Lean, USA: 102–109Google Scholar
  22. 22.
    van Assem M, Gangemi A and Schreiber G (2006) RDF/OWL Representation of WordNet.
  23. 23.
    W3C (2004) OWL Web Ontology Language Overview.
  24. 24.
    Wang X J, Ma W Y and Li X (2004) Data-driven Approach for Bridging the Cognitive Gap in Image Retrieval. In: Proc. of ICME 2004, Taipei, Taiwan: 2231–2234Google Scholar
  25. 25.
    Wang H, Liu S and Chia L T (2006) Does Ontology Help in Image Retrieval? - A comparison between Keyword, Text Ontology and Multi-Modality Ontology Approaches. In: Proc. of ACM Multimedia, Santa Barbara, CA: 109–112Google Scholar
  26. 26.
    Yang J, Liu W, Zhang H and Zhuang Y (2001) Thesaurus-Aided Approach for Image Browsing and Retrieval. In: IEEE Conference on Multimedia and Expo, Tokyo, JapanGoogle Scholar
  27. 27.
    Zinger S, Millet C, Mathieu B, Grefenstette G, Hede P and Moellic P A (2006) Clustering and Semantically Filtering Web Images to Create a Large Scale Image Ontology. In: Proc. of IS&T/SPIE 18th Symposium Electronic Imaging, San Jose, California, CAGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Adrian Popescu
    • 1
  • Gregory Grefenstette
    • 1
  • Pierre-Alain Moellic
    • 1
  1. 1.CEA LIST-LIC2MFrance

Personalised recommendations