Image Sense Classification in Text-Based Image Retrieval

  • Yih-Chen Chang
  • Hsin-Hsi Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5839)


An image sense is a graphic representation of a concept denoted by a (set of) term(s). This paper proposes algorithms to find image senses for a concept, collect the sense descriptions, and employ them to disambiguate the image senses in text-based image retrieval. In the experiments on 10 ambiguous terms, 97.12% of image senses returned by a search engine are covered. The average precision of sample images is 68.26%. We propose four kinds of classifiers using text, image, URL, and expanded text features, respectively, and a merge strategy to combine the results of these classifiers. The merge classifier achieves 0.3974 in F-measure (β=0.5), which is much better than the baseline and has 51.61% of human performance.


Image sense disambiguation Text-based image retrieval Word sense disambiguation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cai, D., He, X., Li, Z., Ma, W.Y., Wen, J.R.: Hierarchical Clustering of WWW Image Search Results Using Visual, Textual and Link Information. In: The 12th Annual ACM International Conference on Multimedia, pp. 952–959 (2004)Google Scholar
  2. 2.
    Loeff, N., Alm, C.O., Forsyth, D.A.: Discriminating Image Senses by Clustering with Multimodal Features. In: The COLING/ACL on Main Conference Poster Sessions, pp. 547–554 (2006)Google Scholar
  3. 3.
    Zinger, S., Millet, C., Mathieu, B., Grefenstette, G., Hède, P., Moëllic, P.A.: Extracting an Ontology of Portrayable Objects from WordNet. In: The 1st MUSCLE/ImageCLEF Workshop on Image and Video Retrieval Evaluation (2005)Google Scholar
  4. 4.
    Fluhr, C., Grefenstette, G., Popescu, A.: Toward a Common Semantics between Media and Languages. In: The 2006 International Workshop on Research Issues in Digital Libraries (2006)Google Scholar
  5. 5.
    Chang, Y.C., Chen, H.H.: Approaches of Using Word-Image Ontology and an Annotated Image Corpus as Intermedia for Cross-Language Image Retrieval. In: Peters, C., Clough, P., Gey, F.C., Karlgren, J., Magnini, B., Oard, D.W., de Rijke, M., Stempfhuber, M. (eds.) CLEF 2006. LNCS, vol. 4730, pp. 625–632. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  6. 6.
    Yarowsky, D.: Unsupervised Word Sense Disambiguation Rivaling Supervised Methods. In: ACL, pp. 189–196 (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yih-Chen Chang
    • 1
  • Hsin-Hsi Chen
    • 1
  1. 1.Department of Computer Science and Information EngineeringNational Taiwan UniversityTaipeiTaiwan

Personalised recommendations