Naming of Image Regions for User-Friendly Image Retrieval

  • Andrea Kutics
  • Akihiko Nakagawa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4141)


This paper presents a novel method for facilitating user-friendly image retrieval by attaching names to image regions. We first detect only the most prominent regions in images when such entities exist, using our own nonlinear image segmentation technique. Besides their visual features, the layout and relations between selected regions are also emphasized. Next, we apply an adaptive and multi-modal classification and naming of image regions using subsequent clustering methods to the features of the regions and related words as well as relevancy information. For both the naming and the testing, we have added a set of illustrations acting as abstract prototypes of the regions to randomly selected natural images. Experiments on 20,000 natural images show the efficacy of using this multilayer region naming model as well as of extensively interacting with users, enabling them to present their queries by a combination of region names, sketches and example images or regions.


Visual Feature Image Retrieval Image Region Natural Image Vector Quantization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smeulders, A., et al.: Content-based image retrieval at the end of the early years. IEEE Trans. PAMI 22(12), 1349–1380 (2000)Google Scholar
  2. 2.
    Zhou, X.S., Huang, T.S.: Unifying Keywords and Visual Contents in Image Retrieval. IEEE Multimedia 9(2), 23–33 (2002)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Zhao, R., Grosky, W.I.: Negotiating the semantic gap: from feature maps to semantic landscapes. Pattern Recognition 35, 593–600 (2002)CrossRefzbMATHGoogle Scholar
  4. 4.
    Barnard, K., et al.: Matching Words and Pictures. Journal of Machine Learning Research 3, 1107–1135 (2003)CrossRefzbMATHGoogle Scholar
  5. 5.
    Hofmann, T.: Learning and Representing Topic. A Hierarchical Mixture Model for Word Occurrences in Document Databases. In: Proc. of CONALD, Pittsburgh (1998)Google Scholar
  6. 6.
    Wang, J.Z., Li, J.: Learning-based linguistic indexing of pictures with 2-D MHMMs. In: Proc. ACM Multimedia, pp. 436–445 (2002)Google Scholar
  7. 7.
    Lim, J.-H., Tian, Q., Mulhem, P.: Home Photo Content Modeling for Personalized Event-Based Retrieval. IEEE Multimedia 9(2), 28–37 (2003)Google Scholar
  8. 8.
    Carbonetto, P., de Freitas, N., Barnard, K.: A Statistical Model for General Contextual Object Recognition. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 350–362. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Gabrilovich, E., Markovitch, S.: Feature Generation for Text Categorization Using World Knowledge. In: Proc. of The 19th International Joint Conf. for Artificial Intelligence (2005)Google Scholar
  10. 10.
    Metzler, D., Manmatha, R.: An inference network approach to image retrieval. In: Enser, P.G.B., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A.F., Smeulders, A.W.M. (eds.) CIVR 2004. LNCS, vol. 3115, pp. 42–50. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Kutics, A., Nakagawa, A.: Detecting Prominent Objects for Image Retrieval. In: IEEE International Conference on ICIP, vol. 3, pp. 445–448 (2005)Google Scholar
  12. 12.
    Hendee, W.R., Wells, P.N.T.: The Perception of Visual Information. Springer, Heidelberg (1997)zbMATHGoogle Scholar
  13. 13.
    Mojsilovic, A.: A method for color naming and description of color composition in images. In: Proc. of the ICIP (2002)Google Scholar
  14. 14.
    Bhusnan, N., Rao, A.R., Lohse, G.L.: The texture lexicon: Understanding the categorization of visual texture terms and their relationship to texture images. Cognitive Science 21(2), 219–246 (1997)CrossRefGoogle Scholar
  15. 15.
    Hanbury, A.: MUSCLE, Guide to annotation, Version 2.0, Tech. Univ. of Vienna (2005)Google Scholar
  16. 16.
    Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Andrea Kutics
    • 1
  • Akihiko Nakagawa
    • 2
  1. 1.Tokyo University of TechnologyTokyoJapan
  2. 2.University of Electro-CommunicationsTokyoJapan

Personalised recommendations