Advertisement

Discovering Phrase-Level Lexicon for Image Annotation

  • Lei Yu
  • Jing Liu
  • Changsheng Xu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6297)

Abstract

In image annotation, the annotation words are expected to represent image content at both visual level and semantic level. However, a single word sometimes is ambiguous in annotation, for example, ”apple” may refer to a fruit or a company. However, when ”apple” combines with ”phone” or ”fruit”, it will be more semantically and visually consistent. In this paper, we attempt to find this kind of combination and construct a less ambiguous phrase-level lexicon for annotation. First, concept-based image search is conducted to obtain a semantically consistent image set (SC-IS). Then, a hierarchical clustering algorithm is adopted to visually cluster the images in SC-IS to obtain a semantically and visually specific image set (SVC-IS). Finally, we apply a frequent itemset mining in SVC-IS to construct the phrase-level lexicon and associate the lexicon into a probabilistic annotation framework to estimate annotation words of any untagged images. Our experimental results show that the discovered phrase-level lexicon is able to improve the annotation performance.

Keywords

phrase-level lexicon image annotation word correlation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cusano, C., Ciocca, G., Schettini, R.: Image annotation using SVM. In: Proceedings of Internet imaging IV, SPIE, vol. 5304, pp. 330–338 (2004) (Citeseer)Google Scholar
  2. 2.
    Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures (2003) (Citeseer)Google Scholar
  3. 3.
    Wang, X., Zhang, L., Li, X., Ma, W.: Annotating images by mining image search results. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(11), 1919–1932 (2008)CrossRefGoogle Scholar
  4. 4.
    Lu, Y., Zhang, L., Tian, Q., Ma, W.: What are the high-level concepts with small semantic gaps? In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008)Google Scholar
  5. 5.
    Sun, A., Bhowmick, S.: Image tag clarity: in search of visual-representative tags for social images. In: Proceedings of the first SIGMM workshop on Social media, pp. 19–26. ACM, New York (2009)CrossRefGoogle Scholar
  6. 6.
    Weinberger, K., Slaney, M., Van Zwol, R.: Resolving tag ambiguity. In: Proceeding of the 16th ACM international conference on Multimedia, pp. 111–120. ACM, New York (2008)CrossRefGoogle Scholar
  7. 7.
    Wang, C., Jing, F., Zhang, L., Zhang, H.: Content-based image annotation refinement. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8 (2007)Google Scholar
  8. 8.
    Li, J., Wang, J.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(9), 1075–1088 (2003)CrossRefGoogle Scholar
  9. 9.
    Blei, D., Jordan, M.: Modeling annotated data. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 127–134. ACM, New York (2003)Google Scholar
  10. 10.
    Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, p. 126. ACM, New York (2003)Google Scholar
  11. 11.
    Feng, S., Manmatha, R., Lavrenko, V.: Multiple bernoulli relevance models for image and video annotation. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2 (2004)Google Scholar
  12. 12.
    Li, X., Chen, L., Zhang, L., Lin, F., Ma, W.: Image annotation by large-scale content-based image retrieval. In: Proceedings of the 14th annual ACM international conference on Multimedia, p. 610. ACM, New York (2006)Google Scholar
  13. 13.
    Wang, X., Zhang, L., Jing, F., Ma, W.: Annosearch: Image auto-annotation by search. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2 (2006)Google Scholar
  14. 14.
    Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence & wordNet. In: Proceedings of the 13th annual ACM international conference on Multimedia, pp. 706–715. ACM, New York (2005)CrossRefGoogle Scholar
  15. 15.
    Wang, C., Jing, F., Zhang, L., Zhang, H.: Image annotation refinement using random walk with restarts. In: Proceedings of the 14th annual ACM international conference on Multimedia, p. 650. ACM, New York (2006)Google Scholar
  16. 16.
    Wang, Y., Gong, S.: Refining image annotation using contextual relations between words. In: Proceedings of the 6th ACM international conference on Image and video retrieval, p. 432. ACM, New York (2007)Google Scholar
  17. 17.
    Jia, J., Yu, N., Rui, X., Li, M.: Multi-graph similarity reinforcement for image annotation refinement. In: 15th IEEE International Conference on Image Processing, ICIP 2008, pp. 993–996 (2008)Google Scholar
  18. 18.
    Liu, D., Hua, X., Yang, L., Wang, M., Zhang, H.: Tag ranking. In: Proceedings of the 18th international conference on World wide web, pp. 351–360. ACM, New York (2009)CrossRefGoogle Scholar
  19. 19.
    Xu, D., Chang, S.: Video event recognition using kernel methods with multilevel temporal alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(11), 1985–1997 (2008)CrossRefGoogle Scholar
  20. 20.
    Han, J., Kamber, M.: Data mining: concepts and techniques. Morgan Kaufmann, San Francisco (2006)Google Scholar
  21. 21.
    Duygulu, P., Barnard, K., De Freitas, J., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 349–354. Springer, Heidelberg (2002)Google Scholar
  22. 22.
    Liu, J., Wang, B., Lu, H., Ma, S.: A graph-based image annotation framework. Pattern Recognition Letters 29(4), 407–415 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Lei Yu
    • 1
  • Jing Liu
    • 1
  • Changsheng Xu
    • 1
    • 2
  1. 1.Institute of AutomationChinese Academy of ScienceBeijingChina
  2. 2.China-Singapore Institute of Digital MediaSingapore

Personalised recommendations