Automatic Image Semantic Annotation Based on Image-Keyword Document Model

  • Xiangdong Zhou
  • Lian Chen
  • Jianye Ye
  • Qi Zhang
  • Baile Shi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3568)


This paper presents a novel method of automatic image semantic annotation. Our approach is based on the Image-Keyword Document Model (IKDM) with image features discretization. According to IKDM, the image keyword annotation is conducted using image similarity measurement based on language model from text information retrieval domain. Through the experiments on a testing set of 5000 annotated images, our approach demonstrates great improvement of annotation performance compared with the known discretization-based image annotation model such as CMRM. Our approach also performs better in annotation time compared with the continuous model such as CRM.


Language Model Visual Feature Image Retrieval Visual Word Latent Dirichlet Allocation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Assfalg, J., Bertini, M., Colombo, C., Del Bimbo, A.: Semantic Annotation of Sports Videos. IEEE Multimedia (April-June 2002)Google Scholar
  2. 2.
    Barnard, K., Duygulu, P., Forsyth, D.: Clustering Art. In: Proceedings of IEEE ICPR (2001)Google Scholar
  3. 3.
    Blei, D., Jordan, M.I.: Modeling annotated data. In: Proc. of the 26th Intl. ACM SIGIR Conf., pp. 127–134 (2003)Google Scholar
  4. 4.
    Berman, A., Shapiro, L.G.: Efficient image retrieval with multiple distance measures. In: Storage and Retrieval for Image and Video Databases(SPIE), pp. 12–21 (1997)Google Scholar
  5. 5.
    Cusano, C., Ciocca, G., Schettini, R.: Image Annotation Using Svm. In: Proceedings of Internet imaging IV, vol. SPIE 5304 (2004)Google Scholar
  6. 6.
    Duygulu, P., Barnard, K., de Freitas, N., Forsyth, D.: Object recognition as machine translation:learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proc. 13th IJCAI, pp. 1022C–1027C (1993)Google Scholar
  8. 8.
    Fountain, S., Tan, T.: Content Based Annotation and Retrieval. In: RAIDER IRSG (1998)Google Scholar
  9. 9.
    Gupta, A., Weymouth, T.E., Jain, R.: Semantic queries with pictures: the VIMSYS model. In: VLDB, pp. 69–79 (1991)Google Scholar
  10. 10.
    Jaser, E., Kittler, J., Christmas, W.J.: Hierarchical Decision Making Scheme for Sports Video Categorisation with Temporal Post-Processing. In: CVPR, vol. II, pp. 908–913 (2004)Google Scholar
  11. 11.
    Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Proc. of 26th ACM SIGIR, pp. 119–126 (2003)Google Scholar
  12. 12.
    Jin, R., Chai, J., Si, L.: Effective Automatic Image Annotation Via A Coherent Language Model and Active Learning. In: Proc. of ACM Multimedia (2004)Google Scholar
  13. 13.
    Lavrenko, V., Manmatha, R., Jeon, J.: A Model for Learning the Semantics of Pictures. In: Proceedings of Advances in Neural Information Processing (2003)Google Scholar
  14. 14.
    Zhang, L., Chen, L., Li, M., Zhang, H.: Automated annotation of human faces in family albums. In: Proc. of ACM Multimedia, pp. 355–358 (2003)Google Scholar
  15. 15.
    Mori, Y., Takahashi, H., Oka, R.: Image-to-word transformation based on dividing and vector quantizing images with words. In: Proc. of MISRM (1999)Google Scholar
  16. 16.
    Muller, H., Muller, W., Marchand-Maillet, S., Pun, T., Squire, D.: Strategies for Positive and Negative Relevance Feedback in Image Retrieval. In: ICPR, pp. 5043–5042 (2000)Google Scholar
  17. 17.
    Lew, M., Sebe, N., Eakins, J.: Challenges of image and video retrieval. In: Lew, M., Sebe, N., Eakins, J.P. (eds.) CIVR 2002. LNCS, vol. 2383, pp. 1–6. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. 18.
    Monay, F., Gatica-Perez, D.: On Image Auto- Annotation with Latent Space Models. In: Proceedings of ACM Multimedia Conf. (2003)Google Scholar
  19. 19.
    Naphade, M.R., Kozintsev, I.V., Huang, T.S.: A Factor Graph Framework for Semantic Video Inexing. IEEE Trans. on Circuits and Systems for Video Technology 12(1) (2002)Google Scholar
  20. 20.
    Wang, W., Zhang, A.: Evaluation of low-level features by decisive feature patterns. In: Proc. of IEEE ICME (2004)Google Scholar
  21. 21.
    Rui, Y., Huang, T.S.: A novel relevance feedback technique in image retrieval. In: Proc. of the 7th ACM Int.Conf. on Multimedia, pp. 67–70 (1999)Google Scholar
  22. 22.
    Smith, J.R., Chang, S.-F.: VisualSEEk: A Fully Automated Content-Based Image Query System. In: Proc. of ACM Multimedia, pp. 87–98 (1996)Google Scholar
  23. 23.
    Tao, J.L., Hung, Y.P.: A bayesian method for content-based image retrieval by use of relevance feedback. In: Chang, S.-K., Chen, Z., Lee, S.-Y. (eds.) VISUAL 2002. LNCS, vol. 2314, pp. 76–87. Springer, Heidelberg (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Xiangdong Zhou
    • 1
  • Lian Chen
    • 1
  • Jianye Ye
    • 1
  • Qi Zhang
    • 2
  • Baile Shi
    • 1
  1. 1.Department of Computing and Information TechnologyFudan University ShanghaiChina
  2. 2.Department of Computer ScienceUniversity of North Carolina at Chapel Hill 

Personalised recommendations