Advertisement

World Wide Web

, Volume 15, Issue 3, pp 233–256 | Cite as

Automatic tagging by exploring tag information capability and correlation

  • Xiaoming Zhang
  • Zi Huang
  • Heng Tao Shen
  • Yang Yang
  • Zhoujun Li
Article

Abstract

Automatic tagging can automatically label images and videos with semantic tags to significantly facilitate multimedia search and organization. However, most of existing tagging algorithms often don’t differentiate between tags used to describe visual content, and neglect the semantic correlation of the assigned tag set. In this paper, we propose a novel automatic tagging algorithm which tags a test image or video with an Informative and Correlative Tag (ICTag) set. The assigned ICTag set can provide a more precise description of the multimedia object by exploring both the information capability of individual tags and the tag-to-set correlation. Measures to effectively estimate the information capability of individual tags and the correlation between a tag and the candidate tag set are designed. To reduce the computational complexity, we also introduce a heuristic method to achieve efficient automatic tagging. We conduct extensive experiments on the NUS-WIDE web image dataset downloaded from Flickr and the MCG-WEBV web video dataset downloaded from YouTube. The results confirm the efficiency and effectiveness of our proposed algorithm.

Keywords

automatic tagging information capability set correlation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ames, M., Naaman, M.: Why we tag: motivations for annotation in mobile and online media. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 971–980 (2007)Google Scholar
  2. 2.
    Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  3. 3.
    Cao, J., Zhang, Y.D., Song, Y.C., Chen, Z.N., Zhang, X., Li, J.T.: MCG-WEBV: A Benchmark Dataset for Web Video Analysis. Technical Report, ICT-MCG-09-001, Institute of Computing Technology (2009)Google Scholar
  4. 4.
    Chen, X.Y., Mu, Y.D., Yan, S.C., Chua, T.S.: Efficient large-scale image annotation by probabilistic collaborative multi-label propagation. In: Proceedings of ACM international Conference on Multimedia, pp. 35–44 (2010)Google Scholar
  5. 5.
    Chua, T., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: Nus-wide: a real-world web image database from National University of Singapore. In: Proceeding of ACM International Conference on Image and Video Retrieval (2009)Google Scholar
  6. 6.
    Cui, B., Tung, A.K., Zhang, C., Zhao, Z.: Multiple feature fusion for social media applications. In: Proceedings of the ACM International Conference on Management of Data, pp. 435–446 (2010)Google Scholar
  7. 7.
    Geng, B., Yang, L., Xu, C., Hua, X.: Collaborative learning for image and video annotation. In: Proceedings of ACM MIR 2008 (2008)Google Scholar
  8. 8.
    Hindle, A., Shao, J., Lin, D., Lu, J., Zhang, R.: Clustering Web video search results based on integration of multiple features. World Wide Web J. 14(1), 53–73 (2011)CrossRefGoogle Scholar
  9. 9.
    Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence&WordNet. In: Proceedings of 13th ACM International Conference on Multimedia, pp. 706–715 (2005)Google Scholar
  10. 10.
    Kennedy, L.S., Chang, S.F., Kozintsev, I.V.: To search or to label: predicting the performance of search-based automatic image classifiers. In: Proceedings of the 14th ACM International Workshop on Multimedia Information Retrieval, pp. 249–258 (2006)Google Scholar
  11. 11.
    Kilian, Q., Malcolm, S., Roelof, Z., Resolving tag ambiguity. In: Proceeding of the ACM International Conference On Multimedia, pp. 111–120 (2008)Google Scholar
  12. 12.
    Lei, W., Linjun, Y., Nenghai, Y., Xian-Sheng, H.: Learning to tag. In: Proceedings of the ACM International Conference On World Wide Web, pp. 20–24 (2009)Google Scholar
  13. 13.
    Lei, W., Steven, C.H., Jin, H.R., Jianke, Z., Nenghai, Y.: Distance metric learning from uncertain side information with application to automated photo tagging. In: Proceedings of ACM Multimedia (2009)Google Scholar
  14. 14.
    Li, H., Wang, Y., Zhang, D., Zhang, M., Chang, E.Y.: Pfp: parallel fp-growth for query recommendation. In: Proceedings of the 2nd ACM Conference on Recommender Systems, pp. 107–114 (2008)Google Scholar
  15. 15.
    Li, X., Snoek, C.G., Worring, M.: Learning social tag relevance by neighbor voting. IEEE Trans. Multimedia 11, 1310–1322 (2009)CrossRefGoogle Scholar
  16. 16.
    Li, X.R., Snoek, C.G.M., Worring, M.: Learning tag relevance by neighbor voting for social image retrieval. In: Proceeding of 1st ACM International Conference on Multimedia Information Retrieval, pp. 30–31 (2008)Google Scholar
  17. 17.
    Li, X., Snoek, C.G.M., Worring, M.: Learning social tag relevance by neighbor voting. IEEE Trans. Multimedia 11(7), 1310–1322 (2009)CrossRefGoogle Scholar
  18. 18.
    Li, X.R., Snoek, C.G.M., Worring, M.: Learning tag relevance by neighbor voting for social image retrieval. In: Proceeding of ACM International Conference on Multimedia Information Retrieval (2008)Google Scholar
  19. 19.
    Li, J., Wang, J.Z.: Real-time computerized annotation of pictures. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 985–1002 (2008)CrossRefGoogle Scholar
  20. 20.
    Liu, D., Hua, X.-S., Wang, M., Zhang, H.-J.: Image retagging. In: Proceedings of ACM Multimedia, pp. 491–500 (2010)Google Scholar
  21. 21.
    Liu, J., Li, M., Ma, W.-Y., Liu, Q., Lu, H.: An adaptive graph model for automatic image annotation. In: Proceedings of 14th ACM International Conference on Multimedia, pp. 61–70 (2006)Google Scholar
  22. 22.
    Liu, D., Wang, M., Hua, X.S., Zhang, H.J.: Tag Ranking. In: Proceeding of the ACM International Conference on World Wide Web, pp. 351–340 (2009)Google Scholar
  23. 23.
    Liu, J., Wang, B., Li, M.J., Li, Z.W., Ma, W.Y., Lu, H.Q., Ma, S.D.: Dual cross-media relevance model for image annotation. In: Proceedings of ACM Multimedia (2007)Google Scholar
  24. 24.
    Liu, D., Yan, S., Rui, Y., Zhang, H.-J.: Unified tag analysis with multi-edge graph. In: Proceedings of ACM Multimedia, pp. 25–34 (2010)Google Scholar
  25. 25.
    Lu, Y., Zhang, L., Tian, Q., Ma, W.-Y.: What are the high-level concepts with small semantic gaps? In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  26. 26.
    Mei, T., Wang, Y., Hua, X.-S., Gong, S., Li, S.: Coherent image annotation by learning semantic distance. In: Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  27. 27.
    Mishne, G.: AutoTag: a collaborative approach to automated tag assignment for weblog posts. In: Proceedings of the ACM international Conference on World Wide Web, pp. 953–954 (2006)Google Scholar
  28. 28.
    Moxley, E., Mei, T., Manjunath, B.S.: Video annotation through search and graph reinforcement mining. IEEE Trans. Multimedia 12(3), 184–193 (2010)CrossRefGoogle Scholar
  29. 29.
    Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., Zhang, H.-J.: Correlative multi-label video annotation. In: Proceedings of ACM Multimedia (2007)Google Scholar
  30. 30.
    Sarkas, N., Das, G., Koudas, N.: Improved search for socially annotated data. In: Proceeding of 35th International Conference on Very Large Data Bases, pp. 778–789 (2009)Google Scholar
  31. 31.
    Shen, Y., Fan, J.P.: Leveraging loosely-tagged images and inter-object correlations for tag recommendation. In: Proceedings of ACM Multimedia (2010)Google Scholar
  32. 32.
    Siersdorfer, S., San Pedro, J., Sanderson, M.: Automatic video tagging using content redundancy. In: Proceedings of ACM SIGIR (2009)Google Scholar
  33. 33.
    Sigurbjrnsson, B., Van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: Proceedings of 17th ACM International Conference of World Wide Web, pp. 327–336 (2008)Google Scholar
  34. 34.
    Wang, X.-J., Zhang, L., Li, X., Ma, W.-Y.: Annotating images by mining image search results. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 1919–1932 (2008)CrossRefGoogle Scholar
  35. 35.
    Wu, F., Han, Y.H., Tian, Q., Zhuang, Y.T.: Multi-label Boosting for image annotation by structural grouping sparsity. In: Proceedings of ACM Multimedia (2010)Google Scholar
  36. 36.
    Xiang, Y., Zhou, X.D., Liu, Z.T., T-Chua, S., Ngo, C-W.: Semantic context modeling with maximal margin conditional random fields for automatic image annotation. In: Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (2010)Google Scholar
  37. 37.
    Yan, R., Hauprmann, A.: Query expansion using probabilistic local feedback with application to multimedia retrieval. In: Proceedings of the 16th ACM Conference on Conference on information and Knowledge Management, pp. 361–370 (2007)Google Scholar
  38. 38.
    Yang, Y., Huang, Z., Shen, H.T., Zhou, X.F.: Mining multi-tag association for image tagging. World Wide Web J. 14(2), 133–156 (2011)CrossRefGoogle Scholar
  39. 39.
    Yang, Y., Yang, Y., Huang, Z., Shen, H.T., Nie, F.: Tag localization with spatial correlations and joint group sparsity. In: Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  40. 40.
    Yates, R.B., Neto, B.R.: Modern Information Retrieval. ACM Press (1999)Google Scholar
  41. 41.
    Xu, Z., Fu, Y., Mao, J., Su, D.: Towards the semantic web: collaborative tag suggestions. In: Proceedings of Collaborative Web Tagging Workshop at ACM International Conference On World Wide Web, pp. 56–65 (2006)Google Scholar
  42. 42.
    Zhou, X., Wang, M., Zhang, Q., Zhang, J., Shi, B.: Automatic image annotation by an iterative approach: incorporating keyword correlations and region matching. In: Proceedings of ACM CIVR (2007)Google Scholar
  43. 43.
    Zhu, Z.-H., Zhang, M.-L.: Multi-instance multi-label learning with application to scene classification. In: Proceedings of NIPS (2006)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Xiaoming Zhang
    • 1
    • 2
    • 3
  • Zi Huang
    • 4
  • Heng Tao Shen
    • 4
  • Yang Yang
    • 4
  • Zhoujun Li
    • 1
    • 2
    • 3
  1. 1.State Key Laboratory of Software Development EnvironmentBeihang UniversityBeijingChina
  2. 2.School of Computer Science and EngineeringBeihang UniversityBeijingChina
  3. 3.Beijing Key Laboratory of Network TechnologyBeiHang UniversityBeijingChina
  4. 4.School of Information Technology & Electrical EngineeringUniversity of QueenslandBrisbaneAustralia

Personalised recommendations