World Wide Web

, Volume 14, Issue 2, pp 133–156 | Cite as

Mining multi-tag association for image tagging

  • Yang Yang
  • Zi Huang
  • Heng Tao ShenEmail author
  • Xiaofang Zhou


Automatic media tagging plays a critical role in modern tag-based media retrieval systems. Existing tagging schemes mostly perform tag assignment based on community contributed media resources, where the tags are provided by users interactively. However, such social resources usually contain dirty and incomplete tags, which severely limit the performance of these tagging methods. In this paper, we propose a novel automatic image tagging method aiming to automatically discover more complete tags associated with information importance for test images. Given an image dataset, all the near-duplicate clusters are discovered. For each near-duplicate cluster, all the tags occurring in the cluster form the cluster’s “document”. Given a test image, we firstly initialize the candidate tag set from its near-duplicate cluster’s document. The candidate tag set is then expanded by considering the implicit multi-tag associations mined from all the clusters’ documents, where each cluster’s document is regarded as a transaction. To further reduce noisy tags, a visual relevance score is also computed for each candidate tag to the test image based on a new tag model. Tags with very low scores can be removed from the final tag set. Extensive experiments conducted on a real-world web image dataset—NUS-WIDE, demonstrate the promising effectiveness of our approach.


image tagging tag completion tag denoising weighted association rule mining 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, R., Imieliński, T. Swami, A.: Mining association rules between sets of items in large databases. SIGMOD Rec. 22(2), 207–216 (1993)CrossRefGoogle Scholar
  2. 2.
    Ames, M., Naaman, M.: Why we tag: motivations for annotation in mobile and online media. In: SIGCHI, pp. 971–980 (2007)Google Scholar
  3. 3.
    Amir, A., Argillander, J., Campbell, M., Haubold, A., Iyengar, G., Ebadollahi, S., Kang, F., M. Naphade, R., Natsev, A., Smith, J.R., Tei, J., Volkmer, T.: Ibm research trecvid-2005 video retrieval system. In: TREC Video Retrieval Evaluation Proceedings (2006)Google Scholar
  4. 4.
    Bailloeul, T., Zhu, C., Xu, Y.: Automatic image tagging as a random walk with priors on the canonical correlation subspace. In: MIR ’08: Proceeding of the 1st ACM International Conference on Multimedia Information Retrieval, pp. 75–82. ACM, New York (2008)CrossRefGoogle Scholar
  5. 5.
    Cao, L., Yu, J., Luo, J., Huang, T.S.: Enhancing semantic and geographic annotation of web images via logistic canonical correlation regression. In: MM ’09: Proceedings of the Seventeen ACM International Conference on Multimedia, pp. 125–134, ACM, New York (2009)CrossRefGoogle Scholar
  6. 6.
    Chang, C.-C., Lin, C.-J.: LIBSVM: A Library for Support Vector Machines (2001)Google Scholar
  7. 7.
    Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: Nus-wide: a real-world web image database from national university of Singapore. In: CIVR, pp. 1–9 (2009)Google Scholar
  8. 8.
    Guan, Z., Bu, J., Mei, Q., Chen, C., Wang, C.: Personalized tag recommendation using graph-based ranking on multi-type interrelated objects. In: SIGIR, pp. 540–547 (2009)Google Scholar
  9. 9.
    Han, Y.: Multi-label boosting for image annotation by structural grouping sparsity. In: ACM Multimedia (2010)Google Scholar
  10. 10.
    Heymann, P., Ramage, D., Garcia-Molina, H.: Social tag prediction. In: SIGIR, pp. 531–538 (2008)Google Scholar
  11. 11.
    Hua, X.-S., Qi, G.-J.: Online multi-label active annotation: towards large-scale content-based video search. In: ACM Multimedia, pp. 141–150 (2008)Google Scholar
  12. 12.
    Kennedy, L.S., Chang, S.-F., Kozintsev, I.V.: To search or to label?: predicting the performance of search-based automatic image classifiers. In: MIR, pp. 249–258 (2006)Google Scholar
  13. 13.
    Krestel, R., Fankhauser, P., Nejdl, W.: Latent dirichlet allocation for tag recommendation. In: RecSys, pp. 61–68 (2009)Google Scholar
  14. 14.
    Li, X., Snoek, C., Worring, M.: Learning social tag relevance by neighbor voting. IEEE Trans. Multimedia 11(7), 1310–1322 (2009)CrossRefGoogle Scholar
  15. 15.
    Liu, D., Hua, X., Zhang, H.-J.: Image retagging. In: ACM Multimedia (2010)Google Scholar
  16. 16.
    Liu, D., Hua, X.-S., Yang, L., Wang, M., Zhang, H.-J.: Tag ranking. In: WWW, pp. 351–360 (2009)Google Scholar
  17. 17.
    Liu, X., Cheng, B., Yan, S., Tang, J., Chua, T.S., Jin, H., Label to region by bi-layer sparsity priors. In: MM ’09: Proceedings of the Seventeen ACM International Conference on Multimedia, pp. 115–124, ACM, New York (2009)CrossRefGoogle Scholar
  18. 18.
    Liu, Y., Wu, F., Zhuang, Y., Xiao, J.: Active post-refined multimodality video semantic concept detection with tensor representation. In: ACM Multimedia, pp. 91–100 (2008)Google Scholar
  19. 19.
    Mei, T., Wang, Y., Hua, X.-S., Gong, S., Li, S.: Coherent Image Annotation by Learning Semantic Distance (2008)Google Scholar
  20. 20.
    Moxley, E., Mei, T., Manjunath, B.: Video annotation through search and graph reinforcement mining. IEEE Trans. Multimedia 12(3), 184–193 (2010)CrossRefGoogle Scholar
  21. 21.
    Noh, T.-G., Park, S.-B., Yoon, H.-G., Lee, S.-J., Park, S.-Y.: An automatic translation of tags for multimedia contents using folksonomy networks. In: SIGIR, pp. 492–499 (2009)Google Scholar
  22. 22.
    Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Mei, T., Zhang, H.J.: Correlative multi-label video annotation. In: ACM Multimedia, pp. 17–26. New York (2007)Google Scholar
  23. 23.
    Rui, X., Li, M., Li, Z., Ma, W.-Y., Yu, N.: Bipartite graph reinforcement model for web image annotation. In: ACM Multimedia, pp. 585–594 (2007)Google Scholar
  24. 24.
    Siersdorfer, S., San Pedro, J., Sanderson, M.: Automatic video tagging using content redundancy. In: SIGIR, pp. 395–402 (2009)Google Scholar
  25. 25.
    Sigurbjörnsson, B., van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: WWW, pp. 327–336 (2008)Google Scholar
  26. 26.
    Sun, K., Bai, F.: Mining weighted association rules without preassigned weights. IEEE Trans. Knowl. Data Eng. 20(4), 489–495 (2008)CrossRefGoogle Scholar
  27. 27.
    Tang, J., Hua, X.-S., Qi, G.-J., Song, Y., Wu, X.: Video annotation based on kernel linear neighborhood propagation. IEEE Trans Multimedia 10(4), 620–628 (2008)CrossRefGoogle Scholar
  28. 28.
    Tang, J., Yan, S., Hong, R., Qi, G.-J., Chua, T.-S.: Inferring semantic concepts from community-contributed images and noisy tags. In: ACM Multimedia, pp. 223–232 (2009)Google Scholar
  29. 29.
    Wang, C., Jing, F., Zhang, L., Zhang, H.-J.: Image annotation refinement using random walk with restarts. In: ACM Multimedia, pp. 647–650 (2006)Google Scholar
  30. 30.
    Wang, C., Yan, S., Zhang, L., Zhang, H.-J., Multi-label sparse coding for automatic image annotation. In: Proceedings of IEEE Int. Conf. Computer Vision and Pattern Recognition, pp. 1643–1650. Florida, USA (2009)Google Scholar
  31. 31.
    Wang, F., Ding, C.H.Q., Li, T.: Integrated kl (k-means—laplacian) clustering: a new clustering approach by combining attribute data and pairwise relations. In: SDM, pp. 38–48 (2009)Google Scholar
  32. 32.
    Wang, N., Parthasarathy, S., Tan, K.-L., Tung, A.K.H.: Csv: visualizing and mining cohesive subgraphs. In: SIGMOD, pp. 445–458 (2008)Google Scholar
  33. 33.
    Wang, X.-J., Zhang, L., Li, X., Ma, W.-Y.: Annotating images by mining image search results. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 1919–1932 (2008)CrossRefGoogle Scholar
  34. 34.
    Weinberger, K.Q., Slaney, M., Van Zwol, R.: Resolving tag ambiguity. In: ACM Multimedia, pp. 111–120 (2008)Google Scholar
  35. 35.
    Wu, L., Hoi, S.C., Jin, R., Zhu, J., Yu, N.: Distance metric learning from uncertain side information with application to automated photo tagging. In: MM ’09: Proceedings of the Seventeen ACM International Conference on Multimedia, pp. 135–144, ACM, New York (2009)CrossRefGoogle Scholar
  36. 36.
    Wu, L., Yang, L., Yu, N., Hua, X.-S.: Learning to tag. In: WWW, pp. 361–370 (2009)Google Scholar
  37. 37.
    Xu, Z., Fu, Y., Mao, J., Su, D.: Towards the semantic web: collaborative tag suggestions. In: Collaborative Web Tagging Workshop. Edinburgh, Scotland (2006)Google Scholar
  38. 38.
    Yang, Y., Xu, D., Nie, F., Luo, J., Zhuang, Y.: Ranking with local regression and global alignment for cross media retrieval. In: ACM Multimedia (2009)Google Scholar
  39. 39.
    Yang, Y., Xu, D., Nie, F., Yan, S., Zhuang, Y.: Image clustering using local discriminant models and global IEEE Trans. Image Process. 19(10), 2761–2773 (2010)CrossRefGoogle Scholar
  40. 40.
    Yang, Y., Zhuang, Y.-T., Wu, F., Pan, Y.-H., Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Trans Multimedia 10(3), 437–446 (2008)CrossRefGoogle Scholar
  41. 41.
    Yin, Z., Li, R., Mei, Q., Han, J.: Exploring social tagging graph for web object classification. In: KDD, pp. 957–966 (2009)Google Scholar
  42. 42.
    Yuan, X., Hua, X.-S., Wang, M., Wu, X.: Manifold-ranking based video concept detection on large database and feature pool. In: ACM Multimedia, pp. 623–626 (2006)Google Scholar
  43. 43.
    Zha, Z.-J., Yang, L., Mei, T., Wang, M., Wang, Z.: Visual query suggestion. In: MM ’09: Proceedings of the Seventeen ACM International Conference on Multimedia. ACM (2009)Google Scholar
  44. 44.
    Zhang, S., Huang, J., Huang, Y., Yu, Y., Li, H., Metaxas, D.N.: Automatic image annotation using group sparsity. In IEEE Conference on Computer Vision and Pattern Recognition, 2010. CVPR 2010 (2010)Google Scholar
  45. 45.
    Zhao, W., Ngo, C.-W.: Scale-rotation invariant pattern entropy for keypoint-based near-duplicate detection. IEEE Trans. Image Process. 18(2), 412–423 (2009)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Yang Yang
    • 1
  • Zi Huang
    • 1
  • Heng Tao Shen
    • 1
    Email author
  • Xiaofang Zhou
    • 1
  1. 1.School of Information Technology & Electrical EngineeringThe University of QueenslandBrisbaneAustralia

Personalised recommendations