Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 3, pp 3659–3676 | Cite as

Social image tag enrichment based on textual similarity modeling

  • Miao ShenEmail author
Article
First Online:

Abstract

In social image sharing websites, users provide several descriptive tags to annotate their shared images. Usually, the user annotated tags are noisy, biased and incomplete. How to improve tag quality is very important for tag based applications. The content relevant tags have certain similarities or connections with each other. Thus from some highly relevant tags, we can infer the other content relevant tags for an image. In this paper, a social image tag enrichment approach is proposed. Considering the diversity of content relevant tags for the image, we first determine some seed tags which are highly relevant to image content and cover wide range of semantics. Then the seed tags are utilized to adopt semantic similarity tags for the input image. Experiments demonstrate the effectiveness of the proposed approach.

Keywords

Tag enrichment Tag ranking Flickr Image Annotation Social image Social Media 
This is a preview of subscription content, log in to check access.

Supplementary material

11042_2017_5184_MOESM1_ESM.pdf (73 kb)
ESM 1 (PDF 73 kb)

References

  1. 1.
    Ames M, Naaman M (2007) Why We Tag: Motivations for Annotation in Mobile and Online Media. In Proc. SIGCHI Conference on Human Factors in Computing SystemGoogle Scholar
  2. 2.
    Chang X, Yang Y (2016) Semi-supervised Feature Analysis by Mining Correlations among Multiple Tasks. IEEE Trans Neural Netw Learn Syst.  https://doi.org/10.1109/TNNLS.2016.2582746 MathSciNetCrossRefGoogle Scholar
  3. 3.
    Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2016) Compound Rank-k Projections for Bilinear Analysis. IEEE Trans Neural Netw Learn Syst 27(7):1502–1513MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chang X, Nie F, Yang Y, Zhang C, Huang H (2016) Convex Sparse PCA for Unsupervised Feature Analysis. ACM Trans Knowl Discov Data 11(1):3:1–3:16CrossRefGoogle Scholar
  5. 5.
    Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2016) Compound Rank-k Projections for Bilinear Analysis. IEEE Trans Neural Netw Learn Syst 27(7):1502–1513MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chang X, Ma Z, Yang Y, Zeng Z (2017) Alexander G. Hauptmann: Bi-Level Semantic Representation Analysis for Multimedia Event Detection. IEEE Trans Cybern 47(5):1180–1197CrossRefGoogle Scholar
  7. 7.
    Chang X, Yu Y, Yang Y, Xing EP (2017) Semantic Pooling for Complex Event Analysis in Untrimmed Videos. IEEE Trans Pattern Anal Mach Intell 39(8):1617–1632CrossRefGoogle Scholar
  8. 8.
    Chang X, Ma Z, Lin M, Yang Y, Hauptmann AG (2017) Feature Interaction Augmented Sparse Learning for Fast Kinect Motion Detection. IEEE Trans Image Process 26(8):3911–3920MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Chua T, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: A real-world web image database from national university of Singapore. In Proc. CIVRGoogle Scholar
  10. 10.
    Datta R, Joshi D, Li J, Wang JZ (2007) Tagging over time: Realworld image annotation by lightweight meta-learning. In: Proc. ACM Mutlimedia, p 393–402Google Scholar
  11. 11.
    Feng S, Lang C, Xu D (2010) Beyond tag relevance: integrating visual attention model and multi-instance learning for tag saliency ranking. CIVR, p 288–295Google Scholar
  12. 12.
    Gao Y, Wang M, Zha Z, Shen J, Li X, Wu X (2013) Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search. IEEE Trans Image Process 22(1):363–376MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Gu Y, Qian X, Li Q, Wang M, Hong R, Tian Q (2015) Image Annotation by Latent Community Detection and Multi-Kernel Learning. IEEE Trans Image Process 24(11):3450–3463MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Han Y, Wu F, Tian Q, Zhuang Y (2012) Image Annotation by Input-Output Structural Grouping Sparsity. IEEE Trans Image Process 21(6):3066–3079MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Jiang S, Qian X, Shen J, Fu Y, Mei T (2015) Author Topic Model-Based Collaborative Filtering for Personalized POI Recommendations. IEEE Trans Multimedia 17(6):907–918Google Scholar
  16. 16.
    Jiang S, Qian X, Fu Y, Mei T (2016) Personalized Travel Sequence Recommendation on Multi-Source Big Social Media. IEEE Trans Big Data 1(2):43–56CrossRefGoogle Scholar
  17. 17.
    Joshi D, Luo J, Yu J, Lei P, Gallagher A (2011) Using Geotags to Derive Rich Tag-Clouds for Image Annotation, Social Media Modeling and Computing. Springer, BerlinGoogle Scholar
  18. 18.
    Kleban J, Moxley E, Xu J, Manjunath BS (2009) Global annotation on georeferenced photographs. In: Proc. CIVRGoogle Scholar
  19. 19.
    Lei X, Qian X, Zhao G (2016) Rating Prediction based on Social Sentiment from Textual Reviews. IEEE Trans Multimedia 18(9):1910–1921CrossRefGoogle Scholar
  20. 20.
    Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Trans Pattern Anal Mach Intell 30(6):985–1002CrossRefGoogle Scholar
  21. 21.
    Li J, Qian X, Lan K, Qi P, Sharma A (2015) Improved image GPS location estimation by mining salient features. Sig. Proc.: Image Comm. 38:141–150Google Scholar
  22. 22.
    Li X, Chen L, Zhang L, Ma W, Lin F (2006) Image annotation by large-scale content-based image retrieval. ACM MMGoogle Scholar
  23. 23.
    Li X, Snoek CGM, Worring M (2008) Learning tag relevance by neighbor voting for social image retrieval. ACM MIR, p 180–187Google Scholar
  24. 24.
    Li X, Snoek C, Worring M (2009) Learning Social Tag Relevance by Neighbor Voting. IEEE Trans Multimedia 11(7):1310–1322CrossRefGoogle Scholar
  25. 25.
    Li G, Wang M, Lu Z, Hong R, Chua T (2012) In-Video Product Annotation with Web Information Mining. ACM Trans Multimed Comput Commun Appl 8(4)CrossRefGoogle Scholar
  26. 26.
    Li J, Qian X, Tang Y, Yang L, Mei T (2013) GPS estimation for places of interest from social users’ uploaded photos. IEEE Trans Multimedia 15(8):2058–2071CrossRefGoogle Scholar
  27. 27.
    Li X, Guo Q, Lu X (2016) Spatiotemporal Statistics for Video Quality Assessment. IEEE Trans Image Process 25(7):3329–3342MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Li X, Mou L, Lu X (2016) Surveillance Video Synopsis via Scaling Down Objects. IEEE Trans Image Process 25(2):740–755MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Liu D, Hua X, Yang L, Wang M, Zhang H (2009) Tag ranking. In: Proc. WWWGoogle Scholar
  30. 30.
    Liu D, Hua X-S, Wang M, Zhang H-J (2010) Retagging social images based on visual and semantic consistency. In: Proc. ACM WWW, p 1149–1150Google Scholar
  31. 31.
    Liu D, Hua X-S, Wang M, Zhang H-J (2010) Image retagging. In: Proc. ACM MultimediaGoogle Scholar
  32. 32.
    Liu D, Wang M, Hua X, Zhang H (2011) Semi-Automatic Tagging of Photo Albums via Exemplar Selection and Tag Inference. IEEE Trans Multimedia 13(1):82–91CrossRefGoogle Scholar
  33. 33.
    Liu D, Yan S, Hua X, Zhang H (2011) Image Retagging Using Collaborative Tag Propagation. IEEE Trans MultimediaGoogle Scholar
  34. 34.
    Lu X, Li X (2014) Multiresolution Imaging. IEEE Trans Cybern 44(1):149–160CrossRefGoogle Scholar
  35. 35.
    Lu X, Wang Y, Yuan Y (2013) Graph Regularized Low-Rank Representation for Destriping of Hyperspectral Images. IEEE Trans Geosci Remote Sens 51(7):4009–4018CrossRefGoogle Scholar
  36. 36.
    Lu X, Wu H, Yuan Y, Yan P, Li X (2013) Manifold Regularized Sparse NMF for Hyperspectral Unmixing. IEEE Trans Geosci Remote Sens 51(5):2815–2826CrossRefGoogle Scholar
  37. 37.
    Lu X, Li X, Li M (2015) Semi-Supervised Multi-task Learning for Scene Recognition. IEEE Trans Cybern 45(9):1967–1976CrossRefGoogle Scholar
  38. 38.
    Lu D, Liu X, Qian X (2016) Tag based Image Search by Social Re-Ranking. IEEE Trans Multimedia 18(8):1628–1639CrossRefGoogle Scholar
  39. 39.
    Lu X, Yuan Y, Zhang X (2016) Jointly Dictionary Learning for Change Detection in Multispectral Imagery. IEEE Trans Cybern.  https://doi.org/10.1109/TCYB.2016.2531179 CrossRefGoogle Scholar
  40. 40.
    Lu X, Li X, Zheng X (2017) Latent Semantic Minimal Hashing for Image Retrieval. IEEE Trans Image Process 26(1):355–368MathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    Mei T, Wang Y, Hua X, Gong S, Li S (2008) Coherent image annotation by learning semantic distance. In: Proc. CVPRGoogle Scholar
  42. 42.
    Moxley E, Mei T, Manjunath B (2010) Video annotation through search and graph reinforcement mining. IEEE Trans Multimedia 12(3):184–193CrossRefGoogle Scholar
  43. 43.
    Qian X, Hua X (2011) Graph-cut based tag enrichment. In: Proc. SIGIR, p 1111–1112Google Scholar
  44. 44.
    Qian X, Hua X, Hou X (2012) Tag Filtering based on Similar Compatible Principle. In: Proc. ICIPGoogle Scholar
  45. 45.
    Qian X, Liu X, Zheng C, Du Y, Hou X (2013) Tagging photos using users’ vocabularies. Neurocomputing 111:144–153CrossRefGoogle Scholar
  46. 46.
    Qian X, Hua X, Tang Y, Mei T (2014) Social Image Tagging with Diverse Semantics. IEEE Trans Cybern 44(12):2493–2508CrossRefGoogle Scholar
  47. 47.
    Qian X, Feng H, Zhao G, Mei T (2014) Personalized Recommendation Combining User Interest and Social Circle. IEEE Trans Knowl Data Eng 26(7):1487–1502CrossRefGoogle Scholar
  48. 48.
    Qian X, Xue Y, Tang Y, Hou X, Mei T (2015) Landmark Summarization with Diverse Viewpoints. IEEE Trans Circuits and Syst Video Technol 25(11):1857–1869CrossRefGoogle Scholar
  49. 49.
    Qian X, Zhao Y, Han J (2015) Image Location Estimation by Salient Region Matching. IEEE Trans Image Process 24(6):4348–4358MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Qian X, Wang H, Zhao Y, Hou X, Hong R, Wang M, Tang YY (2017) Image Location Inference by Multisaliency Enhancement. IEEE Trans Multimedia 19(4):813–821CrossRefGoogle Scholar
  51. 51.
    Qian X, Lu D, Wang Y, Zhu L, Tang YY, Wang M (2017) Image Re-Ranking Based on Topic Diversity. IEEE Trans Image Process 26(8):3734–3747MathSciNetzbMATHCrossRefGoogle Scholar
  52. 52.
    Shen J, Meng W, Yan S, Pang H, Hua X (2010) Effective music tagging through advanced statistical modeling. SIGIRGoogle Scholar
  53. 53.
    Wang C, Jing F, Zhang L, Zhang H (2007) Content-based image annotation refinement. In: Proc. CVPRGoogle Scholar
  54. 54.
    Wang X, Zhang L, Li X, Ma W (2008) Annotating images by mining image search results. IEEE Trans Pattern Anal Mach Intell 30(11):1919–1932CrossRefGoogle Scholar
  55. 55.
    Wang X-J, Yu M, Zhang L, Cai R, Ma W-Y (2009) Argo: Intelligent Advertising by Mining a User's Interest from His Photo Collections. ACM Data Mining and Audience Intelligence for Advertising, p 18–26Google Scholar
  56. 56.
    Wang X, Zhang L, Liu M, Li Y, Ma W (2010) ARISTA - image search to annotation on billions of web photos. In: Proc. CVPR, p 2987–2994Google Scholar
  57. 57.
    Wang M, Yang K, Hua X, Zhang H (2010) Towards a Relevant and Diverse Search of Social Images. IEEE Trans Multimedia 12(8):829–842CrossRefGoogle Scholar
  58. 58.
    Wang M, Ni B, Hua X, Chua T (2012) Assistive Tagging: A Survey of Multimedia Tagging with Human-Computer Joint Exploration. ACM Comput Surv 44(4)CrossRefGoogle Scholar
  59. 59.
    Wang M, Hong R, Li G, Zha Z, Yan S, Chua T (2012) Event Driven Web Video Summarization by Tag Localization and Key-Shot Identification. IEEE Trans Multimedia 14(4):975–985CrossRefGoogle Scholar
  60. 60.
    Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal Graph-Based Reranking for Web Image Search. IEEE Trans Image Process 21(11):4649–4661MathSciNetzbMATHCrossRefGoogle Scholar
  61. 61.
    Weinberger K, Slaney M, van Zwol R (2008) Resolving tag ambiguity. In: Proc. ACM Multimedia, p 111–119Google Scholar
  62. 62.
    Wu L, Hua X-S, Yu N, Ma W-Y, Li S (2008) Flickr distance. In: Proc. ACM Multimedia, p 31–40Google Scholar
  63. 63.
    Wu L, Yang LJ, Yu NH, Hua XS (2009) Learning to Tag. In: Proc. of ACM WWWGoogle Scholar
  64. 64.
    Xu H, Wang J, Hua X, Li S (2009) Tag refinement by regularized lda. In: Proc. ACM MultimediaGoogle Scholar
  65. 65.
    Yan Y, Nie F, Li W, Gao C, Yang Y, Xu D Image classification by cross-media active learning with privileged information. IEEE Trans Multimedia 18(12):2494–2502CrossRefGoogle Scholar
  66. 66.
    Yang K, Hua X, Wang M, Zhang H (2011) Tag Tagging: Towards More Descriptive Keywords of Image Content. IEEE Trans Multimedia 13(4):662–673CrossRefGoogle Scholar
  67. 67.
    Yang Y, Wu F, Nie F, Shen H, Zhuang Y, Hauptmann A (2012) Web and Personal Image Annotation by Mining Label Correlation with Relaxed Visual Graph Embedding. IEEE Trans Image Process 21(3):1339–1351MathSciNetzbMATHCrossRefGoogle Scholar
  68. 68.
    Yang Y, Nie F, Xu D, Luo J, Zhuang Y, Pan Y (2012) A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans Pattern Anal Mach Intell :723–742Google Scholar
  69. 69.
    Yang X, Qian X, Xue Y (2015) Scalable Mobile Image Retrieval by Exploring Contextual Saliency. IEEE Trans Image Process 24(6):1709–1721MathSciNetzbMATHCrossRefGoogle Scholar
  70. 70.
    Yang Y, Ma Z, Nie F, Chang X, Hauptmann AG Multi-class active learning by uncertainty sampling with diversity maximization. Int J Comput Vis 113(2):113–127MathSciNetCrossRefGoogle Scholar
  71. 71.
    Yang Y, Ma Z, Hauptmann AG, Sebe N Feature selection for multimedia analysis by sharing information among multiple tasks. IEEE Trans Multimedia 15(3):661–669CrossRefGoogle Scholar
  72. 72.
    Yuan Y, Zheng X, Lu X (2017) Hyperspectral Band Selection by Discovering Diverse Subset in Multiple Graphs. IEEE Trans Image Process 26(1)Google Scholar
  73. 73.
    Zha Z, Hua X, Mei T, Wang J, Qi G, Wang Z (2008) Joint multi-label multi-instance learning for image classification. In: Proc. CVPRGoogle Scholar
  74. 74.
    Zha Z, Wang M, Zheng Y, Yang Y, Hong R, Chua T (2012) Interactiv e Video Indexing With Statistical Active Learning. IEEE Trans Multimedia 14(1):17–27CrossRefGoogle Scholar
  75. 75.
    Zhang S, Huang J, Li H, Metaxas D (2012) Automatic Image Annotation and Retrieval Using Group Sparsity. IEEE Trans Syst Man Cybern Part B Cybern 42(3):838–849CrossRefGoogle Scholar
  76. 76.
    Zhang D, Han J, Jiang L, Ye S, Chang X (2017) Revealing Event Saliency in Unconstrained Video Collection. IEEE Trans Image Process 26(4):1746–1758MathSciNetzbMATHCrossRefGoogle Scholar
  77. 77.
    Zhao G, Qian X, Lei X (2016) Objective Evaluation for Service by Deep Exploring Social Users’ Contextual Information. IEEE Trans Knowl Data Eng 28(12):3382–3394Google Scholar
  78. 78.
    Zhao G, Qian X, Xie X (2016) User-Service Rating Prediction by Exploring Social Users’ Rating Behaviors. IEEE Trans Multimedia 18(3):496–506CrossRefGoogle Scholar
  79. 79.
    Zhao G, Qian X, Kang C (2017) Service Rating Prediction by Exploring Social Mobile Users' Geographical Locations. IEEE Trans Big Data 3(1):67–78CrossRefGoogle Scholar
  80. 80.
    Zhou N, Cheung WK, Qiu G, Xue X (2011) A Hybrid Probabilistic Model for Unified Collaborative and Content-Based Image Tagging. IEEE Trans Pattern Anal Mach Intell 33(7):1281–1294CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Xi’an Jiaotong University City CollegeXi’anChina

Personalised recommendations

Cite article

Buy options