A survey of tag-based information retrieval

  • Sanghoon Lee
  • Mohamed Masoud
  • Janani Balaji
  • Saeid Belkasim
  • Rajshekhar Sunderraman
  • Seung-Jin Moon
Trends and Surveys


This paper aims to provide a comprehensive survey of tag-based information retrieval that covers three areas: tag-based document retrieval, tag-based image retrieval, and tag-based music information retrieval. First of all, seven representative graphical models associated with tag contents are reviewed and evaluated in terms of effectiveness in achieving their goals. The models are explored in depth based on appropriate plate notations for the tag-based document retrieval. Second, well-established review criteria for two-way classical methods, tag refinement and tag recommendation, are utilized for tag-based image retrieval. In particular, tag refinement methods are analyzed by means of the experimental results measured on different datasets. Last, popular tagging methods in the area of music information retrieval are reviewed for the tag-based music information retrieval. We introduce five criteria: used models, tagging purpose, tagging right, object type, and used dataset, for evaluating tag-based information retrieval methods as a new categorical framework engaging the graphical models as well as the two-way classical methods.


Information retrieval Document retrieval Image retrieval Music information retrieval 


  1. 1.
    Cisco (2016) Cisco Visual Networking Index: Forecast and Methodology 2015–2020. Accessed 1 June 2016
  2. 2.
    Gao Y, Wang M, Luan H, Shen J, Yan S, Tao D (2011) Tag-based social image search with visual-text joint hypergraph learning in Proc. ACM ICM, pp 1517–1520Google Scholar
  3. 3.
    Gao Y, Wang M, Zha Z-J, Shen J, Li X, Wu X (2013) Visual-textual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376MathSciNetCrossRefGoogle Scholar
  4. 4.
    Zanardi V, Capra L (2008) Social ranking: uncovering relevant content using tag-based recommender systems. In: Proc. ACM Rec Sys, pp 51–58Google Scholar
  5. 5.
    Nakamoto R, Nakajima S, Miyazaki J, Uemura S (2007) Tag-based contextual collaborative filtering. In: Proc. 18th IEICE data engineering workshopGoogle Scholar
  6. 6.
    Carmagnola F, Cena F, Cortassa O, Gena C, Torre I (2007) Towards a tag-based user model: how can user model benefit from tags? User modeling. Springer, Berlin Heidelberg, pp 445–449Google Scholar
  7. 7.
    Cai Y, Li Q (2010) Personalized search by tag-based user profile and resource profile in collaborative tagging systems. In: Proc. 19th ACM ICIKM, pp 969–978Google Scholar
  8. 8.
    Chen L, Xu D, Tsang IW, Luo J (2010) Tag-based web photo retrieval improved by batch mode re-tagging. In: Proc. IEEE CVPR, pp 3440–3446Google Scholar
  9. 9.
    Mathes A (2004) Folksonomies cooperative classification and communication through shared metadata. UIC Technical ReportGoogle Scholar
  10. 10.
    Trant J (2009) Studying social tagging and folksonomy: A review and framework J. Digital Information 10-1Google Scholar
  11. 11.
    Gupta M, Li R, Yin Z, Han J (2010) Survey on social tagging techniques. ACM SIGKDD Explor Newslett 12(1):58–72CrossRefGoogle Scholar
  12. 12.
    Dattolo A, Ferrara F, Tasso C (2010) The role of tags for recommendation: a survey. In: HSI, pp 548–555Google Scholar
  13. 13.
    Majid A, Khusro S, Rauf A (2011) Semantics in social tagging systems: a review. In: ICCNIT, pp 191–203Google Scholar
  14. 14.
    Wang M, Ni B, Hua XS, Chua TS (2012) Assistive tagging: a survey of multimedia tagging with human–computer joint exploration. ACM CSUR 44(4):25Google Scholar
  15. 15.
    Deerwester S, Dumais S, Landauer T, Furnas G, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6):391407CrossRefGoogle Scholar
  16. 16.
    Hofmann T (1999) Probabilistic latent semantic indexing. In: Proceedings of the 22nd annual international ACM SIGIR, pp 50–57Google Scholar
  17. 17.
    Blei DM, Andrew YN, Michael IJ (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022zbMATHGoogle Scholar
  18. 18.
    Rosen-Zvi M, Griffiths T, Steyvers M, Smyth P (2012) The author-topic model for authors and documents. In: Proceedings of the 20th conference on uncertainty in artificial intelligence. AUAI Press, pp 487–494Google Scholar
  19. 19.
    Mimno D, McCallum A (2008) Topic models conditioned on arbitrary features with dirichlet-multinomial regression. In: UAI, pp 411–418Google Scholar
  20. 20.
    Ramage D, Hall D, Nallapati R, Manning CD (2009) Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: EMNLP, pp 248–256Google Scholar
  21. 21.
    Ramage D, Manning CD, Dumais S (2011) Partially labeled topic models for interpretable text mining. In: Proc. ACM SIGKDD, pp 457–465Google Scholar
  22. 22.
    Li S, Li J, Pan R (2013) Tag-weighted topic model for mining semi-structured documents. In: Proceedings of the 23th international joint conference on artificial intelligence. AAAI Press, pp 2855–2861Google Scholar
  23. 23.
    Li S, Li J, Huang G, Tan R, Pan R (2015) Tag-weighted topic model for large-scale semi-structured documents (arXiv preprint). arXiv:1507.08396
  24. 24.
    Zamir AR, Ardeshir S, Shah M (2014) GPS-Tag refinement using random walks with an adaptive damping factor. In: CVPR, pp 4280–4287Google Scholar
  25. 25.
    Liu Y, Wu F, Zhang Y, Shao J, Zhuang Y (2011) Tag clustering and refinement on semantic unity graph. In: ICDM, pp 417–426Google Scholar
  26. 26.
    Hou Y, Lin Z (2015) Image tag completion and refinement by subspace clustering and matrix completion (arXiv preprint). arXiv:1506.03475
  27. 27.
    Cui B, Yao J (2012) Tag refinement strategies for social tagging systems U.S. Patent Application 13/980,573, filed March 31Google Scholar
  28. 28.
    Qian Z, Zhong P, Wang R (2015) Tag refinement for user-contributed images via graph learning and nonnegative tensor factorization. IEEE Signal Process Lett 22(9):1302–1305CrossRefGoogle Scholar
  29. 29.
    Wang L, Zhou TH, Lee YK, Cheoi KJ, Ryu KH (2015) An efficient refinement algorithm for multi-label image annotation with correlation model. Telecommunication Systems, pp 1–17Google Scholar
  30. 30.
    Sang J, Liu J, Xu C (2011) Exploiting user information for image tag refinement. In: Proceedings of 19th ACM international conference on multimedia, pp 1129–1132Google Scholar
  31. 31.
    Hua XS, Liu D, Wang M, Zhang HJ (2010) Image Tag Refinement U.S. Patent Application 12/971,880, filed December 17Google Scholar
  32. 32.
    Jin Y, Khan L, Wang L, Awad M (2005) Image annotations by combining multiple evidence and wordnet. In: Proceedings of 13th annual ACM international conference on multimedia, pp 706–715Google Scholar
  33. 33.
    Resnik P (1995) Using information content to evaluate semantic similarity in a taxonomy. In: Proceedings of the 14th international joint conference on artificial intelligenceGoogle Scholar
  34. 34.
    Jiang J, Conrath D (1997) Semantic similarity based on corpus statistics and lexical taxonomy. In: Proceedings of the international conference on research in computational linguisticsGoogle Scholar
  35. 35.
    Lin D (1997) Using syntatic dependency as a local context to resolve word sense ambiguity. In: Proceedings of the 35th annual meeting of the association for computational linguistics, pp 64–71Google Scholar
  36. 36.
    Leacock C (1998) Combining local context and wordnet similarity for word sense identification. In: Fellbaum Christiane (ed) WordNet: a lexical reference system and its application. MIT Press, Cambridge, pp 265–283Google Scholar
  37. 37.
    Banerjee S, Pedersen T (2003) Extended gloss overlaps as a measure of semantic relatedness. In: Proceedings of eighteenth international joint conference on artificial intelligence, pp 805–810Google Scholar
  38. 38.
    Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of 26th annual international ACM SIGIR conference on research and development in information retrieval, pp 119–126Google Scholar
  39. 39.
    Wang C, Jing F, Zhang L, Zhang HJ (2006) Image annotation refinement using random walk with restarts. In: Proceedings of 14th annual ACM international conference on multimedia, pp 647–650Google Scholar
  40. 40.
    Pan JY, Yang HJ, Faloutsos C, Duygulu P (2004) Automatic multimedia cross-modal correlation discovery. In: Proceedings of 10th ACM SIGKDD international conference on knowledge discovery and data mining, pp 653–658Google Scholar
  41. 41.
    Cheng W, Wang X (2011) Image tag refinement using tag semantic and visual similarity. In: ICCSNT 4, pp 2146–2149Google Scholar
  42. 42.
    Wu L, Hua XS, Yu NH, Ma WY, Li SP (2008) Flickr distance ACM multimedia, pp 31–40Google Scholar
  43. 43.
    Wang C, Jing F, Zhang L, Zhang HJ (2007) Content-based image annotation refinement. In: CVPR’071-8Google Scholar
  44. 44.
    Sang J, Xu C, Liu J (2012) User-aware image tag refinement via ternary semantic analysis. IEEE Trans Multimedia 14(3):883–895CrossRefGoogle Scholar
  45. 45.
    Liu J, Zhang Y, Li Z, Lu H (2013) Correlation consistency constrained probabilistic matrix factorization for social tag refinement. Neurocomputing 119:3–9CrossRefGoogle Scholar
  46. 46.
    Liu D, Hua XS, Yang L, Wang M, Zhang HJ (2009) Tag ranking. In: Proceedings of 18th international conference on WWW, pp 351–360Google Scholar
  47. 47.
    Zhuang J, Hoi JCH (2011) A two-view learning approach for image tag ranking. In: WSDMGoogle Scholar
  48. 48.
    Zhu G, Yan S, Ma Y (2010) Image tag refinement towards low-rank, content-tag prior and error sparsity. In: Proceedings of international conference on multimedia, pp 461–470Google Scholar
  49. 49.
    Li Z, Liu J, Zhu X, Liu T, Lu H (2010) Image annotation using multi-correlation probabilistic matrix factorization. ACM Multimedia, pp 1187–1190Google Scholar
  50. 50.
    Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33(3):1065–1076Google Scholar
  51. 51.
    Liu D, Yan S, Hua XS, Zhang HJ (2011) Image retagging using collaborative tag propagation. IEEE Trans Multimedia 13(4):702–712CrossRefGoogle Scholar
  52. 52.
    Zhang S, Tian Q, Hua G, Huang Q, Li S (2009) Descriptive visual words and visual phrases for image applications. In: Proc. ACM Int. Conf, Multimedia, p 7584Google Scholar
  53. 53.
    Xu H, Wang J, Hua XS, Li S (2009) Tag refinement by regularized LDA. In: Proceedings of 17th ACM international conference on multimedia, pp 573–576Google Scholar
  54. 54.
    Fu J, Wang J, Rui Y, Wang XJ, Mei T (2015) Image tag refinement with view-dependent concept representations. IEEE Trans Circuits Syst Video Technol 25(8):1409–1422CrossRefGoogle Scholar
  55. 55.
    Tsai D, Jing Y, Liu Y, Rowley HA, Ioffe S, Rehg JM (2011) Large-scale image annotation using visual synset. In: Proc. IEEE ICCV, pp 611–618Google Scholar
  56. 56.
    Chen L, Xu D, Tsang IW, Luo J (2012) Tag-based image retrieval improved by augmented features and group-based refinement. IEEE Trans Multimedia 14(4):10571067CrossRefGoogle Scholar
  57. 57.
    Wu L, Jin R, Jain AK (2013) Tag completion for image retrieval. IEEE Trans Pattern Anal Mach Intell 35(3):716727Google Scholar
  58. 58.
    Mishne G (2006) AutoTag: a collaborative approach to automated tag assignment for weblog posts. In: Proceedings of 15th international conference on World Wide Web, pp 953–954Google Scholar
  59. 59.
    Ames M, Naaman M (2007) Why we tag: Motivations for annotation in mobile and online media. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 971–980Google Scholar
  60. 60.
    Sigurbj Orsnsson B, van Zwol R (2008) Flickr tag recommendation based on collective knowledge. In: Proceedings of the 17th international conference WWW, pp 327–336Google Scholar
  61. 61.
    Hotho A, Jaschke R, Schmitz C, Stumme G (2006) Information retrieval in folksonomies: search and ranking. Lect Notes Comput Sci 4011:411–426CrossRefGoogle Scholar
  62. 62.
    Semeraro G, Lops P, Gemmis MD, Musto C, Narducci F (2012) A folksonomy-based recommender system for personalized access to digital artworks. JOCCH 5(3):11CrossRefGoogle Scholar
  63. 63.
    Chen HM, Chang MH, Chang PC, Tien MC, Hsu WH, Wu JL (2008) SheepDog: group and tag recommendation for Flickr photos by automatic search-based learning. In: Proceedings of the 16th ACM international conference on multimedia, pp 737–740Google Scholar
  64. 64.
    Lee S, Neve WD, Plataniotis KN, Ro YM (2010) MAP-based image tag recommendation using a visual folksonomy. Pattern Recognit Lett 31(9):976–982CrossRefGoogle Scholar
  65. 65.
    Shen Y, Fan J (2010) Leveraging loosely-tagged images and inter-object correlations for tag recommendation. In: Proceedings of international conference on multimedia, pp 5–14Google Scholar
  66. 66.
    Lindstaedt S, Mrzinger R, Sorschag R, Pammer V, Thallinger G (2009) Automatic image annotation using visual content and folksonomies. Multimedia Tools Appl 42(1):97–113CrossRefGoogle Scholar
  67. 67.
    Silva A, Martins B (2011) Tag recommendation for geo-referenced photos. In: Proceedings of 3rd ACM SIGSPATIAL international workshop on location-based social networks, pp 57–64Google Scholar
  68. 68.
    Eom W, Lee S, Neve WD, Ro YM (2011) Improving image tag recommendation using favorite image context. In: ICIP, pp 2445–2448Google Scholar
  69. 69.
    Anderson A, Ranghunathan K, Vogel A (2008) Tagez: Flickr tag recommendation. Association for the Advancement of Artificial IntelligenceGoogle Scholar
  70. 70.
    Takashita T, Itokawa T, Kitasuka T, Aritsugi M (2010) Tag recommendation for Flickr using web browsing behavior. In: ICCSA, pp 412–421Google Scholar
  71. 71.
    Xu Z, Fu Y, Mao J, Su D (2006) Towards the semantic web: collaborative tag suggestions. In: Collaborative web tagging workshop at WWW2006Google Scholar
  72. 72.
    Kim JW, Candan KS, Tatemura J (2010) Organization and tagging of blog and news entries based on content reuse. J Signal Process Syst 58:407–421CrossRefGoogle Scholar
  73. 73.
    Weinberger KQ, Slaney M, Van Zwol R (2008) Resolving tag ambiguity In: MM, pp 111–120Google Scholar
  74. 74.
    Nguyen QVH, Do ST, Nguyen TT, Aberer K (2015) Tag-based paper retrieval: minimizing user effort with diversity awareness. In: Database systems for advanced applications. Springer International Publishing, pp 510–528Google Scholar
  75. 75.
    Cantador I, Konstas I, Jose JM (2011) Categorizing social tags to improve folksonomy-based recommendations. Web semantics: science, services and agents on the World Wide Web 9(1):1–15CrossRefGoogle Scholar
  76. 76.
    Wang YS (2014) Image tag recommendation algorithm using tensor factorization. J Multimedia 9(3):416–422CrossRefGoogle Scholar
  77. 77.
    Fang X, Pan R, Cao G, He X, Dai W (2015) personalized tag recommendation through nonlinear tensor factorization using Gaussian kernel. In: AAAI, pp 439–445Google Scholar
  78. 78.
    McParlane PJ, Moshfeghi J, Jose JM (2013) On contextual photo tag recommendation. In: Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval, pp 965–968Google Scholar
  79. 79.
    Turnbull D, Barrington L, Lanckriet G (2008) Five approaches to collecting tags for music. In: ISMIR, vol 8, pp 225–230Google Scholar
  80. 80.
    Lamere P (2008) Social tagging and music information retrieval. J New Music Res 37(2):101–114CrossRefGoogle Scholar
  81. 81.
    Marlow C, Naaman M, Boyd D, Davis M (2006) Tagging paper, taxonomy, Flickr, academic article, to read. In: Proceedings of ACM hypertextGoogle Scholar
  82. 82.
    Ames M, Naaman M (2007) Why we tag: motivations for annotation in mobile and online media. In: CHI 07: proceedings of the SIGCHI conference on Human factors in computing systems, pp 971–980Google Scholar
  83. 83.
    Nanopoulos A, Rafailidis D, Symeonidis P, Manolopoulos Y (2010) Musicbox: Personalized music recommendation based on cubic analysis of social tags. IEEE Trans Audio Speech Language Process 18(2):407–412CrossRefGoogle Scholar
  84. 84.
    Lathauwer L, Moor B, Vandewalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21(4):12531278MathSciNetzbMATHGoogle Scholar
  85. 85.
    Hariri N, Mobasher B, Burke R (2012) Using social tags to infer context in hybrid music recommendation. In: Proceedings of 12th international workshop on web information and data management, pp 41–48Google Scholar
  86. 86.
    Nanopoulos A, Karydis I (2011) Know thy neighbor: combining audio features and social tags for effective music similarity. In: ICASSP, pp 165–168Google Scholar
  87. 87.
    Logan B, Salomon A (2001) A music similarity function based on signal analysis. In: ICMEGoogle Scholar
  88. 88.
    Yang YH, Bogdanov D, Herrera P, Sordo M (2012) Music retagging using label propagation and robust principal component analysis. In: Proceedings of 21st international conference companion on WWW, pp 869–876Google Scholar
  89. 89.
    Zhou D, Bousquet O, Lal TN, Weston J, Scholkopf B (2004) Learning with local and global consistency. In: NIPS, pp 321–328Google Scholar
  90. 90.
    Lin Z, Ganesh A, Wright J, Wu L, Chen M, Ma Y (2009) Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix Technical Report UILU-ENG-09-2214Google Scholar
  91. 91.
    Song Y, Dixon S, Pearce M, Fazekas G (2013) Using tags to select stimuli in the study of music and emotion. In: Proc. ICME3Google Scholar
  92. 92.
    Font F, Serra X (2012) Analysis of the folksonomy of freesound. In: Proc. 2nd comp music workshopGoogle Scholar
  93. 93.
    Chen L, Wright P, Nejdl W (2009) Improving music genre classification using collaborative tagging data. In: Proceedings of second ACM international conference on web search and data mining, pp 84–93Google Scholar
  94. 94.
    Font F, Serra J, Serra X (2014) Audio clip classification using social tags and the effect of tag expansion. In: Audio Engineering Society Conference: 53rd international conference: semantic audio. Audio Engineering SocietyGoogle Scholar
  95. 95.
    Levy M, Sandler M (2007) A semantic space for music derived from social tags. In: Proc. ISMIRGoogle Scholar
  96. 96.
    Laurier C, Sordo M, Serra J, Herrera P (2009) Music mood representations from social tags. In: Proc. ISMIR, pp 381–386Google Scholar
  97. 97.
    Saari P, Eerola T (2014) Semantic computing of moods based on tags in social media of music. IEEE KDE 26(10):2548–2560Google Scholar
  98. 98.
    Russell JA (1980) A circumplex model of affect. J Person Soc Psychol 39(6):1161–1178CrossRefGoogle Scholar
  99. 99.
    Ekman P (1992) An argument for basic emotions. Cogn Emotion 6:169–200CrossRefGoogle Scholar
  100. 100.
    Ballan L, Bertini M, Uricchio T, Bimbo AD (2013) Social media annotation. In: 11th international workshop on CBMI, pp 229–235Google Scholar
  101. 101.
    Miller GA, Claudia L, Randee T, Ross TB (1993) A semantic concordance. In: Proceedings of the workshop on human language technology, pp 303–308Google Scholar
  102. 102.
    Bentivogli L, Pamela F, Bernardo M, Emanuele P (2004) Revising the wordnet domains hierarchy: semantics, coverage and balancing. In: Proceedings of the workshop on multilingual linguistic resources, pp 101–108Google Scholar
  103. 103.
    Golder SA, Bernardo AH (2006) Usage patterns of col-laborative tagging systems. J Inf Sci 32(2):198–208Google Scholar
  104. 104.
    Halpin H, Valentin R, Hana S (2007) The complex dynamics of collaborative tagging. In: Proceedings of the 16th international conference on World Wide Web, pp 211–220Google Scholar
  105. 105.
    You D, Antani S, Demner-Fushman D, Thoma GR. (2014) Does figure-text improve biomedical article retrieval? A pilot study. In: 2014 IEEE 27th international symposium on CBMS, pp 471–472Google Scholar
  106. 106.
    Simpson M, You D, Rahman MM, Xue Z, Demner-Fushman D, Antani S, Thoma G (2015) Literature-based biomedical image classification and retrieval. Comput Med Imaging Graphics 39:3–13CrossRefGoogle Scholar
  107. 107.
    Wu H, Bowers DM, Huynh TT, Souvenir R (2013) Biomedical video denoising using supervised manifold learning. In: IEEE 10th international symposium on ISBI, pp 1244–1247Google Scholar
  108. 108.
    Savaris A, Hrder T, Wangenheim A (2014) DCMDSM: a DICOM decomposed storage model. J Am Med Inf Assoc 21(5):917–924CrossRefGoogle Scholar
  109. 109.
    Nemiroff RJ, Bonnell JT (1995) Astronomy picture of the day. Bull Am Astron Soc, 1291.
  110. 110.
    Kipp M (2011) Tagging of biomedical articles on CiteULike: a comparison of user, author and professional indexing. Knowl Org 38:245–261Google Scholar
  111. 111.
    Lee S, Yanjun Z, Mohamed Eid M M, Maria V, Semra K, Saeid B (2015) Domain specific information retrieval and text mining in medical document. In: Proceedings of the 6th ACM BCB, pp 67–76Google Scholar
  112. 112.
    TAC (2016) Proceedings of text analysis conference NIST.
  113. 113.
    Turnbull D, Barrington L, Torres D, Lanckriet G (2007) Towards musical query by semantic description using the CAL500 data set. ACM SIGIR, pp 439–446Google Scholar
  114. 114.
    Knees P, Pampalk E, Widmer G (2004) Artist classification with web-based data. In: Proceeding of ISMIR, pp 517–524Google Scholar

Copyright information

© Springer-Verlag London 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceGeorgia State UniversityAtlantaUSA
  2. 2.Department of NeurologyEmory University School of MedicineAtlantaUSA
  3. 3.Department of Computer ScienceUniversity of SuwonHwaseong-siSouth Korea

Personalised recommendations