Adaptive Tag Selection for Image Annotation

  • Xixi He
  • Xirong Li
  • Gang Yang
  • Jieping Xu
  • Qin Jin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8879)

Abstract

Not all tags are relevant to an image, and the number of relevant tags is image-dependent. Although many methods have been proposed for image auto-annotation, the question of how to determine the number of tags to be selected per image remains open. The main challenge is that for a large tag vocabulary, there is often a lack of ground truth data for acquiring optimal cutoff thresholds per tag. In contrast to previous works that pre-specify the number of tags to be selected, we propose in this paper adaptive tag selection. The key insight is to divide the vocabulary into two disjoint subsets, namely a seen set consisting of tags having ground truth available for optimizing their thresholds and a novel set consisting of tags without any ground truth. Such a division allows us to estimate how many tags shall be selected from the novel set according to the tags that have been selected from the seen set. The effectiveness of the proposed method is justified by our participation in the ImageCLEF 2014 image annotation task. On a set of 2,065 test images with ground truth available for 207 tags, the benchmark evaluation shows that compared to the popular top-k strategy which obtains an F-score of 0.122, adaptive tag selection achieves a higher F-score of 0.223. Moreover, by treating the underlying image annotation system as a black box, the new method can be used as an easy plug-in to boost the performance of existing systems.

Keywords

Image annotation adaptive tag selection ImageCLEF evaluation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    van de Sande, K., Snoek, C.: The University of Amsterdam’s concept detection system at imageclef 2011. In: CLEF (2011)Google Scholar
  2. 2.
    Li, X., Snoek, C.: Classifying tag relevance with relevant positive and negative examples. In: ACM MM (2013)Google Scholar
  3. 3.
    Wang, X.J., Zhang, L., Ma, W.Y.: Duplicate-search-based image annotation using web-scale data. Proceedings of the IEEE (2012)Google Scholar
  4. 4.
    Ballan, L., Uricchio, T., Seidenari, L., Del Bimbo, A.: A cross-media model for automatic image annotation. In: ICMR (2014)Google Scholar
  5. 5.
    Villegas, M., Paredes, R.: Overview of the ImageCLEF 2014 scalable concept image annotation task. In: CLEF (2014)Google Scholar
  6. 6.
    Li, X., Snoek, C., Worring, M.: Learning social tag relevance by neighbor voting. TMM 11(7), 1310–1322 (2009)Google Scholar
  7. 7.
    Tang, J., Hong, R., Yan, S., Chua, T.S.: Image annotation by knn-sparse graph-based label propagation over noisily-tagged web images. In: TIST (2010)Google Scholar
  8. 8.
    Zhu, S., Jiang, Y.G., Ngo, C.W.: Sampling and ontologically pooling web images for visual concept learning. TMM 14(4), 1068–1078 (2012)Google Scholar
  9. 9.
    Borgne, H.L., Popescu, A., Znaidia, A.: CEA LIST@imageclef 2013: Scalable concept image annotation. In: CLEF Working Notes (2013)Google Scholar
  10. 10.
    Jiang, Y.G., Ngo, C.W., Chang, S.F.: Semantic context transfer across heterogeneous sources for domain adaptive video search. In: ACMMM (2009)Google Scholar
  11. 11.
    Cilibrasi, R., Vitanyi, P.: The Google similarity distance. TKDE 19(3), 370–383 (2004)Google Scholar
  12. 12.
    van de Sande, K., Gevers, T., Snoek, C.: Evaluating color descriptors for object and scene recognition. TPAMI 32, 1582–1596 (2010)CrossRefGoogle Scholar
  13. 13.
    Villegas, M., Paredes, R., Thomee, B.: Overview of the imageclef 2013 scalable concept image annotation subtask. In: CLEF (2013)Google Scholar
  14. 14.
    Hua, X.S., Yang, L., Wang, J., Wang, J., Ye, M., Wang, K., Rui, Y., Li, J.: Clickage: Towards bridging semantic and intent gaps via mining click logs of search engines. In: ACMMM (2013)Google Scholar
  15. 15.
    Li, X., Snoek, C., Worring, M., Koelma, D., Smeulders, A.: Bootstrapping visual categorization with relevant negatives. TMM 15(4), 933–945 (2013)Google Scholar
  16. 16.
    Maji, S., Berg, A., Malik, J.: Classification using intersection kernel support vector machines is efficient. In: CVPR (2008)Google Scholar
  17. 17.
    Li, X., Liao, S., Liu, B., Yang, G., Jin, Q., Xu, J., Du, X.: Renmin University of China at ImageCLEF 2013 scalable concept image annotation. In: CLEF Working Notes (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Xixi He
    • 1
    • 2
  • Xirong Li
    • 1
    • 2
    • 3
  • Gang Yang
    • 1
    • 2
  • Jieping Xu
    • 1
    • 2
  • Qin Jin
    • 1
    • 2
  1. 1.Key Lab of DEKERenmin University of ChinaChina
  2. 2.Multimedia Computing LabRenmin University of ChinaChina
  3. 3.Shanghai Key Laboratory of Intelligent Information ProcessingChina

Personalised recommendations