Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Annotation-Based Image Retrieval

  • Xin-Jing Wang
  • Lei Zhang
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_17

Synonyms

Semantic image retrieval; Tag-based image retrieval; Tag-based image search; Text-based image retrieval

Definition

Given (i) a textual query and (ii) a set of images and their annotations (phrases or keywords), annotation-based image retrieval systems retrieve images according to the matching score of the query and the corresponding annotations. There are three levels of queries according to Eakins [ 1]:
  • Level 1: Retrieval by primitive features such as color, texture, shape, or the spatial location of image elements, typically querying by an example, i.e., “find pictures like this.”

  • Level 2: Retrieval by derived features, with some degree of logical inference. For example, “find a picture of a flower.”

  • Level 3: Retrieval by abstract attributes, involving a significant amount of high-level reasoning about the purpose of the objects or scenes depicted. This includes retrieval of named events, of pictures with emotional or religious significance, etc., e.g., “find pictures of a...

This is a preview of subscription content, log in to check access.

Recommended Readings

  1. 1.
    Long F, Zhang HJ, Feng DD. Fundamentals of content-based image retrieval. In: Feng D, editor. Multimedia information retrieval and management: technological fundamentals and applications. Berlin/Heidelberg: Springer; 2013; Part I. p. 1–26. ISSN: 1860-4862.Google Scholar
  2. 2.
    Zhang L, Rui Y. Image search from thousands to billions in 20 years. ACM TOMCCAP 2013;9(1s): Article No. 36. Special Issue on the 20th Anniversary of the ACM MM Conference; New York.Google Scholar
  3. 3.
    Liu Y, Zhang D, Lu G, Ma W-Y. A survey of content-based image retrieval with high-level semantics. Pattern Recogn. 2007;40(1):262–82.zbMATHCrossRefGoogle Scholar
  4. 4.
    Rui Y, Huang TS, Chang S-F. Image retrieval: current techniques, promising directions, and open issues. J Vis Commun Image Represent. 1999;10(4):39–62.CrossRefGoogle Scholar
  5. 5.
    Chang S-F, Ma W-Y, Smeulders A. Recent advances and challenges of semantic image/video search. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing; 2007. p. 1205–8.Google Scholar
  6. 6.
    Blei D, Jordan MI. Modeling annotated data. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2003. p. 127–34.Google Scholar
  7. 7.
    Jeon J, Lavrenko V, Manmatha R. Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2003. p. 119–26.Google Scholar
  8. 8.
    Chang S-F, Chen W, Sundaram H. Semantic visual templates: linking visual features to semantics. In: Proceedings of the International Conference on Image Processing; 1998. p. 531–4.Google Scholar
  9. 9.
    Zhuang Y, Liu X, Pan Y. Apply semantic template to support content-based image retrieval. In: Proceedings of the SPIE, Storage and Retrieval for Media Databases; 1999. p. 442–9.Google Scholar
  10. 10.
    Wang X-J, Zhang L, Jing F, Ma W-Y. AnnoSearch: image auto-annotation by search. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition; 2006. p. 1483–90.Google Scholar
  11. 11.
    Wang X-J, Zhang L, Li X, Ma W-Y. Annotating images by mining image search results. IEEE Trans Pattern Anal Mach Intell. 2008;30(11):1919–32.CrossRefGoogle Scholar
  12. 12.
    Dai LC, Wang X-J, Zhang L, Yu NH. Efficient tag mining via mixture modeling for real-time search-based image annotation. In: Proceedings of the IEEE International Conference on Multimedia and Expo; 2012.Google Scholar
  13. 13.
    Wang X-J, Zhang L, Ma W-Y. Duplicate-search-based image annotation using web-scale data. Proc IEEE. 2012;100(9):2705–21.CrossRefGoogle Scholar
  14. 14.
    Wang X-J, Zhang L, Liu M, Li Y, Ma W-Y. ARISTA – image search to annotation on billions of web photos. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition; 2010.Google Scholar
  15. 15.
    Wang X-J, Zhang L, Liu C. Duplicate discovery on 2 billion internet images. IEEE Conference on Computer Vision and Pattern Recognition; 2013.Google Scholar
  16. 16.
    El-Saban M, Wang X-J, Hasan N, Bassiouny M, Refaat M. Seamless annotation and enrichment of mobile captured video streams in real-time. In: Proceedings of the IEEE International Conference on Multimedia and Expo; 2011.Google Scholar
  17. 17.
    Zhang X, Zhang L, Wang X-J, Shum H-Y. Finding celebrities in billions of webpages. IEEE Transactions on Multimedia. 2012;14(4):995–1007.CrossRefGoogle Scholar
  18. 18.
    Eakins J, Graham M. Content-based image retrieval. Technical report. Tyne: University of Northumbria at Newcastle; 1999.Google Scholar
  19. 19.
    Wang X-J, Yu M, Zhang L, Ma W-Y. Advertising based on users’ photos. In: Proceedings of the IEEE International Conference on Multimedia and Expo; 2009.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Microsoft Research AsiaBeijingChina
  2. 2.Micros FacebookCAUSA
  3. 3.Microsoft ResearchWAUSA

Section editors and affiliations

  • Jeffrey Xu Yu
    • 1
  1. 1.The Chinese University of Hong KongHong KongChina