Abstract
Image annotation, the task of automatically generating description words for a picture, is a key component in various image search and retrieval applications. Creating image databases for model development is, however, costly and time consuming, since the keywords must be hand-coded and the process repeated for new collections. In this work we exploit the vast resource of images and documents available on the web for developing image annotation models without any human involvement. We describe a probabilistic framework based on the assumption that images and their co-occurring textual data are generated by mixtures of latent topics. Applications of this framework to image annotation and retrieval show performance gains over previously proposed approaches, despite the noisy nature of our dataset. We also discuss how the proposed model can be used for story picturing, i.e., to find images that appropriately illustrate a text and demonstrate its utility when interfaced with an image caption generator.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lapata, M. (2010). Image and Natural Language Processing for Multimedia Information Retrieval. In: Gurrin, C., et al. Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12275-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-12275-0_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12274-3
Online ISBN: 978-3-642-12275-0
eBook Packages: Computer ScienceComputer Science (R0)