Automatic Image Annotation Using Maximum Entropy Model
Automatic image annotation is a newly developed and promising technique to provide semantic image retrieval via text descriptions. It concerns a process of automatically labeling the image contents with a pre-defined set of keywords which are exploited to represent the image semantics. A Maximum Entropy Model-based approach to the task of automatic image annotation is proposed in this paper. In the phase of training, a basic visual vocabulary consisting of blob-tokens to describe the image content is generated at first; then the statistical relationship is modeled between the blob-tokens and keywords by a Maximum Entropy Model constructed from the training set of labeled images. In the phase of annotation, for an unlabeled image, the most likely associated keywords are predicted in terms of the blob-token set extracted from the given image. We carried out experiments on a medium-sized image collection with about 5000 images from Corel Photo CDs. The experimental results demonstrated that the annotation performance of this method outperforms some traditional annotation methods by about 8% in mean precision, showing a potential of the Maximum Entropy Model in the task of automatic image annotation.
KeywordsMaximum Entropy Image Retrieval Training Image Machine Translation Image Region
Unable to display preview. Download preview PDF.
- 2.Duygulu, P., Barnard, K., de Freitas, N., Forsyth, D.: Ojbect recognition as machine translation: Learning a lexicon fro a fixed image vocabulary. In: Seventh European Conf. on Computer Vision, pp. 97–112 (2002)Google Scholar
- 3.Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th intl. SIGIR Conf, pp. 119–126 (2003)Google Scholar
- 4.Mori, Y., Takahashi, H., Oka, R.: Image-to-word transformation based on dividing and vector quantizing images with words. In: First International Workshop on Multimedia Intelligent Storage and Retrieval Management (1999)Google Scholar
- 5.Chang, E., Goh, K., Sychay, G., Wu, G.: CBSA: Content-based soft annotation for multimodal image retrieval using bayes point machines. IEEE Transactions on Circuts and Systems for Video Technology Special Issue on Conceptual and Dynamical Aspects of Multimedia Content Descriptions 13(1), 26–38 (2003)Google Scholar
- 7.Berger, A., Pietra, S., Pietra, V.: A maximum entropy approach to natural language processing. In: Computational Linguistics, pp. 39–71 (1996)Google Scholar
- 8.Li, J., Wang, J.A.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Transactions on PAMI 25(10), 1075–1088 (2003)Google Scholar
- 9.Jeon, J., Manmatha, R.: Using maximum entropy for automatic image annotation. In: Proceedings of third international conference on image and video retrieval, pp. 24–31 (2004)Google Scholar
- 10.Malouf, R.: A comparison of algorithms for maximum entropy parameter estimation. In: Proceedings of the 6th Workshop on Computational Language Learning (2003)Google Scholar
- 11.Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: Proceedings of the 16th Annual Conference on Neural Information Processing Systems (2004)Google Scholar