Abstract
In this paper, we introduce a novel high-level visual content descriptor devised for performing semantic-based image classification and retrieval. The work can be treated as an attempt for bridging the so called “semantic gap”. The proposed image feature vector model is fundamentally underpinned by an automatic image labelling framework, called Collaterally Cued Labelling (CCL), which incorporates the collateral knowledge extracted from the collateral texts accompanying the images with the state-of-the-art low-level visual feature extraction techniques for automatically assigning textual keywords to image regions. A subset of the Corel image collection was used for evaluating the proposed method. The experimental results indicate that our semantic-level visual content descriptors outperform both conventional visual and textual image feature models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barnard, K., Duygulu, P., Forsyth, D.: Clustering Art. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 434–441 (2001)
Barnard, K., Duygulu, P., de Freitas, N., Forsyth, D., Blei, D., Jordan, M.I.: Matching Words and Pictures. Machine Learning Research 3, 1107–1135 (2003)
Barnard, K., Forsyth, D.: Learning the Semantics of Words and Pictures. In: Proc. Int. Conf. on Computer Vision, II, pp. 408–415 (2001)
Bimbo, A.D.: Visual Information Retrieval. Morgan Kaufmann Publishers, Inc, San Francisco, California, US (1999)
Brown, P., Pietra, S.D., Pietra, V.D., Mercer, R.: The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics 19(2), 263–311 (1993)
Cascia, M.L., Sethi, S., Sclaroff, S.: Combining Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web. In: Proc. of IEEE Workshop on Content-Based Access of Image and Video Libraries, IEEE Computer Society Press, Los Alamitos (1998)
Duygulu, P., Barnard, K., Freitas, N., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 97–112. Springer, Heidelberg (2002)
Enser, P.G.: Query analysis in a visual information retrieval context. Document and Text Management 1, 25–52 (1993)
Haralick, R.M., Shanmugam, K., Dinstein, I.: Texture features for image classification. IEEE Trans. On Sys. Man, and Cyb. SMC 3(6), 610–621 (1973)
Jianbo, S., Jitendra, M.: Normalized Cuts and Image Segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(8) (2000)
Keister, L.H.: User types and queries: impact on image access systems. Challenges in indexing electronic text and images. Learned Information (1994)
Lew, M.S.: Next-generation web searches for visual content. IEEE Computer 33, 46–53 (2000)
Li, J., Wang, J.Z.: Automatic linguistic indexing of pictures by a statistical modelling approach. IEEE Trans. Pattern Analysis and Machine Intelligence 25(9), 1075–1088 (2003)
Markkula, M., Sormunen, E.: End-user searching challenges indexing practices in the digital newspaper photo archive. Information retrieval 1, 259–285 (2000)
Marques, O., Furht, B.: Content-Based Image and Video Retrieval. Kluwer Academic Publishers, Norwell, Massachusetts, US (2002)
Mori, Y., Takahashi, H., Oka, R.: Image-to-word transformation based on dividing and vector quantizing images with words. In: 1st Int. Workshop on Multimedia Intelligent Storage and Retrieval Management (1999)
Morris, T.: Computer Vision and Image Processing. Palgrave Macmillan Publishers, Ltd, New York, US (2004)
Ornager, S.: View a picture: Theoretical image analysis and empirical user studies on indexing and retrieval. Swedis Library Research 2(3), 31–41 (1996)
Paek, S., Sable, C.L., Hatzivassiloglou, V., Jaimes, A., Schiffman, B.H., Chang, S.F., McKeown, K.R.: Integration of visual and text based approaches for the content labelling and classification of Photographs. In: ACM Workshop on Multimedia Indexing and Retrieval. ACM Press, New York (1999)
Rui, Y., Huang, T.S., Chang, S.F: Image Retrieval: current techniques, promising directions and open issues. Visual Communication and Image Representation (1999)
Smeulder, A.W.M., Worring, M., Anntini, S., Gupta, A., Jain, R.: Content-Based Image Retrieval at the End of the Early Years. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(12) (2000)
Srihari, R.K.: Use of Collateral Text in Understanding Photos. Artificial Intelligence Review. Special Issue on Integrating Language and Vision 8, 409–430 (1995)
Srihari, R.K.: Computational Models for Integrating Linguistic and Visual Information: A Survey. Artificial Intelligence Review, Special Issue on Integrating Language and Vision 8, 349–369 (1995)
Westerveld, T.: Image Retrieval: Content Versus Context. In: Proc. of Content-Based Multimedia Information Access, pp. 276–284 (2000)
Zhou, X.S., Huang, S.T.: Image Retrieval: Feature Primitives, Feature Representation, and Relevance Feedback. In: IEEE Workshop on Content-based Access of Image and Video Libraries (2000)
Zhou, X.S., Huang, S.T.: Unifying Keywords and Visual Contents in Image Retrieval. IEEE Trans. Multimedia 9(2), 23–33 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhu, M., Badii, A. (2007). Collaterally Cued Labelling Framework Underpinning Semantic-Level Visual Content Descriptor. In: Qiu, G., Leung, C., Xue, X., Laurini, R. (eds) Advances in Visual Information Systems. VISUAL 2007. Lecture Notes in Computer Science, vol 4781. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76414-4_37
Download citation
DOI: https://doi.org/10.1007/978-3-540-76414-4_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-76413-7
Online ISBN: 978-3-540-76414-4
eBook Packages: Computer ScienceComputer Science (R0)