Abstract
We introduce in this paper a novel image annotation approach based on support vector machines (SVMs) and a new class of kernels referred to as context-dependent. The method goes beyond the naive use of the intrinsic low level features (such as color, texture, shape, etc.) and context-free kernels, in order to design a kernel function applicable to interconnected databases such as social networks. The main contribution of our method includes (i) a variational approach which helps designing this function using both intrinsic features and the underlying contextual information resulting from different links and (ii) the proof of convergence of our kernel to a positive definite fixed-point, usable for SVM training and other kernel methods. When plugged in SVMs, our context-dependent kernel consistently improves the performance of image annotation, compared to context-free kernels, on hundreds of thousands of Flickr images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Carneiro, G., Vasconcelos, N.: Formulating semantic image annotation as a supervised learning problem. In: Proc. of CVPR (2005)
Li, J., Wang, J.Z.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. on PAMI 25(9), 1075–1088 (2003)
Barnard, K., Duygululu, P., Forsyth, D., Blei, D., Jordan, M.: Matching words and pictures. The Journal of Machine Learning Research (2003)
Monay, F., GaticaPerez, D.: PLSA-based Image AutoAnnotation: Constraining the Latent Space. In: Proc. of ACM International Conference on Multimedia (2004)
Gao, Y., Fan, J., Xue, X., Jain, R.: Automatic Image Annotation by Incorporating Feature Hierarchy and Boosting to Scale up SVM Classifiers. In: Proc. of ACM MULTIMEDIA (2006)
Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Proc. of ACM SIGIR, pp. 119–126 (2003)
Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: Proc. of NIPS (2004)
Feng, S., Manmatha, R., Lavrenko, V.: Multiple Bernoulli relevance models for image and video annotation. In: Proc. of ICCV, pp. 1002–1009 (2004)
Liu, J., Wang, B., Li, M., Li, Z., Ma, W., Lu, H., Ma, S.: Dual cross-media relevance model for image annotation. In: Proc. of ACM MULTIMEDIA, pp. 605–614 (2007)
Wan, X., Yang, J., Xiao, J.: Manifold-ranking based topic-focused multi-document summarization. In: Proc. of IJCAI, pp. 2903–2908 (2007)
Zhou, D., Weston, J., Gretton, A., Bousquet, O., Schölkopf, B.: Ranking on data manifolds. In: Proc. of NIPS (2004)
Liu, J., Li, M., Liu, Q., Lu, H., Ma, S.: Image annotation via graph learning. Pattern Recognition 42(2), 218–228 (2009)
Liu, J., Li, M., Ma, W., Liu, Q., Lu, H.: An adaptive graph model for automatic image annotation. In: Proc. of ACM International Workshop on Multimedia Information Retrieval, pp. 61–70 (2006)
Srikanth, M., Varner, J., Bowden, M., Moldovan, D.: Exploiting ontologies for automatic image annotation. In: Proc. of SIGIR, pp. 552–558 (2005)
Wu, Y., Chang, E.Y., Tseng, B.L.: Multimodal metadata fusion using causal strength. In: Proc. of ACM MULTIMEDIA, pp. 872–881 (2005)
Miller, G.A.: Wordnet: a lexical database for English. ACM Commun. 38(11), 39–41 (1995)
Wang, C., Jing, F., Zhang, L., Zhang, H.J.: Image annotation refinement using random walk with restarts. In: Proc. of ACM MULTIMEDIA, pp. 647–650 (2006)
Jin, Y., Khan, L., Wang, L., Awad, M.: Image annotations by combining multiple evidence & wordNet. In: Proc. of ACM MULTIMEDIA, pp. 706–715 (2005)
Cilibrasi, R., Vitanyi, P.M.B.: The google similarity distance. IEEE Transactions on Knowledge and Data Engineering (2007)
Wu, L., Hua, X., Yu, N., Ma, W., Li, S.: Flickr distance. In: Proc. of ACM MULTIMEDIA (2008)
Wang, Y., Gong, S.: Translating Topics to Words for Image Annotation. In: Proc. of ACM CIKM (2007)
Lu, Z., Ip, H.H.S., He, Q.: Context-Based Multi-Label Image Annotation. In: Proc. of ACM CIVR (2009)
Boser, B., Guyon, I., Vapnik, V.: An training algorithm for optimal margin classifiers. In: Fifth Annual ACM Workshop on Computational Learning Theory, Pittsburgh (1992)
Vapnik, V.: Statistical Learning Theory. A Wiley-Interscience Publication, Hoboken (1998)
Wang, C., Yan, S., Zhang, L., Zhang, H.: Multi-Label Sparse Coding for Automatic Image Annotation. In: Proc. of CVPR (2009)
Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. In: Proc. of CVPR (2006)
Gallagher, A.C., Neustaedter, C.G., Cao, L., Luo, J., Chen, T.: Image Annotation Using Personal Calendars as Context. In: Proc. of ACM Multimedia (2008)
Cao, L., Luo, J., Huang, T.S.: Annotating Photo Collection by Label Propagation According to Multiple Similarity Cues. In: Proc. of ACM Multimedia (2008)
Sahbi, H., Audibert, J.-Y.: Social network kernels for image ranking and retrieval. In Technical Report, N 2009D009, TELECOM ParisTech (March 2009)
Shawe-Taylor, J., Cristianini, N.: Support Vector Machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000)
Yang, Y.H., Wu, P.T., Lee, C.W., Lin, K.H., Hsu, W.H., Chen, H.: ContextSeer: Context Search and Recommendation at Query Time for Shared Consumer Photos. In: Proc. of ACM Multimedia (2008)
Haussler, D.: Convolution Kernels on Discrete Structures. In Technical Report UCSC-CRL-99-10, University of California in Santa Cruz, Computer Science Department (July 1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sahbi, H., Li, X. (2011). Context-Based Support Vector Machines for Interconnected Image Annotation. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6492. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19315-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-19315-6_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19314-9
Online ISBN: 978-3-642-19315-6
eBook Packages: Computer ScienceComputer Science (R0)