Abstract
In many vision problems, instead of having fully annotated training data, it is easier to obtain just a subset of data with annotations, because it is less restrictive for the user. For this reason, in this paper, we consider especially the problem of classifying weakly-annotated images, where just a small subset of the database is annotated with keywords. In this paper we present and evaluate a new method which improves the effectiveness of content-based image classification, by integrating semantic concepts extracted from text, and by automatically extending annotations to the images with missing keywords. Our model is inspired from the probabilistic graphical model theory: we propose a hierarchical mixture model which enables to handle missing values. Results of visual-textual classification, reported on a database of images collected from the Web, partially and manually annotated, show an improvement by 32.3% in terms of recognition rate against only visual information classification. Besides the automatic annotation extension with our model for images with missing keywords outperforms the visual-textual classification by 6.8%. Finally the proposed method is experimentally competitive with the state-of-art classifiers.
Chapter PDF
Similar content being viewed by others
References
Barnard, K., Duygulu, P., Forsyth, D., De Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. Journal of Machine Learning Research 3(6), 1107–1135 (2003)
Benitez, A., Chang, S.F.: Perceptual knowledge construction from annotated image collections. ICME 2002 1, 189–192 (2002)
Grosky, W.I., Zhao, R.: Negotiating the semantic gap: From feature maps to semantic landscapes. In: Pacholski, L., Ružička, P. (eds.) SOFSEM 2001. LNCS, vol. 2234, pp. 33–52. Springer, Heidelberg (2001)
Kherfi, M.L., Brahmi, D., Ziou, D.: Combining visual features with semantics for a more effective image retrieval. In: ICPR 2004, vol. 2, pp. 961–964 (2004)
Blei, D.M., Jordan, M.I.: Modeling annotated data. In: SIGIR 2003, pp. 127–134 (2003)
Gao, Y., Fan, J., Xue, X., Jain, R.: Automatic image annotation by incorporating feature hierarchy and boosting to scale up svm classifiers. In: ACM MULTIMEDIA 2006, pp. 901–910 (2006)
Yang, C., Dong, M., Hua, J.: Region-based image annotation using asymmetrical support vector machine-based multiple-instance learning. In: CVPR 2006, pp. 2057–2063 (2006)
Wang, C., Jing, F., Zhang, L., Zhang, H.J.: Image annotation refinement using random walk with restarts. In: ACM MULTIMEDIA 2006, pp. 647–650 (2006)
Rui, X., Li, M., Li, Z., Ma, W.Y., Yu, N.: Bipartite graph reinforcement model for web image annotation. In: ACM MULTIMEDIA 2007, pp. 585–594 (2007)
Feng, S., Manmatha, R., Lavrenko, V.: Multiple bernoulli relevance models for image and video annotation. CVPR 2004 2, 1002–1009 (2004)
Swain, M.J., Ballard, D.H.: Color indexing. International Journal of Computer Vision 7(1), 11–32 (1991)
Tabbone, S., Wendling, L.: Technical Symbols Recognition Using the Two-dimensional Radon Transform. In: ICPR 2002, vol. 2, pp. 200–203 (2002)
Robert, C.: A decision-Theoretic Motivation. Springer, Heidelberg (1997)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39(1), 1–38 (1977)
Kim, J.H., Pearl, J.: A computational model for combined causal and diagnostic reasoning in inference systems. In: IJCAI 1983, pp. 190–193 (1983)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Barrat, S., Tabbone, S. (2008). Classification and Automatic Annotation Extension of Images Using Bayesian Network. In: da Vitoria Lobo, N., et al. Structural, Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2008. Lecture Notes in Computer Science, vol 5342. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89689-0_97
Download citation
DOI: https://doi.org/10.1007/978-3-540-89689-0_97
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-89688-3
Online ISBN: 978-3-540-89689-0
eBook Packages: Computer ScienceComputer Science (R0)