Abstract
One of the most promising new technologies for widespread application is image annotation and retrieval. Nevertheless, this task is very difficult to accomplish as target images differ significantly in appearance and belong to a wide variety of categories. In this paper, we propose a new image annotation and retrieval method for miscellaneous weakly labeled images, by combining higher-order local auto-correlation (HLAC) features and a framework of probabilistic canonical correlation analysis. The distance between images can be defined in the intrinsic space for annotation using conceptual learning of images and their labels. Because this intrinsic space is highly compressed compared to the image feature space, our method achieves both faster and more accurate image annotation and retrieval. The HLAC features are powerful global features with additive and position invariant properties. These properties work well with images, which have an arbitrary number of objects at arbitrary locations. The proposed method is shown to outperform existing methods using a standard benchmark dataset.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bach, F. R. and Jordan, M. I., “A probabilistic interpretation of canonical correlation analysis,” Technical Report 688, Department of Statistics, University of California, Berkeley, 2005.
Blei, D. M. and Jordan, M. I., “Modeling annotated data,” in SIGIR, 2003.
Bosch, A., Zisserman, A. and Munoz, X., “Representing shape with a spatial pyramid kernel,” in CIVR, 2007.
Carneiro, G., Chan, A. B., Moreno, P. J. and Vasconcelos, N., “Supervised learning of semantic classes for image annotation and retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, 29, 3, pp.394–410, 2007.
Carneiro, G. and Vasconcelos, N., “Formulating semantic image annotation as a supervised learning problem,” in CVPR, 2005.
Duygulu, P., Barnard, K. and Freitas, D. F. N., “Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary,” in ECCV, 2002.
Fei-Fei, L., Fergus, R. and Perona, P., “Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories,” in CVPR Workshop on Generative-Model Based Vision, 2004.
Feng, S., Manmatha, R. and Lavrenko, V., “Multiple Bernoulli relevance models for image and video annotation,” in CVPR, 2004.
Frome, A., Singer, Y., Sha, F. and Malik, J., “Learning globally-consistent local distance functions for shape-based image retrieval and classification,” in ICCV, 2007.
Hardoon, D., Saunders, C., Szedmak, S. and Shawe-Taylor, J., “A correlation approach for automatic image annotation,” in Second Int. Conf. on Advanced Data Mining and Applications, ADMA, 2006.
Hardoon, D. R., Szedmak, S. R. and Shawe-taylor, J. R., “Canonical correlation analysis: An overview with application to learning methods,” Neural Computation, 16, 12, pp.2639–2664, 2004.
Jeon, J., Lavrenko, V. and Manmatha, R., “Automatic image annotation and retrieval using cross media relevance models,” in SIGIR, 2003.
Kato, T., Kurita, T., Otsu, N. and Hirata, K., “A sketch retrieval method for full color image database –query by visual example–,” in ICPR, 1992.
Kurita, T., Kato, T., Fukuda, I. and Sakakura, A., “Sense retrieval on an image database of full color paintings,” Trans. of Information Processing Society of Japan, 33, 11, pp.1373–1383, 1992.
Lavrenko, V., Manmatha, R. and Jeon, J., “A model for learning the semantics of pictures,” in NIPS, 2003.
Lazebnik, S., Schmid, C. and Ponce, J., “Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories,” in CVPR, 2006.
Liu, J., Wang, B., Li, M., Li, Z., Ma, W., Lu, H. and Ma, S., “Dual cross-media relevance model for image annotation,” in ACM Multimedia, 2007.
Makadia, A., Pavlovic, V. and Kumar, S., “A new baseline for image annotation,” in ECCV, 2008.
Metzler, D. and Manmatha, R., “An inference network approach to image retrieval,” in CIVR, 2004.
Mori, Y., Takahashi, H. and Oka, R., “Image-to-word transformation based on dividing and vector quantizing images with words,” in MISRM'99 First Int. Workshop on Multimedia Intelligent Storage and Retrieval Management, 1999.
Nakayama, H., Harada, T., Kuniyoshi, Y. and Otsu, N., “High-performance image annotation and retrieval for weakly labeled images using latent space learning,” in PCM, 2008.
Naphade, M. R. and Smith, J. R., “On the detection of semantic concepts at trecvid,” in ACM Multimedia, 2004.
Otsu, N. and Kurita, T., “A new scheme for practical, flexible and intelligent vision systems,” in IAPR Workshop on Computer Vision, 1988.
Wang, M., Hua, X.-S., Hong, R., Tang, J., Qi, G.-J. and Song, Y., “Unified video annotation via multigraph learning,” IEEE Trans. Cir. and Sys. for Video Technol., 19, 5, pp.733–746, 2009.
Yang, J. and Hauptmann, A. G., “(un)reliability of video concept detection,” in CIVR, 2008.
Yavlinsky, A., Schofield, E. and Ruger, S., “Automated image annotation using global features and robust nonparametric density estimation,” in CIVR, 2005.
Young, T. Y., “The reliability of linear feature extractors,” IEEE Trans. on Computers, 20, 9, pp.967–971, 1971.
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Harada, T., Nakayama, H., Kuniyoshi, Y. et al. Image Annotation and Retrieval for Weakly Labeled Images Using Conceptual Learning. New Gener. Comput. 28, 277–298 (2010). https://doi.org/10.1007/s00354-009-0090-z
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00354-009-0090-z